How to read research like a pro in five minutes cover image

How to read research like a pro in five minutes

Dorian Minors • November 18, 2015

This is an archived article from our predecessor website, The Dirt Psychology. The idea there was to take psychological scholarship and turn it into wisdom. The Armchair Collective tries to go a little further than just psychology. As such, these articles live here in archive form, until they're updated.

Research can be a crock of s**t. Saying that the 'research says' doesn't mean anything if the research is problematic. That's why, here, we always try to link the article so one can evaluate for oneself. Although, if I'm honest, there are plenty of examples in which that has been neglected, and you'd need access to someone at a University a lot of the time to read without paying. The problem is not only that there are now money-making enterprises that will publish research without evaluating it, but that statistics can easily be manipulated to misrepresent the truth. For example:

. Can you sense how much fun this article will be? Stick with it, though. Lest you be convinced somewhere down the line that, say, cracking your knuckles will give you brain cancer.

It's rather easy to confuse the public. So I'm gonna give you an intro into research analysis so you can do it like a pro. Just whip this article out, press ctrl/command+f in your research article and search for the relevant terms (italicised) listed below.

Reliability

Reliability refers to whether one is likely to get the same outcome again and again. We're talking about the consistency of the result. We need to make sure a study has tested that it's reliable; else it's results may not reflect the most common outcome for that subject. So we need to look for one or more of these (I often use the 'Find' function within the PDF to search quickly for the key terms):

We want at least one, if not more of the above.

Validity

Validity means that the study is testing the thing it's supposed to be. So if a study says it's testing the anxiety, we want to make sure it's testing for anxiety and not depression or how sick the person is on that day for example. However a study needs to be reliable to be valid so look for that first. Then look for:

. If you don't feel a sense of achievement for getting this far into the article, you should. This how satisfying it will be to ask your hyperbole-prone friend 'I wonder if that statistic is convergently valid', and watch him try and talk it away.

Last word

Reliability and validity are two of the most important aspects to check for in a study. If you have these, even if the data is misrepresented in other ways, it won't be too far off the mark. Remember though, that everyone has a bias and researchers will often be biased in a specific way so don't just take things for granted (especially if it goes against something that appears to be common knowledge), but don't assume that new research isn't valid either. Check yourself, or risk being wrong because some jerk researcher tricked you. A pretty dense article, I think. Why don't you treat yourself with something a little more straightforward? Like how one simple sentence can double your persuasiveness. Or learn how dodgy sources can be MORE convincing than a legitimate source, by psyching you out. Giving you the dirt on your search for understanding, psychological freedom and 'the good life at The Dirt Psychology.

Turning scholarship into wisdom we can use at The Armchair Collective.

Questions? Comments? Comment sections are a pain to moderate. But this inbox is always read, so send an email. You'll get a reply. Your question might even get a whole article of its own.

More articles? View them all, or check these out:

Dorian Minors

I mostly do brain science. Sometimes I train honeybees. I promise they're related. I made this site because there's no reason why scholars should be the only ones to own knowledge. My special interests are interpersonal relationships, the science of community, spirituality and the brain, and the neural basis of complex behaviour. I hope this stuff is as interesting to you as it is to me. You can find out more about me here.