developing a Critical approach towards reading research

As research students we’re critical about our professors, funding, social media, bollywood, the economy, and ourselves - it’s about time we also get critical about the research that we read! The first step involved in any research process is devouring hundreds of research papers in a week (okay, maybe I exaggerated a little). But reading is such a ‘given’ function, that we forget how integral it is in our understanding and conceptualising of a research project. Reading is embedded throughout the research process for we’re trying to solve one piece of a much larger puzzle. It could range from understanding what other pompous researchers have said in the past, where our work fits in, what still remains unanswered and a host of other important things. And for all of that, you have to READ. 

For students and early-career researchers, it can take some time to move from simply reading research to critically reading research. To critically read means to assess, scrutinize and synthesize the piece of information you encounter so that it makes sense to you and your research. Adding studies that are only minimally relevant to you can confound the specificity of your idea, additionally just reading the limitations sections won’t cut it. When you’re starting out, it’s easy to assume that because something has been published in an academic journal, everything written in it must be taken at face value. With time, you begin questioning what you read (and eventually your decision to get into research). Remember: all that glitters is not gold, and all that’s published is…. definitely not gold. Enough of banter, let’s get to business: 

Before you begin reading a research paper, take a few minute to ask yourself: why am I reading this paper? What is your goal, objective, or intention behind going through this particular piece? Maybe you’re beginning your review of literature, where you read the seminal or the most important/highly cited/classic paper. You would want to read it line by line, multiple times to really get at what it's trying to say. But you’re not going to read every research paper from top to bottom. You could be looking for something: supporting a claim or an argument, trying to identify a gap, considering a methodological approach and its feasibility, just coz you’re bored (yes we exist!) etc. Knowing why you’re reading a paper helps you lock in on what you actually need from it, narrowing down on the really important information. Hence, your purpose determines not only what you read, but how you read it, eventually determining how much time you spend on it.

In psychology, we learn a few broad rules about research designs. A longitudinal design may have certain advantages over a cross-sectional design. A randomized controlled trial (RCT) may offer advantages that a single-group design does not. But the type of design alone does not tell you whether the research was carried out well enough for you to replicate or build upon! An RCT can be poorly implemented. A relatively simple research design can be implemented extremely carefully with interesting results. What really matters is what the researchers actually did; this is why the methodology section deserves your attention.

Start with the sample. It’s an easy target to critique, but for good reason. “Is the sample appropriate for the population or phenomenon the researchers want to draw conclusions about?” How were the participants recruited? What was the sampling method? Is enough demographic information provided to understand who these findings actually apply to? 

Then look at the design and procedures. How exactly was the study implemented? What specific instructions were given to participants? How transparent are the researchers in explaining what they did? Take priming research, for example. Simply saying that a study used “priming” doesn’t tell you enough. Was it implicit, explicit, contextual, or subliminal? How exactly was the manipulation carried out? Different approaches bring different methodological considerations, and you only discover these when you actually get into the details.

The same applies to measurement. If researchers used a self-report scale, was it validated for the population they used it with? Were attention checks included? How were the variables operationalized? And then comes the part many psychology students would happily skip: the results tables.

Quantitative data can stress us out, but take a look at those little tables, spend some deliberate time going through the rows and columns of hell. You don’t necessarily have to understand every statistic immediately but start by asking whether the results being reported actually seem to justify the claims being made later in the discussion. Each section therefore deserves a slightly different kind of scrutiny and so depending on why you’re reading the paper, you might skim parts of the review of literature, Introduction, Conclusion etc. But when you’re trying to judge the quality of a study, the methodology and results deserve particular attention.

This is perhaps the biggest shift that happens when you begin reading research critically: realizing that the paper you’re reading is not the gospel truth. Let’s be honest here, research is carried out by us: mere human beings trying to survive, publish papers, meet deadlines, and get by the day with our hot cup of coffee. And wherever humans are involved, some degree of judgement and bias can creep in. A research paper is evidence; it is one study conducted in a particular context, with a particular sample, using particular methods, producing a particular set of findings (i promise i'm not gonna use ‘particular’ anymore).

And this doesn’t apply only to empirical, quantitative papers. Even review papers and systematic reviews, despite having structured protocols for searching, selecting, and evaluating studies, involve decisions made by humans. Search strategies are constructed by people; inclusion and exclusion criteria are decided by people; quality assessments are conducted by people; relevant research may sometimes be missed. So keep freaking questioning…

Just because something is published does not automatically make it true. Use your judgement, ask questions and compare it with other similar research in that area. And don’t worry if you don’t immediately spot every methodological issue in a paper. This critical ability develops with experience, the more research you read, the more easily you begin noticing things that you probably wouldn’t have noticed earlier.

Acknowledgements, conflict-of-interest declarations, and funding statements are lonely and sit quietly at the end of a research paper. Please offer them your attention because they can change the way you interpret the entire study. Imagine that you’re reading a study demonstrating the effectiveness of a commercial product or an exercise method and then discovering that the research was funded by the company producing that product or service. That doesn’t mean that the findings are wrong but it is information you should know when interpreting them. Who funded the research? “Do the authors declare any potential conflicts of interest?” Were any private organizations involved in the study? These statements take barely a minute to read but they give you important context for evaluating the research. Consider reading Bad Science by Ben Goldacre who highlights how the general public’s understanding of science is sometimes amiss and incomplete, thanks to the media portrayal which lacks criticality.

Finally, look at how transparent the researchers are about their procedures and data.

Is the dataset available? Are the materials or protocols accessible? Is there enough information in the paper for you to understand exactly what was done?

^If not, you can always email the researchers asking for more information and they might just oblige!. This is not solely the case for quantitative papers, there is a slow but steady movement towards greater transparency in qualitative research as well. Qualitative interviews, for example, are built around interpretation, context, and exploration, and there may be genuine ethical and confidentiality reasons why raw data cannot simply be uploaded publicly. But that doesn’t mean you shouldn’t ask what can be made transparent. Where appropriate, qualitative researchers share materials such as interview schedules, information about theme development, codebooks, and, where ethically appropriate and consistent with participant consent, even transcripts.

Never take off your critical hat while reading research. Now I’ll take my writing hat off and go to sleep, until next week…

It’s your always-on revenue engine: agents and workflows build pipeline, chase every buying signal, and move deals forward alongside your team.

Teams like Parallel, Turbopuffer, and Wordsmith build on Attio. Are you one of them?

Try Attio now

“If I were a farmer, I would absolutely use AI.” 

That’s Jensen Huang, founder and CEO of NVIDIA. 

He’s pointing to one of AI’s biggest untapped opportunities: farming. 

DIT AgTech is already putting AI to work with 500+ units deployed and 370,000 head on-platform.

Invest in DIT AgTech Today

^{𝘐𝘯 𝘮𝘢𝘬𝘪𝘯𝘨 𝘢𝘯 𝘪𝘯𝘷𝘦𝘴𝘵𝘮𝘦𝘯𝘵 𝘥𝘦𝘤𝘪𝘴𝘪𝘰𝘯, 𝘪𝘯𝘷𝘦𝘴𝘵𝘰𝘳𝘴 𝘮𝘶𝘴𝘵 𝘳𝘦𝘭𝘺 𝘰𝘯 𝘵𝘩𝘦𝘪𝘳 𝘰𝘸𝘯 𝘦𝘹𝘢𝘮𝘪𝘯𝘢𝘵𝘪𝘰𝘯 𝘰𝘧 𝘵𝘩𝘦 𝘪𝘴𝘴𝘶𝘦𝘳 𝘢𝘯𝘥 𝘵𝘩𝘦 𝘵𝘦𝘳𝘮𝘴 𝘰𝘧 𝘵𝘩𝘦 𝘰𝘧𝘧𝘦𝘳𝘪𝘯𝘨, 𝘪𝘯𝘤𝘭𝘶𝘥𝘪𝘯𝘨 𝘵𝘩𝘦 𝘮𝘦𝘳𝘪𝘵𝘴 𝘢𝘯𝘥 𝘳𝘪𝘴𝘬𝘴 𝘪𝘯𝘷𝘰𝘭𝘷𝘦𝘥. 𝘋𝘐𝘛 𝘈𝘨𝘛𝘦𝘤𝘩 𝘩𝘢𝘴 𝘧𝘪𝘭𝘦𝘥 𝘢 𝘍𝘰𝘳𝘮 𝘊 𝘸𝘪𝘵𝘩 𝘵𝘩𝘦 𝘚𝘦𝘤𝘶𝘳𝘪𝘵𝘪𝘦𝘴 𝘢𝘯𝘥 𝘌𝘹𝘤𝘩𝘢𝘯𝘨𝘦 𝘊𝘰𝘮𝘮𝘪𝘴𝘴𝘪𝘰𝘯 𝘪𝘯 𝘤𝘰𝘯𝘯𝘦𝘤𝘵𝘪𝘰𝘯 𝘸𝘪𝘵𝘩 𝘪𝘵𝘴 𝘰𝘧𝘧𝘦𝘳𝘪𝘯𝘨, 𝘢 𝘤𝘰𝘱𝘺 𝘰𝘧 𝘸𝘩𝘪𝘤𝘩 𝘮𝘢𝘺 𝘣𝘦 𝘰𝘣𝘵𝘢𝘪𝘯𝘦𝘥 𝘩𝘦𝘳𝘦: https://bit.ly/4bzuWCi​}

Most founders have LinkedIn traction with nothing to show for it in the CRM. On Sep 30, Maria Gharib (Mindstream) and Valerie Chapman (Ruth AI) walk through the exact system live.

From AI-assisted content creation to sequenced outreach to booked meeting. You'll leave with a process you can run the same day.

Eligible startups also get the LinkedIn-to-Leads Toolkit: ad credits, Apollo, Captions, and HubSpot's Prospecting Agent.

Get my spot