Quality and rigor in qualitative and mixed methods research
Slide 1If qualitative research doesn't have statistical significance, confidence intervals, or effect sizes, how do we judge whether it's any good?
Qualitative research has its own quality criteria
They reflect a different epistemology
Quantitative research tries to control for bias through design
Qualitative research treats bias as inevitable and demands transparency
Slide 2Qualitative research does have quality criteria. They're different from the ones quantitative research uses, and they reflect a different epistemology. Quantitative research tries to control for bias through design. Qualitative research treats bias as inevitable, and it demands that researchers make their decisions, assumptions, and analytical moves transparent enough that readers can judge them.
Lincoln and Guba
Four criteria, each mapped loosely onto a quantitative concept
Slide 3Most qualitative researchers work from a four-part framework: credibility, transferability, dependability, and confirmability. Each maps loosely onto a quantitative concept: internal validity, external validity, reliability, and objectivity. But the underlying logic is different.
Credibility
Do the findings believably represent participants' perspectives?
Prolonged engagement, triangulation, member checking, peer debriefing
The Syria vaccination study triangulated interviews, a workshop and observations
Convergence on the same governance gaps was the source of credibility
Slide 4Credibility asks whether the findings believably represent participants' perspectives. The classical tactics are prolonged engagement, triangulation across multiple data sources, member checking with participants, and peer debriefing with colleagues who are willing to challenge your interpretations. The vaccination governance study in Syria triangulated key-informant interviews, a validation workshop, and ethnographic observations. When all three converged on the same governance gaps, that convergence was the source of credibility.
Transferability
Do the findings travel?
Thick description lets readers judge applicability to their own setting
Purposive sampling that captures variation strengthens it further
In Goa and Rawalpindi (Singla et al., 2014), peer-worker characteristics mattered differently by setting
Slide 5Transferability asks whether findings travel. The key tactic is thick description: enough detail about context, participants, and processes that readers in different settings can judge applicability themselves. Purposive sampling that deliberately captures variation strengthens transferability further. One peer mental health worker study sampled across two distinct contexts and found that the relative importance of peer-worker characteristics varied by setting, a finding that itself tells readers something about when and where the work might transfer.
Dependability
Could another researcher following the same process reach similar conclusions?
An audit trail of methodological decisions
Codebooks showing how categories were defined and applied
Analytic memos showing how interpretation evolved
"I read the transcripts and themes emerged" does not meet this standard
Slide 6Dependability asks whether another researcher following the same process could reach similar conclusions. The answer hinges on documentation. A serious qualitative project maintains an audit trail of methodological decisions, codebooks showing how categories were defined and applied, analytic memos showing how interpretation evolved, and drafts showing how the argument was sharpened. I read the transcripts and themes emerged does not meet this standard.
Confirmability
Are the findings grounded in data, not the researcher's prior commitments?
Reflexivity: name your assumptions, positionality and potential biases
Clear linkage to data: every major claim traces to a quote, note or document
Negative case analysis: seek out the data that do not fit
Slide 7Confirmability asks whether findings are grounded in data and not in the researcher's prior commitments. Three habits keep work confirmable. The first is reflexivity: explicitly examining your own assumptions, positionality, and potential biases, and naming them in the paper. The second is clear linkage to data, where every major claim traces to a quote, field note, or document excerpt that readers can see. The third is negative case analysis. If your analysis claims all participants experienced stigma but three interviews suggest otherwise, those exceptions are the finding, not a problem to hide.
Meuleman et al., 2025
What rigor makes possible
Systematic collection, transparent analysis, ongoing questioning of assumptions
Then qualitative findings can illuminate causal mechanisms
Identify important subgroups, and explain how associations operate
Slide 8A scoping review of qualitative research's contributions to clinical and epidemiological understanding makes the same point. Rigor emerges through systematic collection, transparent analysis, and ongoing questioning of researcher assumptions. Only when work is conducted that way can qualitative findings reliably illuminate causal mechanisms, identify important subgroups, or explain how associations operate.
Mixed methods studies must also clear the bar of integration
Each strand has to meet its own quality standards
Weak studies leave the numbers and the stories side by side
Without the two strands ever talking to each other
Slide 9Mixed methods studies have to clear the quality bar for both strands, and then clear one more bar that neither strand alone has to: the quality of integration. This is where most weak mixed methods studies fail. The numbers and the stories are each fine in isolation, but they sit side by side on the page without ever truly talking to each other.
Three dimensions of integration quality
Design: does the design fit the question, with timing and priority justified?
Analytical: were the strands brought into contact, for example in joint displays?
Interpretive: does the discussion produce meta-inferences neither strand could make?
Slide 10Three dimensions of integration quality are worth checking. Design integration asks whether the choice of design fits the question. The Syria study used a convergent design because the team needed coverage data and stakeholder perspectives in parallel, and the Ethiopia study used explanatory sequential design because they needed clinicians to explain the capacity gaps. Analytical integration asks whether the team actually brought the two strands into contact and explored convergence, divergence, and expansion. Interpretive integration asks whether the discussion produces meta-inferences: claims that neither strand could have made alone.
In Closing
If you removed either strand, would the rest still tell a complete story?
If yes, the study probably has not integrated
True integration means both strands are necessary for the full picture
Slide 11A useful self-test: if you removed either the quantitative or the qualitative section from your paper, would the remaining section still tell a complete story? If yes, you probably have not integrated. True integration means both strands are necessary for the full picture.