Video 2 of 5
What research is, and what makes it scientific
Research is a systematic investigation designed to produce generalizable knowledge. Four features make it scientific: the approach is empirical, the procedures are public, the goal is inference, and the conclusions are uncertain.
7:20 · 16 slides · printable slides · transcript
Slides
Printable deck →▶Transcript16 sections
Generated from the narration script. Plain text version.
1This video works through the definition of research — a systematic investigation designed to develop or contribute to generalizable knowledge — and then the four characteristics that make research scientific. It also introduces two kinds of inference: descriptive and causal.
2Research is a systematic investigation designed to develop or contribute to generalizable knowledge. That definition has three parts. Systematic means the work follows a documented and repeatable methodology. Investigations include research development, testing, and evaluation. And generalizable knowledge means the investigation intends to advance our scientific understanding, going beyond the collection of facts about a particular sample to conclusions that have relevance for other scholars, practitioners, or policymakers.
3That sounds straightforward enough, but the boundaries of research can get fuzzy in practice. Picture interviewing parents of young children about their use of mosquito nets. You analyze the transcripts and develop new ideas about the barriers to bed net use that you hope to publish. Other scholars read the work, use it to develop new theories of health behavior, and create interventions that promote bed net use. This is research. But what if a journalist wants to write a feature article about the burden of malaria and interviews a few of the same parents? Is this research?
4No, and this distinction matters. For one, the journalist might not follow a systematic method for deciding which parents to approach, how to conduct the interviews, or how to synthesize what they learn. Second, the journalist has a different objective. You wanted to systematically advance our understanding of barriers to bed net use, insights that might apply to different parents in other settings. The journalist intends to inform the public by telling the stories of a few specific parents. Both are valuable, but only one is research.
5Another way a study can advance scientific understanding is by developing and testing scientific methods and procedures. A research team might plan a small pilot test to collect initial data that will inform the design of a larger study. In most cases we would consider these pre-study activities to be research, even if the team does not intend to publish the results, because the pilot study is part of the knowledge generation process.
6But here again, intent matters. Consider another gray area. Let's say Facebook randomly assigns a small percentage of its users to receive email campaign A or campaign B, and tracks which campaign generates the most clicks or sales. This looks a lot like an experiment, and technically it is. But the company's objective is to determine which campaign optimizes their marketing spend, not to say something more general about human perception and behavior. Therefore, it's not considered research under the Common Rule definition.
7If you're wondering whether your own work crosses the line into research, you're asking the right question. It's always a good idea to consult with an Institutional Review Board to determine if your proposed work is considered research, and if it is, whether it falls under policies requiring ethical review. A later chapter takes that question up in full.
8Whether you're designing a study that relies on qualitative methods, quantitative methods, or a blend of both, several main characteristics of scientific research apply to global health. The approach is empirical. The procedures are public. The goal is inference. And the conclusions are uncertain. These four might seem obvious, but each one is worth unpacking.
9Science is built on systematic data collection, and that is what makes it an empirical endeavor. Expert opinion is a form of evidence, but it's not empirical evidence. Empirical evidence comes from systematic observation, and the method of observation can be quantitative or qualitative. Contrary to what some people believe, empirical is not a synonym for quantitative.
10Scientific research uses public methods that can be examined and replicated. Think of a Method section in a scientific paper like a recipe. If you've ever tried to follow a confusing recipe, you can appreciate the importance of good documentation. Your study's recipe must be clear, meaning well written. Thorough, with no dash of this or that. And shared publicly, not a secret passed down to lab members.
11We care about complete and transparent reporting in science for several reasons. First, as consumers of research, we rely on authors' descriptions of their empirical methods to come to our own conclusions about the findings. If research colleagues cannot inspect your methods, they will have little reason to trust your results. Second, no one study should ever rule the day. If the results of your study are robust, another research group should be able to follow the recipe and replicate the findings. Third, sharing your methods makes scientific progress possible.
12So we observe, and we document our observations carefully. But then what? Empiricism is essential to science, but science is more than observation. To create generalizable knowledge, you need data and inference. There are two broad categories. Descriptive inference is using the data we observe to make conclusions about that which we don't or can't observe directly. You might survey 200 people about their health beliefs, but your real aim is to make conclusions about the broader group.
13Causal inference involves a different mental leap, where we ask what if to make conclusions about causes and effects. Consider the case where we want to know which pill works better to resolve an illness, the red one or the blue one. The fundamental challenge to getting an answer is that we can't give someone both pills simultaneously. An individual can only take one pill at a time. In this situation we might be able to randomly assign people to each type of pill as a tool for making a causal inference. But frequently, random assignment isn't possible, and we have to use other strategies for asking what if.
14This last characteristic is perhaps the hardest for new researchers to accept. Every method has limitations, every measurement has error, and every model is wrong to some extent. Take the estimation of maternal mortality rates as an example. Hogan published estimates for 181 countries. Some countries, such as the United States, have vast amounts of data in vital registries that attempt to track all births and deaths. It's not perfect, so we still estimate the maternal mortality rate using a statistical model.
15In the left panel, the United States has a relatively low level of maternal mortality, between about 10 and 20 maternal deaths for every 100,000 live child births. Compared to some countries, the US has a lot of data points for estimating the level and trend in maternal deaths, so the uncertainty band is narrow. Now take a look at Afghanistan on the right. The y-axis scale is much larger, in the thousands, reflecting the fact that many more Afghan women die of causes related to pregnancy or childbirth. The uncertainty band spans a range of more than 3,000 deaths, compared to a range of fewer than 5 deaths in the US.
16The takeaway message is that uncertainty is everywhere in science, and that's okay. No single estimate can be considered the answer. Embrace uncertainty, quantify it when you can, and you will become a better scientist. Two terms carry forward from this video: descriptive inference, using observed data to make conclusions about what we can't observe directly, and causal inference, asking what if to draw conclusions about causes and effects. In the next video we turn to the kinds of research that make up global health.