An Introduction to Research Design

by Martha Brown Menard Ph.D
(Massage Therapy Journal – Summer 1995, v. 34, n. 3, pp. 47-50)

The structure of research

From the previous article, you are already familiar with the most basic element—the research question. It will most likely need to be narrowed down to one (or more) specific and testable hypothesis, or objective. A good research question should be interesting and practical in its scope. In the next article, I will discuss in more detail some considerations in developing your question.

The significance of a research question has to do with the relevance, or point, of the answer to it, and the consideration of its novelty, or what new knowledge it will provide. In order to demonstrate what new light your study will shed on an area, you will need to place your research question into the context of what is already known about it. A review of the literature organized around your question should explain what is known, how answering your question will add to existing knowledge, and why it is important to answer this particular question. You should be able to provide a rationale for your study. You may, once you begin reading about previous studies, decide to modify your original question based upon what you discover.

The next element is the study design. Your research question and how you frame it will largely determine your choice of design. While a complete description of all the different types of research designs is beyond the scope of this article, most can be classified into a few broad categories. The broadest is the choice between a study where the researcher observes the phenomena of interest without interaction or manipulation of the environment, or one where the researcher applies an intervention and then measures the results. The former are called observational studies and the latter are experimental studies. Experimental studies always involve some quantitative comparisons between two or more different groups. Descriptive studies are a type of observational study. These can be either qualitative or quantitative. With either design, the researcher must decide to make her observations on only the occasion or across some period of time. If she chooses a single observation, it is a cross-sectional design; if she observes over a period of time, it is a longitudinal study. When little is known about an area, the pattern of research generally tends toward doing descriptive studies first, and then experimental studies. There are always trade-offs inherent in the choice of a particular design; focusing one aspect of a problem will necessarily exclude other aspects.

What are the variables included in your study? Again, these will be suggested by your research question. If you are interested in how massage affects scar tissue formation, then massage will be one variable and scar tissue formation another. (Of course, you will define these more precisely.) You might also want to know something about the people who would receive the massage. Age, gender and ethnicity are also common variables that are included in many research studies. Variables are classified according to their role in the research design. In the example above, massage would be the independent variable and scar tissue formation the dependent variable. An intervention like massage is always an independent variable. Some researchers, such as epidemiologists, prefer the terms predictor and outcome variables instead, because they find these terms more accurate. Confounding variables relate to both the independent and dependent variables, and necessitate collecting information about characteristics of the people in your study. A confounding variable, such as age, can cause a spurious relationship to appear between the independent and dependent variables, and you must control or assess for this possibility.

Who will be your subjects or participants? They will be the sample from which will try to generalize your results to a larger population, in a quantitative study, or describe and understand a situation or phenomenon in a qualitative study. You will need to consider what are called sampling criteria. It may not be possible or ethically appropriate to measure or interview all of the patients in a hospital unit. You must define the characteristics of the particular subject of your study—which patients are appropriate to the question. Access to subjects is also a major consideration in research design, and you will want to think about where and how you will recruit them.

Statistical issues are also an important element in quantitative studies. If this is a subject which is unfamiliar or uncomfortable, I recomment getting advice from someone with more experience. One of the reasons for narrowning down your research question to a speccific hypothesis is so it can be tested for statistical significance. This is more important for experimental studies than for descriptive ones. The particular statistical test used will depend on your question and how it is framed. In any quantitative study, you will also need to estimate your sample size. This tells you how many subjects you will need in order to find any real differences between the groups in your study, or have an accurate estimate of the phenomena being described. It is important to consult an statistician while you are still planning youtr study. Otherwise, going to a consultant after you have collected your data and asking them to do something with it is like engaging an architect after you’ve already built your house.

The process of research

One purpose of doing a research study is being able to draw some conclusion from the results. How trustworthy is it? Generally with quantitative studies there are two concerns. One is the internal validity of the study, which Krathwol defines as the “capacity of the study to link cause and effect.” The design of the study should exclude other rival hypotheses as plausible alternative explanations for the results. The other concern is the external validity, or the generalizability of the results. An intervention may cause an outcome in particular sample studies, but does that mean it will always cause the same outcome in the larger population from which the sample was drawn? Or to other similar populations? There may be a trade-off in terms of these two. Studies with high internal validity may have low external validity, and vice versa. You will have to consider the relative importance of each to your question as you make design decisions, and try to come up with an appropriate balance. The same concerns in qualitative research are termed credibility and transferability.

Now you are ready to start getting concrete in terms of your subjects and your variables. Going back to the previous example, you will need to decide which people can benefit from massage to reduce scar tissue formations. Since you won’t be able to provide massage ror this entire population, let alone measure the results, you will have to choose a sample to represent them. You will also want to define more precisely what you mean by “massage,” and figure out how you will measure scar tissue formation, in order to assess whether or not your intervention has been effective. These choices involve making the study more practical. They also involve the potential to introduce error. Your actual subjects, variables, and measurements used to test your hypothesis will all differ to some extent from the general research question that they represent. Error can be random, such as chance variations in measurements. Adequate sample size reduces the impact of chance variation. Error can also be systematic, which is termed bias. Your choice of subjects and how you select them can be used to eliminate bias as an alternative explanation. The point of doing statistical tests is to eliminate random error, or chance, as an alternative explanation. It is important to remember that statistical tests alone can never prove causality. While there will always be some amount of error in any study, it is again a trade-off between what is practical, given your resources, and how much proof you need in order to trust your conclusion.

The study outline

All this is a bit much to grasp at once, just as understanding the entire musculoskeletal system is overwhelming at first. The study outline is a way to begin organizing all these different elements. Start with the research question, just one or two sentences, that tells you what you will know as a result of the study. Then begin preparing a one or two-page outline which contains the elements discussed earlier. Following the study question, it should include the significance of the study, or why the question is important; the design, or how the study will be carried out; the subjects, who and how selected; and the variables, or what will be measured. If you can put these together, your outline will provide enough information for a statistical consultant to help with the statistical issues. This section should have the estimated sample size needed and the type of statistical analysis that will be performed.

Summary

Going through the different elements in this sequence will help clarify your question and your thinking. It will take time, as does understanding anatomy and physiology. Look at each element one at a time and then consider how it fits together with the whole. You will most likely need to go back to earlier steps and make changes as yu develop your questions and your outline. Known as an iterative process, it initially takes a good deal of time, but a clear and concise statement of the research question and how you will address it is well worth the effort. In the next article I will discuss some guidelines for choosing a research question.

Resources for further reading

Bass, M., Dunn, E., Norton, P., Stewart, M., and Tudiver, F. (Eds.) (1993). Conducting research in the practice setting. Newbury Park: Sage Publications, Inc. (This book gives an overview of issues in conducting clinical research in a primary care setting, and is relevant to massage therapists interested in doing research within their own practice, or within an organization.)

Gross, R. (1993). The independent scholar’s handbook. Berkeley: Ten Speed Press. (Inspiring stories of individuals who have done important research on their own, along with practical suggestions for those who are interested in researdch but have no previous experience.)

King, G., Keohane, R., and Verba, S. (1994). Designing social inquiry. Princeton: Princeton University Press. (A highly readable work on design issues in qualitative research, and how to improve the quality of inference in research.)