LDS Church Growth, Member Activity, and Convert Retention: Review and Analysis

Return to Table of Contents

Chapter: III-01: Levels of Evidence

Levels of evidence are used in contemporary medical research to separate high-quality research from low-quality research and anecdotal observation, fostering greater awareness in researchers of the need for rigorous study design and assisting readers in assessing the relevance and applicability of published articles. The importance of understanding the level of evidence or quality of research is such that the best major journals in my medical specialty do not publish original articles without specifying the level of evidence. To my knowledge, the concept of levels of evidence has not been previously introduced in the sociologic study of religion, yet I believe that thoughtful researchers will recognize the value these principles offer.

The basic levels of evidence as I have adapted them for the sociologic study of religion are as follows, with level I representing the highest research quality, and level V the lowest:

Level I: High-quality randomized controlled prospective studies
Level II: Original retrospective cohort studies, and low-quality randomized controlled prospective studies (i.e. <80% follow-up). Also systematic reviews of cohort studies.
Level III: Case control studies, non-consecutive studies. Good-quality observational surveys with >80% response rate.
Level IV: Case series and poor quality cohort and case-control studies. Poor quality observational surveys with less than 80% response rate.
Level V: Expert opinion without evidence-based critical appraisal, based on desk research or "first principles."

More detailed discussion of levels of evidence can be found at the Centers for Evidence-Based Medicine and in high-quality medical journals like the Journal of Bone and Joint Surgery.

Almost all previously published sociologic research on LDS growth and member participation has been level III, IV, or V. To a great extent, this reflects the intrinsic limitations of religious social science research. By nature, sociologic data relies upon a variety of assumptions: that survey samples accurately represent the larger population without bias or skew, that individuals surveyed accurately reported religious preferences and other behaviors, and that church-reported data reflects, at least to some degree, meaningful trends in the lives of members.

Conclusions drawn from sociologic data are rarely amenable to the same standards of empirical proof demanded in medicine or the hard sciences, and the same data are sometimes interpreted in very different ways by informed scholars. The difficulty in obtaining full robust empirical data makes meticulous focus on research methodology and careful analysis of existing limitations particularly important, as relatively minor errors which are not recognized may lead one completely down the wrong track.

Some limitations can be remedied; others can only be acknowledged and considered in the data analysis. Nonetheless, thoughtful attention to these issues is essential to ensure that research done is careful and that analysis is meticulous.