Which is more effective, a local intervention carefully modified to account for context or a large-scale program that can be broadly applied to many populations?
The above question and its other variations (e.g. with regards to research design) is an extremely pertinent one that has yet to have a definite answer. However, trade-offs are made constantly as due to limitations in both human and economic resources. While cost-effectiveness analysis (CEA) compares the costs of a program to its benefits in an abstract manner, cost-benefit analysis (CBA) attempts to quantify all relevant measures in order to produce a final value. Both are considered useful tools in program evaluation and yield objective, easily comparable results. Yet, certain benefits such as that of equity in healthcare or cultural suitability of a project that can increase its accessibility to recipients are difficult to quantify. Often requiring contextual information beyond what is easily gatherable in a clinical setting, it can be difficult to justify the associated costs.
Benefits of Context versus Generalisability
The term “generalisability” is primarily used with reference to findings in research, describing whether they are applicable to other settings. In the context of healthcare planning, it can be interpreted as referring to a program that can be applied in many different contexts, provided the presence of requisite infrastructure (e.g. clinics, trained personnel). Because of this, they can be comparatively easier to implement, representing a kind of low-hanging fruit in global health. Furthermore, because measures are standardised, it is potentially easier to compare results between instances where the program has been implemented.
Information about context can be both quantitative and qualitative and can come from a variety of sources such as research in other disciplines (e.g. anthropology), surveillance data. As such, projects and studies aside from the main program may need to be conducted in order to identify valuable but hidden nuggets of information that can clue planners into the unique needs of different communities. With this information, pitfalls can be avoided and local resources maximally employed to ensure the success of the program. However, in economic analyses, contextualised interventions are often regarded as too costly for the health benefits that they accrue because of the additional work required.
So Where Should the Cost-Effectiveness Threshold be Set?
Shown below, the Handbook of Practical Program Evaluation, Fourth edition outlined a ten step process used in CEA and CBA (Newcomer, Harry, and Wholey 2015, 638):
Set the framework for the analysis.
Decide whose costs and benefits should be recognized.
Identify and categorize costs and benefits.
Project costs and benefits over the life of the program, if applicable.
Monetize (place a dollar value on) costs.
Quantify benefits in terms of units of effectiveness (for CEA) or monetize benefits (for CBA).
Discount costs and benefits to obtain present values.
Compute a cost-effectiveness ratio (for CEA) or net present value (for CBA).
Perform sensitivity analysis.
Make a recommendation where appropriate.
Following these steps with values obtained from monetisation of costs and benefits generates a numerical output. In most cases, this is then compared to some calculated threshold to determine cost-effectiveness, which then impacts which programs are selected for implementation. However, rooted in different frameworks and traditions, it is important to recognise that different sectors may set this bar very differently. Which standard is ultimately selected depends on a variety of factors that can be political, social, or economical in nature. In this case, a uni-sectoral approach excludes populations, objectives, and research that do not fall within the particular stakeholder’s interest. A more multi-sectoral approach may distribute the budget so that the stakeholders or interested sectors that incur a higher cost from a program receive a greater portion in compensation or as an incentive (Remme, Martinez-Alvarez, Vassal 2017, 702). However, other obstacles remain such as the inability to monetise certain benefits or costs.
Ultimately, while context and generalisability may not necessarily be at odds in program design, it is important to recognise that the former may prove more decisive in the success or failure of a healthcare campaign. It may also be important to take into account that a CBA or a CEA can yield very different results based on whether it was undertaken from a social, economic, or purely financial perspective. This can cause tension when accounting for these two characteristics in design, especially with regards to equity in healthcare and the need to account for marginalised populations. However, given the many demonstrations of the importance of understanding the recipients of the program, it is of utmost importance that this balance is struck.
This article is written in collaboration with the Health and Human Rights (HHR) subcommittee of the University of Toronto International Health Program. If you found its contents interesting, please consider attending the 2021 HHR Conference and/or submitting an abstract to the 2021 HHR Research Poster Fair.
More information on seminars, speakers, and scheduling can be found on our Facebook page: https://bit.ly/2VHNt7I
Event: UTIHP HHR Research Poster Fair 2021
Time: March 11th, 2021 - March, 13th 2021
Topic: The Future of Healthcare Accessibility Through Telehealth
Presentation format: Online poster fair
Abstract submission: https://forms.gle/8EC3pNZbQmgm3WZi6
Cellini SR, Kee JE. 2015. Cost-effectiveness and cost-benefit analysis. In: Newcomer, KE, Harry HP, Wholey JS, editors. Handbook of Practical Program Evaluation. 4th edition. New Jersey (US): Wiley Online Library. p. 636-672.
Pai M. 2019. Archives of Failures in Global Health [article]. Nature Research Microbiology Community; [cited 2021 Jan 23]. Available from: https://naturemicrobiologycommunity.nature.com/posts/51659-archive-of-failures-in-global-health.
Remme M, Martinez-Alvarez M, Vassal A. 2017. Cost-effectiveness thresholds in global health: taking a multisectoral perspective. Value in Health. 20(4): 699-704. https://doi-org.myaccess.library.utoronto.ca/10.1016/j.jval.2016.11.009.