Preterm birth complications are the leading cause of neonatal mortality, contributing 1 million deaths annually. Stillbirths account for another 3.2 million deaths. Both causes of perinatal mortality are inextricably linked to maternal health and to conditions at birth.
While some community-based interventions have proved effective in controlled settings and specific contexts, the implementation research challenge is to understand how to sustain these interventions at scale in different contexts.
A systematic process based on the Child Health and Nutrition Research Initiative (CHNRI) methodology was used to score and rank implementation research questions regarding community-based maternal–newborn interventions that address prematurity and stillbirths in different contexts at scale.
Of the 55 research questions that were reviewed in this way, the top five addressed equity (e.g., reaching the poor and marginalized, reducing financial barriers), behavioral practices and skills (e.g., engaging with social norms, identifying prematurity), and quality of care provided by community health workers. The top 15 questions encompassed issues pertaining to behavioral interventions, community health workers, referral, and managing health systems.
It is estimated that 3.2 million stillbirths occur each year globally, 1 million of which happen during birth . In addition, complications from preterm birth (before 37 completed weeks of gestation) are the leading cause of death for newborns, contributing an additional 1 million or 12% of child deaths ,. In 2009, more than 200 stakeholders attended the International Conference on Prematurity and Stillbirth convened by the Global Alliance to Prevent Prematurity and Stillbirth (GAPPS, http://www.gapps.org/). The community expert group at the conference included 15 members drawn from technical and funding organizations in addition to program implementers and researchers from around the world (see Acknowledgments section for specific names). In their discussions, the group framed efforts to address preterm and stillbirths within the broader context of maternal–newborn interventions. As most of the evidence supporting these interventions emanates from research projects in controlled settings in specific contexts, the group identified the main challenge being implementing interventions at scale in different contexts. Based on these discussions, the group began a research prioritization exercise for implementation research on community-based maternal-newborn interventions that address prematurity and stillbirths at scale in different contexts. In this paper, we present the results of this exercise.
A number of research prioritization efforts have recently been applied to various health topics and health system themes –. The GAPPS community expert group chose the methodology proposed by the Child Health and Nutrition Initiative (CHNRI) to systematically list and score research questions. The CHNRI methodology was selected because its conceptual framework – has been used in numerous areas by different national and international organizations – (further information on CHNRI methodology, validity, and potential limitations are discussed in Table S1). The group followed three main stages to derive research priorities (detailed in Box 1). Briefly, guided by the CHNRI methodology the group evaluated 55 research questions against five main criteria:
Is the research question answerable in an ethical way?
Does the research question have the potential to reduce the disease burden (due to prematurity and stillbirths)?
Is it likely that the proposed research would address obstacles to scaling up?
Would the proposed research attract funding support and national policy attention?
Would the research results be owned by local actors, including political authorities and elected representatives, health workers, district managers, and communities?
Box 1. CHNRI Process
Stage 1: Defining the research context, questions, and criteria for priority setting
When: May–September 2009
How: Group discussions and subsequent e-mails
Consensus on research context defined by space (developing countries), time (the next 5–10 y), the population of interest (children under five years of age), and disease burden of interest (preterm and stillbirths). Respondents were also asked to keep in mind that all research questions started with the following introduction: “When implementing a community based maternal newborn intervention package that addresses prematurity and stillbirths in different contexts at scale…”
Consensus around 55 implementation research questions grouped according to the following research domains: community engagement, behavioral skills and practices, community health workers, rational drug use, management health systems, and referral.
Consensus on the five criteria used to rank the research questions: ethical answerability, disease burden reduction, ability to support scale-up, likelihood to attract financial and policy support, ownership by local actors.
Stage 2: Enlisting experts to systematically score the research questions
When: October 2009 – March 2010
How: Preliminary e-mails sent to 85 leading experts on community based approaches and maternal-newborn health in developing countries identified through a literature search and through snowballing of program managers. The spreadsheet was also translated into French and Spanish in order to ensure the participation of colleagues from Francophone Africa and Latin America.
42 experts agreed to participate
31 experts were able to complete the spreadsheets, independently scoring the 55 research questions by each of the five criteria by answering “Yes” (1 point), “No” (0 points), undecided (0.5 points), or insufficiently informed to answer the question (missing input).
Stage 3: Computing and writing up results
When: March–August 2010
How: An intermediate score was calculated for each of the five criteria and the RPS computed as the mean of all five intermediate priority scores – (Table S3). AEA scores were computed for each research question as the average proportion of scorers that agreed on the 55 questions asked (Table S1).
29 correctly completed spreadsheets analyzed with all 55 research questions systematically scored and ranked in order of priority and agreement.
Draft circulated to all participants for feedback before being finalized.
Respondents were 39% women and diverse in terms of regional representation (26% sub-Saharan Africa, 16% Asia, 16% Latin America, 10% Europe, 32% North America). While a substantial number of respondents were based in North America, they all work full-time in developing country contexts. Half of the respondents were based in research institutions, whereas the other half were in charge of implementing programs whether through nongovernmental organizations, UNICEF country offices, or USAID headquarters. Nonrespondents were not significantly different from respondents (Table S2).
The research question that was highlighted as the most important out of all 55 reviewed was “Evaluate ways to reduce the financial barriers to facility births at the community level—e.g., user fee exemptions, emergency loans, conditional cash transfers, transportation vouchers, etc.” Other research questions among the top five prioritized also addressed equity issues (reaching the poor and marginalized), but also behavioral practices and skills (engaging with social norms, identifying prematurity) and service delivery (measuring and maintaining quality of care provided by community health workers). The remaining top ten research questions (Table 1) include other behavioral skills and practices (thermal care and feeding for preterm babies, birth planning), concerns about how to best motivate and compensate community health workers and their supervisors, and different dimensions of making referrals more effective. Congruent with the priority need to measure and maintain quality of care by community health workers, rational drug use by community health workers and community engagement with regard to audits was also listed among the top 25 research questions that received an overall research priority score (RPS) of 0.75 or greater (Table 2).
Table 3 shows the ten research questions that were assigned the lowest RPSs. Several broad policy questions (human resource planning, gender profiles, budget flows, accountability, and monitoring systems) are listed here, along with some questions related to the sequencing of community interventions and one specific question regarding private provider practice (delayed cord clamping). Questions from almost all research avenues were found among the bottom ten research questions, suggesting that no one area was completely discriminated against by the scoring. Furthermore, even these lower-ranked research questions received relatively high RPSs compared to those arising from other CHNRI exercises. The RPS for all 55 questions ranged from 0.86 to 0.56, in contrast to other CHNRI exercises, which have generated RPS ranges from 0.90 to 0.25 –. This suggests that respondents collectively considered all implementation research questions as fairly important.
Research questions did vary in specificity. For example, broad questions such as “evaluate community-based strategies to reach the poor and marginalized” were scored alongside very specific questions like “evaluate ways to provide thermal care and feeding the preterm baby.” Both broad and specific questions were ranked in the top and bottom ten implementation research questions, suggesting that no bias existed against the kind of question asked.
The CHNRI methodology evaluates certain dimensions of each research question according to defined criteria. For example, “Evaluate methods and levels of accountability that can be ensured” was not considered to affect disease burden and “Evaluate ways to ensure delayed cord clamping in deliveries assisted by private providers” was not scored as likely to attract funding support or national policy attention. Among the five criteria, the most discriminative was the one related to disease burden reduction, while the least discriminative was the one regarding ethical answerability.
As mentioned, the relatively high mean scores assigned to questions across all criteria (apart from disease burden reduction) indicate that most of the respondents were fairly optimistic about the value of implementation research questions. Average expert agreement (AEA) ranged from 0.82 to 0.49. Similar to other CHNRI exercises, AEA showed a direct positive association with RPSs, indicating that there was more agreement among experts about what were the priority research questions. This is a property that is inherent to the way AEA is measured: very high or very low RPS scores require high levels of expert agreement, while substantial disagreement among experts will lead to RPS moving closer to a mean value -.
To determine whether any systematic bias existed against certain questions due to the profile of the respondent, we analyzed scores for researchers and implementers separately. We found at least a 10% difference in the scoring assigned for 20% of the research questions (Table 4). The 11 questions for which there was a significant difference between researchers and implementers are spread across each research avenue, suggesting no one particular research area was affected by this difference of opinion. In ten out of these 11 questions, implementers ranked the implementation research question as being of higher value than researchers.
The top 25 research questions that have been prioritized span a broad range of issues (Table 2). These implementation research priorities address fostering and sustaining specific behavioral skills and practices at the community level, engaging communities in monitoring service delivery through audits, and improving referral. With regard to service delivery, a host of implementation research questions about the management of community health workers, along with the health system supports they require to function, were stressed. Finally, issues of equity, financing, and referral were highlighted, reflecting how community-based approaches cannot be dealt with in isolation from broader health system concerns.
While many of the implementation research priorities identified can be generalized across community-based maternal, newborn, and child health areas, a few distinctions may be particular to this specific exercise. Issues related to referral were present three times within the top 25 research questions. There is little implementation research on linking families from homes to facilities, or referral more broadly, in low-income countries –. While important gains have been made with task-shifting, effective and equitable referral remains vital, because the most serious cases of prematurity and other birth complications cannot be handled at the community level.
Implementation research questions related to community engagement and some other broader policy concerns central to managing health systems, such as human resource planning and monitoring systems, were overall not given high priority by respondents. Nonetheless, even the bottom ten research questions received high RPSs relative to other CHNRI exercises. This could be because the other exercises had more discriminatory criteria or because previous exercises compared different kinds of research (basic science versus implementation research). It may be easier for experts to discern between very different research areas (basic science versus implementation research) than to discern between areas of implementation research, which they may consider to be of relatively similar importance.
In addition, many of the implementation research questions do not by themselves contribute to improved maternal newborn outcomes. Their value comes forth when combined with other implementation issues that together make a more comprehensive and coherent community-based response with linkages to primary health care service delivery. It might therefore be difficult for respondents to think about specific implementation research questions in isolation from their broader social and health systems contexts.
The partiality toward some areas of implementation research could reflect the profile of respondents. A comparison of scoring by implementers and researchers did show some differences—not across any particular kind of research question, but in the direction of the bias, with implementers ranking implementation research questions higher than did researchers. The reasons for this difference among 20% of the questions are not known, but seem to indicate that implementers perceive the results of implementation research to be more powerful if effectively implemented.
While the CHNRI methodology provides a systematic and transparent way to rank research questions that purposefully avoids biases introduced by group dynamics dominated by powerful individuals, it still is a very lengthy process to undertake. Respondents must score 55 research questions according to five criteria that have three subcomponents each, resulting in 825 dimensions to respond to in the spreadsheet. This makes it a complex spreadsheet and likely does not help response rates. Eliciting participation via e-mail alone was not successful—only 42 out of 85 experts responded to the preliminary e-mail. The 42 experts that did express interest reflected a group that was more familiar with the GAPPS conference and had current working relationships with the lead authors who managed the exercise.
Despite these drawbacks, this exercise represents an important collaboration between researchers and program implementers to jointly identify the key implementation research questions vital to improving community-based maternal and newborn interventions that address preterm and stillbirths. The exercise also developed new criteria deemed more appropriate to implementation research, which require further testing and refinement to improve their discriminatory power.
Success in reducing stillbirth and prematurity rates, and in increasing the survival of preterm infants in low-income countries, is strongly dependent on achieving high and equitable coverage with existing cost-effective interventions ,. Yet coverage of such interventions remains unacceptably low in most countries. For example, across 68 countries with the highest mortality, only 54% of women deliver with a skilled birth attendant and 38% receive a postnatal visit . Furthermore, coverage levels are particularly low among poor and rural families in these countries. Community-based interventions are therefore essential to reach population subgroups whose current access to health facilities is severely limited. The effect of expanding coverage of family and community care to 90% can by itself lead to a 15%–32% reduction in neonatal mortality . Nonetheless, the knowledge gaps around how to sustain these programs at scale in different contexts remain significant.
While important reviews – have helped to spur attention to community-based maternal newborn issues, with intriguing results for specific interventions ,, the implementation research priorities identified in this article will, we hope, help to secure further research attention and financing for this important area. Priority research areas identified include equity concerns (such as removal of financial barriers and responsiveness to the poor and marginalized), specific behavioral skills and practices, and the management of community health workers including referral care. The challenge is now raised; will communities, governments, donors, research institutions, and international organizations respond?
2009 Two million intrapartum-stillbirths and neonatal deaths: Where, why and what can be done? Int J Gynaecol Obstet 107 S5 S19
2010 Global report on preterm birth & stillbirth: The foundation for innovative solutions and improved outcomes. BMC Pregnancy Childbirth 10 (Suppl 1)
2010 Child Health Epidemiology Reference Group of WHO and UNICEF. Global, regional, and national causes of child mortality in 2008: a systematic analysis. Lancet 375 1969 1987
2009 World Health Organization-Global Forum for Health Research - Mental Health Research Mapping Project Group. Mental health research priorities in low- and middle-income countries of Africa, Asia, Latin America and the Caribbean. Br J Psychiatry 195 354 363
2010 Establishing health systems financing research priorities in developing countries using a participatory methodology. Soc Sci Med 70 1933 1942
2010 Priorities for research into human resources for health in low- and middle-income countries. Bull World Health Organ 88 435 443
2010 Defining research priorities for pancreatic cancer in Australia: Results of a consensus development process. Cancer Causes Control 21 729 736
2007 Setting priorities in global child health research investments: Assessment of principles and practice. Croat Med J 48 595 604
2008 Setting priorities in global child health research investments: Universal challenges and conceptual framework. Croat Med J 49 307 317
2008 Setting priorities in global child health research investments: Guidelines for implementation of the CHNRI method. Croat Med J 49 720 733
2007 Setting priorities in child health research investments for South Africa. PLoS Med 4 e259 doi:10.1371/journal.pmed.0040259
2009 Research priorities for health of people with disabilities: An expert opinion exercise. Lancet 374 1857 1862
2009 Setting investment priorities for research in global mental health. Bull World Health Organ 87 438 446
2009 Research priorities to reduce global mortality from newborn infections by 2015. Pediatr Inf Dis J 28 Suppl 1 S43 S48
2009 Setting research priorities to reduce global mortality from childhood diarrhoea by 2015. PLoS Med 6 e41 doi:10.1371/journal.pmed.1000041
1999 Referral revisted: Community financing schemes and emergency transport in rural Africa. Soc Sci Med 49 1473 1487
2006 Maternity referral systems in developing countries: current knowledge and future research needs. Soc Sci Med 62 2205 2215
2009 Linking families and facilities for care at birth: What works to avert intrapartum-related deaths? Int J Gynaecol Obstet 107 Suppl 1 S65 S88
2008 Saving newborn lives in Asia and Africa: cost and impact of phased scale-up of interventions within the continuum of care. Health Policy Plan 23 101 117
2005 BMJ 331 1107
2005 Evidence-based, cost-effective interventions: How many newborn babies can we save? Lancet 365 977 988
2005 Community-based interventions for improving perinatal and neonatal health outcomes in developing countries: A review of the evidence. Pediatrics 115 Suppl 2 519 617
2008 Alma-Ata: Rebirth and Revision 6 Interventions to address maternal, newborn, and child survival: What difference can integrated primary health care strategies make? Lancet 372 972 989
2009 60 Million non-facility births: Who can deliver in community settings to reduce intrapartum-related deaths? Int J Gynaecol Obstet 107 Suppl 1 S89 S112
2010 Lay health workers in primary and community health care for maternal and child health and the management of infectious diseases. Cochrane Database Syst Rev 3 CD004015 doi:10.1002/14651858.CD004015.pub3
2010 Improving newborn survival in low-income countries: Community-based approaches and lessons from South Asia. PLoS Med 7 e1000246 doi:10.1371/journal.pmed.1000246
2005 Home-based neonatal care: Summary and applications of the field trial in rural Gadchiroli, India (1993–2003). J Perinatol 25 S108 S122
2010 Effect of scaling up women's groups on birth outcomes in three rural districts in Bangladesh: A cluster-randomised controlled trial. Lancet 375 1193 1202
2010 Effect of a participatory intervention with women's groups on birth outcomes and maternal depression in Jharkhand and Orissa, India: A cluster-randomised controlled trial. Lancet 375 1182 1192