CSR Home>Cancer Health Disparities Poster
Applying the Fidelity Index (FI) to Evaluate a Community-Based Participatory
Research (CBPR) Program Aimed at Reducing Cancer Health Disparities
John Foster-Bey, Marcia Salazar,Tiffani Jeffries, Lynn D. Disney, and Timothy J. Willis; CSR, Incorporated funded by the National Cancer Institute (NCI) as the National Evaluator
The Fidelity Index was applied to an approach developed and implemented by CSR for other initiatives.
CNP-CBPR Model
NCI, through the Center to Reduce Cancer Health Disparities (CRCHD), awarded funds to 25 university-based and independent cancer health research institutions to develop and implement a program that:
- Forms sustainable community-researcher partnerships
- Increases the utilization of beneficial cancer health interventions
- Trains junior researchers in CBPR and cancer health disparities research, and,
- Disseminates research findings through academic and non-academic publications.
The CNP-CBPR model was implemented in three phases:
Phase I Logic Model: Implementation of the Model
Phase II Logic Model: Initiate CBPR
Phase III Logic Model: Sustainability
CNP Evaluation-
Fidelity Index-Key Goal
- A key goal of the CNP evaluation was to measure how well each grantee adhered to the program model for the NCI-CNP Initiative and whether fidelity to the model was related to success in achieving other outcomes.
- Program outcomes include utilization of beneficial cancer interventions, training new researchers in CBPR and cancer health disparities research, and disseminating research findings through academic and non-academic publications.
- As the National Program Evaluator, CSR used a fidelity index, a tool it had developed and used in other contexts, to help assess program fidelity for the CNP Initiative.
Elements of the Fidelity Index
Phase I
- During Phase I, the capacity building phase, grantees were expected to implement the following elements of the model:
- Develop a core organizational infrastructure to support community-based participatory activities to reduce cancer health disparities by establishing a staff of multidisciplinary professionals.
- Create partnerships with communities that suffer cancer health disparities and with other organizations with an interest in reducing cancer health disparities among members of these communities.
- Form collaborations with NCI centers/divisions/offices outside of CRCHD for the purpose of adequately addressing cancer health disparities.
- Perform community-based participatory educational activities that address cancer health disparities by increasing cancer education, awareness and use of beneficial cancer interventions by community members.
- Leverage CNP activities to obtain non-CRCHD funding for community-based participatory activities directed at reducing cancer health disparities.
Phases II & III
- During Phase II, each grantee was expected to implement the following components of the model:
- Perform community-based participatory research activities, focusing on primary and secondary prevention activities;
- Conduct needs assessments;
- Conduct policy assessments—with an emphasis on developing effective community-based participatory interventions to reduce cancer health disparities.
Composing the Index
- Fifteen (15) components were identified from the original Request for Applications (RFA).
- Each component of the index was a value between 0 (did not meet the requirement), and 1 (fully met the requirement), and was given equal weight in the final index.
- The Index summarizes all of the components and produces a fidelity score that ranges from 0 to 100. A score of 100 on the Index indicates that the program incorporated all of the mandatory components of the CNP program model exactly as the program planner specified.
Translating the CNP Model into a Measurable Fidelity Index
| Index Component ID |
Index Component Description |
Calculation | Value Between |
|---|---|---|---|
| FI1 | Key Staff (PI, Project Manager, and Research Coordinator | Yes/No | (0-1) |
| FI2 | Did CNP have Steering Committee | Yes/No | (1,0) |
| FI3 | Did Steering Committee have Academic, Community and NCI Representatives | Yes/No | (0-1) |
| FI4 | Did Steering Committee Meet at Least Once During the Year | Yes/No | (1,0) |
| FI5 | Community Advisory Group | Yes/No | (1,0) |
| FI6 | Did Community Advisory Group Meet at Least Three Times During the Year | Yes/No | (0-1) |
| FI7 | Clinical Partners | Yes/No | (1,0) |
| FI8 | Non-clinical Partners | Yes/No | (1,0) |
| FI9 | Clinical Agreement to Clinical Partner ratio | Yes/No | (0-1) |
| FI10 | Non-clinical Agreement to non-clinical partners ratio | Yes/No | (0-1) |
| FI11 | Non-CRCHD NCI Partnerships | Yes/No | (0-1) |
| FI12 | Cancer Educational Activities | Yes/No | (1,0) |
| FI13 | Non-CRCHD Funding | Yes/No | (1,0) |
| FI14 | Needs Assessment | Yes/No | (1,0) |
| FI15 | Policy Work | Yes/No | (1,0) |
| Fidelity Index | [Sum(FI…FI15)/ # of Components (15)]*100 |
Ranges from 0 to 100% |
*Value calculated for each CNP grantee by program year.
Example of Results: CNP A
| cnp | Fidelity Index 2006 | Fidelity Index 2007 | Fidelity Index 2008 | Fidelity Index 2009 | Other Fidelity Index | Fidelity Index Elements |
|---|---|---|---|---|---|---|
| A | 0.67 | 0.67 | 0.67 | 0.67 | 0.67 | Key Staff |
| 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | SC Members | |
| 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | SC Representation | |
| 1.00 | 1.00 | 1.00 | 0.00 | 0.75 | SC Meetings | |
| 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | CAG Members | |
| 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | CAG Meetings | |
| 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | Clinical Partners | |
| 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | Non-Clinical Partners | |
| 1.00 | 0.55 | 1.00 | 1.00 | 0.89 | Clinical agreement to clinical partners ration | |
| 1.00 | 0.97 | 1.00 | 1.00 | 0.99 | Non-clinical agreement to non-clinical partners ratio | |
| 0.00 | 0.50 | 1.00 | 1.00 | 0.63 | Non-CRCHD NCI Collaborations | |
| 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | Cancer Education Activities | |
| 0.06 | 0.21 | 1.00 | 1.00 | 0.57 | Non-CRCHD funding to CRCHD Funding ratio | |
| 1.00 | 1.00 | 1.00 | 1.00 | Needs Assessment | ||
| 1.00 | 1.00 | 1.00 | 0.67 | Policy Assessment | ||
| 82.56 | 86.00 | 91.13 | 91.13 | 87.73 | Total |
Fidelity Index Score by Year
Note:Data presented here is not actual evaluation data but data developed to help illustrate how the FI would work in this context.
Preliminary Results of Fidelity Index
Fidelity Index Score by Funding Level
Note:Data presented here is not actual evaluation data but data developed to help illustrate how the FI would work in this context.
Fidelity Index Score by Non-SPN/SPN
Note:Data presented here is not actual evaluation data but data developed to help illustrate how the FI would work in this context.
Analyzing Findings
- Analysis by funding level of CNP
- Group 1: $1 - $1.4 million
- Group 2: $700,000 - $950,000
- Group 3: $350,000 - $520,000 - Analysis by participants in a precursor program, The Special Population Networks (SPN) vs. grantees who were not SPNs
Summary of Key Findings and Conclusions
- The index will allow us to ascertain how well each CNP grantee adhered to the program model, how long it took them to attain the model’s specifications, and whether they were able to maintain all the CNP model requirements over time.
- The Index also will assist in determining whether CNP outcomes, including beneficial cancer interventions, publications, and training new researchers, are associated with fidelity to the CNP model.
Acknowledgements:
We would like to thank Drs. Chu, Taylor, Cooper, Canto and the rest of the staff of the NCI-CRCHD for their support. Special thanks to the 25 CNPs for their participation in this project.