How Should a Neighborhood Improvement Initiative Measure Its Impact?

By Deena Chisolm, PhD
Director, Center for Child Health Equity and Outcomes Research, Nationwide Children’s Hospital

Nationwide Children’s Hospital is one of the most advanced pediatric campuses in the United States. And we’re located on the South Side of Columbus, which is one of the most economically challenged communities in our city.

In 2008, when the hospital was significantly expanding its footprint and capabilities, our leadership and board realized we needed to do a better job of figuring out how we “fit where we sit.” We needed to better use our resources to help our neighbors. That’s why we partnered with the faith based Community Development for All People and other local groups to create Healthy Neighborhoods Healthy Families, which initially focused on building, rehabbing and fixing homes in the community near the hospital. It has now expanded to multi-family dwellings, workforce development, creating educational opportunities and broadly addressing social determinants of health.

I am a researcher -- an epidemiologist -- interested in how many factors affect health. I have had the interesting experience of watching the community around Nationwide Children’s change because of the hospital and Healthy Neighborhoods Healthy Families. Among the many projects my colleagues and I are now engaged in is measuring that change.

How should we measure success? How can we figure out which community improvements work and which don’t?

Those are questions that many health systems interested in community investment are now asking. Healthy Neighborhoods Healthy Families had initial neighborhood improvement goals such as reducing vacancy, and we have achieved many of them – the vacancy rate dropped from approximately 30% to 6%, more than 450 homes were impacted, nearly $50 million of investment has been brought to the initiative. It’s reasonable for a health system that is considering a similar approach to wonder, though, whether a community investment program makes the community’s overall health better.

Our experience at Nationwide Children’s has shown us that many health outcomes goals have a very long timeline – longer than funders or boards of directors may be used to. Healthy Neighborhoods Healthy Families is well into its second decade, and we feel that we are still in the beginning stages of improving health. A study I led that was published last year showed decreases in emergency department visits and inpatient admissions in the neighborhood, but similar neighborhoods showed similar (if not quite as great) improvement.

A health system interested in beginning a community investment initiative needs to be clear about their short term, mid-term and long-term goals, and what to measure, how to measure, when to measure, and what to compare those measurements against.

Healthy Neighborhoods Healthy Families initial goals focused on home improvement, and we had mechanisms to measure that. The initiative had less ability to look across time at specific health outcomes for residents. We know now that it’s important to figure out how to collect that data over time, and we’re thinking harder about that as we expand the initiative to a second Columbus neighborhood.

There are so many challenging parts to a neighborhood improvement initiative: funding; building trust within the neighborhoods; finding partners; making it all sustainable over time. But one that must not get lost is measuring the good you to hope to achieve. Like all of those other issues, it’s challenging, but it’s necessary.