Do Health Impact Assessments Help Promote Equity Over the Long Term?

Overview

The 2020 COVID-19 pandemic and the rise of social movements to advance racial justice have highlighted the wide disparities in health outcomes between White people and Americans of color, particularly Black and Indigenous populations. These disproportional outcomes can be linked to inequities in determinants of health—environmental, social, and economic factors, such as housing, income, employment, and education, that shape health and well-being.

To begin to mitigate health disparities and improve public health broadly, policymakers will need to implement and then evaluate interventions across diverse sectors. However, they face a challenging task in identifying specific policies that can promote health equity—the guiding principle that disparities in health outcomes caused by factors such as race, income, or geography should be addressed and prevented, providing opportunities for all people to be as healthy as possible.

Health impact assessments (HIAs) are one tool that can help. HIAs use a standardized six-step process to investigate how decisions—such as whether to develop a transit system, build a park, or construct a natural gas plant—could affect a community’s health and to promote the consideration of health factors as variables in decision-making. They also present recommendations on how to boost the benefits and mitigate the risks of any potential health effects.

Nationwide, government agencies, educational institutions, and nonprofit and for-profit organizations have conducted more than 440 HIAs since 1999.¹ And research shows that, in the short term, HIAs can promote public health, increase residents’ capacity to effect change in their communities, and boost decision-makers’ knowledge about the potential health effects of proposals.²

This report details the findings from the last phase of a first-of-its-kind study that explored whether and how much HIAs influence determinants of health and health equity, and provides important insights—especially for HIA practitioners, funders, evaluators, and their partners—on improving these assessments’ effectiveness. The study was conducted by Harder and Co. Community Research on behalf of the Health Impact Project, a collaboration of the Robert Wood Johnson Foundation and The Pew Charitable Trusts. It examined HIAs’ impact on the degree to which policymakers considered health in decision-making; decision-maker knowledge of the role of environmental, social, and economic factors in health and health equity; and changes to determinants of health and equity over time.

The research featured an online questionnaire and phone interviews with HIA stakeholders, and reviewed  62 HIAs on decisions that could affect one of three selected health determinants: employment; safe, affordable, healthy housing; and access to healthy food. The key findings are:

• HIAs can boost community members’ capacity to participate in decision-making, policymakers’ consideration of equity, and residents’ and policymakers’ awareness of the connections between health equity and public decisions. Despite evidence of these effects, the analysis also shows that many HIAs took place in communities where residents and decision-makers were already cognizant of health determinants, limiting the assessments’ value related to resident and policymaker awareness. Practitioners and funders could maximize these impacts by concentrating HIA use and resources in communities where stakeholders have a more limited awareness of health determinants and how they drive outcomes.
• HIAs may promote systemic changes that could advance health equity. Some evidence suggests that HIAs can encourage policymakers to target more resources to lower-income communities or generate more sustained decision-maker attention on issues affecting communities, such as communities of color, that disproportionately face poor health outcomes. However, this impact was not demonstrated for all studied HIAs, so further examination is needed to measure its extent.
• HIAs can influence policy change and result in health-promoting changes to decisions, confirming the findings of previous research. However, the study also highlights that HIAs are only one of several factors policymakers consider and that issues such as timing of the assessment, political will and context, and feasibility of the recommendations can influence the chances that HIA recommendations will be adopted.
• HIAs were only minimally associated with long-term changes in the availability of well-paying jobs; access to healthy foods; or housing affordability, availability, or quality. The study highlights the complex financial, political, and social pressures that shape policy outcomes and underscores that HIAs are only one of many factors that inform decision-makers.
• More work is needed to identify effective ways to track the long-term effects of HIAs. Without extensive primary data collection and costly, protracted research studies, the data available to document HIAs’ effectiveness over time is incomplete and of relatively low quality.

Based on these findings, the Health Impact Project offers the following recommendations for HIA practitioners, funders, and evaluators:

• Build on the growing evidence base documenting, and institutional momentum around, the importance of equity as a public health issue. Leading national and international entities, such as the American Public Health Association and the World Health Organization, have demonstrated the health and societal costs  of social and economic disparities, and highlighted the need for action at all levels of government to advance health equity.³ HIAs can support these efforts by increasing both community engagement in decision-making and policymaker awareness of the relationship between health equity and decisions in  a range of sectors.
• Ensure that HIA recommendations prioritize determinants of health equity, including identifying ways that policymakers can support community-driven decision-making. Several HIAs examined in this study included recommendations that focused on mechanisms such as creating ongoing community engagement, adopting tools and processes that emphasize the experiences of historically marginalized communities, or establishing funding for community-driven initiatives.
• Ensure that an HIA is appropriate given the practitioner’s stated goals, and manage stakeholders’ expectations about what HIAs can achieve. This study demonstrated that HIAs are only one of many factors that inform decision-making. Therefore, practitioners should clearly define with their HIA team and stakeholders what the objectives are and what would constitute success—for example, is the goal policy change or increased community capacity to participate in decision-making?—and evaluate the HIA’s effects according to those objectives.
• Develop a range of recommendations that require various levels of decision-making authority, action, and resources. Providing decision-makers and community members with options can allow some recommendations to move forward even if decision-makers do not want to implement them all.
• Enhance monitoring and evaluation. All HIAs end with a phase for evaluation and monitoring, but practitioners often have limited resources to ensure ongoing measurement of an assessment’s impact and the effects of the final implemented decision. Development and consistent implementation of feasible monitoring plans could yield higher-quality data on outcomes after the HIA.

The study’s limitations include a smaller-than-expected sample of HIAs and data, variable data quality, and external factors such as demographic variation.And though the findings are mixed regarding HIAs’ ability to promote systemic change that advances health equity and to influence determinants of health over the longer term, these assessments have demonstrated near-term benefits. In particular, the study finds that HIAs encourage community members’ engagement in decision-making and improve decision-makers’ knowledge about the potential health effects of proposed programs, policies, projects, and plans. Because of the urgent need to improve health and health equity over the short and long terms, this research was an important step toward determining whether and how much HIAs can contribute to positive systemic change.

Glossary

Determinants of health: Environmental, social, and economic factors, such as employment, housing, education, and transportation, that affect health and the quality and length of people’s lives.

Determinants of health equity: Systemic dynamics—for instance, the allocation of resources to communities at high risk for poor health outcomes—that can drive health disparities among populations.

Food deserts: Areas that the U.S. Department of Agriculture has identified as being low income and offering residents limited or no access to healthy and affordable food, specifically where at least 500 people or 33% of the population lives at least 1 mile (in an urban area) or 10 miles (in a rural area) from a grocery store.

Health disparities: Disproportionate disadvantages in prevalence of disease, injury, or violence or in opportunities to achieve optimal health among certain racial, gender, socioeconomic, geographic, or  other populations.

Health equity: The guiding principle that disparities in health outcomes caused by factors such as race, income, and geography should be addressed and prevented, providing opportunities for all people to be as healthy as possible.

Health in All Policies: Defined by the World Health Organization as “an approach to public policies across sectors that systematically takes into account the health implications of decisions, seeks synergies, and avoids harmful health impacts in order to improve population health and health equity.”⁴

Institutional racism: Laws, policies, and practices of society and its institutions that benefit White people while oppressing, discriminating against, disadvantaging, or neglecting other racial and ethnic groups.

Interpersonal racism: When individuals—with or without intent—allow their biases to influence the  way they treat members of certain racial groups, often resulting in emotional, financial, and other types of distress and harm.

Methodology

This study examined 62 of the 414 HIAs listed in the Health Impact Project’s online database in December 2017, when the first phase of the study began. The assessments were sorted into three sets based on completion date—2009-13, 2014-15, and since 2016 or underway at the time of the study—and classified according to the most relevant health determinant:

• Access to healthy food (food). These HIAs examined the potential effects of a decision on the cost or availability of high-quality, healthy foods such as fresh fruits and vegetables.
• Safe, affordable, healthy housing (housing). HIAs in this group evaluated a decision’s potential impacts  on the availability of safe, well-maintained affordable housing.
• Employment. These HIAs assessed the possible effects of a decision on the availability of high-quality jobs that offer adequate pay, benefits, and stable hours.

The researchers selected these three to facilitate a comparison of determinants with a relatively strong research base demonstrating short-term impacts of HIAs, specifically access to healthy food and safe, affordable, healthy housing—and one without such evidence: employment. The researchers intentionally categorized HIAs into determinant rather than sector groups because decisions across a range of sectors can affect the same health determinant. For example, decisions in sectors such as labor, transportation, and criminal justice can affect employment. The study team assigned each HIA to the most appropriate determinant group—based on the team’s familiarity with each assessment, the information in the final reports, and practitioners’ questionnaire responses (see “Data sources”)—even if the HIA considered more than one determinant.

HIAs were eligible for inclusion in the study if they were rapid, intermediate, or comprehensive and focused on a program, policy, plan, or project implemented at the neighborhood, city, county, or state level.⁵ HIAs in the 2009-13 and 2014-15 groups had to meet two additional criteria: They had to be finished, and the assessed decision needed to have been made. Harder and Co. used two online questionnaires to confirm eligibility and enroll HIAs in the study.

The 62 HIAs spanned 31 states and covered a range of decision-making levels: city or county (48%), neighborhood (27%), state (23%), and regional (2%). They examined plans (34%), policies (32%), projects (21%), and programs (13%), and were split fairly evenly across the three health determinants: food (32%), employment (31%), and housing (37%). About a third of the practitioners leading the HIAs had never done one before. Further, 44% of the studied HIAs were completed between 2009 and 2013, 31% in 2014 or 2015, and 26% were completed since 2016 or were still underway.⁶

Outcomes studied

This analysis examined the HIAs’ effectiveness across six outcomes, specifically the extent to which:

1. Decision-makers implement HIA recommendations.
2. The HIA process affects awareness of determinants of health among decision-makers and  community residents.
3. HIAs can strengthen the capacity of communities facing health inequities to influence decisions that  affect them.
4. HIAs influence governments and institutions to include communities facing health inequities in  decision-making processes.
5. HIAs are associated with changes in determinants of health.
6. HIAs are associated with changes in determinants of health equity.

Data sources

The research team used a mixed-methods approach to analyze five data sources:

• Questionnaires completed by practitioners representing the 62 HIAs. The researchers determined whether HIAs met the criteria for inclusion by collecting questionnaires from practitioners about the perceived influence of each studied assessment on the opinions and actions of decision-makers. Those that did so were enrolled in the study.
• Documents related to 57 completed HIAs in the study sample. The research team examined the studied HIAs’ final reports and monitoring and evaluation plans, as well as news articles and other background sources, to collect information about the assessments, such as where they took place, the types of decisions they reviewed, whether the communities had prior HIA experience, and political factors that may have influenced the decision-making process.
• Interviews with 44 stakeholders representing 30 HIAs. Through formal conversations with people—some of whom had decision-making authority related to the policies, programs, projects, or plans assessed by the HIAs—the study team explored stakeholders’ perceptions of and opinions about changes in determinants of health and health equity over time, contextual factors influencing policy choices, and the HIAs’ roles in decision-making.
• Public data pertaining to the 14 HIAs for which such information was available. The study team examined American Community Survey (ACS) and the USDA’s Food Access Research Atlas data related to five indicators:
  1. Poverty rate.
  2. Unemployment rate.
  3. Share of housing units with residents who spend 30% or more of their household income on housing.
  4. Proportion of housing units that are vacant.
  5. Proportion of census tracts within the HIA study area that are food deserts.   

The study team used the data to identify evidence of changes to the three health determinants  since completion of the studied HIAs. (See “Study limitations” for more information on the availability  of public data.)

• Questionnaires completed by 287 community residents for three of the HIAs. The questionnaires explored residents’ perceptions of local changes in the three determinants and, among respondents who were familiar with or had participated in the HIA process, the extent to which the assessment contributed to those changes. The questionnaires targeted three groups of residents: those who were actively involved with the HIA, those who lived in the community when the assessment occurred but may not have participated in or been aware of the process, and those currently living in the community regardless of their involvement with the HIA. Some questions required prior familiarity with the HIA, while others did not.  (See “3 HIAs Selected for Community Questionnaires” for more information.)

Analysis approach

To synthesize data across the five sources, the study team used a rubric approach, establishing criteria that formed the bases for two scores assigned to each of the six studied outcomes. One score represented the HIAs’ impact and the other gauged the quality of the data available for use in the study. The research team developed two rubrics, each with four possible scores:

• Impact. Assessed the degree to which the available evidence suggests that the HIAs contributed to improvements in the outcomes measured. The possible impact scores were:
  • Minimal. HIAs have made limited contributions to improvements.
  • Emerging. HIAs are beginning to contribute to improvements.
  • Established. HIAs have contributed to a moderate level of improvement.
  • Advanced. HIAs contributed to a high level of sustained improvement. The research team calculated impact scores for each outcome and the three determinants of health.
• Quality. Assessed the completeness and relevance of data available for each HIA and provided additional insights into HIA practice and evaluation. The scores included:
  • Strong. High-quality data was available for most HIAs.
  • Good. High-quality data was available for some HIAs.
  • Fair. High-quality data was available for a few HIAs.
  • Poor. High-quality data was available for very few or no HIAs.

The team separated the impact and quality scores in order to gauge how HIAs have contributed to improvements in determinants of health and health equity, while simultaneously examining the challenges and limitations related to data availability. Additional information on rubric scoring and analysis can be found in the separate methodological appendix available on the webpage for this report.

Implementation of HIA recommendations

A primary goal of HIAs is to identify feasible actions that can minimize the health risks and maximize  potential health benefits of a decision. Therefore, determining the extent to which decision-makers implement HIA recommendations provides one way to gauge effectiveness.

Prior research shows that when HIA recommendations are clearly articulated and politically, economically, and technically feasible, decision-makers are more likely to act on them.¹⁰ In addition, tailoring the language of recommendations to the decision-maker and other target audiences can increase the chances that they will  have the desired impact.¹¹

The study found many instances in which decision-makers implemented HIA recommendations to improve health and equity. (See Table 1.) For example, in the food HIA group, adopted recommendations included:

• Development of “food hubs”—businesses or organizations that manage aggregation, distribution,  and marketing of fresh locally and regionally produced food.
• Creation or revision of zoning and other policies to allow small farms or farmers markets to operate  in a community.
• Establishment of affordability programs to enable markets to price fresh food on a sliding scale for residents facing income, transportation, and other barriers to healthy eating.
• Development of a program in which participating restaurants provide smaller portion size options.
• Building of a grocery store.
• Changes to zoning laws to limit fast-food restaurants.¹²

Table 1

Decision-Makers Adopted Recommendations From at Least Half of HIAs With Available Data

Frequency and scale of implementation among 29 HIAs by determinant

Determinant of health	Number of HIAs studied with available data on implementation of recommendations	All HIA recommendations implemented	One or more HIA recommendations implemented
Food	11	27%	64%
Employment	6	17%	50%
Housing	12	8%	92%

© 2020 The Pew Charitable Trusts

One questionnaire respondent described the influence that the HIA process had on a larger community planning effort, stating: “I do know that [the property developers] asked for our report. ... I also think that the neighborhood folks that were involved in the inner circle in decision making, that they had the report. ... I do  think [the HIA] probably was at least high in the pile of information used to make a decision.”

Examples of recommendations implemented in housing-related HIAs include:

• Developing new bills to require rented properties to maintain certain quality standards, known as a warranty of habitability.
• Revising comprehensive plans, which serve as guiding documents for decision-making about the built and natural environments within a jurisdiction, to include affordable housing goals and objectives.
• Enhancing climate and pollution control, mitigating noise, and improving health and safety issues in housing units, such as updating ventilation systems or mitigating lead paint conditions.¹³

One employment-related HIA recommendation that was implemented involved improving long-term transit planning to ensure that routes serve employment centers. As one interview respondent said, “They went  away from a spoke-and-hub system to transfer points, and it expanded routes toward some of the employers  that are in little bit different areas. So they were trying to increase access to people being able to get to their places of employment.”

Several factors influence the likelihood that HIA recommendations will be implemented

Interview respondents across the determinant groups identified several factors that affected decision-makers’ ability or willingness to implement HIA recommendations, including:

• Feasibility. Interview respondents shared examples of recommendations that decision-makers considered but found to be too complex or impractical to implement. For instance, one HIA recommended historic property tax credits, which would have limited changes that could be made to housing unit exteriors.
• Political will and turnover. Some respondents noted that implementing their recommendations would require legislation and so was “dependent on political climate and budgetary considerations.” Others explained that leadership changes affected implementation, sometimes dramatically. As one person said, “All of the work with the county judge and executives was left to the wayside after [the decision-maker] was ousted shortly after the HIA.”
• Alignment of HIA findings and recommendations with stakeholders’ preferred course of action. Some HIAs yielded data and findings that bolstered support for a course of action that key decision-makers and stakeholders already sought, which made implementation of the HIA recommendations comparatively easy to achieve. One interviewee noted that decision-makers took up the HIA recommendations “because they [already] supported the changes of transit route.”
• Timing. Interview respondents associated with two food HIAs and at least one employment HIA specifically noted that the decision had been made before the HIA was conducted and therefore the HIA did not play a role in decision-making.

Even when decision-makers do not implement the recommendations, HIAs often generate discussion or communication materials that can spur dialogue among stakeholders and policymaker action. Across the determinant groups, the study found evidence that HIAs raised awareness among stakeholders and decision-makers of aspects of the proposed plan, project, program, or policy that could affect health and equity. For example, one interview respondent noted that the HIA stressed negative health and equity effects on young people of color of certain local law enforcement practices, and the ensuing discussions between advocates and policymakers contributed to the legislature striking a provision concerning underage tobacco possession. Similarly, another respondent explained that the HIA “didn’t change the big picture of what we were doing. ... What it really changed were some of the specifics around programming.” Further, a respondent said that the HIA had raised the importance of housing quality as a critical health and equity issue and that as a result, “housing and the warranty of habitability were the major part” of the subsequent city board of directors election.

In addition, many HIA recommendations touch on multiple health determinants, so implementation of recommendations may not affect the determinant to which the HIA was assigned for this study. Determinant group assignments were based on a review of the primary focus area of each HIA, as well as input about the HIA’s areas of focus from the primary contact, typically the lead practitioner. Therefore, each determinant  group includes some HIAs that concentrated narrowly on the assigned determinant and some that looked at broader policies. For example, HIAs that examined changes to local transit systems, comprehensive plans, and housing developments can be found in all three groups because they often looked at a wide range of potential health impacts. As one interview respondent observed, “Even when we place people into housing as part of the case management and ongoing support ... employment is part of the strategy.” Therefore, some HIAs were classified as having recommendations implemented even when the results did not directly relate to the determinant to which the HIA was assigned.

Awareness of determinants of health

Any integrated approach to implementing health-promoting policy changes across multiple sectors requires that all stakeholders understand and consider determinants of health.¹⁴ HIAs can be an important tool in raising community members’ and decision-makers’ awareness of the wide range of economic, social, and environmental conditions that shape health and drive disparities in outcomes among racial, income, geographic, or other demographic groups.

HIAs appear to help deepen community members’ awareness of health determinants

Across the determinant groups, interview respondents reported that before the HIAs, community members already had a high, general level of awareness about how housing, employment, and food access affect health. This sentiment was consistent among interview respondents, regardless of their level of participation in or familiarity with the HIA. (This varied substantially among interviewees based on their roles. For example, some were involved in analysis and HIA report writing while others, generally government agency personnel, were responsible for the decision the HIA sought to inform but did not directly participate in the HIA process.)

Some interview respondents provided information about specific changes in determinant awareness and the contribution of the HIA. For example, a food HIA in the Midwest examined how a proposed amendment to a decade-old farm ordinance would affect the health of residents through changes in the availability and price of fresh fruit and vegetables, among other impacts resulting from increased regional food production. The amendment sought to increase access to fresh foods for schools, farmers markets, corner stores, and other community sites by broadening investments in the production of fruits, vegetables, and meat on small and organic farms. One respondent noted that “awareness has gone up since we did the HIA, and it’s still on  people’s radar. ... People are still talking about it.”

Unsurprisingly, HIAs were less likely to contribute to changes in awareness of health determinants in communities that already had a high level of such knowledge. In many of these locations, stakeholders instead used HIAs to build on existing momentum around a proposed decision or to deepen community members’ understanding of how decisions in a range of sectors can affect health. For example, a Midwestern HIA focused on development of a full-service grocery store in a food desert where community members were already substantially aware of the importance of access to healthy food to public health. One interview respondent observed that access to healthy food “was an issue that was important to this neighborhood, which was why we thought it was worth doing this [HIA], and then we got in on the conversation. It really ... started from the neighborhood. [The assessment] might have built awareness on some of the specifics of how to define the problem of not having a decent grocery store, but the access-to-food issue was already on the front burner.”

In another example, an interviewee described how an employment HIA broadened community understanding of what constitutes adequate employment: “People think that if people are employed, that they should be fine. ... [But] it’s not just employment; it’s the type of employment: Are people getting a living wage? Are they able to afford child care? Are they able to afford transportation?”

Further, a few interviewees described how engaging specific partners could help HIA practitioners capitalize on the potential to increase awareness through an HIA. For example, one interviewee explained that community members would have been more aware of employment issues if the HIA had included local organizing groups in addition to public health professionals and researchers: “I think if it had been conducted by different primary institutions, it might have been better suited to [raising awareness]. My recollection is that it was the department of health and [an independent research group], and neither of those are community organizing groups or are particularly rooted in community.”

Some HIAs helped decision-makers better understand their role in promoting health

Some decision-makers said in interviews that, as a consequence of the HIA process, they were starting to recognize connections in their communities between health outcomes and social, economic, and environmental factors, such as employment, housing, and food access. For example, an interviewee from Florida said: “We’re looking at health more globally in our profession and in policies. ... I think that [the planning department’s] awareness was heightened. ... I think that probably added to some of [the department’s] knowledge base and experience.”

Similarly, a stakeholder involved in an HIA that focused on raising living wages in a city in the Northeast shared that, before the HIA, he:

“hadn’t considered health equity as an argument you could use for raising wage standards and raising labor standards. [An] obvious example of why we should be paying better wages is because ... there are actual health outcomes associated with how much you have in your pocket and how much you can provide for your family and what it means when you don’t know when your next meal is [going to] come from. In some ways, I think it’s a stronger argument than making a policy-based or legal-based argument for increasing wage standards. ... It’s hard to argue against health.”

Employment HIAs had less impact on determinant awareness than housing and food HIAs did

Several interview respondents from the employment group noted that awareness of employment issues among decision-makers was already high before their HIAs were conducted and that this heightened awareness was the impetus for, rather than an outcome of, many employment HIAs. For example, an HIA in California evaluated a county ballot initiative to fund programs to reduce homelessness, including employment and workforce readiness initiatives. The catalyst for the HIA was a January 2015 count of homeless people in the county that showed a 6% uptick and a subsequent declaration making homelessness one of the county’s top priorities. All 34 departments in the county “were told that homelessness was now within their portfolios; they had to highlight and work on ways to address homeless clients who impacted their systems.” This emphasis on homelessness and employment led to the HIA, instead of being prompted by it.

Similarly, another HIA in California explored the extent to which a proposed wage theft ordinance would affect the health of workers and their families. The decision-maker who had proposed the ordinance and was already highly aware of employment as a health determinant said, “I don’t know that [the HIA influenced awareness of employment]. ... I was already highly motivated to address wage theft, and I was working on the issue long before the study.”

By contrast, interview respondents associated with housing and food HIAs consistently reported that awareness of the health determinants sometimes increased after the HIAs. For example, one interviewee described how a housing HIA in Arkansas focused on ensuring that rental properties are fit to occupy and helped further boost a growing public recognition of the importance of housing quality to tenants’ health. “I think [awareness is] a lot higher than it was, let’s say, five years ago,” the interviewee said. “We just had a very public election for mayor, and [habitability] was a political issue with the mayor and all of our board of directors. ... They spent a lot of  the last decade growing national and local attention to those issues, which gets people talking about housing a  bit more.”

In some instances, interview respondents who discussed housing HIAs said that overall awareness about housing issues was already high. However, they also observed that the HIA promoted awareness around specific aspects of housing that were affecting the community but were not part of policy discussions, such as how the surrounding built environment, including parks, sidewalks, and transportation, can affect health.

Lexey Swall for The Pew Charitable Trusts

The public’s role in policy change and decision-making

This study examined two interrelated outcomes that affect residents’ ability to participate in decision-making that affects them: the capacity of communities, particularly those facing health inequities, to influence decisions and the degree to which governments and institutions collaborate with residents on shaping decision-making processes and outcomes.

Community engagement is one mechanism by which HIAs may affect determinants of health and health equity, particularly by enhancing trust between government institutions and residents. Previous evaluations have found that HIA processes often provide meaningful opportunities for community input using a range of techniques, including publicly posted information, focus groups, public meetings, advisory committees, and resident votes on HIA recommendations.¹⁵

Community participation in the HIA process aims to enhance opportunities for residents, particularly those  facing health inequities, to participate in decision-making processes and increase their capacity to influence policies, decisions, and institutions during and after an HIA. Although the extent and quality of community involvement can vary widely across HIAs, research shows that effective engagement augments the benefits of an assessment by empowering residents with the skills and experience to participate in political processes and decisions that affect their lives and livelihoods.¹⁶ And by engaging decision-makers alongside residents, the HIA process can help illuminate ways that governments and institutions can alter their practices to support community-driven decisions.

Most community engagement was preliminary and did not result in significant resident participation in decisions

The research team asked interviewees to describe the opportunities that their HIA created for community members to engage in the decision-making process and assessed the examples offered according to the International Association for Public Participation’s Spectrum of Public Participation, which outlines five levels of community participation:¹⁷

1. Inform. Community members receive information and updates from decision-makers.
2. Consult. Residents and local business owners provide feedback on options to decision-makers.
3. Involve. Decision-makers consider community member input and concerns at every stage.
4. Collaborate. The community actively participates in every phase of decision-making.
5. Empower. The community has final decision-making authority.

Most decision-makers described community engagement that fell within the “inform” or “consult” levels, with only a few in “involve” and “collaborate,” and none in “empower.” In one example of participatory engagement, an interviewee shared how community members got “involved in other projects [beyond the HIA]. I think neighbors, really, if they join their neighborhood associations, they have a lot of opportunity to sort of leverage their voice in the bigger picture. These few neighborhood associations ... are called upon when city planners or policymakers are looking for input from the community.”

Although nearly all the HIAs presented multiple opportunities for community engagement, interview participants did not specify whether communities experiencing inequities were involved. And many of the engagement opportunities preceded or did not arise from the HIA.

Some HIAs increased community capacity to participate in decision-making

A prominent theme among respondents from the housing and employment determinant groups was boosting both the immediate and longer-term ability of residents to effect change in their local communities. For example, one person explained how the HIA created a system that enabled residents from three counties to overcome geographic distance and a history of misunderstanding and come together to participate in decision-making processes: “The HIA has brought together community members from all three of these different counties to have a conversation about the commonalities of what they’re experiencing. ... I feel like that’s a really powerful part of the process of this HIA.”

Another interview respondent said that tenants’ involvement in decision-making developed throughout the HIA process. Tenants not only helped inform the HIA process, but some also became community leaders and continued to be involved in local decision-making. The respondent explained that the tenants’ participation in the HIA process empowered them to become involved in ongoing community decision-making, such as by serving on steering committees.

These findings are consistent with data from the three HIAs selected for community questionnaires.  (See Table 2.) Of the respondents who were involved in or aware of the HIA in Columbia, Missouri, 79%  agreed that the HIA empowered residents to get involved in policy issues affecting their community, and  63% and 61%, respectively, agreed that the HIA encouraged residents to participate in volunteer or civic engagement activities, such as serving on a board or voting in local or national elections. And among the questionnaire respondents from Los Angeles, 59% agreed that the HIA empowered residents to get involved  in volunteer activities in their community, and 64% agreed that the HIA showed people how to contribute to efforts to improve community conditions. 

Table 2

HIA Helped Increase Residents’ Capacity to Participate in Policy-Related, Volunteer, or Civic Activities

Perceived change among respondents involved in or aware of 3 HIAs

The HIA …	Employment HIA in Columbia, Missouri	Housing HIA in Los Angeles	Food HIA in Indianapolis
	Strongly agree/agree	Strongly agree/agree	Strongly agree/agree
Empowered residents to get involved in a policy issue	30 (78.9%)	34 (58.6%)	11 (52.4%)
Showed people how to participate in efforts to improve conditions in the community	28 (73.7%)	37 (63.8%)	11 (52.4%)
Encouraged people to participate in volunteer activities	24 (63.2%)	34 (58.6%)	10 (47.6%)
Encouraged people to participate in civic engagement activities	23 (60.5%)	19 (32.8%)	10 (47.6%)

© 2020 The Pew Charitable Trusts

Some interviewees provided examples of changes to decision-making structures that had empowered community members and stemmed from HIAs, including:

• Adding community engagement staff. One respondent noted that a local government agency created a position to focus specifically on promoting participation among residents, particularly marginalized populations, in shaping policies and programs.
• Having more community representatives in advisory bodies. Based on recommendations from an HIA, a local transportation advisory council revised its guidelines to ensure inclusion of a representative from the business community to better consider the potential impacts that changes to the transit system could have on residents’ access to employment opportunities.
• Integrating a Health in All Policies approach. HIA is one way to advance Health in All Policies. One respondent said that in part because of the HIA in which she participated, her city government developed a Health in All Policies working group to integrate health considerations into decision-making processes across sectors, with an emphasis on health equity.¹⁸ (See the glossary for more information.)

Even when making sincere efforts, governments and institutions can struggle to effectively collaborate with community residents on decision-making. Many HIAs in this study contributed to short-term increases in community engagement and resident participation in decision-making. However, sustained changes in decision-making structures to facilitate ongoing and routine community engagement were more difficult to achieve. Interview respondents noted several challenges in transforming decision-making processes, including:

• Low interest among decision-makers in changing structures. Some interview respondents noted that, in certain instances, policymakers opposed changes that would have made the decision-making process more inclusive of community members or populations that had traditionally faced inequities or been excluded. One respondent noted that, although some individuals were prioritizing equity in decision-making, “it was definitely uphill to make [equity] a priority for the people with the most decision-making power.” And according to another, this in turn led to the departure of the most vocal advocates for change. “Some people are gone now that were very instrumental. ... They may have felt that when they spoke up ... it’s almost like to be too passionate wasn’t a good thing.”
• Limited opportunities or resources to sustain community engagement efforts. One interviewee indicated that community engagement increased as a result of the HIA but that “we don’t have these [community] meetings all the time unless they’re connected with a project.”
• Historically inequitable decision-making. Some study participants noted that local government  and other decision-making institutions would always have room to improve the way they collaborate  with residents and support community-driven decision-making. For example, when asked about  whether decision-makers provided opportunities for community members to get involved, one  interviewee noted that few government institutions in the community were creating opportunities  for and sustaining stakeholder engagement.

Changes in determinants of health

Because HIAs seek to influence decisions in a way that promotes health and reduces inequities, jurisdictions, community advocates, and public health officials often turn to them when decisions are likely to affect groups that have historically experienced health disparities or pose the risk of creating or exacerbating those gaps.

The findings from the first phase of this study, which the Health Impact Project published in 2019, suggest that HIAs improve trust between decision-makers and community residents, promote equitable access to health-promoting resources, and curtail disproportionate exposure to environmental hazards.¹⁹ However, research measuring HIAs’ impact on health determinants is scant, in part because such analysis is methodologically complex and because changes in determinants of health may not be an appropriate or realistic goal for HIAs in light of the many other economic, political, and social factors that can affect policymaking. Given these challenges, this study explored whether and how HIAs’ impact on health determinants could be measured on a national scale by examining each HIA across three key dimensions:

1. Changes in the selected social determinant of health.
2. The role of HIA in any observed changes.
3. Changes in disparities related to the selected determinant.

The studied HIAs contributed somewhat to improving access to healthy food

Overall, the determinant changes in the food HIA group were rated as emerging, indicating that some HIAs showed evidence that access to healthy food is improving in the communities where the assessments took place.

The community questionnaire for the food determinant group was conducted in the Meadows neighborhood of Indianapolis, with 91 respondents participating. The questionnaire asked residents, among other things, about any changes that they perceived to have occurred communitywide or that they experienced personally in the cost and availability of fresh fruits and vegetables and in access to a full-service grocery store. Although respondents differed in their views about whether the cost of fresh fruits and vegetables had improved, most agreed that access—defined as the ability to find fresh fruits and vegetables and get to a grocery store—had gotten better, in terms of both their own experiences and the community as a whole. (See Table 3.)

The researchers’ review of publicly available data on food access—specifically, the change in the number of census tracts that meet the USDA’s criteria for food deserts—did not reveal specific evidence that any of the six food HIAs for which data was available contributed to greater access to healthy food. The data was inconclusive as to whether HIAs decreased food deserts overall, with no discernable pattern emerging across the six HIAs. (See the methodological appendix for more information.) 

Lexey Swall for The Pew Charitable Trusts

Table 3

Most Residents of an Indianapolis Community Said Access to Healthy Food Increased After the HIA

Perceived change in availability and cost of fresh produce and access to grocers

	Better/somewhat better	Stayed the same	Worse/somewhat worse	No response
How have the following conditions changed for the community since the year the HIA took place?
Ability to find high-quality fresh fruits and vegetables	68 (74.7%)	10 (11.0%)	12 (13.2%)	1 (1.1%)
Cost of high-quality fresh fruits and vegetables	49 (53.8%)	22 (24.2%)	19 (20.9%)	1 (1.1%)
Ability to get to a full-service grocery store, fresh produce market, or other store with high-quality fresh fruits and vegetables	65 (71.4%)	10 (11.0%)	15 (16.5%)	1 (1.1%)
How have the following conditions changed for you since the year the HIA took place?
Ability to find high-quality fresh fruits and vegetables	63 (69.2%)	17 (18.7%)	10 (11.0%)	1 (1.1%)
Cost of high-quality fresh fruits and vegetables	46 (50.5%)	24 (26.4%)	19 (20.9%)	2 (2.2%)
Ability to get to a full-service grocery store, fresh produce market, or other store with high-quality fresh fruits and vegetables	62 (68.1%)	20 (22.0%)	7 (7.7%)	2 (2.2%)

© 2020 The Pew Charitable Trusts

In interviews about the food HIAs, the researchers asked about participants’ perceptions of and opinions about whether access to healthy food had changed in their communities. Most respondents did not discuss the magnitude of any change they perceived or whether it had led to a decrease in disparities. But some said that, although changes had not yet materialized, they felt the HIA had indirectly affected local decision-making and plans to improve systems that would eventually boost access to healthy food. For example, one respondent said the HIA had “changed the way [city decision-makers] thought about how they invested” money intended to promote fresh food retailers, while another reported that “the outcomes have not occurred yet. The implementation has not occurred, but the planning processes have.” He added that a new food-focused nonprofit has been launched since the HIA.

Even when gains in access to healthy food were cited, respondents still said food security policies and outcomes had room to improve and were cautious about linking any observed changes directly to the HIA. For example, one person noted that, although access to healthy food was getting better, some of the improvement predated the HIA and that several persistent problems remained: “The last 10 years, there’s been growth of some access. It’s still not good enough. ... We do definitely have [a] food desert. ... Our transportation and people’s ability [to get to grocery stores], it’s not available 24/7. ... So there are still issues.”

HIAs had little effect on housing affordability, availability, or quality

This analysis rated HIAs focused on housing as minimal, indicating a general lack of evidence that HIAs contribute to changes in housing affordability, availability, or quality. The research team was able to identify sufficient publicly available data to support examinations of impact for only two HIAs in the housing group. The analysis indicated a very small improvement in the affordability and availability of housing. However, the extent to which the HIAs contributed to those changes is difficult to determine because a wide range of external factors, such as economic growth, national and state policies, and demographic shifts, probably influenced housing trends in those communities. (Additional information on the methodology for this analysis can be found in the appendix available on the webpage for this report.)

The case study questionnaire for housing examined an HIA in Los Angeles and asked 112 respondents to share their perceptions and experiences regarding changes in the number of housing units available, their condition, and the amount of income spent on housing. An overwhelming majority (94%) of respondents said that the cost of housing had increased since the HIA was conducted, and most (over 60%) said that housing availability and conditions had decreased, with similar results across respondents’ perceptions of communitywide changes and personal experiences. (See Table 4.)

Table 4

Most Residents Said Housing Worsened in Their Los Angeles Community After an HIA

Perceived change in neighborhood housing access, quality, and affordability

	Better/somewhat better	Stayed the same	Worse/somewhat worse	Don’t know
How have the following conditions changed for the community since the year the HIA took place?
Number of housing units available	9 (8.0%)	27 (24.1%)	76 (67.9%)	0 (0.0%)
Housing conditions	11 (9.8%)	30 (26.7%)	71 (63.4%)	0 (0.0%)
Amount of income spent on housing	0 (0.0%)	6 (5.4%)	105 (93.8%)	1 (0.9%)
How have the following conditions changed for you since the year the HIA took place?
Number of housing units available	1 (0.9%)	16 (14.3%)	95 (84.8%)	0 (0.0%)
Housing conditions	14 (12.5%)	29 (25.9%)	69 (61.6%)	0 (0.0%)
Amount of income spent on housing	0 (0.0%)	7 (6.3%)	105 (93.8%)	0 (0.0%)

© 2020 The Pew Charitable Trusts

The researchers also asked interview participants who were associated with HIAs in the housing group to share their perceptions of and opinions about housing in their community. Most respondents did not say whether they had observed widespread changes in housing availability, quality, or affordability in their community, but some did mention specific changes related to housing that came about after an HIA took place. For example, one respondent said that metrics of housing quality and safety had improved as a result of the HIA, in addition to the development of new housing projects, noting that the “community is more receptive to multifamily developments than they were in the past.” Another interviewee said the HIA had “helped organizations [in my community] be more effective” and noted improvements in local housing indicators after the HIA.

Some respondents who reported no changes in housing indicators said they predicted or anticipated improvements in the future. For instance, one interviewee noted that the new city budget included funds for housing and code enforcement, that the mayor-elect would prioritize housing, and that the housing commission was “looking for ways to provide more affordable housing and loosen some of the more strict rules against those who have been incarcerated—always with the goal of supplying more safe, clean, and affordable housing to the citizens” of the town. Another respondent described how a community survey indicated that residents’ perceptions of housing quality and safety improved.

HIAs did not measurably drive change in the availability of high-quality, well-paying jobs

HIAs focused on employment demonstrated minimal effects on the availability of high-quality, well-paying jobs. Although the secondary data indicated falling unemployment in the communities that were the focus of the six HIAs for which data was available, the changes mirrored national trends. Thus, the extent to which the individual HIAs may have contributed to the decline in unemployment is difficult to assess.²⁰ Another key employment indicator, poverty, did not show a clear trend across the HIA communities.²¹

The study team conducted the community questionnaire for the employment determinant in Columbia, Missouri, asking 84 respondents to share their perceptions about and experiences with changes in employment income and the numbers of available full-time and “high-quality” jobs available. Respondents’ perceptions varied on all three indicators, but a slightly higher percentage said the number of full-time jobs had improved (45%, compared with 35% who reported an increase in the number of high-quality jobs and 37% who said income in their community had risen). Participants’ responses about their personal experiences were similarly mixed, although 45% said their income had increased. (See Table 5.)

Table 5

Columbia, Missouri, Residents Had Conflicting Views of Changes in Employment

Perceived differences in income, job availability, and quality

	Better/somewhat better	Stayed the same	Worse/somewhat worse	Don’t know	No response
How have the following conditions changed for the community since the year the HIA took place?
Amount of money earned	31 (36.9%)	30 (35.7%)	18 (21.4%)	5 (6.0%)	0 (0.0%)
Number of full-time jobs available	38 (45.2%)	25 (29.8%)	20 (23.8%)	1 (1.2%)	0 (0.0%)
Number of high-quality jobs available	29 (34.5%)	26 (31.0%)	26 (31.0%)	3 (3.6%)	0 (0.0%)
How have the following conditions changed for you since the year the HIA took place?
Amount of money earned	38 (45.2%)	25 (29.8%)	20 (23.8%)	0 (0.0%)	1 (1.2%)
Number of full-time jobs available	31 (36.9%)	24 (28.6%)	21 (25.0%)	6 (7.1%)	2 (2.4%)
Number of high-quality jobs available	26 (31.0%)	23 (27.4%)	30 (35.7%)	4 (4.8%)	1 (1.2%)

© 2020 The Pew Charitable Trusts

With one exception, the interview respondents for this determinant all indicated that their local HIAs did not affect employment outcomes, and even the lone person who said the HIA influenced some changes in employment reported that those contributions were indirect.

Interview respondents expressed a variety of opinions about the status of employment in their areas after the HIA. Among those who reported changes in employment, some felt that this positive change was due to broader economic forces over time. For example, one interviewee noted that the local community had already been engaged in concerted efforts related to job growth: “I think it [employment] definitely changed. There are folks who have been doing a lot of work around earned sick leave, family leave. Given the fact that we have strong labor leaders, if nothing had changed, that would be a problem.”

Two interviewees felt that the decisions the HIAs sought to inform led to improvements in employment,  but they did not specifically name the HIA as a contributor to the changes, while others cited positive changes but said the HIA was not a contributing factor. As one interview respondent said: “I would argue employment  has obviously increased in our community because our community has grown. I don’t think the HIA had anything to do with that.” 

Data related to changes in disparities in health determinants was limited

A lack of available indicators that the research team could analyze by race, ethnicity, and income left the study with little data on disparities related to the food and housing determinants, and robust demographic data was available for only one employment indicator: the proportion of the population within the HIA study area living below the federal poverty level.²²

The researchers looked at this indicator and found changes in poverty level rates by race, but the data suggests that the differences are linked to broader national trends. The team’s analysis found that poverty rates generally increased for all racial groups in communities where HIAs were conducted immediately after the 2008 recession, but they tended to decline for all races in communities where HIAs were undertaken during the economic recovery period. However, the data also showed that disparities between Whites and non-Whites generally decreased over time, regardless of whether overall poverty rates rose or fell.

This analysis highlights the complex factors affecting health determinants such as poverty and the challenges in gauging HIAs’ local impact using national data, rather than locally generated metrics that are more relevant to the decision the HIA sought to inform.

Changes in determinants of health equity

This study examined three indicators of progress toward improvements in determinants of health equity:

1. Government-community relationships. The minimum practice standards for HIAs encourage practitioners and decision-making bodies to engage community members throughout the process.²³ Previous research suggests that intentional inclusion of local residents in the HIA process can repair historically tenuous relationships and build trust between residents and government agencies.²⁴ Community member involvement can also ensure that HIA recommendations are realistic, feasible, and supported by residents, which in turn demonstrates the value of community engagement to decision-makers.²⁵ In this study, government-community relationships are measured in terms of residents’ perceptions of government and decision-makers’ perceptions of the community.
2. Systematic allocation of resources. HIAs include recommendations intended to change how resources are distributed, how decisions about those resources are made, and whose needs are considered in those decisions. These changes can stretch beyond the decision assessed by an HIA, shifting decision-maker opinions about what and whom to consider in their broader processes.²⁶ This study looked for changes in the systematic distribution of resources, including to the criteria used for allocation decisions and the extent to which vulnerable populations are considered in policy and program development.
3. Experiences of racism. An established body of literature points to racism, both interpersonal and institutional, as a key driver of health inequities.²⁷ HIAs may contribute to reductions in experiences of racism through improved government-community relationships, which may reduce the likelihood that community members would experience racism in interactions with decision-makers or other community members, and by advancing changes in policies or decision-making processes that help correct histories of institutional racism.

Food HIAs positively affected determinants of health equity, but more research is needed

HIAs in the food category had an “established” effect on changes in the studied determinants of health equity, meaning some contributed to improvements in at least one determinant. Respondents for three food HIAs described changes in the systematic allocation of resources after the HIA.²⁸ One indicated that one Midwestern city’s public health department hired an urban planner after an HIA examining the potential development of a grocery store and that “part of [the urban planner’s] job will be to push Health in All Policies.” She added, “This health impact assessment helped us to start building a foundation for what kind of health data is useful and interesting to others.”

Another respondent described several changes to contract and funding decisions after an HIA of a neighborhood plan, noting, “We have language that is being put into [requests for proposals] and into contracts that is requiring the contractors to [abide by] a local hire commitment that will have goals attached to it.” In addition, the person indicated that the coalition of neighborhood associations involved in the HIA has secured funding to establish a community land trust. The money “does not go very far, but it’s allowing [the associations] to leverage other resources, and work to try to create that stability for people in the neighborhood.” Although this progress is not directly related to access to healthy food, the respondent underscored the importance of the HIA in examining “the big picture from a health perspective and ... [allowing] those various factors or issues to be looked at, addressed, and mitigated in different ways than has ever been done.”

The community questionnaires from Indianapolis provided further insights into changes in the determinants of health equity arising from the food HIAs. Among the respondents who were familiar with the HIA, 62% indicated that it strengthened relationships between community residents and local decision-makers, and 52% reported that it enabled community members to more easily participate in efforts to address local concerns. (See Table 6.)²⁹ A lower percentage reported improvements in marginalized communities’ ability to access resources (48%) and reduced experiences of racism from other residents (43%) and from people in positions of power (43%).

Employment and housing HIAs contributed to initial steps toward improvements in determinants of health equity

HIAs in the housing and employment groups demonstrated an “emerging” impact on determinants of health equity. This rating indicated that some HIAs with available data played a role in preliminary efforts to improve at least one determinant of health equity. However, these contributions had not yet resulted in an improvement to any given determinant.

Interview respondents highlighted several examples of steps taken toward positive changes in determinants  of health equity, including:

• Grassroots organizing efforts to increase community participation in decision-making.
• Development of a scorecard to assess how well equity was addressed in the implementation of an organization’s programs.
• Increased attention from local officials on the inclusion of community members in decision-making.
• Proactive development of relationships with organizations that work to promote equitable access to community resources.
• Proactive consideration of data on health disparities, inequities, and determinants in the decision- making process.
• Standardization of a process to examine the effects on health equity of decision-making in an agency or sector.

More work is needed, however, to strengthen government-community relationships, reduce experiences of racism, and improve the systematic allocation of resources to benefit communities most at risk of inequities.

Although housing and employment HIAs received the same impact score, results from the community questionnaires for these determinant groups differed. Among Los Angeles respondents who were familiar with the Crossings at 29th and San Pedro streets HIA, 48% reported that the process yielded stronger relationships between community residents and local decision-makers, and 69% said the HIA enabled community members to more easily participate in efforts to address local concerns. A lower percentage reported improvements in marginalized communities’ ability to access resources (29%), and reduced experiences of racism from other residents (28%) and from people in positions of power (22%).

For the questionnaire conducted in Columbia, Missouri, 74% of respondents familiar with the HIA reported stronger relationships between residents and decision-makers, and 68% said the HIA made it easier for residents to participate in efforts to address concerns in their communities.³⁰ A slightly lower percentage indicated that marginalized communities were better able to access resources (63%) and that the HIA reduced experiences of racism from other residents (68%) and from people in positions of power (61%). These results differed from the other two community questionnaire sites in two ways: The percentage of respondents reporting a positive impact on the determinants of health equity was higher, and the variation of percentages across determinants was smaller.

Jon Tyson/Unsplash

Table 6

HIAs Contributed to Perceived Changes in Some Determinants of Health Equity

Select responses from study participants involved in or aware of 3 HIAs

The HIA …	Food HIA in Indianapolis	Housing HIA in Los Angeles	Employment HIA in Columbia, Missouri
	Agree/strongly agree	Agree/strongly agree	Agree/strongly agree
Strengthened relationships between community residents and local decision-makers	13 (61.9%)	28 (48.3%)	28 (73.7%)
Made it easier for community members to participate in efforts to address local concerns	11 (52.4%)	40 (69.0%)	26 (68.4%)
Made it easier for people who are often left out of decision-making (such as people of color, women, LGBTQ communities, or people with disabilities) to access resources	10 (47.6%)	17 (29.3%)	24 (63.2%)
Reduced experiences of racism from people in positions of power	9 (42.9%)	13 (22.4%)	23 (60.5%)
Reduced experiences of racism from other community residents	9 (42.9%)	16 (27.6%)	26 (68.4%)

Note: The questionnaire did not define “resources,” so the responses reflect participants’ own interpretation of that term.

© 2020 The Pew Charitable Trusts

HIAs contributed to more consideration of health equity in decision-making

Across the three HIA categories, interview respondents reported promising movement toward long-term changes in determinants of health equity. In particular, interviewees described increased consideration of health equity in the decision-making process, including through:

• Greater awareness of health equity. One respondent, for example, indicated that the HIA process was directly responsible for highlighting health equity in the decision-making process: “The education and research we’ve done has helped to educate us. We had some knowledge [before]. I think our knowledge has increased, and it’s health equity that we look at in every situation.” Another stakeholder from the housing group noted that the HIA “made me a lot more aware of the impacts [of housing on] public health. I didn’t know how nuanced it was. ... Participating in the HIA improved my knowledge around this space.”
• More commitment among decision-makers to discuss health and health equity. One decision-maker noted: “I certainly talk about [health equity] a lot more. Anytime I’m giving a presentation about health and transportation planning, I try and include an equity piece in there.”
• Better integration of health and health equity into other decision-making processes. Some respondents suggested that the HIAs had encouraged other decision-making bodies to consider health and health equity more closely in their own processes. One interviewee stated, “I strongly believe that the HIA led to [another report], which was the health element of the county comprehensive plan.” Another respondent said the HIA led to the inclusion of chapters on health disparities and inequities in the city’s community health improvement plan.

Although several interview respondents noted that recent prioritization of health equity reflected “broader industry knowledge” rather than an outcome specific to the HIA process, HIAs nevertheless appear to help increase the focus on equity within and beyond public health.

More work is needed to improve health equity

Some interviews revealed an ongoing need to educate policymakers about health and health equity, as well as the ways in which people in power can improve health equity. For example, an elected official said health equity was the purview of the local health department, not her decision-making, which reflects a limited awareness of the role that decisions across a range of sectors and agencies play in health outcomes and inequities. Similarly, two interviewees from the same HIA observed that, though the HIA and decision-making process had addressed health disparities and inequity, not all decision-makers understood the issues. Interview respondents also said efforts to improve health equity require a commitment of resources and energy that extends beyond an individual HIA or decision point.

Study findings align with prior research on HIA outcomes

The findings from this analysis are consistent with previous studies showing that HIAs can influence decision-making; increase decision-maker knowledge about health impacts; boost people’s participation in decisions that affect them; raise awareness among decision-makers of the potential health impacts of proposed policies, programs, plans, and projects; and contribute to health-promoting changes to decisions.³¹

For this study, data on implementation of recommendations was available for 29 of the 62 HIAs in the sample and revealed that 21 HIAs (72%) had at least some recommendations implemented and five (17%) had all recommendations implemented. The 72% finding is substantially higher than those of two previous studies  that determined that 37% and 48%, respectively, of HIAs directly affected changes to proposed decisions.³² To explore this discrepancy, the Harder and Co. researchers also calculated this proportion across the full study sample, with the assumption that HIAs for which limited or no data was available were less likely than those with data to have their recommendations implemented. This resulted in an estimate that at least some recommendations were implemented for 34% of the studied HIAs, which is more consistent with prior research.

Further, research has suggested that HIAs can increase civic agency in communities by strengthening residents’ skills and ability to influence decisions beyond those considered by the HIA, enhancing the level of contact between community members and decision-makers, and elevating community voices in the decision-making process.³³ Similarly, this study suggests that HIAs can increase community capacity to participate in decision-making during and after the HIA process and, in some cases, help drive policymakers to collaborate with community members in shaping decisions.

The available research literature on HIAs also describes the substantial challenges of evaluating these assessments’ effects over time.³⁴ For example, disentangling the impacts of the HIA process on communities from those of the implemented decision is extremely difficult. Further, because social changes can take years to manifest and may not be clearly attributable to HIAs, given other economic, political, or social influences, this can also hinder efforts to study HIA impacts. The current study faced these same challenges as well as issues regarding data availability and quality, but it was nevertheless the first, to the study team’s knowledge, to explore potential associations between HIAs and changes in determinants of health and health equity, and as such contributes to the research base by testing different approaches to examining these complex relationships and by identifying opportunities for future evaluations.

Recommendations

This study provides insights into ways that HIAs can contribute to changes in social determinants of health and health equity. HIA practitioners, funders, evaluators, and their partners can consider the following recommendations to enhance HIA practice to better influence decisions and promote health equity:

• Build on the growing evidence base documenting, and institutional momentum around, the importance of equity as a public health issue. Leading national and international entities, such as the American Public Health Association and the World Health Organization, have demonstrated the societal costs of economic and health disparities and highlighted the need for action at all levels of government to advance health equity.³⁵ To support these efforts, HIA practitioners should focus on involving stakeholders, such as community organizing groups, that could most benefit from tools that help increase community engagement in decision-making and policymaker awareness of the relationship between health equity and decisions in a range of sectors. HIAs promote community engagement to build communities’ capacity to effect change and to increase inclusion of community members in decision-making. Building on the momentum generated through an HIA process can help spur additional community and decision-maker action to advance health equity.

The Society of Practitioners of Health Impact Assessment’s “How to Advance Equity Through Health Impact Assessments” (formerly known as the “Equity Metrics for Health Impact Assessment Practice”)— a resource that helps practitioners plan their approach to addressing equity in an HIA—suggests that recommendations can help promote equity by responding to community concerns and emphasizing actions to protect or bolster health in communities facing inequities.³⁶ Practitioners can strengthen existing elements of HIA practice by ensuring that those impacts include determinants of health equity, such as experiences of racism, government-community relationships, and allocation of resources.

• Ensure that HIA recommendations prioritize determinants of health equity, including identifying ways that policymakers can support community-driven decision-making. A number of the HIAs in this study included recommendations that focused on mechanisms to help address health equity, such as creating ongoing community engagement, adopting tools and processes that emphasize the experiences of historically marginalized communities, or establishing funding for community-driven initiatives.

When appropriate and within the scope of an HIA, recommendations that focus on improving health equity can also highlight opportunities for policymakers and community residents to collaborate on shaping decision-making processes and outcomes and creating more inclusive decision-making structures and policy systems. Practitioners must also ensure that such recommendations are feasible and that decision-makers sustain and carry out commitments to change their processes.

• Ensure that an HIA is appropriate given the practitioner’s stated goals and manage stakeholder expectations about what HIAs can achieve. Practitioners should clearly define with the HIA team, decision-makers, and community members what qualifies as success—for instance, policy change or increased community capacity to effect change—and evaluate the HIA’s impacts according to those established goals. Additionally, practitioners and funders should manage expectations regarding what HIAs can accomplish. As the National Research Council noted in its foundational report on HIAs, health is only one factor in a decision-making process, so “it is not reasonable to consider HIA successful only if it changes decisions.”37 Additionally, in many instances, HIAs result in recommendations to minimize health risks of a proposal, but the decision at hand may still pose risks and yield overall negative health and health equity outcomes.

• Develop a range of recommendations that require various levels of decision-making authority, action, and resources. Providing options can allow some recommendations to move forward even if the overall policy does not or when decision-makers do not want to implement them all. Practitioners can strengthen HIA recommendations by involving decision-makers, agencies responsible for implementation, and affected communities in their development.

• Enhance monitoring and evaluation. The HIA process concludes with a dedicated phase for monitoring and evaluation, but because this step occurs at the end of the project, it often suffers from inadequate resources to ensure ongoing measurement of impact. Strategies for strengthening monitoring and evaluation include ensuring that a portion of the HIA funding is allocated to (or securing a separate grant for) this phase ahead of time; publicly publishing monitoring and evaluation plans that can be pursued even if the original practitioners are no longer involved; and convening committees that include residents and local organizations, in addition to practitioners, to measure outcomes at specific milestones after the HIA. Using these strategies could help HIA practitioners, funders, and evaluators:
  • Track outcomes using HIA-specific monitoring and evaluation plans. Focusing more closely on the relationship between the decision the HIA sought to inform and the desired outcomes may yield more concrete evidence about immediate as well as long-term results. Practitioners, funders, evaluators, and their partners should work to develop and consistently implement feasible monitoring plans to help yield higher-quality data on the relationship between the decisions HIAs seek to inform and changes in health and health equity over time.
  • Incorporate measures related to determinants of health equity into evaluation. This study highlighted direct and indirect ways that HIAs could contribute to changes in determinants of health equity. By studying how HIAs contribute to these gains, evaluators can further demonstrate ways to consider equity in decision-making at the local level. Useful metrics could include measuring changes in decision-making processes that enhance collaboration among community residents and policymakers, the strength of government-community relationships, the allocation and distribution of resources, and people’s experiences of racism.
  • Measure long-term impacts throughout the HIA process. Although studying HIAs’ long-term impacts can be challenging, identifying and seizing opportunities to do so as the HIA progresses can ensure that outcome measures are woven into the evaluation plan instead of being developed after the report and data collection are completed. Further, when evaluation teams have the resources to measure changes after an HIA is over and decisions are made, they must define outcome measures and identify indicators and data sources at the outset of the HIA to help avoid tracking indicators that are not linked to the HIA goals or for which data on the appropriate geography, time, or populations cannot be collected. 

Study limitations

This study was an ambitious attempt to expand research on the effectiveness of HIAs across the United States and to examine the extent to which individual HIAs contributed to changes in social determinants of health and health equity. However, the study had several limitations that readers should consider when reviewing and interpreting the findings. The original study design anticipated a larger sample of HIAs and greater availability of data, which would have supported examination of longer-term impacts on determinants of health and health equity. In addition, the quality of the available data was not strong overall, ranging from poor to fair, and varied significantly across the determinant groups and by outcome. This low data quality stemmed from several challenges, including interviewees’ and questionnaire respondents’ ability to answer questions about specific outcomes and a lack of suitable, publicly available data.

Other limitations included:

• Design. The research team derived the study sample from a convenience sample of HIAs that responded to an enrollment questionnaire, not from a random selection of all HIAs conducted within the study period. Although the team invited all HIAs in the Health Impact Project’s database from the appropriate time frame to participate, additional unknown HIAs may have been conducted but not invited. Further, some HIA contacts did not respond to outreach attempts. The available time and resources for the study affected the total number of HIAs included in the sample, which may weaken the strength of evidence that can be drawn from the sample. Additionally, the sample included only HIAs that had a complete enrollment questionnaire and focused on one of the three selected determinants of health, so the results are not necessarily generalizable to all HIAs or even all those focused on food, housing, or employment.
• Cohort assignment. The evaluation team assigned each HIA to a social determinant of health cohort based on the information provided by the HIA’s primary contact during enrollment. Some misclassification may have occurred based on contacts’ recollection of the HIA, which may have affected how well the study questions aligned with the original intent or purpose of some HIAs.
• Response rate. The researchers were unable to identify at least one interview participant for 32 of the 62 HIAs in the sample. Additionally, conducting community questionnaires for three HIAs proved to be very resource-intensive and introduced quality issues related to tracking whether respondents were in fact residents of the target area or understood each questionnaire item. The study team therefore decided not to conduct the community questionnaire for additional HIAs.
• Cohort analysis. The research team split the sample HIAs into three time cohorts, based on the year they were conducted, to examine whether different trends could be observed for HIAs in progress versus those that were complete. Further, the design initially anticipated follow-up interviews for the most recent time cohort—HIAs completed since 2016 or underway at the time of the study—to examine changes over time. But because of limited response rates among stakeholders for those HIAs, the researchers did not have sufficient data to compare responses over time as originally planned.
• Self-reported information. The interviews and questionnaires in this study relied on stakeholders’ self-reported information, and the study team had limited ability to confirm those claims. For example, practitioners may have been inclined to overestimate the level of community engagement during an HIA, and questionnaire responses were mixed in terms of how they reported the level of changes to determinants of health. Recall bias, in addition to contextual factors such as political motivations, awareness of social determinants of health, or personal experiences, also could have influenced  these responses.
• Proximity to the decision. Responses to interview or community questionnaire questions depended on familiarity with the relevant HIA and proximity to the decision it sought to influence. Most interviewees were not decision-makers and so had limited ability to respond to questions related to how much the  HIA influenced policy decisions or whether recommendations were implemented. When decision-makers were interviewed, they were able to speak to the decision the HIA sought to influence, but they often were less familiar with the content of the HIA. In retrospect, a better model would have been to interview decision-makers and HIA practitioners for each HIA, but that was not feasible given study resources and response rates.
• Contextual factors. A goal of the study was to examine how population measures related to changes in social determinants of health within the communities where HIAs took place. However, several external factors influence population indicators, which can obscure the extent to which individual HIAs contributed to the trends observed. Further, limitations in population data hindered efforts to examine trends in the first place. Although a set of indicators for each health determinant group was set at the start of the study, data for those indicators was often difficult to obtain within the time and geographic boundaries of each HIA. For population metrics, the study team considered only HIAs that examined decisions directly relevant to those metrics, in order to ensure a close fit between the metrics and the outcomes the HIA sought to influence.  In addition, each data source had specific limitations. For example, because the study methodology used ACS one-year estimates, the analysis was limited to HIAs with study populations of 65,000 or more. Further, to compare population indicators before and after an HIA took place, the study team had to select a time period for each HIA for this part of the analysis that included data from before the HIA started and after it was completed. Because ACS data was available only from 2009 to 2017 (as of the writing of this report), the researchers excluded HIAs that were completed before 2009 or after 2017. Finally, the researchers distributed the community questionnaire in only three HIA communities, rather than the 10 originally planned. Ultimately, each HIA had contextual factors that affected whether recommendations were implemented and the longer-term influence of the HIA. 

Conclusion

This study was the first attempt, to the research team’s knowledge, to explore potential associations between HIAs and changes in determinants of health and health equity. And though it found minimal evidence that  HIAs were associated with long-term changes in the availability of high-quality, well-paying jobs; access to healthy foods; or housing affordability, availability, or quality, it did reinforce previous research demonstrating  that HIAs can influence policymakers to consider health equity when making decisions in relevant economic  and public sectors.

In addition, the study highlighted how HIAs can drive changes in decision-making structures to better engage community members and be more inclusive, and found promising evidence that HIAs can contribute to systemic or structural changes that advance health equity, such as targeted allocation of resources to lower-income communities. Further, the analysis revealed that, absent extensive primary data collection and costly study designs, the data available to document HIAs’ effectiveness over time is incomplete and of relatively low quality.

Taken together, these findings argue strongly for enhanced monitoring and evaluation of HIAs and for practitioners and funders to manage and communicate expectations for HIAs and their expected outcomes. 

Endnotes

1. The Pew Charitable Trusts, “HIAs and Other Resources to Advance Health-Informed Decisions: A Toolkit to Promote Healthier Communities Through Cross-Sector Collaboration,” accessed Jan. 27, 2020, http://www.pewtrusts.org/healthimpactproject/toolkit.
2. E. Bourcier et al., “An Evaluation of Health Impact Assessments in the United States, 2011-2014,” Preventing Chronic Disease 12 (2015): E23, https://www.ncbi.nlm.nih.gov/pubmed/25695261; C. Davenport, J. Mathers, and J. Parry, “Use of Health Impact Assessment in Incorporating Health Considerations in Decision Making,” Journal of Epidemiology and Community Health 60, no. 3 (2006): 196-201, https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2465566; F. Haigh et al., “What Makes Health Impact Assessments Successful? Factors Contributing to Effectiveness in Australia and New Zealand,” BMC Public Health 15 (2015): 1009, https://bmcpublichealth.biomedcentral.com/articles/10.1186/s12889-015-2319-8; Center for Community Health and Evaluation and Human Impact Partners, “Community Participation in Health Impact Assessments: A National Evaluation” (2016), https://humanimpact.org/wp-content/uploads/2018/10/Full-report_Community-Participation-in-HIA-Evaluation.pdf; Health Impact Project, “Is Health Impact Assessment Effective in Bringing Community Perspectives to Public Decision-Making? Lessons From 4 Case Studies in California” (The Pew Charitable Trusts, 2016), https://www.pewtrusts.org/-/media/assets/2016/10/tce_brief.pdf.
3. World Health Organization, “Closing the Gap in a Generation: Health Equity Through Action on the Social Determinants of Health” (2008), https://www.who.int/social_determinants/thecommission/finalreport/en; American Public Health Association, “Creating the Healthiest Nation: Advancing Health Equity,” https://www.apha.org/-/media/files/pdf/factsheets/advancing_health_equity.ashx?la=en&hash=9144021FDA33B4E7E02447CB28CA3F9D4BE5EF18.
4. World Health Organization, “Health in All Policies (HiAP): Framework for Country Action” (2014), https://www.who.int/healthpromotion/hiapframework.pdf; World Health Organization, “Health in All Policies: Helsinki Statement—Framework for Country Action” (2014), https://www.who.int/iris/bitstream/10665/112636/1/9789241506908_eng.pdf?ua=1.
5. HIAs can be fairly quick, using a “rapid” or “desktop” model over a few weeks or months. Or they can be longer, employing either an “intermediate” approach using available data or a “comprehensive” approach involving primary data collection, both of which take several months to more than a year to complete.
6. Percentages may not total 100% because of rounding. At the time of the study, five of the 62 HIAs were still underway.
7. Human Impact Partners, “29th St./San Pedro St. Area Health Impact Assessment” (2009), https://www.pewtrusts.org/en/research-and-analysis/data-visualizations/2015/hia-map/state/california/the-crossings-at-29th-and-san-pedro-st-south-central-redevelopment.
8. Indiana University Richard M. Fairbanks School of Public Health and Marion County Public Health Department, “Health Impact Assessment: Development of a Full-Service Grocery Store Within a Food Desert” (2013), https://www.pewtrusts.org/en/research-and-analysis/data-visualizations/2015/hia-map/state/indiana/potential-full-service-grocery-store-development-in-a-food-desert.
9. Columbia/Boone County Public Health & Human Services, PedNet, and Central Missouri Community Action, “Assessing the Impact of a Transportation Utility Fee in Columbia, MO” (2013), https://www.pewtrusts.org/en/research-and-analysis/data-visualizations/2015/hia-map/state/missouri/potential-utility-fee-to-fund-columbia-transit.
10. A.L. Dannenberg, “Effectiveness of Health Impact Assessments: A Synthesis of Data From Five Impact Evaluation Reports,” Preventing Chronic Disease 13 (2016): E84, https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4951082.
11. Ibid.
12. J. Barham et al., “Regional Food Hub Resource Guide” (U.S. Department of Agriculture, 2012), https://www.ams.usda.gov/sites/default/files/media/Regional%20Food%20Hub%20Resource%20Guide.pdf.
13. A. Ricklin, W. Klein, and E. Musiol, “Healthy Planning: An Evaluation of Comprehensive and Sustainability Plans Addressing Public Health” (American Planning Association, 2012), https://planning-org-uploaded-media.s3.amazonaws.com/publication/download_pdf/Healthy-Planning.pdf.
14. L.M. Gottlieb, J.E. Fielding, and P.A. Braveman, “Health Impact Assessment: Necessary but Not Sufficient for Healthy Public Policy,” Public Health Reports 127, no. 2 (2012): 156-62, https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3268800.
15. International Association for Public Participation, “IAP2 Spectrum of Public Participation” (2007), http://rethinkurban.com/wp-content/uploads/2012/06/Spectrum.png; Center for Community Health and Evaluation and Human Impact Partners, “Community Participation in Health Impact Assessments.”
16. A.-L. Tamburrini, K. Gilhuly, and B. Harris-Roxas, “Enhancing Benefits in Health Impact Assessment Through Stakeholder Consultation,” Impact Assessment and Project Appraisal 29, no. 3 (2011): 195-204, https://www.tandfonline.com/doi/abs/10.3152/146155111X12959673796281.
17. International Association for Public Participation, “IAP2 Spectrum of Public Participation.”
18. L. Rudolph et al., “Health in All Policies: Improving Health Through Intersectoral Collaboration” (2013), https://nam.edu/wp-content/uploads/2015/06/BPH-HiAP.pdf. 
19. Health Impact Project, “Do Health Impact Assessments Promote Healthier Decision-Making? Findings From a National Study of the Perspectives of HIA Stakeholders” (2019), https://www.pewtrusts.org/-/media/assets/2019/03/do-health-impact-assessments-promote-healthier-decision-making.pdf.
20. “Unemployment rate” is defined by the ACS as the proportion of working-age individuals currently unemployed but seeking employment.
21. “Poverty” is defined as the proportion of the population within the study area in households determined to be below the federal poverty level.
22. The researchers used ACS data on poverty that aligned with the time period for each HIA. Historical poverty thresholds are available at https://www.census.gov/data/tables/time-series/demo/income-poverty/historical-poverty-thresholds.html.
23. R. Bhatia et al., “Minimum Elements and Practice Standards for Health Impact Assessment, Version 3,” North American HIA Practice Standards Working Group (2014), https://hiasociety.org/resources/Documents/HIA-Practice-Standards-September-2014.pdf.
24. E.K. Sohn et al., “Avenues of Influence: The Relationship Between Health Impact Assessment and Determinants of Health and Health Equity,” Journal of Urban Health 95, no. 5 (2018): 754-64, https://www.ncbi.nlm.nih.gov/pubmed/29948783.
25. National Research Council, “Improving Health in the United States: The Role of Health Impact Assessment” (2011), https://doi.org/10.17226/13229.
26. Bourcier et al., “An Evaluation of Health Impact Assessments.”
27. D.R. Williams, J.A. Lawrence, and B.A. Davis, “Racism and Health: Evidence and Needed Research,” Annual Review of Public Health 40 (2019): 105-25, https://www.annualreviews.org/doi/abs/10.1146/annurev-publhealth-040218-043750.
28. Although HIAs in the food determinant group had a higher impact score than those in the other groups, the data quality for the food HIAs was poor because interview respondents were less likely to answer this question, to be knowledgeable about the topic, or be proximal to the decision that the HIA assessed.
29. Because these questions required familiarity with the HIA process, they yielded answers from only a subset of the total respondents to the questionnaires.
30. Because these questions required familiarity with the HIA process, they yielded answers from only a subset of the total respondents to the questionnaires.
31. Dannenberg, “Effectiveness of Health Impact Assessments”; Health Impact Project, “Do Health Impact Assessments Promote Healthier Decision-Making?”; B. Rogerson et al., “A Simplified Framework for Incorporating Health Into Community Development Initiatives,” Health Affairs 33, no. 11 (2014): 1939-47, https://www.healthaffairs.org/doi/pdf/10.1377/hlthaff.2014.0632; Sohn et al., “Avenues of Influence”; E. Bourcier et al., “An Evaluation of Health Impact Assessments”; Center for Community Health and Evaluation and Human Impact Partners, “Community Participation in Health Impact Assessments”; Davenport, Mathers, and Parry, “Use of Health Impact Assessment”; Haigh et al., “What Makes Health Impact Assessments Successful?”; Bhatia et al., “Minimum Elements and Practice Standards”; Health Impact Project, “Is Health Impact Assessment Effective?”
32. Dannenberg, “Effectiveness of Health Impact Assessments.”
33. Center for Community Health and Evaluation and Human Impact Partners, “Community Participation in Health Impact Assessments.”
34. Dannenberg, “Effectiveness of Health Impact Assessments.”
35. World Health Organization, “Closing the Gap in a Generation”; American Public Health Association, “Creating the Healthiest Nation.”
36. Society of Practitioners of Health Impact Assessment, “How to Advance Equity Through Health Impact Assessments” (2016),  https://humanimpact.org/wp-content/uploads/2016/11/SOPHIA_EquityMetricsV2_2016.11.pdf.
37. National Research Council, “Improving Health in the United States.”