Authors

  1. Carey, Timothy A.

Article Content

It is long-standing and well documented that health inequities increase the further one travels from a major center.1 Surprisingly, despite how readily these inequities are acknowledged, the routine evaluation of health services in rural, remote and very remote locations is still not standard practice. For example, a recent systematic review reported that the evaluation of effectiveness, and demonstration of impact, of visiting primary healthcare services in very remote contexts had received little attention.2 While effectiveness and impact are closely related terms, impact is a more general consideration of the consequences of an intervention or service. Effectiveness may refer to whether there has been an improvement in the targeted outcomes of a program, whereas impact includes the ongoing effects of a program as well as the influence it may have had even for people not directly involved with it.

 

Despite the scant attention impact has received regarding visiting services in very remote contexts, the importance of impact is widely acknowledged. The assessment of impact must become standard practice so that limited financial resources are used in the most prudent way possible to reduce inequities outside metropolitan locations. Understanding "impact" as an acronym to identify its key components may help. When referring to the IMPACT acronym, capitalization will be used.

 

To maximize the impact of health and psychosocial programs in general, service providers and health service managers should ensure that what is provided is: important to the community; monitored and measured so that genuine improvement can be demonstrated; planned so that initiatives develop through systematic, cohesive collaboration; adequately and appropriately resourced; courageously challenging the status quo, prevailing attitudes and existing wisdom; and telling a story that affects people's hearts as well as their heads. These criteria could be used by funders of services, policy and other decision makers, and health service managers to ensure that service providers are maximizing the impact of the interventions they are delivering. The criteria are explained and expanded below.

 

If programs and services are to be embraced by their intended beneficiaries, then these interventions must address a need that is a priority from the perspective of the beneficiaries. For example, Hudson3 reported that a community in East Arnhem in Australia's Northern Territory was provided with suicide prevention training even though they had no experience of suicide and people from the community had already attended a regional suicide prevention training session at considerable cost. It is important that the relevant "community" is identified and defined before the service is provided and that the community's health priorities are established. The community, for example, might not be a geographic location but rather a group defined by a particular condition (e.g. type 2 diabetes) or certain circumstances (e.g. homelessness, domestic violence).

 

To determine with some degree of certainty that genuine improvements are occurring systematically with the provision of the intervention, the initiative should be monitored and measured on a regular and routine basis. A system of ongoing monitoring and evaluation needs to be planned and embedded into the program before implementation commences. The value of ongoing monitoring and evaluation has been repeatedly demonstrated across a variety of settings.4 Regularly and routinely monitoring and evaluating an intervention invites a reconceptualization of program effectiveness - that is, effectiveness transitions from an intrinsic property of the program to an outcome that is cocreated by the service provider and the service recipient through their interaction with the resource of the service or program.4

 

Any intervention must be planned from the outset to achieve a desirable impact. Programs and services should be provided systematically and deliberately, with cohesive collaboration being an essential and easily identifiable feature of the planning.

 

Both the sustainability and ultimate impact of a program depend heavily on how the intervention is resourced. Initiatives that are introduced into defined communities must be adequately and appropriate resourced.4 This resourcing should include the necessary training for staff as well as consideration of evaluation costs.

 

Any intervention requires a change in some aspect of the status quo. Chronic diseases, for example, are chronic because certain behavior patterns have been occurring over a long period of time. Achieving impact under these conditions, therefore, requires courageously challenging the status quo, accepted wisdom and prevailing attitudes. One recent study found that peer pressure was not a reason that youth in remote Australia gave to explain their alcohol use.5 Rather, these youth considered peer pressure to be a concept that adults had created to influence and manage youth behavior.5

 

An essential, but often neglected, aspect of maximizing program impact is to "tell the story". Not only is it important to ensure the quality and integrity of what information is disseminated, it is also essential to consider how the information is conveyed. To ensure impact is being fully exploited, it is important to tell the story in a way that affects people's "hearts as well as their heads".4

 

Conceptualizing impact using the IMPACT acronym accords well with findings from systematic reviews and could be used to guide the structure of future systematic reviews. It could also be used in conjunction with findings from systematic reviews to identify and address areas where impact could be improved in particular fields. For example, a systematic review of patient involvement in quality-improvement initiatives identified challenges of engaging patients, tokenism and budgetary constraints.6 These findings suggest that, in this area, greater attention needs to be devoted to ensuring the following: the initiative being proposed is important to the targeted patient group; the story is told in a way that reaches people's hearts and heads; and that appropriate resourcing is provided. Similarly, a systematic review of the experiences of Aboriginal health workers and non-Indigenous health professionals identified numerous factors that affected the success of these collaborations.7 If the collaboration between Aboriginal health workers and non-Indigenous health professionals is considered the intervention, the results of this systematic review suggest that, from an IMPACT perspective, improvements in health care could be realized by ensuring that the intervention is important to all relevant people; systematically planning the way the intervention will develop; and routinely monitoring and measuring the effects of the intervention.

 

To eliminate health inequities, it is critical that IMPACT is at the forefront of service providers' minds at all times. It is also imperative that health service funders and policy makers insist on the demonstration of IMPACT. Conceptualizing impact in this way may help with the development of comprehensive strategies to realize the greatest impact for the benefit of the communities served.

 

References

 

1. Carey TA, Wakerman J, Humphreys JS, Buykx P, Lindeman M. What primary health care services should residents of rural and remote Australia be able to access? A systematic review of "core" primary health care services. BMC Health Serv Res 2013; 13:178. [Context Link]

 

2. Carey TA, Sirett D, Russell D, Humphreys JS, Wakerman J. What is the overall impact or effectiveness of visiting primary health care services in rural and remote communities in high-income countries? A systematic review. BMC Health Serv Res 2018; 18 (1):476. [Context Link]

 

3. Hudson S. Mapping the Indigenous program and funding maze. Sydney: The Centre for Independent Studies; 2016. [Context Link]

 

4. Carey TA, Fauth JM, Tremblay GC. Rethinking evaluation for improved health outcomes: implications for remote Australia. Am J Eval 2019; doi: 10.1177/1098214018824040. [Context Link]

 

5. Carey TA, MacGregor M. Understanding why youth in remote Australia use alcohol in the ways that they do. Aust J Rural Health 2019; doi: 10.1111/ajr.12523. [Context Link]

 

6. van Deventer C, McInerney P, Cooke R. Patients' involvement in improvement initiatives: a qualitative systematic review. JBI Database System Rev Implement Rep 2015; 13 (10):232-290. [Context Link]

 

7. Mercer C, Byrth J, Jordan Z, Stern C. The experiences of Aboriginal health workers and non-Aboriginal health professionals working collaboratively in the delivery of health care to Aboriginal Australians: a systematic review. JBI Database System Rev Implement Rep 2014; 12 (3):234-418. [Context Link]