Learning from the MDGs

Despite considerable international agreement about MDG aims and interventions to implement them, decisions were inconclusive about which outcome measures should evaluate MDG goals. Effective MDG interventions included training village health workers, free insecticide treatment of bed nets, eliminating basic health service fees in low/middle-income countries, improving access to sexual and reproductive care and offering combination drug therapies for HIV infection.4 One line of thinking is that outcome measures should be intervention-specific, and hence different assessments are needed to address heterogeneous interventions designed to improve quality of life (QoL) and survival.4 Although specific outcome measures are suited to evaluating single, and sometimes similar interventions, their data limit conclusions when outcomes from diverse targets need to be compared. Despite broad agreement that quality, timely and reliable disaggregated data are essential to measuring progress, and recognition that gross domestic product (GDP) alone would be insufficient to evaluate the SDGs, in 2015 the UN2 reported low agreement about which alternative and additional measures to select. Although good generic measures are available to compare many different populations, conditions and settings, rather than endorsing one of these, the UN-SDG panel decided to go ‘beyond GDP’ by developing new measures of progress; consequently, they proposed subjective well-being (SWB) indicators, for example, positive mood5 (Sustainable Development Solutions Network, p69)5. Without generic subjective data from global cultures, it will not be possible to draw conclusions about progress in 2030 when SDG outcomes are finally and formally evaluated. Is it possible to find out whether ‘anyone has been left behind’3 (ICUN, UNEP, WWF, p3–35)3, without asking about perceptions?

Within the context of this debate, we consider an international, well-developed, multilingual, generic tool, with an established track record. Acceptable and feasible to assess subjective QoL in many cultures worldwide, the WHOQOL-BREF was developed through an international collaboration convened by the WHO, Geneva. The WHOQOL group aimed to advance standards in cross-cultural measurement, and produce a global measure that would complement ‘objective’ indices on standard of living (eg, GDP) and health (eg, mortality rate).6 Multiple language versions of the WHOQOL were simultaneously developed in 15 cultures worldwide, from an internationally agreed protocol which has since been used by a network of developers in around 100 cultures. Endorsement of a suitable subjective measure to evaluate SDG goals will enable ongoing projects to take immediate advantage of evaluating key outcomes from interventions, treatments and major life events, using a common metric.

Requirements for evaluating the SDGs

A good subjective measure for SDG purposes would need to show (a) measurement properties of reliability and validity, at global, regional and national levels, and moreover that its scores respond sensitively to changes in people’s lives; (b) a capacity to detect positive and negative changes, reflecting experience; (c) acceptability to different cultures, and feasibility to use; (d) easy administration and scoring, with scores that are readily interpretable by decision makers and practitioners; (e) a capacity to assess well people, including those receiving health promotion or disease prevention interventions, and unhealthy populations of all types (eg, chronic mental health, neglected tropical diseases, intractable infections, non-communicable diseases, road traffic injuries); (f) that the results can indicate where inequalities between groups exist, so that ‘nobody is left behind’; and (g) standard procedures to enable cultural adaption, translation and standardisation when new language versions are needed, potentially for 7300+ cultures worldwide. Such properties will enable sound comparisons of diverse cultures, for example community happiness in Bhutan; berry picking in Sweden; ‘feeling fed-up’ in Britain. The availability of this type of tool opens up opportunities for ‘invisible’, indigenous groups (eg, ‘first nations’), to give ‘voice’ to their QoL views when engaging with SDG projects, and perhaps for the first time, to communicate them to policymakers.

Although generic economic indices like GDP and population health estimates (eg, disability-adjusted life years) were used in MDG evaluations, no single measure can fulfil this challenging brief. When it comes to SWB and health, there is no substitute for obtaining meaningful information directly from individuals about their personal experiences, and then testing whether the intervention significantly changes it. Although the UN acknowledged the importance of assessing experience in 2015, and designated WB (defined as positive mood) as an indicator for SDG 17 among 100 indicators in total,5 by March 2016, WB had disappeared from the new final list of 230 UN approved SDG indicators.7 Furthermore, all 230 are exclusively objective. Without subjective empirical data, comparing SDG achievements remains partial and therefore contentious. Incomplete data for MDG targets frustrated global policymaking during the planning period leading to publication of the SDG plan in 2015: they reported ‘progress was limited and hard won’.2 A global evaluation strategy of using a common metric is urgently needed to record the personal experience of all communities participating in SDG projects in the next 12 years. A standardised subjective measure added to existing population-based indicators would provide a more comprehensive and conclusive evaluation. Without sound data it will be impossible to conclude whether WB has been significantly improved by the SDG programme, wasting more years, and considerable human and financial resources.

WB or QoL?

Although promoting WB is a specified aim of SDG 3, the UN2 did not say whether WB or QoL should be measured alongside health. Conceptual distinctions between subjective QoL and SWB are opaque, raising questions about whether these concepts are identical, ‘nested’ within each other or different.8 Lack of clarity confounds policymakers and practitioners who seek to make informed choices about the best measure to use. QoL assessment is well suited to measuring SDG 3 outcomes, with its reputation for rigorously developed multidimensional instruments that are relevant to mental and physical health. Furthermore, the five dimensions of mood (positive and negative), cognition and evaluation that typically compose SWB9 can be readily mapped onto some of the 25 QoL dimensions assessed by the WHOQOL. New international empirical findings10 strongly suggest that SWB is ‘nested’ within QoL. QoL as measured by the WHOQOL offers a scale with exceptional multidimensionality and conceptual integration, so is highly appropriate to address the very wide range of life qualities addressed by the 17 SDGs.

WB has also been viewed as purely ‘objective’, and assessed by counting material household goods (eg, TVs, cars, house size),11 but some indices show a loose relationship with subjective measures. Over a decade of analysis examining the relationship between GDP and SWB, Easterlin and Sawangfa12 showed that this association is neither strong nor linear, as generally assumed. While SWB in lower income countries does increase as GDP rises, among higher income countries rising GDP has little or no association with SWB,12 illustrating that objective GDP indicators are no substitute for subjective measures of WB or QoL.

Although the history of health-related QoL assessment reveals several high-quality generic measures (eg, European Quality of Life Scale-five dimensions (EQ-5D), Short Form-36 Health Survey (SF-36)), the unique conceptual breadth of the WHOQOL with its 25 facets scored in four domains enables WB to be evaluated for many SDGs, for health goals where WB is intrinsic (SDG 3), and for other SDGs (SDG 11) eg, where QoL evidence could show that people feel physically safer and secure in sustainable cities. In SDG 17, better QoL could strengthen ways to implement and revitalise the global partnership for sustainable development. Multiple dimensions of a profile are especially valuable for unpacking the impact the of QoL dimensions where trade-offs are unknown; for example, action to conserve biodiversity may enhance psychological QoL, and may also conflict with more immediate human needs for adequate housing, better nutrition and being employed (relating to SDG 15, SDG 2 and SDG 16),2 so potentially abrading environmental QoL. Including a quality generic, subjective profile could complement objective economic measures (eg, GDP), and balance the evaluation portfolio.

Advances in QoL assessment

The WHOQOL suite of instruments offers a series of patient and person-reported outcome measures (PROMs) that provide insights into thoughts, feelings and personal experiences relating to QoL and health. The WHOQOL was the first generic, cross-cultural PROM designed explicitly for global use. Focus groups of patients, health professionals and community members were convened in 15 cultures, pooling their proposed concepts and wording for the questionnaire. Due to the way it was developed, it is highly acceptable to users. Acceptability promotes lower survey refusal rates and reduces missing data, which in turn improve the accuracy that decision makers need when drawing conclusions and taking action. Improvements to validity that arise from using PROMs methods have accelerated routine use in randomised controlled trials (see https://commonfund.nih.gov/promis/overview).

A strength of the WHOQOL development was the innovatory ‘spoke-wheel’ methodology that was devised by a multidisciplinary, multinational collaboration to simultaneously create many highly equivalent language versions from a commonly agreed concept.13 Issues of national importance to QoL were identified locally, and then appended to WHOQOL-BREF core items, to complete the QoL concept for that country,14 for example, eating and appetite in Hong Kong. This process tailored the evaluation to the culture, and hence resonates with the UN Inter Agency Expert Group policy7 to allow countries to devise culturally appropriate indicators that are not on the approved international SDG list. National progress in SDG targets will be tracked then later aggregated up to regional and international levels.

Over 20 years, a WHOQOL-BREF manual has facilitated around 100 culturally adapted translations globally, completed by over 60 000 sick and well adults. This process potentially enables universal QoL comparisons between cultures as diverse as Assam tea pickers, Wall Street bankers and Peruvian mineworkers, by producing highly equivalent cross-cultural data. Similarly, the WHOQOL-BREF’s globally agreed importance items are clustered and scored in physical, psychological, social and environmental QoL domains. This breadth of dimensions uniquely extends the WHOQOL concept well beyond the physical and psychological, commonly found in health-related QoL measures, as it includes QoL related to social relationships, the environment (eg, physical safety, financial resources, home environment), and an unusual spiritual, religious and personal beliefs component within the psychological domain (see detail in references 6 15 16). These features extend its application to innovative settings beyond health. A profile of scores offers a fine-grained multidimensional analysis, and a priori predictions can be made about which domains and items will likely respond most to the impact of an intervention. Furthermore, the WHOQOL-BREF together with its importance ratings has been recently used at a one-to-one level to assess QoL, and prioritise individual patient healthcare.17 18 This research is underpinned by the WHO definition of QoL: ‘An individual’s perception of their position in life, in the context of the culture and value systems in which they live, and in relation to their goals, expectations, standards and concerns’ (The WHOQOL Group,p. 43)19. Briefly, QoL is subjective and in the eye of the beholder, as it is invisible to observers; hence, it is best judged by the person themselves.20

The WHOQOL-BREF results are an aid to policymaking and allocating budgetary resources in the US State of Connecticut Department of Mental Health and Addiction Services (DMHAS).21 Since 2007, 14 000+ adult mental health service users have been invited to complete the WHOQOL-BREF annually, which takes about 6 min in English. This evidence is disaggregated by service, demographics and health, enabling policymakers to prioritise service delivery to disadvantaged groups with the aim of redressing QoL deficits. Following a 3-year pilot, DMHAS annually allocates budgetary resources to promote effective interventions, and make service improvements,21 illustrating how disaggregated WHOQOL-BREF data (eg, by age, education, gender, culture) can be used to reduce inequalities. It would therefore be valuable for delivering SDG 3 targets to relevant groups, being eminently suited to solving global policy issues.

The importance of responsiveness

Global psychometric properties of the WHOQOL-BREF show that scores are reliable and valid across diverse cultures,15 and international reference data are available for gender, age and health status groups.16 For some countries, national norms are available (eg, Australia).22 Other studies show that the scores distinguish sick people from well, and across a wide range of physical and psychological disorders, and social and environmental conditions;23 this is evidence of good discriminant validity.

Among key psychometric properties, responsiveness (also sometimes referred to as sensitivity to change) is perhaps the most important requirement. In clinical healthcare and interventions, responsiveness shows the degree to which score changes are clinically relevant. Responsiveness is the ability of a measure to detect clinical change, particularly where differences in outcomes occur, including small ones. Responsive instruments have scores with the capacity to change sensitively and appropriately, in accordance with the person’s condition or situation, and this information needs to be interpreted. However, there are many different ways of measuring responsiveness. In a longitudinal study of depressed patients in primary care in six cultures who received anti-depressant medication over 9 months, changes in WHOQOL-BREF domain scores were examined in relation to changing depressive symptoms to see how much QoL scores would increase if depression improved, and decrease if deterioration occurred.24 Without such measurement information about change and its relevance, the success of an intervention or trial remains unknown.

The systematic review (SR) technique used in this study can assess responsiveness to change by combining and assessing known evidence in meta-analysis. As methods of calculating responsiveness are controversial,25 we followed Norman et al.’s recommendation26 to limit calculations to Cohen’s effect size.27 Through evaluating collective knowledge about WHOQOL-BREF responsiveness, we shall draw conclusions about the clinical and social relevance of this measure. For instance, this information has clinical value when managing long-term treatment for chronic conditions, such as Parkinson’s disease.20

Responsive instruments can then be used to monitor new services, modes of service delivery, the impact of policy changes, reactions to political crises and environmental disasters, such as those outlined as SDGs. For SDG targets, it will be essential to determine whether QoL has improved significantly following planned interventions for instance, providing adequate and reliable nutrition to improve QoL. This question could be investigated in Addis Abba today using the Amharic version of the WHOQOL-BREF, and support action in this low income country. As no other generic global health-related QoL measure has the same capacity to embrace the plethora of situations and conditions outlined in the SDGs, the WHOQOL-BREF should be the subjective measure of choice.

Establishing the responsiveness of social and environmental domains in the WHOQOL-BREF is important to understanding the breadth of QoL impacts from adverse conditions (eg, earthquakes) and afterwards. As yet, the WHOQOL-BREF’s responsiveness has not been comprehensively examined with regard to a variety of situations and populations, despite several studies which assessed outcomes after diverse interventions and major life events. Examples include radiation exposure,28 earthquake survival,29 housing elderly Sichuan Chinese in an earthquake zone,30 Palestinian conflict in Gaza,31 Nigerian refugees,32 women torture survivors,33 slum upgrades34 and poverty in low/middle-income countries.35 By using the WHOQOL-BREF in these challenging fields, this indicates that social and development practitioners view it as a suitable tool.

The ability of instrument scores to respond to changing conditions is perhaps the most important psychometric property of health and WB measures. This property is no more applicable than when monitoring change induced by environmental and social interventions, including many addressing SDG targets. Gathering multidimensional subjective information about how QoL is affected provides much needed information when trying to understand why some interventions succeed, and if they fail, how that intervention might be adjusted to become more effective.

The main aim of the present research was to conduct a systematic review and meta-analysis of global evidence on the responsiveness of the WHOQOL-BREF to change. Furthermore, to assess the relevance or meaning of WHOQOL-BREF domain scores in clinical and social settings.