Why hasn't more use been made of ACSA for measurement of progress?

What is ACSA? It seems to be an acronym for a lot of different things, but the particular ACSA I am referring to is Anamnestic Comparative Self-Assessment. This is an approach to measuring progress which was first suggested by Jan Bernheim about 30 years ago.

The distinctive feature of ACSA is that it asks survey respondents to rate their current wellbeing by comparison with their memory of the best and worst periods of their own lives (with the best period being given a rating of +5 and the worst period being given a rating of -5).

ACSA is an alternative to the conventional question which asks people to rate their current lives using abstract universal anchors. For example, the Cantril scale gives ‘the best possible life’ a rating of 10 and ‘the worst possible life’ a rating of zero.

In terms of measuring progress, ACSA has the merit of using anchors that could reasonably be expected to more stable over time than perceptions of the best possible life. As explained in recent posts (here and here), when people are asked to rate their own lives relative to the best possible life, they are likely to be making that assessment relative to a moving target. If they see their own lives improving in line with their perceptions of the best possible life, they can be expected to give similar ratings to their lives in successive surveys. It should be obvious to everyone that it is a mistake under those circumstances to interpret stable ratings as implying an absence of progress.

A major study comparing results obtained using ACSA and a conventional measure of life satisfaction for a large number of adult hospital patients suggests that ACSA is indeed less subject to biases of various kinds. For example, the results obtained using ACSA were more responsive to a major objective change in the prospects of end-stage liver disease patients following liver transplantation. The conventional measure of life satisfaction did not capture adequately the impact on wellbeing of the life-threatened situation of these patients prior to transplantation, or the fact that transplantation restored them to an almost normal life. The study is reported in Jan Bernheim et al, ‘The potential of anamnestic comparative self-assessment (‘ACSA) to reduce bias in the measurement of subjective well-being’, Journal of Happiness Studies (2006). An ungated article providing a brief discussion of ACSA is available here.

The potential strengths of ACSA relative to conventional measures of life satisfaction are most obvious where the focus of research is on changes in the wellbeing of individuals over time. A potential weakness of ACSA arises in comparing ratings of different individuals, even though research findings suggest that there are common elements in memories of different people concerning the best and worst periods of their lives (the best periods often involve such things as birth of a child and the worst periods such things as unemployment). It seems likely that many people in high-income countries would perceive that the worst periods in their lives were not as bad as those experienced by vast numbers other people in the world. They might also perceive that the best periods of their lives were better than those of people with fewer opportunities.

One possible way to combine the ACSA ratings of different people would be to place them on the same scale as conventional ratings using the Cantril scale.  When I did that for myself, I gave a rating of 8.5 to my current life, a rating of 9.5 to the best period of my life and a rating of 6.0 to the worst period of my life. That implies an ACSA rating of about 2 [10*(8.5-6.0)/(9.5-6.0) – 5]. That is also the ACSA rating I gave to my current life when I asked myself the ACSA question directly. Such introspective exercises don’t necessarily mean much, but this one suggests to me that the underlying concepts used in ACSA are compatible with the Cantril scale. I urge other people to do the exercise to see if they also get sensible ACSA estimates. 

As far as I can see there is no reason why surveys could not ask people to give a rating to the best and worse periods of their own lives on the Cantril scale, immediately after asking them to rate their current lives on that scale. The Cantril scale is far from perfect as a methodology for making interpersonal comparisons of well-being, but the results it provides in that context seem to make more sense than in making comparisons over time. The calculation of ACSA scores in conjunction in longitudinal surveys using the Cantril question provides potential for development of meaningful measures of perceptions of progress.

I don’t know the answer to the question I asked at the beginning of this post. More use should be made of ACSA. It seems to me that including ACSA type questions in longitudinal studies, such as HILDA, has potential to provide useful information.

Why have happiness researchers been so slow to recognize the problems in using surveys to measure progress?

In my last post I pointed out that it is not possible to measure perceptions of progress accurately by using surveys to measure average life satisfaction at different times and then observe to what extent it has risen or fallen. As a result of changing reference norms, people who value an expansion of economic opportunities cannot necessarily be expected to show rising satisfaction with their lives in successive happiness surveys.

I have just discovered that a similar point was made by Francis Heylighten and Jan Bernheim over a decade ago, in an article that seems to have attracted little attention. The authors made the point as follows:

‘Progress could in principle be measured through the change over time of average scores of subjective well-being. However, the existing longitudinal data show little improvement. These survey results are intrinsically insensitive to developments over time, because SWB is typically evaluated relative to proximate, and therefore salient, reference points, such as peers or expectations based on recent experience’. See: Heylighen F. & Bernheim J.(2001): "Measuring Global Progress  Through Subjective Well-Being", in: Proceedings of the III Conference of the ISQOLS.

One of the suggestions that Heylighten and Bernheim made to correct this distortion was to develop a progress indicator from variables that explain a high proportion of cross-country differences in life satisfaction.

If that approach was followed to develop an indicator to measure perceptions of  progress, recent research by John Helliwell and Christopher Barrington-Leigh suggests that the relevant variables to include might be: the log of household income; whether the respondents had relatives or friends to count on if needed; whether the respondents were satisfied with their freedom to choose what to do with their lives; whether corruption was widespread in business and government; and whether they had donated money to a charity in the past month. Their analysis suggests that people in both high-income and low-income countries place about the same value on log income (use of logs allows for declining marginal utility of income) but people in high-income countries place more value on variables other than income. See: ‘Measuring and Understanding Subjective Well-Being’ Canadian Journal of Economics, 43 (3), 2010.

However, I’m not sure that the suggested approach would entirely solve the problem. It seems likely that perceptions that people in low-income countries have of the best possible life would involve a less opulent life-style than the perceptions of people in high-income countries i.e. perceptions of the best possible life rise with increasing wealth (and the marginal utility of income may not decline as rapidly as cross-country regressions seem to imply). In my view, that means it would be preferable to measure perceptions of progress directly using the method suggested in my last post, i.e. by comparing the answers that survey respondents provide when asked to rate their past lives at the same time as their current lives. An even better approach to measurement of progress, as suggested in the book I am writing, would be to identify the characteristics of good societies and measure to what extent societies were adopting those characteristics.

There may be a case to be made that the well-being of people in high-income countries would be higher if the move toward post-materialistic societies was more rapid. But the people who want to make that case should argue it openly, rather than pretending that responses to happiness surveys indicate that most people do not place much value on material progress.

Can happiness surveys help us to measure progress?

I have written about similar questions here before, but I’m not sure that I managed to get the message across to many people. The issues are not all that complex. I probably just need more practice in trying to explain them in simple terms.

The most obvious way to use happiness surveys to measure progress would be to use such surveys to measure average life satisfaction at different times and then observe to what extent it has risen or fallen.

Where is the problem in that? The main problem is that as a result of changing reference norms people who value an expansion of economic opportunities cannot necessarily be expected to show rising satisfaction with their lives in successive surveys.

What do I mean by changing reference norms? When we are asked to rate our satisfaction with life we do so relative to reference norms, such as by comparing our standard of living with that of people we know. Some surveys ask people to rate their lives relative to ‘the best possible life’, but our perceptions of ‘the best possible life’ may also change. For example, education may cause people to expand their horizons so they become less satisfied with a modest standard of living. The same kind of thing can happen when people move from rural to urban areas or obtain access to TV and the internet.  

So, if education tends to make people less satisfied with a modest standard of living, does that mean that they do not value the opportunities that education provides? It obviously doesn’t. Some people make large sacrifices to obtain educational opportunities, so it would be difficult to argue that they don’t value them.

The same reasoning applies to the benefits of technological progress. No-one could expect that people living in 1950 could have felt unhappy or dissatisfied - or sad, or angry even - because they did own personal computers or any of the numerous other amenities of modern life that had not then been invented.

The fact that we do not feel dissatisfied that we do not yet possess the products of future technological progress does not mean that such products will not enhance our future wellbeing and that of our descendants. It just means that we are fortunate to have emotional systems that enable us to give a high rating to our current lives if we can attain a standard of living that is somewhere near the upper bound of what it is currently possible for humans to attain.

Changing reference norms help our emotional system to adapt to changes in external circumstances, but that doesn’t mean that we should allow them to bias our judgements about changes in the quality of our lives.

Derek Bok, former president of Harvard, unwittingly provided a good example of the distorted perception that can arise when we ignore changing reference norms when he wrote:

‘As Americans adapt and yesterday’s luxuries turn into today’s necessities, people are naturally unwilling to give them up, but that does not mean that they are any happier than they were before the process began. Neither does it suggest that the products they yearn for in future will bring them any greater pleasure. What then is the justification for future economic growth?’ See: ‘The Politics of Happiness’, 2010, p 67.

The fallacy in that argument becomes obvious if it is applied to advances in medical science. Does the fact that people in high-income countries have adapted to advances such as the development of antibiotics, and now tend to view them as a normal part of life, mean that such advances have no value? In deciding whether or not we would be happier without advances in medical science, or any other product of technological change, the pertinent question to ask is whether we are obtaining a net benefit from it now. Adaptation may cause us to take for granted the benefits of technological progress, but it is our judgement of where our interests lie that makes us unwilling to give up the those benefits.

One way to eliminate the possible impact of changing reference norms is to ask survey participants to rate their lives at some point in the past (for example, five years ago) at the same time as they are asked to rate their current lives. Responses to such questions enable levels of individual flourishing to be gauged against historical benchmarks to show to what extent people feel that their lives have improved over time. My analysis of such data collected by the World Gallup Poll suggests that people tend to perceive the greatest improvement in their lives over the previous five years in countries where a high percentage of people consider that the national economy is ‘getting better’ and where rates of economic growth have been relatively high.*

Happiness surveys can help us to measure progress if they are used in the right way.

 ----------------------------------------------------------------------------------------------

*The estimated regression equation is as follows:

LIFETODAY = -0.330 + 1.003*PASTLIFE + 0.015*ECONOMY + 0.037*GROWTH + 0.299*IMPGOV

                       0.290)  (0.044)                     (0.003)                      (0.017)                    (0.243)

Adjusted R² = 0.84. The figures in brackets are standard errors of the estimated coefficients.

102 countries were included in the analysis.

LIFETODAY is the average rating ‘life today’ from the Gallup World Poll (around 2008) which asks respondents to rate their current lives on a ladder scale with the ‘best possible life’ as the top rung.

PASTLIFE is the average rating of ‘life five years ago’ from the Gallup World Poll.

ECONOMY is percentage of participants in the Gallup World Poll who perceive that economic conditions in their country are getting better.

GROWTH is the estimated rate of growth in per capita GDP (rgdpl) from Penn World Tables over the preceding five years (2002-07).

IMPGOV is the improvement over the period 2002-07 in the average of the six World Bank Governance Indicators.

How should we describe the current imbalances within western democracies?

Nicholas Eberstadt’s answer to this question is fairly clear from the title of his recently published book, ‘A Nation of Takers, America’s Entitlement Epidemic’. Eberstadt describes the growth of welfare payments in the US, the decline of stigma against accepting help from the government and the growth of dependence on entitlements. He establishes that about half of the US population now live in households receiving some government benefits and more than 30 percent now receive means-tested benefits. He suggests that, with the growing numbers living on disability benefits, ‘gaming and defrauding of the entitlement system have emerged as a mass phenomenon in modern America’. He also suggests that the ‘taker mentality’ has gravitated toward ‘taking from a pool of citizens who can offer no resistance to such schemes: the unborn descendants of today’s entitlement-seeking population’.

The book also presents two dissenting views. William Galston argues that although many people have come to depend on entitlements to fund their living expenses, they have not become ‘dependent’ in the way that children are dependent on their parents. He suggests that much of the growth of welfare entitlements rests on ‘temporally extended interdependence’. One generation consents to helping to fund the retirement of their parents, with the expectation that the next generation will do the same for them. He acknowledges, however, that ‘something has gone awry’ when the current generation discharges its obligations by imposing heavier sacrifices on the next generation. He suggests that the moral issue is ‘generational selfishness’ rather than dependence. He agrees with Eberstadt that disability benefits are subject to serious abuse, but suggests that the willingness of people to take advantage of the system is not necessarily evidence of deep cultural change.

The main point made by Yuval Levin is that differences in vision about the relationship between government and the citizen – collectivism versus radical individualism – overlook the importance of the ‘space between the individual and the state’, which is occupied by the family, civil society and the private economy. He argues that the state gravely threatens the space for private life. He suggests that rather than dependence, the problem is more ‘a draining away’ of ‘civic energies by the effort required to sustain the liberal welfare state. The country ‘is increasingly exhausting itself’ not just because of the size of the entitlement and benefit regime but also because of its ‘immense inefficiency’. Levin suggests that rather than a nation of takers, America is ‘a nation at risk of becoming incapable of rising to the challenge of self-government’.

The different viewpoints presented in this book are highly relevant to some issues discussed in the book I am writing. One of the points I am making is that when governments relieve us of the need to exercise our power of self-direction, then our skills in running our lives will not develop properly and we are likely to remain dependent on government throughout our lives. That means I am in sympathy with the points that Nicholas Eberstadt is making. At the same time, the US does not seem to me to be a particularly promising place to look for evidence of dependence on welfare having a widespread adverse impact on the social fabric.

I also suggest in the book I am writing that there is a growing gap in many wealthy countries between the responsibilities that many people expect democratic governments to discharge and what governments are actually capable of delivering. Perhaps it could be described as a problem of dependence, in the sense of governments becoming addicted to ever more spending (despite rising debt levels or increased reliance on unstable revenue sources).

At times, I have described the problem as an expectations gap, implying that it has arisen because of inflated public expectations of what governments can do. But it isn’t particularly helpful to blame ‘the public’. The underlying problem is that political leaders who seek to place responsibilities on government that are beyond its capability do not suffer appropriate political consequences. So, we should be thinking about how political leaders could be persuaded to moderate their promises and focus more effort on delivering efficient government.

The diagram presented below seems to me to be a useful way to think of the issues involved.

It is interesting to consider where particular countries should be located on the diagram. The countries of southern Europe should obviously be placed near the bottom right hand side and the Scandinavian countries would be at the top right. Hong Kong might be toward the left at the top. But where should we place the US, or Australia?