CHAPTER THREE: BEYOND EDUCATIONAL PERFORMANCE INDICATORS: MOVING TOWARDS OVERALL SOCIAL INDICATORS

3.1 INTRODUCTION
3.2 BACKGROUND
3.3 SOCIAL INDICATORS SYSTEMS
3.4 COMPOSITE QUALITY OF LIFE INDICES
3.5 DEVELOPING A MODERN FRAMEWORK: MONITORING SOCIAL CONDITIONS TOP-DOWN?
3.6 THE IMPACT OF EDUCATION UPON HEALTH
3.7 THE IMPACT OF EDUCATION ON EMPLOYMENT IN DEVELOPING COUNTRIES
3.8 MONITORING BASIC NEEDS AT THE LOCAL LEVEL
ANNEX 3 A: Opportunity Cost and Valuations
ANNEX 3 B: Technical Problems in Developing Indicators
ANNEX 3 C: VALUING HEALTH AND MEASURING (UN) EMPLOYMENT
ANNEX 3 D: EXERCISES IN LOCAL PLANNING

3.1 INTRODUCTION

This chapter moves beyond the discussion of educational performance indicators, to consider both overall system of social indicators and ways of implementing them, both at a macro and micro level. Methods of assessing impacts of education upon other systems are discussed, in particular, the impact of education on health and employment.

In order to understand the rationale for socio-economic measurement, it is important to review it's history. In the first two sections, the origins and purposes of socio-economic measurement in the sixties and seventies are reviewed. It is concluded the major problem then was with the quality of the data available.

There have been several attempts to construct composite quality of life indices ranging from the PQLI in the 1970s to the UNDP's Human Development Index in the 1990s. These are analysed and rejected in section three. On this basis, the next three sections analyse the principles and criteria for constructing a flexible framework of social indicators and exemplify this in terms of assessing the impact of education on health and employment. Recognising the difficulty of implementation, the requirements and principles of monitoring basic needs at the local level are detailed in the final section.

This chapter is supported by several annexes which detail further background issues: opportunity cost and valuations; technical problems in developing indicators and concepts of employment and unemployment; and experience with local planning.

3.2 BACKGROUND

3.2.1 Origins and purposes of systems of socio-economic data
3.2.2 The Social Indicators Movement of the 1960s
3.2.3 Approaches: A Thousand Flowers Bloom
3.2.4 Factors contributing to the decline of social indicators

3.2.1 Origins and purposes of systems of socio-economic data

Although the recording of socio-economic data systematically can be traced back at least to William the Conqueror's Domesday Book, modern interest in reporting on (the lack of) social progress can probably be attributed to people like Quetelet (a Belgian statistician working in the middle of the nineteenth century) concerned to document the downside of industrialisation. Moreover, whilst the art of social measurement developed either as a consequence of increasing trade (Petty 1690) or as offshoots of the development of administrative systems required by the embryonic welfare states (Quetelet), or concerns of liberal reformers to measure poverty (Booth, 1890), the attempt to develop socio-economic data systems is recent: it is a delayed consequence of the development of the national accounts during the 1930s and 1940s.

During the twenty years that followed the Second World War, the success of Western capitalist economies in raising living standards meant that, although there was some interest in modifying national accounts in order to provide a better picture of economic welfare - for example, Net National Welfare (NNW) - there was little specific concern over social data. Indeed, the only systematic attempt was that promoted by the United Nations, the System of Social and Demographic Statistics (SSDS). It was proposed to monitor the progression of people through their life cycle in a systematic fashion. Not only did the system imply very complex and exhaustive data collection, however, the reliance on the length of time spent in a status as the single dimension for assessing the value or weight to be attached to that status was very limiting.

3.2.2 The Social Indicators Movement of the 1960s

Social Indicators to Monitor 'Progress'

The resurgence of interest in a system of socio-economic measurement across a broad range of subjects in the 1960s can be traced to a number of factors coming together:

· The re-documentation of poverty in several Western European countries and the parallel questioning of trickle-down theories;

· A growing concern about the environmental and other social costs of economic growth in general and more specifically, the upheavals generated by the space programme in the United States;

· The emergence of a youth culture questioning the purpose of economic growth.

Social Indicators for Managing Change

The concern with managing social change is one which has been clearly expressed by proponents and critics of the 'social indicator movement' (so called by many authors in the 1960s) alike. Bauer, working for the American National Academy of Science on research funded by NASA, was concerned with anticipating the 'second order' effects of innovations such as the space programme, which may be more important than 'first order' effects. By this, he meant not only sensitivity to feedback but prediction of the consequences of feedback, which he saw as one of the main purposes of social indicators (Bauer, 1966).

Wiles (1971) was more specific about the kind of social change involved, seeing 'crisis prediction' as the goal behind the demand for new kinds of social information. This objective he condemned on two grounds. Firstly, because of the 'totalitarian implications' of 'efficient social information' which will 'stabilise whatever group is in power' by enabling it to be more flexible, buy off opposition more efficiently, increase its ideological hold, etc. Secondly (and 'fortunately'), crisis prediction never is efficient, whether one considers specific crises or looks at major underlying causes (particularly technological development). It is clear therefore that, for both these authors, the purpose of social indicators is maintaining social control, seen as threatened by a rapid rate of social change, complex and so far unpredictable, brought about by technological innovation.

Hence the interest of social indicators from the point of view of social control by the state is clear; if all the consequences of technologically innovatory programmes can be predicted, then programmes can be chosen not only according to their estimated profitability, but also according to their value in terms of preserving the system. Of course the implementation of such a system raises a number of problems. There are particular difficulties with measuring social change compared with measuring a stable economic system. Thus, not only is there a lack of any agreed theory of social change while the economic system comes - as it were -apparently readily quantified, it is often argued against social indicators that they are an attempt to quantify the unquantifiable; and the lack of a common unit of measurement makes it difficult to weight the social indicators so as to produce one, or a small number of indices (see Sheldon and Moore, 1968.)

An Uneasy Balance

The distinction between whether measures are to be used in the research and evaluation context or in a planning mode is important: for the planning purpose, almost any estimate is better than no estimate at all; while for research and evaluation purposes, the premature publication of proxy data may be counterproductive (Murray 1991).

But, whether the aim is reporting on the outcomes of social programmes, quality of life, social progress or simply social trends, or in enlarging social control à la Bauer, the choice of what components of well-being are to be measured by indicators is essentially a political one. Obviously, any combination of these indicators via weights that would themselves be the subject of debate - would be at least as politicised.

3.2.3 Approaches: A Thousand Flowers Bloom

Since the 1960s, there has been a flowering of different methods of measuring the quality of life in industrialised societies. A classic division might be in terms of methods, theory and policy. Among the methodologists, there are those who have followed the survey route: there have been large number of surveys to document the 'objective' living conditions of individuals and households ('objective' in the sense that the measurement is made by someone else). Equally, there have been a large number of attitudinal and/or opinion surveys and, sometimes, more systematic attempts to assess 'happiness' or 'satisfaction' in terms of scales. Other methodologists have directly addressed the problem of measuring the quality of life through a variety of valuation techniques. The theoreticians, unsurprisingly, have been less concerned with data collection procedures and more concerned with monitoring collective attributes of a society - sometimes seen as national characteristics - such as the level of autonomy, political participation, etc.; or with elaborating a model of social systems in order to identify its key attributes. The 'policy' group is concerned with providing simple policy tools - and so have focused on possible ways of modifying GNP to provide a better measure of (economic) welfare; or on combining 'objective' indicators (usually) into a composite in parallel to national income per capita.

There have, therefore, been several strands in this history often with very little communication between them. For the purposes of this review, the division into methods, theory and policy is not very useful because we are searching for social indicators that are theoretically grounded, for appropriate methods of data collection, and for relevant policy applications. Instead, we discuss the various strands in two main groups.

There are those who emphasise the importance of a uniform method of valuing welfare. Much of this work has been based on the financial nexus and have led to proposals either for methods of extending GNP to reflect better economic welfare; or for ways of valuing other, currently non-monetarised, components, using the measuring rod of money. There has also been interesting work, however, carried out using time as the basis for valuation and these are also discussed in detail in the Technical Appendix 1 under the general rubric of 'opportunity cost'.

Others have argued that we must escape from a system of data which are independent of the national accounts or of opportunity cost; and those working within that paradigm form the 'social indicator movement'.

In both cases, there has been further tension between the emphasis on the techniques of data collection and analysis, and on the validity of the data being collected for measuring the phenomena concerned.

3.2.4 Factors contributing to the decline of social indicators

Several factors have been identified as contributing to the decline in social indicators:

· There was a 'tendency for indicators to become vindicators' (Bulmer, 1990: 410) and for the reports to be 'rather bland compromises, deliberately presented without text that might link the data to policy' (Innes, 1990: 430).

· The system became divorced from the policy context, focusing on the measurement task, often to the exclusion of the political and institutional one (Innes, 1990: 431).

· There was the lack of general social scientific theories to measure the theoretical constructs.

Carley's ahistorical argument ignores the fact that social indicators were developed in reaction to the perceived chaos of the 1960s, environmental concerns - both nuclear and pollution - and incomprehensible youth. It was because these threats had receded towards the end of the 1970s (they didn't disappear), or had been replaced by other concerns (e.g. over energy) that people lost interest.

The basic problem of understanding the social benefits to be derived either from corporate or government activity remains, however. While temporarily replaceable by crude indices based on recorded financial transactions such as GDP, serious analysts have always searched for some other system of measurement.

3.3 SOCIAL INDICATORS SYSTEMS

3.3.1 Early Examples in the Seventies
3.3.2 Basic Needs Approach
3.3.3 Moving Towards the Individual
3.3.4 Conclusion

3.3.1 Early Examples in the Seventies

Three early examples illustrate different approaches to the wider policy integration intended by the development of social indicators.

Net National Welfare Measurement Committee of Japan

Established in May 1971 by the Economic Council of Japan, the NNW Measurement Committee proposed an index of "welfare national income" or "net national welfare" which would "allow for... various plus and minus factors that do not appear in present GNP figures (OECD, 1976: 35). They included the following items in NNW: government consumption; private monetary consumption; services from government capital stocks; services from personal consumer durables; leisure hours (at average wage rate) and non-market activities such as housework. Subtracted were environmental maintenance costs and pollution reparation expenses as well as losses due to urbanisation in commuting and accidents; and net investment.

The initial calculations by the NNW Measurement Committee suggested that the average annual rate of growth in NNW was some 20% below that of Net Domestic Product (Economic Council of Japan, 1973). The NNW - probably for that reason -was never adopted as a measure of welfare.

The Physical Quality of Life Index (PQLI)

The Overseas Development Council looked for a measure to assess nutritional success in very poor countries. They said "The ultimate test, obviously, was the physical quality of life achieved." (Morris, 1976). A useful measure should meet the following criteria:

· the measure should not depend too heavily (if at all) on market performances (or on GNP data);

· the index should avoid measures that assume that LDCs will inevitably develop along lines followed by developing countries;

· the index should avoid measures that are excessively ethnocentric;

· measures of performances probably should not be based on absolute minima;

· the index or indicators must be sensitive to distribution results; and

· the measure must be simple.

This latter point - the way in which one overall index tends to obscure rather than clarify issues is, of course, one of the original objections to the GNP measure being used as an index of (economic) progress. Proposing the PQLI - and it's latter-day derivative, the Human Development Index - is rather like jumping out of the frying pan into the fire. (see Section 3.4 on composite indices).

Active Life Sequence.

Seers (1975) has proposed that the well-known indicator of life expectancy should be extended to calculate expectations of the time spent in various 'life states' - for example, attending school, working, retirement, and so on. Later he argued that this technique would facilitate a statistical description of a community in terms of the typical life cycle of members of that community (Seers, 1979) based on the system of social transition matrices developed by Stone (1972).

Notwithstanding the basic practical problem of collecting a detailed record of movements by members of a population from any one state to another, the theoretical presumption of this approach is that the most important aspect of each and every life state is the length of time spent on it. Although the basis of some indices¹ ' it actually seems rather implausible as a general rule. It is indeed important to recognise that time - like resources - is limited and so has to be allocated and shared. But our concern with policy integration is wider than this.

Of the three approaches, PQLI is obviously far too simplistic to be useful for tackling the kind of problems that arise. Moreover, it has no theoretical foundation and it is therefore surprising to see that such a "stitched-together" measure is proposed in the academic literature as a 'useful' index (Cereseto and Weintztein, 1986) - and of course, as mentioned above, has spawned the HDI (see also 3.4.1). In contrast, both the NNW approach and the Active Life Sequence technique are interesting partly because they have some conceptual foundations but they are, in consequence, limited in their applicability. Thus, the NNW approach depends on being able to assign a monetary value to each of the dimensions included, with attendant problems of valuation (see Annex 3A); and the Active Life Sequence approach considers only the allocation of time to various activities and not the value of that time (see also Annex 3A). Nevertheless both are theoretically based and may shed light on some of the issues involved in integrating policies and programmes.

3.3.2 Basic Needs Approach

The notion of a hierarchy of needs - and therefore of basic needs - was developed by Maslow (1944). An emaciated version of this idea was taken up in the basic needs approach to development as defined in the Programme of Action at the 1976 ILO World Employment Conference. The 'basic needs approach' aimed to take account both of what goods and services are available and who were the beneficiaries in terms of consumption. Thus the 'definition of a set of basic needs, together constituting a minimum standard of living, would at one and the same time assist in the identification of these (poorest) groups and provide concrete (production) targets against which to measure progress' (ILO, 1976: 31). Basic needs were taken to include two elements:

( ) certain minimum requirements of a family for private consumption, as well as certain household equipment and furniture; and

( ) essential services provided by, and for the community at large, such as safe drinking water, sanitation, public transport, health, educational and cultural facilities.

· security, food and water, clothing and shelter, sanitation (the survival needs);

· access, knowledge, mobility and skills (to function in society);

· quality, justice and self-reliance (to express a fundamental identity).

The main thrust of a basic-needs strategy must be to ensure that there is effective mass participation of the rural population in the political process in order to safeguard their interests.' (Sheehan and Hopkins, 1979: 59)

The Basic Needs Approach has fallen into disrepute; critics variously argue that it is Utopian, romantic, populist, anarchic, inefficient, slow, and that it obscures the fundamental international problems that frequently underlie poverty. Instead, they point to the need for freer trade or for fair prices (take your pick) in order to resurrect growth-oriented development (see Wisner, 1988).

But the special feature of the basic needs approach was the very belated recognition of the fact that welfare must be measured in concrete terms for each individual prior to homogenisation in monetary or any other units and prior to any aggregation across people in terms of a 'growth' index. No monetary measure of income per capita can ensure that essential goods and services are produced in the right quantities at the right time and actually reach the right people. Just as the policy implications of the approach were soon perverted (see Wisner, 1988), however, so were the statistical implications.

Despite the initial affirmation of the importance of identifying basic needs in physical quantities, those applying the approach in the latter half of the 1970s were eager to find a one dimensional yardstick (Hicks and Streeton, 1979) and sometimes a monetary equivalent. Indeed, the empirical papers in Employment, Growth and Basic Needs (ILO, 1976) are expressly designed to show what rates of growth in the measured economy would be required to satisfy a given collection of basic needs, leaving aside all kinds of questions about changes in the structures of the economies.

The Problem of Precedence

Maslow, based on then current theories of self actualisation, had promulgated a strict priority ordering among the different needs. In contrast, liberal eclecticism led the ILO to make no judgements at all about the relative importance of different needs (although of course, they crept in via the calculation of a Basic Needs Income). Yet, from any given perspective, there is an order of priority among 'needs': the Aristotelian and Marxist will tend to emphasise indicators of activity, whilst the Platonic liberal will tend to emphasise indicators of status, and so on.

Similarly, the development of a set of performance indicators is context dependent and more generally, the assessment of the impact of a set of policies is always from a particular perspective. This situation has led to proposals for a theory of social reporting (Johansson, 1976) so that one could purposefully choose indicators which are relevant for human development (Miles, 1985), taking account of the inter-linkage between areas. In this case, we are examining the potential for indicators of performance in basic education in developing countries and the impact of that education upon other sectors.¹⁹

3.3.3 Moving Towards the Individual

This emphasis on physical quantities rather than aggregate indices, however, led to measuring an individual's met needs, and thus, an individual's subjective needs were added to the otherwise, physical list of basic needs.

Subjective Happiness/Satisfaction

One such set of measures is concerned with deriving satisfaction measures. A systematic approach to measuring happiness and/or satisfaction has been developed by the Michigan school, the major exponents being Andrews and Withey (1976). They argue, on the basis of small-scale survey work, that several domains contribute to the final outcome of happiness, and that responses to questionnaires about satisfaction in respect of each of these domains can be used to generate a happiness scale. (The current version of this approach is the series of EuroBarometer surveys.)

The problem is that, if direct questions about satisfaction are asked, nearly everybody responds satisfied, and a large proportion 'very satisfied': this is partly because responses appear to measure social norms (of the 'can't complain' variety) rather than self-ratings of well-being.

Citizen Surveys

Another, Scandinavian approach (Johannson, 1976) has been to argue that objective data on living conditions should be available to all citizens as a prerequisite in a representative democracy. While clearly objective data have to be included, reliance on citizen reporting raises all the usual technical problems of social surveys.

Indeed, the crucial issue is whether several of the important outcome indicators can easily be collected via the survey method of either type. This is not only because of the difficulties over response rates, etc., but also because the measurement of some of the most crucial characteristics of modern society is not amenable to the survey method.

For example, during the 1980s in the UK, an indicator which was used as a highly political football, was the numbers of deaths from hypothermia during the winter months; yet dead people tell no tales, even to a survey methodologist. Other difficult indicators that have been attempted have been the number of individuals in precarious (or without) housing; a further type of data which may be seen as an important monitor of the extent to which basic needs are satisfied is net nutritional intake. Again, data on activities or counts of events over a period raise problems of recall that can only potentially be solved via an expensive longitudinal panel design.

3.3.4 Conclusion

There are a variety of approaches and theories of what constitutes well-being, and while it might be difficult to claim that one perspective is superior to another, it has to be recognised that different perspectives generate different sets of priorities.

At the same time, what appear to be radically different viewpoints, tend to converge on a similar list of main constituents, while of course varying in the way these are organised, the emphasis, weight or rank given to each, and so on.

Table 10: Possible Framework of Social Concerns

Proposed Social Concerns	Content (for illustration)
HEALTH	Length & Health-Related Quality of Life Children's Future Development
LEARNING	Experience of School Levels of Ignorance
HUMAN ACTIVITIES	Use of Time Quality of Activities
WORKING CONDITIONS	Security Quality of Working Life
NECESSITIES	Basic Needs Fulfilment Poverty Lines
PHYSICAL ENVIRONMENT	Overconsumption of Energy Pollution
	The Family Victimisation The Wider Community
RELATING/SOCIAL ENVIRONMENT	The Family Victimisation The Wider Community
PEOPLE & THE LAW	Restrictions on Movement Interference with Liberty

The OECD programme was intended to provide a 'system' of indicators to be used in the reporting of social issues and the formulation of social policy. In practice, however, the goal areas they proposed such as 'health', 'individual development through learning', 'employment and quality of working life' etc. have been treated as being in almost one-to-one correspondence with the 'appropriate' ministries of health, education, employment etc. Moreover, the enumeration of the indicators themselves does not force the necessary collaboration between different administrative units.

While precise measurement of these various aspects would require new data systems, a major focus should be on improving the comparability and coverage of data collection systems that are already in place. Within such a broad framework, however, it may well be sensible, in any one period, to choose a few key indicators that could be the focus of efforts to improve data collection systems. Table 10 details some of the common social concerns that could provide a possible framework for the creation of social indicators. The next section reviews some of the more recent composite indicators, while in the following two sections, the problems of defining indicators of impact on education on other sectors are considered.

3.4 COMPOSITE QUALITY OF LIFE INDICES

3.4.1 The Human Development Index (HDI)
3.4.2. Other Innovatory Composites
3.4.3 Conclusion

The purpose of composite quality of life indices sometimes is to provide a better measure of national or regional performance with regard to human welfare and development - what one might call real poverty reduction. They are usually seen, however, as a complement to, rather than as a substitute for GNP, and are constructed from among components of a broader system.

In fact, there have been a whole series of attempts to construct such composites both in developing countries and in industrialised societies. The earlier PQLI has been discussed above. The review here will only consider a selection, focusing in particular on the potential of adopting or modifying the Human Development Index (HDI) and on the problems of synthetic indices.

3.4.1 The Human Development Index (HDI)

The HDI was proposed by the United Nations Development Program (UNDP). It was developed as a composite of longevity (measured by life expectancy); literacy (usually measured in terms of a minimum number of years of schooling); and command over resources for a decent living (measured by GNP per capita). The index is based on the average relative deprivation of each country on each of these three dimensions. The similarity in content and method with the PQLI is striking.

There are obvious problems with the HDI that have been reviewed by several commentators (e.g. Lind (1992); Murray (1991)). Following the OECD approach to the analysis of social well-being, these can be grouped as:

· the choice of components and the relative weights to be attached to them;

· the analysis of how those components can best be measured; and

· whether or not the desired series can actually be measured and, if not, what new data procedures are required.

Few would dispute that the three components chosen - measures of mortality, of education, and of economic activity - are three of the most important components of the quality of life; but the choice is ultimately arbitrary. Equally, even though it may seem plausible, the choice of weights is arbitrary. These and other technical' problems are common to all composite indices (see Annex 3B).

Measuring the Components

Life Expectancy (at Birth)

As the measure of mortality, UNDP chose life expectancy which, in fact, summarises mortality experiences over the last 60 years. In a developing country context, indicators of child mortality tend to be more popular (e.g. UNICEF rank countries by the under-5 mortality rate in presenting data on countries in their annual publication, The State of the World's Children. This is because they are easier to measure in the absence of vital registration systems, and it is assumed (without much solid evidence) that all age-specific measures are highly correlated. Because of the attention being given to child survival, infant mortality is often a poor predictor of life expectancy in the developing country context (Murray 1988). The advantage of life expectancy is precisely that it does take account of mortality at all ages.

There are still problems of comparability because the implicit age-weighting differs according to the country's pattern of mortality - an economist might want to weight lives according to social output thereby priviledging certain age groups; an advocate of fundamental human rights would want equal weights; etc.. Whichever view is taken, the variability of weights in current indices seems inequitable (Murray 1991).

Lind further argues that this choice does not take into account disability which is an important consideration for industrialised societies (Lind, 1992). Indeed, in many developed countries, increasing attention is being paid to measures of Disability Free Life Expectancy rather than Life Expectancy (see ODA, 1996).

Education

The UNDP chose adult literacy because:

Literacy is a person's first step in learning and knowledge building, so literacy figures are essential for any measurement of human development (UNDP, 1990: 12).

Murray (1991) further argues that investments in primary (and secondary) schooling are much more important than the attainment of literacy. While this is questionable in less developed countries, this is obviously true in 'developed' industrial societies. However, Murray's suggestion of using the educational attainment of 15-29 year-olds in terms of years of schooling is a reversion to input indicators which, is what we are trying to get away from.

Economic Activity

This is measured by GDP up to a fixed maximum ($4861 in 1994). The presumption is that above a certain floor value, individuals (or households) are no longer in poverty. This ignores the very large literature on absolute vs relative poverty and the processes of exclusion and marginalisation developing in industrialised countries. A direct measure of poverty (however defined) would appear preferable.

Moreover, even for developing countries below the ceiling, and leaving aside the problem of collecting accurate population data, it is unlikely that GDP is the best macro-economic variable for comparisons across countries.

(a) The coverage of GDP is different in terms of:

· the definition of economic territory

· the recording of taxes and subsidies

· the comparability and reliability of estimates for dwelling services (mortgages and rents)

· the comparability of estimates of the borderline between intermediate and final consumptions²⁰

· the completeness and recording of the activity of financial institutions

· the quality of estimates for the transition from GDP to GNP.

( ) The extent and size of the informal sector (or parallel economy) is unknown. Both the incomes generated and the profits realised are not declared for tax purposes; however regardless of origin, the income is diffused throughout the whole economy, and this influences the level and distribution of total household disposable income.

3.4.2. Other Innovatory Composites

The use of composite indices is intended to summarise the overall impact of development changes on the quality of life for citizens of a developing country. This assumes that a statistical change can be interpreted as a real change; but there are real doubts over the validity of the data (Murray, ¹⁹91). Various other indices have been proposed as alternatives.

Johnston (I988) proposes a 'chain index' which takes its numeric values from observed year to year changes in the value of the indicators that are included in the composite index. He selects 21 statistical variables to represent prevailing conditions in nine major areas of social concern (with at least two indicators in each of the nine areas); calculates year to year percentage changes; introduces 'multipliers' to make observed variability in each of indicators roughly comparable and allowing for different weights for positive and negative movements); and adds the resulting index values to give a total score.

Lind (1993) proposes a composite index which is wider than the HDl. The HDI can record large improvements in socio-economic conditions but simply miss others. Clavijo (1992) provides an illustration for Colombia where, according to the HDI, vast improvements were recorded, ignoring the permanently high homicide rate. He goes on to propose a Right to Safety Index; and a rate of reduction of insufficiencies (RRI).

Humana (1986) has based an index on the UN Declaration of Human Rights (1948), attempting to follow the progress of these rights in 120 countries by applying various measures; and computing an overall score for the satisfaction of human rights.

Many authors have attempted to produce an index for the quality of city life: for example, the European Foundation for the Improvement of Living and Working Conditions have developed an index of the collapse of the urban environment (EFILWC, 1993).

3.4.3 Conclusion

The problems of agreeing on index components and then on their appropriate weights are extremely difficult even between a small elite group of consenting adults (for example, those sat round a central committee table). They are insurmountable if the intention is to reflect the complexity and diversity of social development, or if we are also seriously attempting to involve the community. For these reasons, a range of more pragmatic approaches are considered from 'top-down' to 'bottom-up'. In the next section, we describe the procedures suggested by Frances Stewart for developing a flexible and functioning top-down system. This is exemplified in the following two sections which assess the impact of education on health and employment. The final section considers the problems involved in designing 'participatory', 'bottom -up' scheme for monitoring and evaluation.

3.5 DEVELOPING A MODERN FRAMEWORK: MONITORING SOCIAL CONDITIONS TOP-DOWN?

3.5.1 A Framework?
3.5.2 Designing More Relevant Statistics?
3.5.3 An Excess of Estimates, a Data Drought

Few now disagree that the impact of economic restructuring on social programmes should be considered independently of the purely economic consequences of the economic restructuring efforts. Economic development should serve human and social goals. In a statistical context, the emphasis must shift from counting products and things, to counting people, their activities, and what happens to them.

The main reason for concentrating first on the direct measurement of well-being is simply that, in recent history at least, it has been neglected relative to the total statistical efforts of governments which instead concentrate on monitoring the activities and transactions of state agencies. This does not mean that measures of the activities or outputs of government policies are irrelevant in the context of social reports; the question is to place them appropriately within a framework organised around the intended impact of policies, that is, the well-being of the population affected by those policies.

3.5.1 A Framework?

The problem is that the policies adopted in one sector may have unintended consequences upon conditions or trends in another. Thus, policies within the education sector have an impact upon health, the physical and social environment and the distribution of power as well as upon educational achievement. Vice versa, policies adopted in the health sector - sectoral policies concerned with the physical and social environment, and the existing distribution of power between various groups in society - affect purely educational objectives. What kind of instruments can be developed which take these inter-relationships into account in the process of planning and in monitoring the implementation of programmes?

The basic problem posed by this representation is to provide a framework within which information about progress towards objectives can be related to data about inputs, process and outputs of social programmes. But, as emphasised in a previous section, there are no consensually agreed and substantiated causal models linking measures of corporate or government activity (whether of input, process or output) to states of, or changes in, social welfare. The inclusion of data on corporate or government activity in such a framework, therefore, needs to be presented as the context of, rather than the explanation for the state of, or changes in, the state of social welfare; variations in the former should not be taken as the explanation for variations in the latter. Baster (1985) reviews many of the proposals and also cautions against:

grandiose schemes, which attempt to capture socio-economic reality as an integrated totality, because they push more urgent and more useful exercises in statistical bridge-building between sectors out of the limelight.

Neither can the choice and definition of what to measure and how to report it be treated as a technical issue. For example, one could argue, on practical and technical grounds, that nutrition status measurements may well be the most reliable and cost-effective source of information about social and economic conditions in countries where the health and social and statistics systems are inadequate - and that comment refers equally to many developed countries with over-developed administrative statistical systems. But the promotion of nutritional status as a major indicator is also a political judgement: thus it may be much easier to achieve a consensus in favour of using nutritional rather than socio-economic status as a basis for comparing needs, regardless of the (undoubted) value of the latter indicator.

These political influences upon the collection and interpretation of statistics must not be ignored. A framework is not a neutral, technical device: as Seers (1975) argued:

The series a statistical office chooses to prepare and publish exercise a subtle but pervasive and lasting influence on political, social and economic development. This is why the apparently dull and minor subject of statistical policy is of crucial importance (Seers, 1975: 5).

3.5.2 Designing More Relevant Statistics?

There are virtually no statistics anywhere on most aspects of life that really matter - the average distance people have to carry water a food; the number without shoes; the extent of overcrowding; the prevalence of violence; how many are unable to multiply one number by anothendr, or summarise their own country's history... (Seers, 1983: 5-6)

Stewart (1987) suggested the following features of an ideal monitoring system:

· data to be available regularly and speedily;

· provision of indicators of the status of the population at large and of vulnerable groups, broken down by major socio-economic category and by region, to record how vulnerable groups are being affected by changing economic circumstances and by policies;

· provision of indicators of the immediate as well as the underlying causes of the changing status of the various groups;

· a well-functioning system for the timely reporting, aggregation, analysis and diffusion of the information collected (Stewart, 1987: 259).

Regularity and Speed

It has been recognised for many years now that the (official) statistics available to monitor human welfare status are badly deficient in most countries (OECD, 1970;

McGranahan et al, 1972; UNESCO, 1979; UNRISD, 1977). This is due as much to a bureaucratic concentration on the development and maintenance of existing systems of administrative statistics as to any (intellectual or political) reluctance to consider what kind of measures would be most appropriate for monitoring human welfare status. Whatever the principal cause, however, the deficiencies of the official data are not easy to remedy (see, for example, the discussion of employment statistics in the Technical Annex).

It is worth recalling Carlson's conclusion about population data after reviewing data from 52 selected developing countries.

"Given the limited potential of measuring infant mortality trends on an annual basis, world-wide progress in reducing infant mortality could meaningfully be reported only once every 3 or 4 years" (Carlson, 1985).

In other words, while rapid feedback is important, so is precision. If we demand that a statistical system be able to monitor sensitively the fine details of changes, we are likely to overlook the biases and distortions due to inadequate data collection procedures. There is, of course, a danger in over-emphasising the difficulties of data collection and our ignorance, but too many recent publications have erred in the opposite direction (e.g. de Kadt, 1989).

Status Indicators

There should be some overall sense of demographic trends. For example, it is estimated that almost three quarters of the increase in the population aged 65 and over by the year 2025 will occur in developing areas of the world. For example, in Nigeria, the population aged 65 and over is projected to increase from 1.3 million in 1950 to 16.0 million by 2025 (more than 12 fold), and whilst the absolute increase in the USA is much larger - from 18.5 million in 1950 to 67.3 million in 2025 (Kalache, 1986) - it is relatively much smaller (less than 4 fold).

Stewart's brief, of course, was to suggest status indicators for children. She proposed the following list:

· Prime indicators of health and nutritional status: infant mortality rates; child death rates; indicators of nutritional status for the under-5s*; low birth weights*; indicators of morbidity (disease prevalence and incidence).

· Indicators of educational status: literacy levels; primary school completion rates; drop out rates*; repetition rates.

· Additional indicators of child welfare: rates of child labour; number of street children*.

It is difficult to organise the collection of accurate data in many developing countries; and it is especially difficult to rely on the accurate collection of data for many key variables in the educational, health and nutritional field. Assuming, however, that these difficulties can be overcome, Stewart's proposed list is an interesting commentary on the claim that statistical series can be fitted into a causal framework relating adjustment policies to the impact upon children, for some of her suggested indicators might be more appropriate for adults. For example, low birth weight rates are mostly used as indicators of the health and nutritional status of women of child-bearing age (Sterky and Mellander, 1973; Tanner, 1982) rather than as indicators of the child's own status; and literacy levels are, surely, more appropriate as indicators of adult status rather than of child status. In certain circumstances, therefore, adopting too definitive a framework might not generate the appropriate policy recommendations.

Indicators which are specific to adults' well-being are:

Health and nutritional status

· height relative to average height of tallest 10% or of wealthiest group in society
· indicators of morbidity
· mothers giving birth to low birth-weight children
· adult mortality rates (adjusted for age and sex distribution)

· literacy levels
· primary school completion levels

· adequate resources
· numbers of dependent children

· crucial but only sensibly measurable at each local level. Indices have been developed for use in specific community projects (see for example. Section 2.3 above).

It should be emphasised that, from a basic needs perspective, the issue is not relative inequality as such, but the lower and unacceptable status of certain groups.

Causal Indicators, Process and Input Indicators

Stewart suggests choosing only a small subset of indicators, those which:

· are of the greatest importance - in magnitude in determining child status;
· may be amenable to special policies for the protection of vulnerable groups; and
· are affected by changes in the economic environment and macro adjustment policies.

For example, Stewart's choice of process indicators jumbles-up behavioural with environmental indicators. While the prevalence of breastfeeding and primary school enrolment rate can be treated as mainly behavioural (or at least informed by culture), the distribution of oral rehydration, immunisation level, availability of potable water and health services access rate are mostly independent of the individual. This does not make for clarity.

Her choice of input indicators is much more contentious. The focus on real incomes and real government expenditures would be almost irrelevant in some subsistence economies. Although, in earlier chapters it is acknowledged that real income data are often deficient (see Cornia, 1987: 26), Stewart nonetheless recommends using other, more accessible data on basic food prices, levels of money income or expenditure in money terms, and employment and unemployment (Stewart, 1987: 263).

More sensitive and useful indicators can be developed, but they depend on the particular context. While we might all agree on the kinds of things that ought to be measured, the precise definitions of indicators and the importance attached to specific data series will differ. For instance debates over the relative importance, for literacy, of the expansion of adult education or of primary schools, will be reflected in the kinds of indicator we generate.

At the same time, there are structural conditions that affect the possibility of making progress towards the attainment of basic needs. For example, there are changes in ecology and the environment, the relative accessibility of primary health care, the resource base for adult education. Some structural conditions depend upon national and international policies and provision. In addition, we must also not forget the actual and potential impact of war.

As with the status indicators, it is important to devise indicators of inequitable state inputs to services used by different groups such as the relative proportion of education expenditure on the different levels of education or the relative proportions of health care expenditure on primary health care or on the hospital system. Similarly, we would want to compare conditions between rural and urban areas and between men and women. As MacCormak (1988) comments:

'a dynamic historical perspective would add weight to any analysis, but how is one to bridge the gap between inadequate, male-based statistics and informants who are very vague about the past?'

A Well-Functioning System to Analyse and Give Feedback on the Data Collected

Stewart cites this as one of the major stumbling blocks to analysis; developing country statisticians could reasonably reply that much of their time is occupied in completing regular questionnaires from the international agencies.

Stewart is correct in pointing to the organisation of data collection and the publication of the results as one of the most difficult tasks (Stewart, 1987: 264). It was emphasised at the conference on Statistical Policy in Less-Developed Countries held at the Institute of Development Studies in 1979 (Dasgupta and Seers, 1979); it cannot be repeated too often.

There is, however, a question as to the appropriate level of analysis and discrimination. The preference here is for this structure to be as close as possible to the community providing the data. While the attempts by UNRISD to develop a local area monitoring system have not spread, the general approach is correct. Officials record information about and from villagers and then collate the data in a form for onward transfer; they should also be able to provide a minimum interpretation back to the community, on the basis of their report, without waiting for the Ministry. Lourié (1987) also comments that the studies and surveys which do exist of the learning process and of motivation need to be published and disseminated (Lourié, 1987). They also need to be synthesised.

There are, of course, many conditions for which a local response is not adequate. Whilst a national surveillance system such as that of Botswana or Indonesia is ideal, it cannot always be replicated, or, if it could in principle, there are frequently practical obstacles. In this kind of situation, it is worth investigating the possibility of obtaining an overall view by other methods.

3.5.3 An Excess of Estimates, a Data Drought

Finally, the demand for data sometimes seems never-ending. Where, one might reasonably ask, does it stop?

It has been recognised for some time, at least by decision-makers, that there is a problem of information overload which needs to be reduced through a focus on relevant items and supporting data. This was, precisely, one of the motivations for the social indicator movement which focused on the problem of selecting a small set of data series.

The fundamental point which should be self-evident is that the collection of data is not an end in itself: it ought to be collected to answer some questions (Carr-Hill, 1987). But there are considerable pressures to extend the nature and range of data collected, including the growth of research specialisations which have meant the generation of new sets of questions (however trivial). Second, the spreading regulatory functions of the (international) state in parallel with the spreading tentacles of multilateral corporations, implies an extension of administrative activity and therefore the generation of extensive data bases.

It is, of course, true that the request for further data or further research is often a means of delaying positive social action. At the same time, it is important to recognise that the possibilities of gigantism do not obviate the need to think. Doubtful evidence proliferates while crucial data are not collected.

No-one would dispute that social conditions in many developing countries - and especially in much of Africa - are appalling and that every effort, both national and international, should be made to improve them. But global forecasts of further doom and gloom from outside, based on macro-estimates of unknown reliability, are of little help in formulating appropriate policies. In the short- and probably medium-term, it would be better to devote some statistical resources to building up a picture based on reliable evidence from local studies, rather than to continue to proliferate international 'guesstimates'.

In the long term, the development of a comprehensive national statistical framework could, in principle, provide appropriate data. But disputes over what is an appropriate pattern of growth mean that for the foreseeable future, the elements of such a system would be imposed from the centre. In the short-term, local-level monitoring with community participation is a realistic and potentially more democratic approach, and that is the subject of the final section. Before that, we consider ways of assessing the impact of education upon health and upon employment in developing countries.

3.6 THE IMPACT OF EDUCATION UPON HEALTH

3.6.1 Introduction
3.6.2 Brief Review of Evidence of Possible Impacts
3.6.3 The Measurement Problem in Evaluating Projects
3.6.4 Conclusion

3.6.1 Introduction

The purpose of this section is not to provide an extensive overview of the relationships between education and health; and in particular the extent to which education affects health²¹. For that the reader is referred to (Caldwell, 1993). Instead, the intention is to suggest indicators for monitoring the impacts of education upon health. In Chapter One, however we have already discussed the difficulty of assessing the impact of, say, the delivery of educational services upon attainment. The problem is multiplied in attempting to assess the 'added value' of education in affecting aspects of health (because of all the other factors which affect health). While it may therefore be possible to suggest indicators of the state of health upon which there is a presumed impact (and those will be considered here), it is unlikely that we can, in general, identify that part of any change in the indicator which is attributable to the educational intervention.

Nevertheless, many of the arguments for extending universal primary education are in terms of the improved health of school 'graduates'. No one - at least we think not - is suggesting that increased education will per se improve the health of the individuals involved. Instead, the presumption is that there are direct and indirect effects upon the health of an individual: direct effects are effects upon fertility, infant mortality, parenthood, attitude to positive health, preventive health; indirect effects are effects through their employability and sustainability (see section 3.7 for effects of education on employment).

3.6.2 Brief Review of Evidence of Possible Impacts

Fertility

The mission of the United Nations Family Planning and Population Association (UNFPA) is to promote the possibilities of family planning. The bulk of their activities have been through educational programmes - whether or not carried out in school.

But - unless there are large-scale tracer studies over the reproductive careers of those exposed to such programmes - the evidence of impact has to rely on estimates of changes in the population fertility rates. These are not easy to interpret.

The World Fertility Surveys carried out by the International Statistical Institute (ISI) during the 1970s and 1980s showed large differences from indirect (census-based) estimates of either fertility or child mortality. In general the ISI estimates of mortality were substantially lower, with the most likely source of discrepancy being the misreporting of the age of mother at census.

Blacker (1987: 197) concluded:

there is still no method of estimating either fertility or mortality in developing countries lacking reliable registration which can be guaranteed to give accurate results.

The presumption of a linear relationship between female literacy and the infant mortality rate has been the basis for UN and World Bank projections of population for nearly 15 years now. We need hard longitudinal evidence to confirm this - and one possible route is through tracer studies that could be part of a monitoring system.

The evidence from micro studies is less clear. Oni (1988) reports on a household survey in llorin, Nigeria at the end of 1983. Out of a sample of 932 households in three residential strata, 913 married women were interviewed. He analyses for the effect of a wide range of socio-economic variables (woman's education, husband's education, type of union, religion, woman's occupation, parity, contraceptive use, area of residence, presence of indoor tap water, presence of refrigerator) on child mortality. Although all the inter-correlations are statistically significant, in a multivariate analysis, the only 'cultural' variable that is retained is husband's education (all the other cultural variables are dropped).

Parenthood

There is substantial evidence that parents (and especially mother's education) affects child development. For example, a longitudinal case-control study in Aberdeen compared 120 low birth weight babies with 120 normal weight babies matched on gender, parity and social class (Illsley and Mitchell, 1984), and showed that the most important variable discriminating stage of development (and school achievement) at age 10 was the interviewer's assessment at birth of the competence of the mother.

It is extremely difficult to see how this could be converted into indicators of the impact of education. The 'obvious' suggestion - that one could track the number of early pregnancies - will not do, as many of the early pregnancies are associated with girls who leave school because they are pregnant.

Positive Health

One interesting sideline is the likelihood that education might lead to the collective promotion of positive health. In developed countries, an example is the socially differentiated pattern of smoking over this century. During the 1920s and 1930s, when it was fashionable, middle classes smoked more than working classes. After the Second World War, working class groups began to catch up; and then when the link between smoking and lung cancer was suggested, doctors were the first group to give up smoking, followed by middle classes. In developing countries, one can cite the (loose) links between the literacy campaigns in Andhra Pradesh and the banning of alcohol sales (Crinnion, Shotton and Carr-Hill, 1999).

Whilst these might well be some of the most important effects, it is difficult to define appropriate indicators.

Prevention of Illness

The obvious argument is in terms of the impact of education upon health behaviours; an obvious associations is the relation between education and liability to HIV/AIDS and vice versa (Oulai and Carr-Hill, 1994).

Following the Knowledge, Attitudes, Behaviours and Practice (KABP) model of health education, for example, there have been surveys of people's knowledge, attitudes and behaviours, comparing between those who have been on a literacy programme with those who have not. Carr-Hill et al (1991) showed in Tanzania how those who had been enrolled in literacy classes gave more correct answers to knowledge questions and appeared to have more 'modern' attitudes and practices.

We need the following:

· routine ways of measuring the extent to which health practices have been absorbed in any particular programme, whether in or out of school; and

· ways of measuring impact among out-of-school youths and adults.

3.6.3 The Measurement Problem in Evaluating Projects

The above discussion has focused on the nature of the influences of education upon different aspects of health at an individual level. When we are working at a project level, the concern is usually with assessing the benefits of any project compared with the costs. It is not possible to assign financial values to improvements of health status. In some cases, the issue is to compare the impact upon, say, rates of infant mortality, of spending on education services with, say, spending on health services. Then one can compare the technical efficiency of the two possible projects.

This is only possible, however, when the benefits of the approach can be assessed one-dimensionally. You cannot compare projects aimed at reducing infant mortality with those aimed at reducing tuberculosis among adults.

A similar problem has arisen in developed countries, where the issue is to compare between two (or more) medical technologies and interventions. Traditionally, improvements were assessed in terms of the reduction in the death rate (or the increase in life expectancy or survivorship following an intervention). But many interventions are now aimed at improving the healthfulness of the remaining life years as well as - and, in some cases rather than - increasing life expectancy or the chances of survival. In that context, economists in the UK and the USA have been concerned to establish a 'measuring rod' for comparing the benefits of different interventions which combines quality and quantity (Weinstein and Stason, 1977; Williams, 1985). Based on questionnaires, values between 0 and 1 have been attached to a large number of descriptions of health statuses and then physicians have assigned different conditions to those descriptions so that there is now a 'quality' rating attached to patients' status before and after most common interventions. These ratings can be combined with the expected years of life to yield a Quality Adjusted Life Year (QALY); and then interventions compared in terms of their costs per QALY.

In developing countries, the World Bank have proposed a Disability Adjusted Life Year (DALY) where the indices are based on the views of 75 eminent international public health specialists. These ratings are the basis for the proposals they made in the 1993 World Development Report.

There are several drawbacks, many of which are summarised in ODA (1996) and are reviewed briefly in Annex 3C.

3.6.4 Conclusion

Measuring the impact of a set of services upon performance in a different domain is going to be difficult because of the (more directly relevant) service activities within the corresponding sector. While there is a large body of evidence that education does affect fertility, infant mortality, parenthood, attitudes to positive health and to prevention of illness, the problem of quantifying those impacts is made more complex because of the difficulty of defining health. The economist's proposed solution mystifies more than it illuminates.

3.7 THE IMPACT OF EDUCATION ON EMPLOYMENT IN DEVELOPING COUNTRIES

3.7.1 Current Practice
3.7.2 What Improvements Could be Made in Indicators?

The purpose of this section is to examine briefly, current indicators used to measure the impact of education in developing countries on employment and to suggest possible areas for improvements.

3.7.1 Current Practice

How is Employment/Unemployment Currently Measured?

The labour force framework supported by the ILO and the one generally accepted world-wide, gives rise to three main categories - unemployment, employment and economic inactivity. The first two are looked at, in some detail, in Annex 3C. Since the three concepts are mutually exclusive, all those who are neither employed in some sense nor unemployed are said to be economically inactive, and this is, therefore, the residual category. Developing countries use some variation of this framework, but definitions still vary widely between them. Consequently, employment and unemployment data rarely mean the same thing from country to country.

Impact of Education on Employment

Normally, for primary and early secondary education the rate of return approach is used to assess the impact of education on labour market outcomes. As the student ages and progresses through the education system, more indicators, and more complex systems, are used to match education to the labour market so as to prevent mismatch. Thus, while rate of return (ROR) analysis is commonly used for basic education, ROR and most of the other methods described below, are used for secondary, tertiary and vocational education.

The types of labour market data that would be required by these approaches are: wages, household income, labour force status, unemployment, employment status (self-employed, employee, informal sector worker), occupational status, educational mismatch between supply and demand of qualified labour.

Rate of Return Approach (ROR)

The ROR approach calculates the net returns on educational expenditure (ILO, 1984), measured as the increase in net income that an individual will be able to command throughout his/her life in relation to the income he/she would have received if he/she had not reached that educational level.

For each specific educational programme, the present value of the flow of future net income is calculated on the basis of the above definition. Those programmes which show positive returns should be promoted, while those showing zero or negative net present value should be reduced or possibly abandoned.

If the flow of net income is calculated as the difference between the income of the individuals after tax, and the costs to them include both the direct costs paid for the education and the indirect costs in terms of income not earned because of participation in educational programmes, for a given discount rate, this gives the private rate of return. If the income is calculated before payment of tax and the costs include all the resources utilised (by the individual and by the state) to implement the education programme, for a given discount rate, this gives the social rate of return.

According to Richards (1994), the "weapon" wheeled out to overcome the alleged negative effects of manpower forecasting on the allocation of educational resources was this "rate of return" approach. There are, however, at least five main objections to the approach. First, it neglects external effects, since the only gains quantified are those accruing to the individuals who had received the education in question. Second, the analysis cannot shed light on the extent to which households needed to be encouraged to undertake "human capital investments". Thus, for example, the persistence of primary school drop-outs co-existing with high private rates of return could be caused either by a family decision on the relative priorities of work or schooling, or by insufficient government resources to primary education. Third, the basic assumptions - that observed wages reflect the marginal product of labour, and that the content of the marginal years of schooling an individual undertakes is responsible for the marginal increase in income - are questionable. Patronage often accounts for high incomes in many developing countries, and even though the level of education might be correlated with this -rich people will send their children to schools whatever the quality of the education or of the pupil - the incomes do not reflect the education received. Fourth, it assumes that total employment remains constant. Dougherty (1985) argues that most rate-of-return studies of manpower-development programmes implicitly assume that the old post of a trained individual is not filled by an unemployed worker and that the trained individual does not displace any other worker. Hence it is implicitly assumed that total employment remains constant. Fifth, it gives no guide to the quality of education currently being given. One would have to wait at least a decade to see whether the quality of the education delivered was reflected in the wages given, which is hardly a basis for improving the quality of education today.

Several authors critical of ROR analysis, nonetheless see it as a useful technique with limitations (Bennell, 1996; Lauglo, 1996). Lauglo (1996) writes, "To give guidance for present decisions, one needs what is never available: information on future earnings associated with different types of education. Data from the past are the best we can do, and reliable estimates of lifetime income streams are only available for those educated many years ago. The problem is that labour markets and the supply of educated persons to those markets can change so as to make past income streams poor predictors of future ones."

Manpower Requirements Approach (MRA)

The Manpower Requirements Approach is another way of tailoring education to employment needs. It first came to widespread prominence in the OECD's Mediterranean Regional Project (MRP) in the early 1960s. The three major steps in manpower forecasting are: (a) projecting the demand for educated manpower; (b) projecting the supply of educated manpower; and (c) balancing supply and demand. It has fallen into disrepute because of the plethora of assumptions made and its inflexibility due to technological change.

Labour Market Information Systems (LMIS)

The seeming failure of both the RoR and the MRA to assess mismatches on the labour market has led some authors to concentrate on the preparation and organisation of labour in Labour Market Information Systems (LMIS) as an "alternative" to forecasting. (See, for example, Mason, 1979 and International Labour Review, 1994.) As Richter (1989) notes "labour market information means nothing more nor less than what it says - information about labour markets". Indeed, at best they present a shopping list of items to be collected without providing an analytical framework within which to collect and then to analyse data for planning or policy formulation.

LMIS publications are potentially useful, however, in a developing country context, to delineate the main variables of interest for manpower planning and to arrive at consistent definitions. Unfortunately most of them do not do this. Another disadvantage of LMIS, as described in the recent publications by the ILO on the subject, is that they ignore, to a large extent, the demand side of the equation. This is because data on macro-economic planning is largely the preserve of non-labour-market specialists. Since demand projections for labour depend on the economic growth rate, however, these should hardly be ignored in LMIS.

The contradiction in LMIS has, belatedly, been realised by one of its leading proponents, Lothar Richter, who notes that "the volume of labour market information produced is likely to show an upward tendency; and...as a result manpower projections of the scenario-building type... are likely to be the main beneficiaries"(!) (Richter, 1989)

Cybernetic and Pragmatic Approaches

Manpower forecasting, as we have seen, has been largely concerned to date with supply side policies and, in particular, implications for education and training. This is because the outcomes from the two main approaches to manpower forecasting, the manpower requirements approach (MRA) and the rate of return method (ROR), both concentrate on education and training policies.

The disillusionment with the two analytical tools most widely used in manpower planning has led to the development of combinations of qualitative and quantitative methods. This has been advocated by Dougherty (1985). He argues for the systematic use of all available information as feed-back for planning. Such a system should be pragmatic and eclectic through using previously neglected or non-existent types of labour-market information: vacancy/unemployment ratios, trends in relative wages, the use of key informants, etc. Such a system should be monitored by manpower planners on a continuous basis. Although he gives the name "cybernetic" to such an approach, perhaps what Dougherty really means is an "heuristic" approach to employment and manpower planning, a system to enable planners and policymakers to find out what things are necessary for it to operate effectively, given the lack of precise definition.

This heuristic approach has a number of advantages over other approaches. First, it helps to organise existing data; second it focuses attention on the labour market and the need for new research in that area; third, it focuses attention on precisely those data required to understand the labour market; and last the experiments with the system are performed in terms of scenario analysis. Thus the system avoids point estimates through giving a range of estimates that depend on a number of supply and demand scenarios.

An approach that followed a heuristic approach is the MACBETH model (Hopkins et al 1985). The system is embedded in a user-friendly package and results are presented in graphical form allowing a dialogue to be maintained with even the most numerically illiterate policy maker. The system is heuristic because it produces results quickly, provokes discussion on the results emanating from the scenarios and leads the inquisitive into the search for new data sources, and better ways of understanding the labour market.

Further Controversies

Among several other controversies surrounding the different approaches to measuring education's impact on employment are two important ones not yet mentioned. First, the screening hypothesis, that education merely filters individuals to appropriate occupations on the basis of their ability, rather than the education received, undermines the use of all the approaches covered above. Secondly, the old chestnut, that schooling doesn't matter, that it is, rather, social class that determines future earnings, is given some substantiation, by various studies such as Fergany (1994). Fergany finds in Egypt, that social class is a bigger determinant of future earnings, when adjusted for on-the-job experience than education per se.

3.7.2 What Improvements Could be Made in Indicators?

In their review for the World Bank s Wapenhans report, Sigurdsson and Schweitzer (1995) examined the use of performance indicators in World Bank lending for education. Little was mentioned of the relation between education and the labour market or poverty. The report noted, however, that quality in vocational education and training (VET) was usually evaluated in terms of the employment and earnings potentials of graduates in the labour market.

What sorts of indicators could be used to measure the impact of education on the labour market and on poverty? There are two general sets of indicators: those that examine earnings and employment status today which are based on education imparted up to twenty years earlier and look at averages rather than specific individuals; and those that relate current employment or poverty status with actual education some years earlier as estimated by household surveys or tracer surveys. In each of the two cases the indicators required are likely to be similar: it is just that the methods of data collection and analysis are not entirely the same.

What improvements could be made to existing DFID practices? Judging from the Kenya study (see Chapter 2) where no direct account was taken of eventual labour market outcomes - admittedly this was for primary school projects where eventual poverty and labour market outcomes are far into the future - some improvements could be made. The following table suggests the types of indicators (i.e. general categories rather than specific indicators) that could be collected to measure different aspects of different levels of schooling.

Type of education	Suggested types of indicators
Primary education	earnings (wages, other earnings); rate of return (private and social); employment status (not in labour force unemployed, underemployed, employed); poverty status (less than food-based poverty ultra poor
Secondary education	same as primary+ mismatch between job obtained and job qualified for
VET	same as secondary plus responses from employers on appropriateness of training received; indicators on satisfaction gained from work
Tertiary education	same as VET

3.8 MONITORING BASIC NEEDS AT THE LOCAL LEVEL

3.8.1 Some Basic Principles of Indicator Construction for Democratic Monitoring
3.8.2 An Appropriate Local Information Monitoring System

The purpose of this section is to examine the possibilities of monitoring distance from, and progress towards meeting basic needs at the local level. A number of basic principles are first proposed: an ideal system can be specified, quite easily, on paper, but that specification raises a number of issues about aggregation and about the nature of the indicators required which are discussed. Reference should also be made to the various annexes.

3.8.1 Some Basic Principles of Indicator Construction for Democratic Monitoring

First, data are produced not collected: they depend on underlying concepts and on a system of processing in which different agents have different interests and tasks. Equally, the historical and social context of measurement is important: for example, the ready access of quantitative measures and techniques for aggregates of things has dominated the way in which statistical systems have developed. In sum, measurement work and statistical work are not socially or theoretically autonomous activities.

Consequently, the activity of measurement itself is a potential agent for change. Indeed, the potential of data measurement to influence policy often leads to its suppression, even when no-one disagrees about the concepts or definitions. We also have to recognise the political role of information. For example, educational and health professionals do not necessarily use data to present their case to the right people. This is graphically illustrated by Gordon (1979), who relates her experiences as Director of the Bakwu Applied Nutrition Programme in Ghana over a five-year period,

Second, everyone might agree that a particular phenomenon is worth measuring, but the actual indicator chosen would vary according to the clientele. Consider, for example, school attendance, which everyone wants to know something about. The government planner, typically, will be interested in enrolment, repeater and dropout ratios, pupil-teacher ratios, construction costs and so on; people would be more interested in access to different types of educational facilities, what they or their children can learn in different institutional contexts (it need not, of course, be a building, or even a formal programme); and concerned pedagogues in the type of resources that are needed to impart the type of knowledge which is socially useful.

Third, the same indicator can be used in various ways. Thus, an indicator of individual well-being may reflect a current condition, membership of a risk group, or a trend in the causative factors. Accordingly,

a change in the use of an indicator from, for example, the diagnosis and treatment of malnutrition in the individual, to the quantifying of risk for families or communities, or to the analysis of trends and changes, requires a change in definition and significance of that indicator. This dependence raises fundamental questions about the procedure for defining indicators, about who should be involved in the process, and about the role and objectives of research (Dowler et al, 1982: 101-102).

Finally, since social change can only be carried out by people, measures and statistical activities should be on the human level and, as far as possible, organised around their possibilities for change. In principle, this means that we have to understand how people develop their own goals in their social environment and how they develop their own measurement criteria. In practical terms, many authors have' remarked that the validity of data depends upon the extent to which the informant understands and agrees with the motivations and objectives of those collecting the data, and at least consents to the use to which the data will be put. Even this pragmatic approach imposes severe constraints on the viability of surveys which are centrally designed and executed.

A corollary is that measures and statistical procedures should be transparent. There is a need for a middle way between everyone having to become a statistician overnight, and the statistician's responsibility for explaining the assumptions as well as the implications which varied assumptions would have. The obvious example here, is the ease with which an economic statistician slips from talking about economic welfare to measuring GNP per capita, without explaining the limitations of using the latter as a proxy for the former.

3.8.2 An Appropriate Local Information Monitoring System

The basic proposal is not, in principle, complex, assuming agreement at a political level about what are minimum standards. It would include the following:

· Agreement at a political level about what are minimum standards and about statistical specification of these standards. The same people, of course, might be involved in both the political debate and the technical development.

· An accurate picture of the present situation. This is not easy, but the information needs to be presented in a form that is digestible but not pre-digested (already interpreted according to a particular schema). Unfortunately, present statistical skills are not oriented towards the task of presentation: it is not only a task of instigating numeracy among the non-numerate; it is equally important for the data manipulator to learn how to communicate.

· A documentation of appropriate resources. This is by far the most difficult because of the lack of substantiated theory as to what kind of resources (of land, labour and capital) are most effective with different social organisations in attaining which levels of welfare for individuals. We have a general idea as to which resources are necessary, but not those which are sufficient to attain desired ends.

The Nature of the Data

The most appropriate form of data collection will vary. Thus, finding out what are people's objectives is, at best, organised collectively. Relevant background data would be made available, and new data might be required; but the appropriate method cannot be decided in advance. Even in a less than ideal system, we would envisage that informal appraisal, inspection, or the interviewing of key persons would be more appropriate than an attitude survey.

There are two difficulties with the traditional survey approach: first, the answers to questions of the form 'which of the following do you think are the most important?' when asked by a project team with a specific interest, are almost certainly biased. Faniran (1986) reported on a survey of rural households perceptions of water quality in three rural communities around Ibadan, by a water project team, that people gave highest priority to the provision of potable water among their preferred amenities, and 80% gave water as first and second choice. Another project team working in a different sector, would have obtained a very different answer.

Second, the answers to general questions about objectives and political strategies for attaining them are open to a wide variety of interpretations. Consider, for example, the assessment of whether or not communities really have control over the decisions which affect their daily life. Although their perceptions may not, of course, tally with those of the researcher, people usually do have quite clear perceptions about the extent to which they have control over their daily lives. But they are conditioned, and sometimes informed, by the external constraints upon them and, despite the sophistication of the various scales which have been developed by social psychologists purporting to measure autonomy, control and fatalism, these are related to the individual and not to their social context nor to the collectivity which is the issue here. Moreover, there is no obvious way in which communities can express their lack of control other than by affirming it - often in very banal ways (although a modest proposal is made in Annex 3B).

We can be more specific with the next two stages. An assessment of the present situation requires a household survey carefully designed at the local level. An informal appraisal could be wildly erroneous, and administrative data are likely to be systematically biased. Finally, an inventory of existing and potential resources would basically rely on collective discussion at a local level supplemented by household survey data. The only role for administrative records would be in providing background data on the existing situation and for an inventory of possibilities in respect of the more macro objectives, such as the equalisation of disparities between population groups.

Much of the data required by such a statistical system need not be very sophisticated. Far too much effort is often invested in both developing and developed countries to obtain precise numerical information about a state of affairs where nominal data would do. For example, it is interesting to know how well someone can cook, but what is important is that we will not be poisoned by what we eat. Similarly, the well-known tendency for the qualifications required for a given occupation to be upgraded over time (Dore, 1976) is often accompanied by an increasing degree of differentiation in the level of qualifications obtained and therefore in the complexity of administrative records. Yet, all we really want to know is whether the person can do the job or not.

The problem of aggregation

When such a system is spelt out in detail, it will appear sensible to aggregate and compare according to some common denominator (whether in terms of resources such as labour or capital, or at an intermediate level in terms of particular levels of goods and services). The mistake which has been made all too often is to aggregate too far, too fast. For, although the physical quantities (whether of land, labour and capital, or of goods and services) can be converted into a common monetary or other unit and then aggregated, this conversion assumes a whole structure of production, distribution and exchange. It is therefore completely illegitimate to suppose that this aggregation has any meaning outside that particular structure. A more homely (North) example might help here. With fiscal harmonisation of the EEC, the problem of harmonising government subsidies to physical and social infrastructure has arisen. In turn, this requires ways of assessing the quality of services that are provided. How do you trade off, for example, the exceptional convenience of the Paris Metro as compared to the London Underground against the (relative) lack of violence on the latter? The answer is that those kinds of issues have to be decided not by the parachuted-in-social scientist, but as part of a democratic political process.

Obviously, it is sensible to bring together and compare findings from surveys in different areas (Edingbola et al, 1986), but it does not follow that the results should be aggregated to present an overall picture. For the particular mode of aggregation - including simple addition - will benefit some groups and disadvantage others. Compromises are made in the process of decision making, depending on the relative power of different groups. For a post-hoc analyst, these compromises imply a mode of aggregation and a set of weights; but they cannot be taken as the appropriate basis for future decision-making. The pressure to substitute a technical form of appraisal (e.g. cost-benefit analysis) for the essentially political process of decision making must be resisted.

Utopian - or An Essential First Step?

Obviously, one could criticise these suggestions as being impossibilist and Utopian. But the experience with the failures of development planning where mistakes are made simply because we do not know very basic facts about the target populations - or because what we do think we know is very partial - suggests that ensuring that there is a firm base for knowledge is vital not a luxury. When this is allied to substantive democratisation and community based participation, then there have to be some moves in the direction suggested.

Of course, there have to be caveats about the tendency for local elites to control and thereby generate more inequity; but that is true of all attempts at localisation. While we do not know the answers, we are at least aware of the right questions to ask: thus it is important to know who decides which kind of information is to be collected and for what purpose. Nevertheless, the argument here is that we cannot know the true extent and nature of poverty (or progress) unless those immediately concerned are also involved in measuring and monitoring their own status.

ANNEX 3 A: Opportunity Cost and Valuations

Opportunity Cost and Valuations

The purpose of this appendix is to review briefly the development of synthetic indicators based on the National Accounts and those developed by using valuation methods based on opportunity cost, whether measured in terms of monetary values, or in terms of time.

Extending GNP: Modifying the National Accounts

National Income can be regarded as a kind of index which (in principle) adds together all goods and services produced in an economy, using as weights either their market prices or the cost of inputs used to produce them (their factor cost). Although not originally intended as a measure of welfare, even economic welfare, it is often used in that way, in spite of many criticisms.

In attempting to avoid the arbitrary choice of weights associated with the HDI type of index (see Chapter Three, section 3.1.1.), one approach is to modify National Income so that it better reflects welfare. This can entail including non-marketed production and other "goods" such as leisure, making deductions for production which does not contribute to welfare or for social or environmental costs, reclassifying items among consumption and investment, or among intermediate and final production, and so on.

This approach requires that all the goods and bads introduced into the modified National Income be assigned a monetary value. Where, as in most cases, they are not traded in markets, some indirect valuation method has to be used. The development of cost-benefit analysis and, more recently, increasing concern with integrating environmental costs into policy making has led to the development of a number of valuation methods, mostly based on the "willingness to pay" principle.

Attempts at Adjusting National Income

The development of national accounting was marked by a number of debates, most notably between neo-classical economists interested in arriving at a measure of economic welfare and Keynesians concerned to create the tools required for demand management (Seers, 1975). The consensus which developed around Keynesian policies after the second world war meant that the Keynesian view was the one embodied in the national accounting system adopted by most governments. National Income was not intended as an index of welfare (or indeed as a measure of income).

Nordhaus and Tobin (1973) were the first to propose and estimate a modified version of National Income, intended more fully to reflect economic welfare. They proposed three kinds of modification. First, there was a reclassification of expenditures, with health care and education treated as investment in human capital, and certain expenditures, such as on the police and on defence, treated as "intermediate", that is, not in themselves generating welfare. Second, imputations were introduced for the services of capital goods such as owner-occupied dwellings and durable consumer goods, for leisure time, and for some forms of non-market production. Third, some costs of urbanisation were deducted.

While maintaining that GNP is deficient as a measure of welfare, Nordhaus and Tobin argued that it is sufficiently well correlated as to make separate measurements unnecessary. This view is disputed by Daly and Cobb (1990), however, who argue that shorter time periods than the 1929-65 period examined by Nordhaus and Tobin, reduce this correlation (Daly and Cobb, 1990: 76-80). Further, such a correlation is a likely consequence of the imputation methods used.

Other versions of measuring national welfare were implemented, such as the NNW for Japan (already discussed in Chapter 3, section 2.1). Another measure was constructed by Zolotas (1981), an Index of the Economic Aspects of Welfare, which took into account pollution costs and resource depletion.

The most recent attempt at a modified National Income is Daly and Cobb's Index of Sustainable Economic Welfare (Daly and Cobb, 1990). This is an attempt to draw on the best of previous attempts, while improving on their limited treatment of environmental issues and sustainability. The ISEW starts from personal consumption, which is weighted by an index of distributional inequality. Based on a search to quantify the annual welfare flow, and excluding the stock of capital welfare, a number of items are added: household labour, services derived from consumer durables and highways, public spending on health and education, and net capital growth. Other items subtracted include: spending on consumer durables, health and education, advertising, various urbanisation costs, and pollution and resource depletion costs.

The Proposed UN Satellite Accounts

Another approach to modifying the National Accounts was through the construction of satellite accounts linked to the main national income accounts, as discussed among international organisations for at least twenty years. This discussion has coalesced into proposals for a "Satellite System for Integrated Environmental and Economic Accounting" (SEEA), described by Bartelmus et al. (1991).

Satellite accounts represent a compromise: definitions, accounting identities, production boundaries, etc. are consistent with the core accounts, so that the two can be combined, or not, according to intended use.

The focus of the SEEA is narrower than that of the other welfare measures discussed above, since it is concerned only with environmental issues. If and when implemented, however, it will provide a measure of environmentally adjusted National Income that, in principle, is a better index of welfare than national income itself. The development of other satellite accounts (incorporating for example other social costs of production), while at least as difficult, would make it possible to arrive at a better index.

Evaluation of Using the National Accounts

There are a number of problems faced by attempts to base a welfare index on modifying the national accounts, some common to welfare indices in general (which are considered below), but most of them to do with the difficulties of monetary valuation. We attempt here a pragmatic evaluation.

Familiarity with aggregates such as GNP, and with its use as an (imperfect) welfare indicator, obscures the full extent of its shortcomings. Hueting (1991), for example, discusses some fifteen objections to National Income as a measure of welfare. Many of these objections have been part of the debate about GNP since the 1960s and are addressed, at least to some extent, by the indices discussed in Chapter Three, section 2. They include expenditures currently classified as final but which should be treated as intermediate, a major part of government expenditure, as well as all the aspects of welfare which are not measured by human production of goods and services (environmental goods, inequality, leisure, working conditions, security, etc.)

But some of the objections (although described by Hueting as being of a "technical" nature) are in a sense more fundamental, because they undermine the basis for using market prices at all: national income omits the consumer surplus, although this is part of welfare; it involves adding different people's utilities, which is unjustified particularly where there is inequality of income; it ignores diminishing marginal utility, of individuals and of the economy as a whole; real national income requires the use of price indices which can only be calculated correctly for a constant basket of goods; and the movement of activities from the unpaid to the paid sector is ignored.

Other problems arise in relation to the policy relevance (or otherwise) of the national accounts. Seers (1976) emphasises, for instance, the "monistic" character of the accounts, with their treatment of the whole nation as an appropriate object of analysis and policy. In a period characterised by halting GNP growth, combined with increased inequality and poverty, restructuring of production, globalisation, and environmental problems which cut across national boundaries, this national emphasis is unjustified. It is more important to consider how the accounts should be extended to monitor developments in the international economy on the one hand, and with breakdowns by income classes and by production sectors on the other. This applies to the current national accounts, and equally to attempts to create welfare indices out of them.

Imputing Monetary Values

There is now a large literature (barely touched on here) concerned with imputing monetary values to goods and bads for which there is no market and thus no market price, with particular emphasis on environmental costs and benefits. The methods proposed are mostly based on the "willingness to pay" (WTP) principle (or its twin, "willingness to accept compensation"), which rests on the assumption of "rational" economic people, spending their incomes in such a way as to maximise their welfare. In the case of marketed goods, the observed willingness to pay 1 Euro for a good is taken as evidence that the good procures the buyer at least 1 Euro's worth of welfare. Summing over goods and individuals then gives a measure of welfare, and this is essentially the rationale for treating National Income (Nl) as such a measure (although it ignores several problems discussed above, and others).

Deriving Estimates for Willing ness to Pay (WTP)

Where goods are not marketed, WTP is not directly observable and can only be estimated indirectly. One popular approach to environmental valuation, for instance, is to design an econometric model which incorporates all the factors affecting price, of, say, housing, and an estimate is obtained of the variation in price - and hence WTP - which results from differences in the amount of an environmental cost or benefit. Particular problems arise from the need to include all factors affecting price (hence there are huge data requirements), and from the fact that in practice environmental factors are highly correlated. At best the method can be used to estimate WTP for some aspects of some environmental goods.

A more experimental approach, contingent valuation, consists in simply asking a sample of people about their WTP, either directly or through a more elaborate game playing strategy. This method is cheap and generally applicable; but there is a major problem of realism, and substantial observed differences between willingness to pay and to accept compensation. In the latter case people may give infinite values.

There are many other problems with WTP indices, not least of which is their close relation in practice to ability to pay. Thus, utilising an index based on WTP as a basis for policy targeting is likely effectively to assume that poorer countries, regions and people are "underpolluted" relative to richer ones.

To arrive at an index of welfare, WTP methods have to be applied not just to environmental factors, but also to social and economic factors operating outside the market. This raises a whole host of problems of definition and interpretation.

Problems of Definition and Interpretation

Before valuation methods can be applied to the various costs and benefits relevant to welfare but presently omitted from national income, these have to be agreed on. There are problems of definition, of what to include, and of uncertainty. As an example, "defensive expenditures" - spending designed to offset environmental or social degradation - is currently included in national income, but should be deducted. Here valuation is not a problem, but agreement on which expenditures lead to a genuine increase in welfare and which expenditures offset declining welfare is difficult to reach.

Uncertainty raises further difficulties, particularly in relation to environmental costs. The likely future costs of global warming, for example, are widely thought to be very high, but there is great uncertainty about how high, what they will be, and how they will be distributed (e.g. among nations). Simply omitting costs we are uncertain about may lead to serious overestimation of welfare. Introducing these costs as knowledge of them improves will create problems of stability and comparability.

Valuation Methods Based on Time

The same kinds of methods are being applied to combine other sets of indices, one example being the development of time budgets. Thus, modifications of National Income focus on valuations of economic activity. Another approach, taking the same 'utilitarian' framework, starts out with the 24 hours in a day as the fundamental unit.

Much of society and social theory defines people in terms of what they do as paid employment, and many social scientists and particularly psychologists regard paid employment as a key determinant of individuals' well-being - or lack of it. To an extent this is understandable since what a person does to earn a living is often the second largest single occupier of their time every week (sleep being the largest).

The reality of work is, of course, far more complicated: many jobs require a variety of different manual or intellectual skills; some are physically exhausting but require little or no intellectual or emotional input; others have no apparent worth for the individual worker beyond a wage-packet at the end of the week. Since it is also true that about half of the population does not do paid work in a formal sense, it is clear that to obtain a full picture of human activities and well-being in our society we must broaden our perspective.

In recent decades, and particularly in the light of the work of the feminist movement there has been a marked increase in the recognition of the importance of housework and childcare as significant occupiers of time for many people. Perspectives, and indeed practices, have significantly changed in the last three or four decades, particularly with regard to the division of unpaid labour between men and women, the status of childcare in terms of work and leisure, and the benefits and drawbacks of technological innovation with regard to unpaid work.

Aside from these compulsory occupiers of time (i.e. 'work' in its broadest sense) there are also changes occurring in 'free' or 'leisure' time. Social commentators have begun to take an increasing interest in 'leisure' whether as a source of emancipation and self-directed activity or of passive boredom.

Time Trade-Off Methods

Richard Stone developed a System of Social and Demographic Statistics based on the length of time spent in any one status; that approach was discussed briefly in Chapter 3, section 2.1. There have been some recent attempts along similar lines discussed below.

Quality Adjusted Life Years

Disability Adjusted Life Years (DALYs), and their functional equivalents such as Healthy Life Years (HLYs), for example, are calculated by subtracting from life expectancy the average length of time spent incapacitated (or seriously ill). However, as in practice in Japan and the United States, they involved only marginal adjustments to life expectancy itself, little further effort was made to develop them.

Using Time as the Numeraire for Modifying GNP

A totally different approach has been suggested by Lind (1993) as an alternative formulation to the Human Development Index. The argument is that instead of a simple average of relative deprivation, the best way of combining life expectancy and GDP is with the compound:

L= bw* e^1-W

in which 'b' is a measure of income, 'e' is life expectancy, and 'w' is the proportion of time spent in employment.

However, this formulation makes the implicit presumption that the quality of non-work time is independent of income - which is a little doubtful.

Collecting Time Budget Data

Time budgets consist in recording at fixed time intervals what activities a sample of individuals are engaged in, so that the proportion of people engaged in different activities at any one time, or the length of time spent on different activities, can be estimated. Since all activities take place over time, time budgets provide a framework for considering activities systematically, although of course the welfare associated with activities cannot be reduced to the time (any more than the money) spent on them. Time budgets allow us to assess the importance of a broad range of human activities: anything that occupies a significant amount of one's time is likely to be of personal psychological and social importance.

Time budgets also provide an opportunity to get away from the emphasis on employment time, which is a feature of most existing statistics, as well as the dichotomy between "work" and "leisure", which increasingly fails to capture the reality of how most people spend their time.

The available data, however, draw a clear distinction between paid employment and other activities. Thus while information about the conditions of paid employment has been of increasing interest to government for a century, statistics about other activities are comparatively scattered and few.

An Index of Economic Welfare

Three approaches have been reviewed: extensions to the national accounts; imputing monetary valuations; and using time budgets.

It should be emphasised that, although there has been considerable intergovernmental discussion, the SEEA (Satellite System for Integrated Environmental and Economic Accounting) is only a proposal: the other measures are accompanied by estimates, however heroic the assumptions on which they are based. While many of the data required by SEEA have been the subject of research studies, no country comes near to having the range of estimates required. Bartelmus et al illustrate the proposals with data from "a realistic, but fictitious country" (1991: 140).

Moreover, as Daly (1989) emphasises, the purpose of the reform of the SNA to include satellite accounts is in order to create a better measure of income, making it clear that much else needs to be done to arrive at a measure of welfare,

In practice, even the best attempts so far at indices of welfare have been forced into arbitrary assumptions. For example Zolotas (1981) assumes that exactly half advertising is of a "persuasive" kind (and thus should be excluded). Daly and Cobb assume that half higher education spending is consumption and half investment. There are many other examples.

Attempts at monetary valuation of non-marketed goods have to make a lot of assumptions about how people actually reach decisions which are not borne out by empirical data. Reaching valuations usually involves ignoring 'irrational' responses, and/or averaging across a very wide range.

On the other hand, as we have seen, there is often a high degree of correlation between the conventional aggregates (Nl, GNP) and the modified versions. This results partly from their common core (most marketed production). In addition, large items such as the value of housework or of leisure are estimated essentially by multiplying the time involved by the average wage rate or something closely correlated with it. The latter is likely to change much more quickly than the former, and is of course highly correlated with Nl. Yet the pervasive use of money and prices as a method of valuing welfare may becoming less and less relevant.

Using time budgets is, of course, completely different; but it also relies on a single method of valuation. It is this presumption - that there is a single method of valuation that can be applied to the whole range of human welfare - which is most questionable, and was one of the main reasons for developing social indicator systems in the first place. Finally, none of the proposals tackle any of the problems of monetary valuation or of relying on a single index discussed in Chapter Three.

ANNEX 3 B: Technical Problems in Developing Indicators

Technical Problems in Developing Indicators

"The series a statistical office chooses to prepare and publish exercise a subtle and pervasive influence on political, social and economic development. That is why the apparently dull and minor subject of statistical policy is of crucial importance' (Seers, 1975: 3)

Although the title of the appendix is 'technical' problems, the concern here is to identify the policy implications of some apparently technical choices. In particular, we should repeat that the concern is with measuring outcomes rather than inputs or activities. It is therefore, absolutely crucial to establish what we are trying to measure before elaborating statistical procedures for collecting data for indicators.

Three issues will be dealt with: first the pitfalls involved in deriving indicators: second, the difficulties of constructing composite indices; and third, the special difficulties of monitoring the disadvantaged.

Deriving Indicators

There are several technical problems. We consider just four: objective and subjective data; issues of comparability; reliance on existing data; formulating indicators.

There is a general failure to incorporate both 'objective' and 'subjective' components into indices; or to know how to combine them. This is partly because of an over-rigid division between 'qualitative' and 'quantitative' methods of collecting data; and partly a failure to recognise that much of human behaviour is governed mainly by how a situation is perceived, not by objective circumstances (e.g. Keynes' analysis of the behaviour of markets; Thomas' analysis of the behaviour of Polish peasants).

For example, a concern with 'exclusion' - although resulting from objective circumstances - is, at least in part, a subjective matter in that the criteria which identify the excluded are socially determined. There is a need for both quantitative and qualitative data, and they must be open to revision and disaggregation.

The usual difficulty with subjective data is that respondents are being asked their opinions about questions framed by the policy-maker/researcher. While there are other possible approaches, they are time-consuming.

Comparability

The attempt to derive a set of social indicators or a societal index which is relevant across all developing countries raises issues of comparability and the possibility of comparative research. Living conditions can only be monitored and compared if the macro frameworks are similar, including 'the nature of economic and social restructuring, the role of the state at a central and local level, the relationship of bureaucratic to market allocation mechanisms, the ideologies which underpin the social institutions of family, religion, education and so on.' (Harloe and Martens, 1984). They go on to argue that, at the micro level, the situation within each country or region 'can only be understood within the context of the particular legal and financial arrangements, financial structures, cultural and religious traditions and geographical settings'. While, as a consequence, 'each observation is unique and generalisation impossible', at the higher level, 'common cross national trends may be identified even if they manifest themselves in very different forms in different countries'.

This has consequences for the kinds of social indicators systems that should be developed. For example, a cross-national concern with the 'informal sector', unless specified more precisely - may be translated differently in different contexts.

Reliance on Existing Data

The content of most quality of life measures is dictated by data and measures that are readily available rather than the demands of prior theory. Etzioni and Lehman (1967), at the beginning of the 'social indicators movement', pointed to the problems this can cause, for example:

· 'fractional measurement', where there is a lack of correspondence between a concept and its operational definition (as with 'unemployment' and its measurement using social security statistics);

· 'indirect measurement' (e.g. measuring educational attainment by years of school), especially prevalent where data is used which was collected for some other purpose.

Formulating Indicators

In many or most cases the main concern is not with averages but with the proportion of people who achieve an acceptable minimum standard - that is, with avoiding poverty and marginalisation. This concern moreover is not simply with ensuring a basic income for everyone; it cuts across all the other concerns and includes, for example, basic literacy, avoiding early death, and the availability of transport providing access to basic facilities.

Agreeing on minima within any one cultural group is usually possible but reaching agreement on such minima across groups or societies - other than for the prerequisites for survival - appears to be very difficult; whilst attempts to define a level relative to each society (e.g. the bottom 10% of that group's or that society's income distribution as poor) are rather vacuous. Minima have to be context specific.

It is also important to establish an "optimal need satisfaction", beyond which level, more is not necessarily better (eg. food, housing, etc.). Indeed more can even mean worse not only in physical terms (eg. vitamins A and D in excess) but also in terms of excessive security and overpowering relationships. This means that in many cases, neither a set of 'positive' indicators ('the proportion who are literate') nor a set of purely 'negative' indicators ('the proportion without adequate housing') will be sufficient: instead, it implies relatively sensitive measurement of the proportions who are substantially below and substantially above that group's critical optimum.

It is also important to point out that in many or most cases the main concern is not with averages but with the proportion of people who achieve an acceptable minimum standard - that is, with avoiding poverty and marginalisation. This concern moreover is not simply with ensuring a basic income for everyone; it cuts across all the other concerns and includes, for example, basic literacy, avoiding early death, and the availability of transport providing access to basic facilities.

Difficulties with Composite Indices

There are many difficulties with devising an overall quality of life index. In fact, there are two distinct sets of problems: establishing a coherent set of component indicators; and interpreting combinations.

Choice of Components

There is no consensus over what should be the components or the weighting procedures that should be employed in 'composite' quality of life indices:

· whilst everyone needs/wants a certain minima of several conditions, few can agree on what is the optimum level or what combination is required;

· whilst nodding in the direction of consumer sovereignty as the mechanism for choosing components and how to combine them, few have actually attempted to take that position seriously.

Trade-Offs are Obscured

Although there is a very close correlation between life expectancy and per capita income (also of PQLI and per capita income), the relationship of economic growth and development is not so simple e.g. isolation of elderly and certain forms of child abuse prevail more in high income nations.

For example, Etzioni and Lehman (1967) argued against 'formalistic-aggregative measurement of collective attributes', as with the US crime index, which simply adds up a broad range of crimes, giving the same weight to a murder and a $50 theft. Seers (1985) goes so far as to suggest that synthetic indexes are often constructed without understanding the system that generates them.

The emphasis is on individuals, looking at extremes rather than averages, and not lumping them all in together for a set of bland results.

The point is that not only is well-being multi-dimensional, its aspects are incommensurable in that although they are inter-related, they are not substitutable for each other. For example, a sufficient income to ensure good nutrition increases life expectancy, but you cannot compensate early deaths with high income. Indeed, although an index, through continued use, can be presented as being simple -as has, one might argue, GNP itself - the underlying presumptions are often quite complex and obscured.

Lack of Disaggregation

Few quality of life indices address distributional aspects of the different components of the 'quality of life' or 'well-being' of particular population groups.

This is principally because of the difficulty of collecting sufficient nationally comparable data to yield meaningful estimates at the community level or for small groups; such indexes can usually only be calculated for highly aggregated and often inappropriate geographic units of analysis.²²

Monitoring the Marginalised

There are many practical difficulties with assembling data whether we are talking about administrative systems or about social surveys. But there is a general difficulty which should be mentioned, given:

· increasing - or increasingly recognised (because of the 'new social movements') - diversity within the population;

· the purpose of many of these systems is to monitor the living standards/quality of life of those at the bottom of the heap or who are marginalised; and

· our overall concern with empowerment - or providing information important and useful to the concerned citizen.

More Sensitive Social Surveys

The problems of 'monitoring the disadvantage' is only an acute form of the general problem of monitoring the quality of life where there is increasing attention being paid to user orientations.

One possible proxy approach, in general, increasingly used in UK and USA is that adopted in the survey conducted for the ITV in 1989, published as Poor Britain. Respondents were asked to select from among a long list those items they thought were essential/necessary for civilised living in today's Britain. They were then asked whether they themselves had access to those items. This approach has been adopted in developing part of the study design and questionnaires to evaluate the impact of devaluation of the CFA upon poverty in the Ivory Coast (Carr-Hill, 1995).

An extension of that approach - which has been tried within the health care context · is to ask people to name which are the most important sub-components of any area of concern; and then to ask them how they rate their own situation or status in respect of each of these components (Ruta et al, 1994). Whilst this methodology is still in its infancy and, as currently designed, tends to favour the articulate middle class respondent who is used to reflecting upon and rating options, it might be worth exploring.

Whichever avenue is adopted, the importance of focusing on user orientations and views about what constitutes the quality of life should not be ignored.

A Local Approach

A local approach to developing quality of life indices should include:

(1) observations on the locality ranging from total area and population, to number of post and telegraph offices to number of tea shops, to prices of goods and clothing;

(2) listing of all households in the locality (c. 2000) with standard socio-demographic information; and

(3) an omnibus survey of a sample of 100 households in each locality (consumption, education, health, etc.).

ANNEX 3 C: VALUING HEALTH AND MEASURING (UN) EMPLOYMENT

VALUING HEALTH AND MEASURING (UN) EMPLOYMENT

Valuing Health

Theoretical

People do not agree on the definition of health. Even in the basic questionnaires, doctors and patients have different views (Kind and Rosser, 1977). The basic problem is that people have different definitions of health. Blaxter (1981) has identified at least five dimensions. Even if one agrees that, in principle, this is a worthwhile exercise, in practise this makes valuation very difficult.

Should we value patients or groups of patients equally?

In fact, both the QALY and the DALY give decreasing weights monotonically as people age (because they have lower life expectancies). However people are not so simple: there is survey evidence that the young and old alike place more weight on parents with young children and less on the old and very young (Wright, 1985)

Does the same value of the index mean the same thing to different people? People who suffer from illness generally adapt; and even if they do not adapt, their perceptions of health and illness may change. Some people may be more independent than others; etc.

Despite being sometimes thought of as a utility measure, the QALY or DALY is at best a pseudo-utility measure, (Culyer, 1990) because it does not include any consideration of non-health welfare.

Practical

The way in which the index is constructed means that quality scores tend to be compressed towards one; so that even though health deteriorates with age, older people when asked will still claim to be satisfied with their health (Wright 1985).

The approach uses a discount rate - the DALY was discounted at 3% - but there is little survey evidence to support this. High discount rates lead to a bias against educational or other interventions at young ages.

Who should decide on the quality scores? The QALY procedure is supposed to be 'democratic' because people are asked to rate health status: in fact they are entirely constrained by the design of the questionnaire instruments (Carr-Hill, 1992).

Planning Context

Given the move towards public participation, is it appropriate to introduce an index as a basis for decision-making where only a limited numbers of 'experts' are - or could be - conversant with the criticisms?²³

The costs per QALY or cost per DALY figures are based on average costs. Such figures may not be optimal; and may not be easily transferred from one context to another (let alone from one country to another). Marginal costs of an additional intervention may be different from average costs.

The approach may give you some idea of what to aim for - the best package of services - but no idea how to get there.

Employment/Unemployment Concepts

Unemployment

The conventional definition, and the one most widely used today, was agreed upon by labour statisticians at the 1982 ILO conference on the subject. Briefly, it sets three separate criteria for classification as unemployed. The unemployed must first be without work; second, currently available for work during the reference period; and third seeking work, that is, they must have taken specific steps in quest of a job during a specified recent period.

Thus, the unemployed comprise all persons above the age specified for measuring the economically active population who were: (a) "without work' (and not self-employed); (b) "currently available for work"; and (c) "seeking work".

Special provisions are made for persons without work who have made arrangements to start work at a date subsequent to the reference period and for persons whose employment contract is temporarily suspended.

The "without work" criterion draws attention between employment and non-employment. "Without work" should be interpreted as total lack of work, or, more precisely, as not having been employed during the reference period. The purpose of the "without work" criteria is to ensure that employment and unemployment are mutually exclusive. A person is classifiable as unemployed only if it has already been established that he or she is not employed. Thus persons who were engaged in some casual work while seeking employment should be classified as employed, in spite of the job search activity. The other two criteria of the standard definition of unemployment, "current availability for work" and "seeking work", serve to distinguish those of the non-employed population who are unemployed from those who are not economically active.

There are many difficulties and subtleties in the application of these seemingly simple rules. Perhaps the most contentious is what is meant by "without work". Internationally this is accepted to be a person in the reference period, which is usually one week or one day before the survey question is posed, who did not work for pay or in-kind earning for even one hour.

Recognizing that too exclusive a focus on a single measure may distort the view of other developed nations in comparison with that of the United States, the US Bureau of Labour Statistics (BLS) has published since 1976 seven alternative measures for unemployment. Applying these definitions to nine countries Sorrentino (1993) found that the unemployment rate varied in 1989 for the US, for example, from 1.2% to 7.9%.

In developing countries, especially the poorer ones such as those in SSA, it is rare to find an adequate labour survey that follows precisely ILO labour standards. Data are normally available from ten-yearly-conducted censuses since resources are usually not available to carry out household surveys to map labour market statistics in the intervening periods. Consequently, almost without exception, unemployment figures that appear from time to time in the least developed countries are estimates of a very poor nature. To a certain extent this is understandable since the main labour issue in poor countries is normally underemployment, not the type of unemployment that results from the lLO's rather strict definition of unemployment. Yet, even rich countries do not always observe ILO definitions playing, for instance, with eliminating older people less than 65 who may not have a chance of a job from the register of the unemployed. EUROSTAT produces comparable unemployment estimates through taking the employment surveys from each member state and then re-calculating unemployment rates using the strict ILO definition.

Employment

According to the lLO's 1982 international definition of employment agreed at the Thirteenth International Conference of Labour Statisticians (Thirteenth ICLS) (see Hussmanns et. al., 1990), the "employed" comprise all persons above the age specified for measuring the economically active population, who, during a specified, brief period (one week or one day) were in the categories: (a) paid employment: "at work" or "with a job but not at work"; (b) self-employment: "at work": persons who performed some work for profit or family gain, in cash or in kind; (b) "with an enterprise but not at work". The international standards further specify that, for operational purposes, the notion of "some work' may be interpreted as work for at least one hour\

Under-employment

According to the 1966 ICLS resolution, under-employment exists "when a person's employment is inadequate, in relation to specified norms of alternative employment, account being taken of his occupational skill (training and work experience)". (Report of ICLS, 1966) Two principal forms of under-employment are distinguished: visible under-employment, reflecting an insufficiency in the volume of employment; and invisible under-employment, characterised by low income, underutilisation of skill, low productivity and other factors. The 1982 ICLS resolution weakened the emphasis on under-employment by noting that "for operational reasons the statistical measurement of under-employment may be limited to visible under-employment".

This means that visible under-employment is defined as a subcategory of employment and that there are three criteria for identifying, among persons in employment, those who are invisibly employed: (1) working less than normal duration; (2) doing so on an involuntary basis; and (3) seeking or being available for additional work during the reference period.

The 1982 ICLS noted that compared to visible under-employment, which is a statistical concept directly measurable by labour force and other surveys, invisible under-employment is "primarily an analytical concept reflecting misallocation of labour resources or a fundamental imbalance as between labour and other factors of production". To measure invisible under-employment, whether in respect of income, levels of skill or productivity, it is necessary to establish thresholds below which the income is considered abnormally low, the skill underutilised, or the productivity insufficient. However, the concept has not so far been endorsed by the ICLS because of the difficulty in defining international standards

Informal Sector Employment

The concept of the informal sector has played a growing role over the past three decades, in particular, in developing countries for its alleged role in absorbing vast numbers of unskilled labour in a dualistic economy. It is surprising, therefore, to note that no clear and generally accepted definition of the concept exists. The ILO, one of the first to introduce the concept in its Kenya Report in the early 1970s (ILO, 1972), has begun to take the concept even more seriously²⁴ in recent years as it attempts to develop a universally accepted definition and its measurement. Deliberations have taken place under the form of the Meeting of Experts on Labour Statistics (MELS). They note that one may broadly characterise it as the aggregate of activities that result from the need for generating one's own employment to earn a living because other sectors of the economy - agriculture, large modern firms and the public service - are unable to provide a sufficient number of adequate employment and income opportunities for a rapidly growing labour force and there are no - or only rudimentary - social benefits from the state to fall back on. It is by no means a marginal phenomenon - rough estimates put it at 300 million in developing countries (ILO, 1991). In developed countries, the labour surplus is smaller and social protection systems exist and therefore the informal sector that does exist (eg small-scale units outside the formal economy or services rendered by one household to another) is relatively small. Additionally, many activities exist in the black or concealed economy. In Central and Eastern Europe and some of the Southern European countries, the black economy is widespread.

The informal sector in developing countries typically consists of very small-scale units established and owned by self-employed persons either alone or in partnership with others. They often have very little capital, equipment, technical know-how or managerial skills, and use simple, labour-intensive technology. As a result, most of these units work at low levels of organisation, productivity and income. They tend to have little or no access to organised markets, credit institutions, formal education and training, or public services and amenities. The vast majority of these activities are legal in themselves, e.g. handicrafts, vehicle repair, taxi driving, food selling, etc., contrasted with criminal activities or illegal production, e.g theft, extortion, smuggling, production and distribution of drugs, prostitution etc.

On the other hand, governments often tolerate the existence of informal sector activities performed outside or at the fringe of laws and regulations because they lack the means to cover the whole economy with legislation or because they realise that many informal sector activities will one day become legal. Indeed, among informal sector entrepreneurs there is a desire to legalise their operations whenever possible, since this would enable them to have access to some institutional support, such as credit, or to the protection of the law in such matters as enforcement of contracts (Tokman, 1991).

In a draft resolution produced in its January meeting the 1993 ICLS adopted most of the recommendations for a definition made by MELS. (Report of ICLS, 1993) Much is still left to individual surveys or governments to decide on criteria, and further investigative work is currently underway at the ILO.

ANNEX 3 D: EXERCISES IN LOCAL PLANNING

EXERCISES IN LOCAL PLANNING

The purpose of this annex is to illustrate two exercises in local planning, efforts, in principle, dedicated to supplying useful information to those who require it at the local level: first, an early attempt by UNRISD and second, some of the procedures instituted by the ILO in their attempts to implement their "basic needs" approach to planning.

The UNRISD Programme

The programme to "measure real progress at the local level" instituted by the United Nations Research Institute for Social Development arose out of:

dissatisfaction with the kind of information available to national planners and decision makers... on progress in actually levels of living of the masses of the population... The project aimed to improve the information systems for the benefit of development planners by making more use... of information on conditions and changes at the local level... and where locality studies are being carried out... exploiting them more systematically for the use of planners, (quotes from Scott, 1973: 1)

...the formulation of new indicators (or the reformulation of old ones),... the organisation that is required for the continuous collection, processing, analysis and presentation of information from the local level. (Scott, 1973:ii)

They proceed mainly by analysing cross-sectional data collected from interview or observation, cantering through a range of multivariate procedures which can be used to combine a wide range of items into a few policy-relevant indicators. But, because they recognise the importance of the choice of indicators and the difficulty of interpreting observed associations between them, they also draw attention to the possibility of using more complex methods to assess the objectives and aspirations of local populations.

More interesting, perhaps, are the range of methods they proposed for using data available at a local level, which would be easier and cheaper than national census, sample surveys and administrative files which have been subsequently taken up by the Rapid Appraisal advocates. These were:

· key persons reporting on selected and predetermined communal facilities and social arrangements;

· population and housing census enlarged in selected localities:

· making more use of administrative registers such as new housing starts, tax records at the local level;

· better use of sample surveys, especially of the conventional two-stage type, involving first selection of localities and then selection of respondents for interviewing.

On another level, local socio-economic observatories as have been set up in France: collating a wide variety of data about an area into a 'single' data base whilst the first ones were instituted for economic data, more recently, a similar network has been established for health data (Observatoires Regionaux de la Sante).

Techniques used by the ILO to Assess the Education-Employment Nexus at the Local Level

Those connected with the ILO have proposed a number of "techniques" (the inverted commas are used because they are really rather too derivative of other procedures to be separately named).

· calculating the "Basic Arithmetic" of Youth Employment simply by comparing the proportion of each age cohort with primary and secondary school qualifications with the proportions for whom there are modern sector 'jobs' available (Dore, 1976);

· relying on Key Informants (rather solemnly labelled as "a sociological -anthropological term") to provide household information on present economic activity, income levels, potential employment, future training and capital requirements in villages (McGranhan et al, 1982);

· local and regional analyses of the wage employment sector distinguished between subsistence self-employment and entrepreneurship, between casual, temporary labour, contracted and permanent workers in the formal sectors, apprentices, casuals and journeymen in the informal sector, and between cottage home industry, local firms and external enterprises (Kind, 1980).

But those categories she suggests (citing Gill, 1977), whilst possibly constituting a useful checklist, do not provide criteria for knowing which learning needs should be satisfied in which socio-economic contexts, let alone whether the education being provided might satisfy those learning needs.

These initial suggestions by the ILO - drawing on the UNRISD approaches - led to the subsequent development of the Rapid Rural Appraisal (RRA) family of techniques (again being solemn). These include using/seeking out existing information; identifying and learning from key informants; direct observations and asking questions about what is seen; guided interviews, group interviews with informal or selected groups. It means becoming a field researcher rather than a desk worker; the policy maker and planner learn from the villager; the villager becomes the subject not object of development. In particular, it highlights the importance of using existing information rather than commissioning a special survey (Myers 1985 cited in Thaker et al, 1988).

These procedures have proved useful in certain circumstances for local planners. Their usefulness will obviously depend on the extent to which the information collected answers questions which are seen as relevant by someone or some group. This raises the issue of who decided what information should be collected and what should be done with it when it has been collected. It should never be forgotten that control over local information can be an important means of consolidating and retaining power.

Radicalising Survey Methodology

Freire (1972: 13) argued:

There is no such thing as a neutral educational process. Education either functions as an instrument which is used to facilitate the integration of the younger generation into the logic of the present system and bring about conformity to it, or it becomes the 'practice of freedom', the means by which men and women deal critically and creatively with reality and discover how to participate in the transformation of their world.

The problem with Freire's method (or similar prescriptions for radical education) is that they only work if people come to them, whereas most people's experience of education (whether or not radical) is that the involvement and motivation by subjects of the learning process, which is crucial, cannot be assumed ab initio. They have to be 'drawn in' to education before they can be 'drawn out' by education. In fact, one main reason why such methods work (as indeed they do) is because of the ideals and enthusiasm of the committed, highly skilled and motivated educators who employ them. Indeed, it is arguable that someone like Freire would have been be a successful educator - in terms of raising consciousness so that people can participate in the 'transformation of their world' -with almost any method.

The problem, therefore, is to develop a low-key method of awakening people's interest in the first place (the subsequent level and nature of their motivation cannot, of course, be determined in advance); a tool which will focus people's attention on the issue, without imposing counter-productive discussion on those who do not see any problem or who have no hope of effecting any change. Any approach to uninterested and unmotivated people must, inter alia, examine their education experiences and the part these institutions have played in producing their immediate situation and particularly their understanding of, and reactions to, that situation. An early attempt was made by the author to use an interview/questionnaire approach as this requires a very low level of involvement on the part of the respondent and yet introduces ideas to him/her.

The difficulty is, that when people are asked their reaction to the education they have received or that they would desire for their offspring, the purpose and content of the slab of education being offered are often indeterminate. Even if made precise, there is little incentive for people to reply because they sense that the final decisions will be made elsewhere.

We can go some way towards compiling data on needs in a non-alienating and non-exploitative fashion by involving a selection of the population at each stage. In a pilot study in Brighton, England, in 1974, the following procedure generated considerable discussion about the purpose and content of educational programmes:

· an informal discussion with groups of individuals from the projected population, eliciting the categories in which people perceive the reality of their own lives and possible futures;

· a more directed semi-structured interview with the same group of individuals about the relevance of present and possible educational careers to their own lives; and

· a self-completion questionnaire for the population designed so as to compare the purpose and results of present formal educational systems and other forms of socialisation, with the way in which they live their own life and their hopes for improvement in its quality, (taken from Carr-Hill, 1984b)