Home > Decision Support > Appraisal & evaluation techniques > Key considerations

Key considerations

  1. Which techniques are most appropriate in order to consider non-market impacts?
  2. What impacts should we include when appraising decisions?
  3. How can we achieve a consistent approach?

 

1.   Which techniques are most appropriate in order to consider non-market impacts?

The most widely used appraisal and evaluation techniques tend to be based on various forms of cost-benefit or cost-effectiveness analysis (the cost-benefit analysis used in the public sector is the parallel of the profit-and-loss analyses used by businesses, but it is used for public decisions and for non-market outputs). Cost-benefit analyses quantifies in monetary terms as many of the costs and benefits of a proposal as possible; cost-effectiveness analysis compares the costs of alternative ways of producing the same or similar outputs.

Cost benefit analysis tends to follow the following stages.

  1. defining the project / policy / other decision
  2. identifying the impacts
  3. valuing those impacts
  4. discounting the value flows
  5. applying a Net Present Value test
  6. analysing the sensitivity of the result to relevant parameters

Impacts for which the market does not provide a satisfactory measure of economic value are assigned values in order that they can be included directly in cost or benefit calculations. Common techniques for establishing values for non-market impacts are ‘willingness to pay’ and ‘willingness to accept’. These are outlined in Annex 2 of the Treasury’s Green Book, which also explores various other techniques such as ‘quality-adjusted life years’ for evaluating health impacts and the ‘social cost of carbon’ for valuing any impacts on greenhouse gas emissions.

Even where such supplementary techniques are used, results from cost-benefit analysis are heavily dependent on the assumptions made, particularly regarding:

  • the actual choice of impacts considered (i.e. whether the impacts considered are comprehensive);
  • the difficulties in valuing non-market goods;
  • the choice of discount rate to represent the time value of money; and
  • ecosystem complexity (i.e. whether all impacts throughout an ecosystem can actually be identified).

Given the difficulties in making assumptions about these issues, it is very possible that even if non-market impacts are considered and monetised, the result of cost-benefit or cost-effectiveness analysis may well not be a sustainable outcome. For example, the optimal level of pollution as decided by the analysis may not equate to a sustainable level of pollution.

A range of new sustainability valuation and appraisal methods have been, and continue to be, developed. Six methods are explored in a 2007 Sustainable Development Research Paper. The methods covered all ‘account for different types of knowledge (monetary and non-monetary; quantitative and qualitative data); consider seriously the issue of inter-generational equity; provide opportunities for learning during the appraisal process; ensure transparency of each step of the appraisal process; and have a strong element of public and stakeholder engagement.’

 2.  What impacts should we include when appraising decisions?

As stated in the Green Book relevant costs and benefits are those that can be affected by the decision at hand.

Defra’s Stretching the Web tool was developed with the aim of helping policy makers consider the full impacts of their policy, project or programme work. Though developed for civil servants, it could be applied to almost any organisation. Users rank their decision outcomes which are then displayed as a simple web which allows for the easy identification of ‘bites’ or negative impacts. This enables decision makers to identify areas for further work, such as research or engagement with partners.

Some organisations prefer the use of matrixes, checklists or lists of questions to ensure that all impacts on society, environment and economy have been considered. Examples include:

– though deriving from within the regional government sector, they represent good examples of how to consider a wide range of impacts that other sectors might learn from.

Impacts can be considered in isolation, but must at some point be brought together for comparison. For information on assessing the impact of decisions on equality and human rights, see the Equality and Human Rights Commission. For resources on assessing health impacts, see the Health Impact Assessment Gateway. A research document setting out range of resources on the integration of social, economic and environmental impact assessments has also been produced by Northumbria University.

3.  How can we achieve a consistent approach?

Organisations can move towards a consistent approach by:

  • setting out clear guidance to staff who evaluate or appraise;
  • training staff in the principles of sustainability (see Skills module);
  • scrutinising decisions that have been made to understand why unsustainable options were taken, and if necessary, revising appraisal methods accordingly; and
  • Establishing organisational systems and processes which enable organisations to deliver more sustainable development, for example, based on Accountability Rating criteria 2007.

Are there key considerations we haven't included?  If so, please let us know.