Paying Doctors For Quality: Intended And Unintended Consequences

01 Feb 2010

The NHS 'quality and outcomes framework' (QOF), which introduced greater incentive pay for doctors, led to modest improvements in the quality of primary health care. But GPs were also able to increase their incomes, reduce their working hours and manipulate the indicators by which they are assessed.

These are some of the findings of research by Professor Hugh Gravelle and colleagues, published in the February 2010 Economic Journal. The authors explore first whether the QOF has led to higher pay and job satisfaction – both affirmative – and then its behavioural consequences, both intended and unintended.

The intended impact on treatment outcomes is at best unclear, partly due to poor data on the situation before the reform was introduced. But there is evidence of 'gaming' by some doctors to increase pay without increasing treatment. This mirrors other evidence of incentive payments often having unintended consequences when the targets are not perfectly aligned with objectives.

The QOF, which the NHS introduced in April 2004, was one of the most elaborate quality incentive schemes ever introduced in any health care system. It resulted in payments of over £1,000m a year to general practices. But it is difficult to estimate its effect on the quality of care. The NHS only collected routine data on most of the activities incentivised by the QOF after it was introduced. There was no piloting or trialling of the scheme, which was introduced simultaneously for all practices in all four constituent countries of the UK.

The research finds that in a sample of 500 practices, there was an upward trend in quality indicators before the QOF was introduced and any effect of the QOF was modest. Comparison of trends in indicators for coronary heart disease (CHD) and other diseases confirms that there was an underlying upward trend in both indicators that were incentivised by the QOF and those that were not, and the QOF led to a small additional increase in the incentivised indicators.

The effects on general practitioners

Practices had high QOF point scores. In England, they achieved 91.3% of the 1,050 available per practice in 2004/5, 96.2% in 2005/6 and, after some minor amendments to the scheme, 95.4% of the 1,000 available in 2006/7.

Practices received considerable increases in gross income. Most are partnerships of GPs and have to meet the costs of running the practice out of their gross income before distributing their net profits to the partners. The rate of growth of net profits per GP (including both full and part-timers) increased markedly during the first year of the QOF.

At the same time, GPs' normal hours worked and job pressure fell sharply. Unsurprisingly their job satisfaction increased. The QOF appears to have been a very good deal for GPs.

Unintended consequences

In the first two years, around half of potential QOF revenue was attached to 65 indicators of clinical quality in 11 disease areas. The indicators are measured as the ratio of the number of patients for whom the practice has achieved some outcome divided by the number reported eligible for that outcome.

Expressing a clinical indicator as a ratio is intended to provide an incentive to practices to increase the numerator, that is, to increase the number of treated patients. But the denominator for an indicator in a disease domain –&##160;the number eligible for treatment – is the number of patients with the disease minus the number of patients the practice chooses to deem ineligible for that indicator ('exception reporting').

Patients can be reported as unsuitable for an indicator on a variety of grounds, for example, if they are terminally ill, frail or cannot tolerate the medication. The practices can also exception report patients who have failed to attend. So a practice can increase the proportion of patients for whom an indicator is achieved by increasing the number of patents it exception reports.

The research uses the structure of the incentive scheme to test whether some practices deliberately increased the number of exception reports to increase their indicator scores. It finds that 'gaming' the system in this way did occur.

The researchers conclude:

'The UK's experience with the QOF suggests that major reforms to payment systems should be trialled and evaluated before being rolled out nationally. Such pilot schemes with baseline measurements would have revealed if quality was already improving and may have enabled the taxpayer to get better value for money from the scheme.

'More considered design would also have revealed potential problems with exception reporting and other peculiar details of the scheme, such as the lack of transparency in the link between effort and reward.'

'Doctor Behaviour under a Pay for Performance Contract: Treating, Cheating and Case Finding?' by Hugh Gravelle, Matt Sutton and Ada Ma is published in the February 2010 issue of the Economic Journal. Hugh Gravelle is at the Centre for Health Economics at the University of York. Matt Sutton and Ada Ma are at the Health Economics Research Unit at the University of Aberdeen.

Hugh Gravelle