When Statistics Need a Health Warning

7th July, 2011

It is commonplace for research findings, especially those containing statistics, to be picked up by PR and media companies for the purposes of publicity and ‘newsworthiness’.

For example, the BBC News website recently reported upon official statistics from the Office for National Statistics (ONS) that suggested public sector workers enjoy higher pay and better employment conditions than workers in private sector organisations. The headline statistics suggest the salary for the average public sector worker is 7.8% higher than for those working in the private sector. The ONS figures also suggest that on average public sector workers have a shorter working week and are considerably more likely to have a company pension; 88% compared to 14% in the private sector.

At face value the statistics support continuing claims from some quarters that public sector workers are not experiencing the same economic pain as those in the private sector and arguably adds further weight to those calling for a radical overhaul of public sector pensions. However, when interpreting statistics we need to exercise caution to ensure that any claims made are supported by the evidence and this can only be achieved by ensuring that the data is complete and any comparisons made are on a like for like basis.

The BBC report noted that the period used for comparison (April 2010) was such that ‘banker bonuses’ were not included; nor were the earnings of the self-employed. Some might argue that the claims being made are based upon incomplete data and as such suffer from a lack of validity.

Then the issue of making sure you are comparing people on a like for like basis; that is comparing people doing exactly the same job. In many instances this may be possible but it is unlikely that it can be done for all jobs within the public and private sectors.  While it may be possible to make a direct comparison between a social carer in the private sector to one in the public sector it is likely to be considerably more difficult to find a direct public sector comparison for the person operating a till at the checkout in your local supermarket. Like comparing oranges and pears – both are fruits but apart from that they have little else in common.

Despite those challenges, the ONS statistics did produce some interesting findings. For example, a higher proportion of public sector workers have a degree (38% vs 23%); though conversely the statistics also showed that public sector graduates earn 5.7% less than their private sector peers. Other findings drawn from the ONS statistics indicate that:

– The public sector has a higher proportion of high-paid jobs because of a trend to out-source lower-skilled occupations;
– There is a higher proportion of older people working in the public sector that have seen their wages increase over time;
– That lower paid workers within the public sector get paid more on average than their private sector counterparts but this trend is reversed for the top public sector earners who received 6% less than their private sector counterparts.

Leaving aside the rights and wrongs of the often contradictory claims based upon the ONS statistics there are a number of important lessons we can learn from examining the claims themselves and relating them back to the statistics. For example:

– The contradictory claims being made shows that it is possible to form different interpretations from a single set of statistics;
– Recognise that statistics in themselves do not always present the full picture and should be viewed in their wider context;
– Appreciate that it can often be difficult to make direct comparisons between different groups and that therefore you are not always comparing like for like;
– And perhaps of greatest importance is recognising any limitations that may exist within the statistics and / or the manner in which they are collected.

Following these simple lessons may not always give definitive answers but they provide a good basis from which to evaluate claims made from statistical data.