While crude mortality rates are very important, it is very hard to use this information to compare and contrast what’s happening between hospitals. This is because every hospital is different, both in the treatments and operations that it offers and the make-up of its local population.
A hospital that carries out higher-risk operations, such as organ transplants, has a higher number patients who are elderly and/or come from areas of greater poverty, will have a crude mortality rate that is very different from one that doesn’t provide such higher-risk operations and whose local population is generally younger and more affluent.
This is why several years ago statisticians interested in comparing mortality rates between hospitals sought to find a new statistical number to allow them to do just that. The one now used most commonly is called the hospital standardised mortality ratio – or HSMR for short.
The HSMR scoring system works by taking a hospital’s crude mortality rate and adjusting it for a wide variety of factors – population size, age profile, level of poverty, range of treatments and operations provided, etc. The idea is that by taking these facts in to account for each hospital, it is possible to calculate two scores – the mortality rate that would be expected for a NHS hospital and the observed rate for an individual hospital.
The Trust’s main hospitals – the Lister and QEII – are described as district general hospitals. Nationally the expected HSMR score for such hospitals is set as a score of 100. It is important to remember that this figure does not represent deaths – it is just a baseline number that statisticians use against which to compare observed performances. The Trust’s published HSMR scores for the last few years are set out in the charts below.
When thinking about HSMR scores, it’s tempting to view these figures simplistically. You know, a Trust like with a score of, say, 115 has 15% more deaths than average, while one with a score of 95% has 5% less deaths on average.
Not only is this an over-simplification, it’s also probably an entirely wrong conclusion to draw. There are several reasons why any individual HSMR score needs to be treated with caution, namely:
- The quality of the clinical coding – every clinical procedure undertaken in the NHS has its own unique code and unless these are used properly on our computer records, this can have a direct skewing effect on the resulting HSMR score. As you’ll see later on, this was an important issue behind the Trust’s own HSMR scores it received during 200/09, which related in part to inadequate clinical coding – an issue that has now largely been resolved.
- Where a patient dies – compared to other parts of the country, Hertfordshire has fewer hospice beds and community-based services that help people to be with their families and loved ones when they die. As a result, we have more people who end up dying in our hospitals when that doesn’t need to happen. Again this can affect our HSMR score.
- Clinical quality issues – thankfully not a big issue for the Trust, but this is the one that most people get concerned about – and rightly so too. Running a service where more patients may be dying than would be expected is a key clinical quality issue to which the Trust pays particular attention. A rising HSMR for a particular clinical procedure is an early warning indicator that something might not be right. In most cases it proves to be caused by a coding or other non-clinical issue. But sometimes, it can be a pointer to something more serious, which then allows us to take corrective action quickly.
- Through the combination of the complexity of the data being measured, along with natural random variation that occurs, HSMR scores, are never absolute figures. Indeed the experts behind the HSMR system suggest that any individual score could vary by as much as +/- 7%. So statically speaking, a NHS Trust with a HMSR score of 94 could well have an identical performance to one with a score of 106 and vice versa.
So what does this performance tell us?
First of all we’re no longer dealing with actual deaths, but rather whether or not what’s recorded against an individual hospital looks to be above or below average. And the important word here is looks. Why? Because experts in using the HSMR scoring system always give a big health warning about how it should be used and interpreted.
For example, scores well above 100 suggest that there may be a need to investigate whether or not there is an underlying clinical problem that needs to be addressed. This does not mean that people can or should assume that a real problem exists at all. It could just be that the data on which the calculation was based wasn’t as accurate as it should have been. But there again, it could point to a specific clinical issue that needs attention.
Until investigated thoroughly, it is often impossible for anyone to tell what the true reason is behind a lower or higher than expected HSMR score. Confusing, isn’t it?
Working through the Trust’s scores
But it is possible to shed light on what might be going on behind this important statistic by looking at the Trust’s annual HSMR performance for the last few years.
For each of the two years to 2007/08, the Trust’s annual HSMR score has been recorded at in or around the national average that would be expected for district general hospitals such as the Lister and QEII.
During 2008/09, however, the Trust began to notice that its quarterly HSMR score was rising month-on-month and needed to be investigated. This was done and what was uncovered was a significant deterioration in the quality of the data recorded against patients treated at the Trust’s hospitals. While this had no direct effect on the quality of care received, it did skew adversely the resulting HSMR score. Once this data entry problem was fixed, the Trust’s HSMR rating began to improve quite quickly.
But that’s not the whole story, however. By improving the quality of the Trust’s data, it did allow us to see one or two clinical areas where the HSMR score was still higher than it should have been – for example for patients with broken hips, where the death rate was higher than expected.
By looking at the clinical pathways involved, changes were made that had a real impact on the chances that this vulnerable and often frail group of patients had in not just surviving their surgery, but going on to recover a good quality of life afterwards.