While crude mortality rates are important, it is very hard to use this information to compare and contrast what’s happening between hospitals. This is because every hospital is different, both in the treatments and operations that it offers and the make-up of its local population.
A hospital that carries out higher-risk operations, such as organ transplants or see more patients who are elderly and/or come from areas of greater poverty, will have a crude mortality rate that is very different from one that doesn’t provide such higher-risk operations and/or whose local population is generally younger and more affluent.
This is why several years ago statisticians interested in comparing mortality rates between hospitals sought to find a new statistical way to allow them to do just that. The one now used most commonly is called the hospital standardised mortality ratio – or HSMR for short – which is published nationally by Dr Foster Intelligence.
The HSMR scoring system works by taking a hospital’s crude mortality rate and adjusting it for a variety of factors – population size, age profile, level of poverty, range of treatments and operations provided, etc. The idea is that by taking these factors in to account for each hospital, it is possible to calculate two scores – the mortality rate that would be expected for any given hospital and its actual observed rate. It is the difference between these two rates that is important when it comes to HSMR.
The Trust’s main hospitals – the Lister and QEII – are described as district general hospitals. Nationally the expected HSMR score for such hospitals is set as being 100. It is important to remember that this figure does not represent deaths or percentages – it is just a baseline number that statisticians use against which to compare observed performances. The Trust’s published HSMR scores for the last few years are set out in the charts below.
Data source – Dr Foster Toolkit – July 2015 refresh
When thinking about HSMR scores, it’s tempting to view these figures simplistically. For example a hospital like with a score of, say, 115 has 15% more deaths than average, while one with a score of 95% has 5% less deaths on average. This is not the case as HSMR scores are not percentages, rather an indication to help clinicians and managers understand how their services are performing.
Indeed there are several reasons why any individual HSMR score needs to be treated with caution, namely:
- The quality of the clinical coding – every clinical procedure undertaken in the NHS has its own unique code and unless these are used properly on our computer records, this can have a direct skewing effect on the resulting HSMR score.
- Compared to other parts of the country, Hertfordshire has fewer hospice beds and community-based services that help people to be with their families and loved ones when they die. As a result, we have more people who end up dying in our hospitals when that doesn’t need to happen. Again this can affect our HSMR score.
- Clinical quality issues – thankfully not a big issue for the Trust, but this is the one that most people get concerned about – and rightly so too. Running a service where more patients may be dying than would be expected is a key clinical quality issue to which the Trust pays particular attention. A rising HSMR for a particular clinical procedure is an early warning indicator that something might not be right. In most cases it proves to be caused by a coding or other non-clinical issue. But sometimes, it can be a pointer to something more serious, which then allows us to take corrective action early and quickly.
- Through the combination of the complexity of the data being measured, along with natural random variation that occurs, HSMR scores, are never absolute figures. Indeed the experts behind the HSMR system suggest that any individual score could vary by as much as +/- 7%. So statically speaking, a NHS trust with a HMSR score of 94 could well have an identical performance to one with a score of 106 and vice versa.
So what does this performance tell us?
First of all we’re no longer dealing with actual deaths, but rather whether or not what’s recorded against an individual hospital looks to be above or below average. And the important word here is looks. Why? Because experts in using the HSMR scoring system always give a big health warning about how it should be used and interpreted.
For example, scores well above 100 suggest that there may be a need to investigate whether or not there is an underlying clinical problem that needs to be addressed. This does not mean that people can or should assume that a real problem exists at all. It could just be that the data on which the calculation was based wasn’t as accurate as it should have been. But there again, it could point to a specific clinical issue that needs attention.
Until investigated thoroughly, it is often impossible for anyone to tell what the true reason is behind a lower or higher than expected HSMR score. Confusing, isn’t it?
Working through the Trust’s scores
But it is possible to shed light on what might be going on behind this important statistic by looking at the Trust’s annual HSMR performance for the last few years. Since 2010/11, the Trust’s annual HSMR score has fallen steadily to a point where statistically-speaking, it is consistently below the 100 average mark. Although the HSMR score did rise slightly in 2013/14, it dropped back in 2014/15 and is on course to fall further again in 2015/16 once the final data comes through.