Control and monitoring of the MOT Scheme
Britain’s MOT Testing Scheme, now over 50 years old has changed significantly over time. The inspection itself has expanded beyond all recognition from the early 1960s, yet whilst some equipment is more sophisticated, its still not as ‘clever’ as it could be. And the way in which the authorities seek to control Testing garages has altered considerably – especially since after 2005 when the ‘Risk Assessment’ system that we have today took effect. Now, with full web-based MOT computerisation literally just around the corner, perhaps that too will result in a significant change to the MOT. Here we have a closer look at what has been done, is being done, and perhaps could be done by DVSA to better control and improve the MOT Scheme.
Local knowledge and inconsistencies
An unchanged aspect of the MOT Scheme is DVSA’s geographical MOT Areas. Every MOT Testing Station has a ‘local’ office responsible for the formal administrative aspects of the operation of that business – although there are fewer areas than there used to be.
Originally the areas were pivotal. Every Testing Station had a local Vehicle Examiner allocated to the business – a ‘beat bobby’ MOT policeman who knew the ‘patch’ and annually visited each Test Station, checking formalities, chatting with the Authorised Examiner and closely observing an MOT inspection – passing on pearls of wisdom over a cup of tea. One sugar or two?
Discipline was a ‘two strikes and you’re out’ system. A formal warning, followed by withdrawal if a second problem occurred within five years. This was initiated by the VE, endorsed by his boss and finally a disciplinary letter sent via the Area Manager. A benefit of the system was the Vehicle Examiner’s extensive ‘local knowledge’ – after all, he made an informal ‘risk assessment’ every year! The downside was that some became a bit too friendly with the Testing staff they were supposed to be monitoring.
Disciplinary inconsistencies, however, occurred across VOSA Areas. A transgression attracting a ‘warning letter’ to a Testing Station in one Area could result in withdrawal of Authorisation in another. The disciplinary points system emerged to address the problem, but inconsistencies continued, so now a ‘Central Review Team’ at VOSA HQ reviews every disciplinary case submitted by a local Area before it is finalised and sent to the VTS concerned.
Risk Assessment
DVSA’s publication “Guide to MOT Risk Reduction” is a useful document, but will have to change since Testers’ ‘Performance Reports’ won’t be available on the web-based MOT Computer – which when fully developed will have more flexibility. Perhaps that’s an opportunity to fully review DVSA’s whole ‘Risk Assessment’ system for controlling and monitoring Testing Stations.Three yearly ‘risk assessment’ of VTSs commenced in March 2008 following the Hampton Report of 2005 on the regulation of small businesses. VOSA decided, or were told to monitor and control Test Stations using such a system, which arguably they were already doing annually, albeit on an informal basis via those annual VE visits. Annual ‘Observed Tests’ were also ended.
The new system formalised three yearly ‘risk assessment’ visits by a VE who might know little or nothing about the VTS. The concept of a local VE was gone. MOT advice now came via the MOT Computer ‘help-line’, and a large proportion of VEs were now employed conducting three yearly ‘risk assessment’ visits on those increasing numbers of VTSs, or focussing on extra visits to Amber and Red garages.
Other DVSA activities
Whilst ‘risk assessment’ is currently DVSA’s primary tool to monitor MOT garages, there are other things they do to keep an eye on what is happening at MOT businesses, albeit not affecting so many VTSs:
Compliance data: Every year VEs visit about 1,800 Testing Stations randomly selected by computer, and inspect a recently Tested vehicle to measure MOT quality, for DVSA’s annual ‘Compliance Report’.
Intelligence led inspections: This largely comes from ‘tip-offs’ about bad Testing. It can be a mystery shopper visit, checking a recently Tested vehicle or covert surveillance.
Appeal Tests: Every appeal Test has the potential of providing DVSA staff with an opportunity to monitor a VTS.
Red/amber visits: These VTSs get additional checks, perhaps a mystery shopper visit or have a recently Tested vehicle re-inspected.
Taken altogether, ‘risk assessment’ visits plus those 1,800 ‘Compliance checks’ – require at least 10,920 VE visits to Test Stations each year, not including those additional checks on Red and Amber VTSs. Unfortunately, as we’ve seen in DVSA Matters in practice such visits fall well short of that target.
MOT Testers
The average Green garage MOT Tester only meets a Vehicle Examiner for fifteen minutes or so once every three years during a risk assessment visit if asked a set of MOT specific questions.
For businesses with more than one Tester, some Testers may not see a VE for six or maybe even nine years. That, coupled with a five yearly refresher training cycle (currently suspended), means Green garage Testers are largely neglected by the DVSA.
Clearly there are conscientious Testers out there. They read DVSA’s Matters of Testing or publications like ours regularly, and also contribute to MOT Forums like that on our MOT Testing website, and may be members of trade bodies – but how many others just carry on Testing, and don’t necessary realise when they are becoming careless or complacent…
Reduced monitoring – less control?
Since three yearly ‘risk assessment’ has been formalised, despite the greater formality, and increased scrutiny during the visit, ‘green’ Testing Stations are monitored less frequently. And we know that any action following an annual ‘desk-based’ assessment is unlikely! How then, after monitoring VTSs via the ‘risk assessment’ system do DVSA control Testing Stations?
In practice, between assessment visits, the only way DVSA can ‘control’ a green VTS (80% of MOT garages), is by monitoring the computer data – Test times and pass/fail rates – and if things seem to be going seriously awry to set up a ‘mystery shopper’ visit, or a visit to re-Test a recently Tested vehicle. As to what’s actually happening at the Testing Station day to day, there’s no control at all.
Provided they keep to a reasonable pass/fail rate, and ensure that they don’t do ten minute Tests, a new ‘rogue’ Tester, or manager at a ‘green’ VTS could get away with all sorts of unacceptable practices until the next risk assessment visit.
Red and amber – closely monitored, and controlled
Conversely, Red and Amber garages (6% and 14% respectively), are closely scrutinized with nine-monthly visits – ‘mystery shopper’ visits, or re-inspecting a recently Tested vehicle. Yet, perhaps ironically, with more visits, they are more likely, statistically, to fall foul of the DVSA disciplinary system. Perhaps green garages having the same level of disciplinary scrutiny as that received by red garages, might have just as many problems discovered…
Maybe the extra attention could better be ‘tutorial’ rather than disciplinary? Nevertheless, that is DVSA’s preferred system of control. Ultimately, however what matters is the quality of MOT Testing. So, has the ‘Risk Assessment’ system improved Testing quality, as it is designed to do?
MOT quality
The 2013/14 MOT error rate was a 5-year high.Last year’s MOT Test error rate was the highest since 2009/10, despite VOSA and DVSA having ‘tweaked’ the ‘risk assessment’ system to effect improved MOT quality over time – which would suggest that the current ‘risk assessment’ system isn’t working as it should. If it were that error rate will have fallen.
This doesn’t mean a risk-based system of assessment can’t work, or even to say it hasn’t improved matters – things might have been worse under the old system. Even so, extrapolating the numbers from DVSA’s own Compliance data to all MOTs, in the year to March 2014, suggests that over 4 million MOTs probably got the wrong pass/fail result, and altogether perhaps 8.5 million probably had some form of error or another.
Road safety
In road safety terms, however, the situation is difficult to assess. Just over 3 million vehicles were issued with a pass where they should have failed – about 10% of all MOTs, but some may relate to trivial non-safety critical items. What’s difficult to find in the data, though, is how many failure items were missed on already failed cars, and so hadn’t been repaired when a pass was ultimately issued, a further, but unknown threat to road safety. What can be said for certain though is that poor Testing quality has the potential to seriously endanger road safety.
Conclusions
So here’s what can we conclude about the current system the DVSA use to control and monitor MOT Testing:
Green Testing Stations: Are inadequately monitored, controlled or supported by DVSA between ‘risk assessment’ visits and must also be contributing to poor MOT Testing – but accurate figures are unavailable.
MOT Testers: Are largely neglected by the DVSA. Five yearly refresher training is inadequate, and the questions posed to Testers over a few minutes during a ‘risk assessment’ visit make but a minor contribution to improving MOT Test quality. Some Testers have no attention from DVSA for five years, now longer with refresher training suspended.
Amber and Red garages: Get a structured system of monitoring and control with defined and dire consequences if they fail to improve their Test quality.
The role of the MOT Trade
As pointed out by a reader in the last edition of MOT Testing, the MOT Trade should have a role to play in ensuring MOT quality is maintained. Arguably, irrespective of the DVSA’s somewhat lax approach to monitoring and controlling green garages, the garages themselves should ensure that MOT Test quality is maintained.
So, given poor MOT quality, why doesn’t that happen? Nobody seems to know. Poor Testing has three basic underlying causes:
Incompetence: Testers who have insufficient experience, and/or knowledge, having failed to ‘keep up’ with the changing regulations over the years.
Carelessness and/or complacency: Testers who have become so used to carrying out MOT Tests, they are going through the motions without realising that their skills and attention have eroded with time.
Dishonesty: Testers and AEs who, for financial gain do not do Testing properly – for example, issuing a pass certificate and trusting the owner to fix failure items. Or issuing a certificate without even seeing the vehicles, and so on.
The last of these, dishonesty, is indeed something the MOT Trade itself should try to police or discover. Yet whilst the best AEs can put security systems in place, a crafty Tester might still beat that system. In practice it would be for that AE to convince the DVSA there was little more that could have been done to thwart a dishonest Tester.
Incompetence, carelessness and complacency, however should be addressed by Testing Station managers, and many do have systems in place to ensure quality Testing, either doing it themselves or obtaining the assistance of a trade body or MOT consultant. Too few, however, do very little, and the MOT Trade should be concerned about that, but so too should the DVSA. Their risk assessment system is designed to discover such Testing Stations, but while they have rigid percentages of Green, Amber and Red garages, too many Green garages, when left alone for three years will be less competent than the DVSA would like.
A better way…?
What then could DVSA do to improve things given they are bonded to a risk assessment system of monitoring and control? At MOT Testing we believe the key is to change the culture within MOT garages. Here are some suggestions that might help:
Better focus the ‘risk assessment’ process at the garage so there’s a real emphasis on the MOT operation and management activity designed to keep Testers sharp.
Focus more on the Testers. We know the DVSA are keen on what they have called ‘Continuous Professional Development’ for Testers, which could in future involve a periodic check of their knowledge and performance.
Drop the annual desk-based assessments and divert the resources to provide better support to Green garages.
Increase the number of visits to Green Garages and ensure a fresh ‘risk assessment’ visit is conducted if the management changes, or a new Tester arrives where there’s only one Tester.
Why not ‘risk assess’ Testers on the basis of their disciplinary history, experience and number of tests conducted? For example, a Tester moving from a specialised dealership to an independent garage may have limited experience of inspecting all makes of vehicles.
Retain the compliance visits, the intelligence led activities and maintain the focus on Red and Amber VTSs – perhaps shifting the emphasis slightly from harsh disciplinary activity to mentoring and support.
In this piece we have not sought to criticise the DVSA – but to explain how their system of monitoring and controlling the MOT Scheme currently operates, and to provide evidence that it has been relatively ineffective in improving MOT quality in recent years. We’ve also put forward some ideas of our own as to how things could be improved.
Better with a new computer?
But we do sympathise with the DVSA who could do much more but for being saddled with an outdated ‘dial up’ MOT Computer entirely unsuitable to install interactive features to better communicate with Testers and Testing Station managers on an ongoing basis.
The new web-based system, should better communicate with Testers and managers to effect a real improvement in MOT quality, with significant financial saving – some of which surely, should be used to better support green garages.
Unfortunately, however, that will take time. With refresher training suspended and severe restrictions on the management information Testing Stations will receive until the system is further developed and refined at some future date, MOT Test quality may well initially decline before getting better…!
This article was published last May in MOT Testing Magazine, before the new MOT computer was up and working, which should now be providing the DVSA with a lot more data. In the next edition of MOT Testing Magazine we will be shining a spotlight on DVSA’s Enforcement and Disciplinary policy – and how it could (should?) be better developed as part of DVSA’s ‘Modernisation’ programme.