I am amused that people are slow to realise that large language models (ChatGPT etc) do not understand what they are saying, or that they make things up — that is, they hallucinate. Performance on “surface layer” testing does not equate to competence. Anybody who has taught medical students knows that humans are quite capable of exhibiting the same behaviour. It was one of the values of the old fashioned viva. You could demonstrate the large gulf between understanding (sense)on the one hand, and rote — and fluent rote at that — simulation on the other (garbage).
The medical educationalists, obsessed as they are with statistical reliability, never realised that the viva’s main function was for the benefit of teachers rather than learners. It is called feedback.
The medical student as ChatGPT
One of my favourite papers about medical education was an article Geoff Norman wrote called ‘Medical expertise and mashed Potatoes’. In it he recounts a meeting with the famous chef Albert Roux. Norman uses the encounter to point out the similarities between expertise in what seems like very different domains. Since I have recently almost lost the will to live having tried to gorge on an ultimately inedible diet of pseudo-competency based descriptions of what doctors ‘are’ (really, the Danish devote pages to an exegesis of the CanMEDS, and tweaking of where to put ‘professionalism’ in a schema of what doctors ‘are’!) They all need to get a dose of Wittgenstein to see the folly of their ways..) Anyway I digress.
Medicine was historically an apprenticeship, but our problem (well, actually the students problem) is that in large part this is not mirrored in the way we organise it at the undergraduate level. As Alice Gopnik, the psychologist once remarked, at university we tend to think the way to teach people how to cook, is to lecture then for three years and then, and only then, allow them to crack an egg. Here is a nice video from the NYT of people who understand education much better than we do. The video, and Norman’s article, say more than the ever enlarging girth of the working groups.
Derm has been a very very popular specialty in the US for many decades. Some of this reflects work / life stuff, the ability to mix surgery and physician-like diagnostic skills, and because skin disease matters to patients a great deal, and if they purchase their own care, they put a big value on it. (And in truth the fact that acute medicine is deeply unpopular and unsustainable as a longterm career). I have been told similar changes are playing out in the UK, but I hadn’t seen any figures. Here is a summary for ‘medical’ (sic) specialities from the RCP just published in ‘Clinical Medicine’ showing, as for lots of domains, the winds blow east across the Atlantic.
Well, imagine if this catches on:
Q.I want a refund?
A.We believe in you. We believe in Rosh Review. We believe that if you use Rosh Review, then you will pass the board exam. If you don’t pass, we will refund you the full amount of your subscription costs. Guaranteed. Your ABEM Score report must be emailed, faxed, or mailed to Rosh Review, LLC within 30 days of the score report being published.
Well, lets see if the NHS Postgraduate Deaneries will follow suit.
Someone asked me this week whether I thought the skills mismatch really exists. The idea of the skills mismatch is that despite persistent unemployment, there are high-skilled or specialized jobs that go unfilled in large numbers because people with the skills aren’t available. My response was that the existence of hundreds of colleges and universities is de facto evidence of a skills mismatch. So when a publication like the Globe and Mail calls the skills mismatch a fairytale, it is referring to one specific statistic in one particular industry, which may or may not be misleading. But in fact, there have always been skills mismatches, and while we can’t predict precise job markets, we can do our best – government, industry and education – to prepare people to adapt and grow into the new needed skills. “What you can try and do is ensure that you are as resilient as you can be and that you have the broad set of flexible skills that allow you to take advantage of an opportunity when it comes along.”
One definition of education (as compared with training ..perhaps) is of course is it is that which provides resilience in the face of change. Better still we might consider using Nassem Taleb’s term, anti-fragility. Humans are not fitted to one ecological niche, rather evolution has selected us for the ability to fit into many niches. We can also not just respond to the environment, but becomes masters of it (and therein lies on occasions hubris). This is one framework in which to view education. Those who believe in medial education as well as medical training, will warm to this approach. The difficulty is defining what aspects of education really do facilitate the ability to adapt. The usual mantras of ‘teaching lifelong learning skills’ I am deeply sceptical of. You can cultivate those attributes, you can signal you value them, but you can’t expect to ‘tick-box’ them.
For me, several years removed from training, the stony bureaucracy and burdensome administration are astonishing. Modern shift patterns have broken teams where peer and consultant rapport made being a junior doctor survivable. Now, rather than judge juniors through a colleague who has taught them and seen them on the job we use anonymous comments from someone on a feedback form. This causes huge distress. The moral contract of medicine, of choosing to do a job out of care and love, has been fractured because of the unyielding training structures that senior doctors have bequeathed.
Margaret McCartney on the dismal and worsening state of postgraduate education and training in the UK. The title ‘Frankly, it is not much fun’ are her words, but the words that spring to my weary mind are those of William Blake: ‘A dog at the master’s gate, predicts the ruin of the state.’ And so it will come to pass.
Of the 401 residents who received the e-mailed survey, 226 (56.3%) completed it. Of these, 97.7% reported spending at least one hour per week engaging in extracurricular education, and 34.5% reported spending two to four hours per week (P < .001). Time listening to podcasts was the most popular (reported by 35.0% of residents), followed by reading textbooks (33.6%) and searching Google (21.4%; P < .001). Residents endorsed podcasts as the most beneficial (endorsed by 70.3%) compared with textbooks (endorsed by 54.3%), journals (36.5%), and Google (33.8%; P < .001). Most respondents reported evaluating the quality of evidence or reviewing references “rarely” or less than half the time. A majority (80.0%) selected the topics they accessed based on recent clinical encounters.
[simnor_button url=”http://ovidsp.tx.ovid.com/sp-3.11.0a/ovidweb.cgi?QS2=434f4e1a73d37e8c5a069e886e3737f20a1958d49d4875d6d54b2eaffb8643848c17ab581dd41da4398e4d1a9bca3fe39df3ef677700f63097b58846b2f14930c6c27000c3a4968ed9f6db7fb7e8ff1674cfd10f35fc05f9c37dde20d4602cd63c0595a84ba784fac9a60dd2931acca1ed2e12e7cfca0705ec62fa4a87962aaa240de814a45fc7116eaf5d39ae0c7abecee0b2aeee5df0ef6609d054fe3fbeab6a9b3da8e9a8b0af003dd3dd4915d4d08f32983dc724fee852ccb3fcac6dc63ac5d1ecd8e862a93caf996907ebc5f752c77041894b6456548e0c23fdb0fe14ab362549dcadfa55830b6bc38cad1dc9ad0f95d39b9597121343ecf5da05e773ef ” icon=”double-angle-right” label=”DOI: 10.1097/ACM.0000000000000170″ colour=”white” colour_custom=”#fff” size=”medium” edge=”straight” target=”_self”]
“What of attempts to improve skin cancer diagnostic skills in primary care, or to develop GP specialists as seen in Australia or the UK? There are various points to make here, and perhaps a lot of wishful thinking about how the problem could be solved if only ‘GPs’ knew more about this or that subdomain of medical knowledge. In truth, such blandishments, must be frustrating to many GPs: there are only so many hours in the day. There are studies showing that it is possible to improve diagnostic skills over the short term following organised tuition (cited in Rees (16)). To find anything else would of course be surprising: if we expose intelligent people to formal tuition or learning, we expect short-term performance to improve. But, the critical point is whether this improvement is maintained, and what aspects of performance suffer because they have been replaced by training in another domain (16). There is no free lunch. If we run a course on skin cancer, then the rheumatologists, cardiologists etc. will all want to run courses. And much of what we know about such one off tuition is that in the absence of consolidation and feedback, the benefits are short lived only. How many of us remember all the history and geography we learned at school?” Here.
It is hard to find anything in the Scotsman worth writing about. Newspapers and good journalism are victims of the medium I am writing in. But here is something. Sir Harry Burns, is stepping down from his role as Chief Medical Officer for Scotland, and taking up a position in Global Public Health at the University of Strathclyde. I am a little bit cynical about the academic bandwagon of ‘Global Public Health’. Much of it seems to miss the point that Sydney Brenner with characteristic insight made many years ago: the most common disease on this planet is MDD. Otherwise known as money-deficiency-disease. Correcting this is not straightforward— and medicine has an important, but limited role— but, as Bruce Charlton pointed out, industrialisation and capitalism has lifted more people out of poverty in the last quarter century (and hence cured more people with MDD) than public health research funding. Too often I am suspicious academics are trying to mine away at a newly exposed seam of research funding, rather that solve problems. Exceptions all around; Paul Farmer and the like, I accept. What however is worthy of mededed.me, is that Burns takes a broadside at the way we train doctors.
Looking back on his time as CMO, Sir Harry said there were still issues needing addressed. “I am not happy with the way we train young doctors now,” he said. “As I look at young people training in medicine now, I think their opportunity to do the kind of things that I have done, which has been a very varied career, I think those opportunities are harder to deliver now. “How they are trained now is very rigid. What I did in moving from surgery to public health would be really difficult now. “There’s an over-emphasis on ticking boxes, as opposed to encouragement of innovation and new ways of looking at your career.”
Well, I couldn’t agree more. But walk down any hospital corridor in Scotland and many doctors will tell you postgraduate training is a mess, and has been getting worse. Yet, each year, another report will spew forth from one of the various Bunkers in London, and yet more tick-boxing will be enforced, alongside more debasement of the English language. And the position in Scotland, under Burns’ watch, is arguably worse than in other parts of the UK. I used to wonder how people in the former eastern bloc melded official doctrine, with their private thoughts. How did they reconcile their inner beliefs with all the bullshit. I now know— as do a lot of doctors. Too much of medical education is about producing widgets for the NHS, wrapped up in an unwholesome diet of newspeak and doublethink. The problems are that we do not know what sort of widgets we might want, and second, many of us do not think medical education should be in the widget business. There is plenty left to do in the UK.