2016, looking backwards.
‘Every age thinks it’s the modern age, but this one really is.’ [Tom Stoppard]
The saying goes that you can tell something about a person by the company he keeps. If books are your company, then what you read may say something about what makes you tick (although the history of the 20th century provides plenty of examples of the love of high culture by many barbarians and butchers — medical humanists, please note). So, if I look back over the year, what sort of company have I been keeping?
Well, start with the Glenn Greenwald book about Ed Snowden and move onto Laura Poitras’s brilliant film, ‘Citizenfour’. Then look at why governments like surveillance, and why surveillance likes information monopolies, with a book I have been putting off reading for many years, Tim Wu’s magisterial treatise on the history of communication technology, The Master Switch. The latter not only serves as a primer on the relation between the control of communication, the state and the communication corporations, but opened by eyes to the way Bell (Labs/ AT&T) blocked human progress: the Kronos effect, whereby advance is retarded because it threatens those who have had a hand in creating it. If you think the internet really is different, read on and worry. Along the way, for those interested like me in tech and learning, remember:
The invention of film, wrote D. W. Griffith in the 1920s, meant that “children in the public schools will be taught practically everything by moving pictures. Certainly they will never be obliged to read history again.” In 1970, a Sloan Foundation report compared the advent of cable television to that of movable type: “the revolution now in sight may be nothing less … it may conceivably be more.”
I have read lots on education. Alison Wolf (Does Education matter?) demolishes lazy thinking relating education to innovation and wealth creation. I wish the GMC would read a little more on this theme. Rich DeMillo’s most recent book (Revolution in Higher Education) was not such an eye-opener for me as his earlier book, Abelard to Apple, but that is probably because I came to it later. For all the naysayers re MOOCs and tech, HigherEd is in a problem because it no longer represents good value for many of those who have been encouraged to go, and because its business model is falling apart. It does some things well — even very well — in comparison with many organisations, but ‘basic’ higher education — is not one of them. The finances all round are not going to get any better, and as even I got into academic print, many university VCs have a lot in common with the much derided CEOs of record companies when Napster and Steven Jobs and iTunes changed things a little. On this theme, Michael Crow’s ‘Designing the New American University’ is up there, too.
Universities, as well as financial problems, have existential problems. They have become lazy corporations, that have left ‘Donnish Dominion’ behind, and replaced it with a facade of consumerism, whilst pretending that they can still be guardians of integrity (and are worth our support for that reason, alone). Well, this is what was called Kissingeritis, or in anther context, the criticism Robert Oppenheimer received: you can be at the centre of power and ‘influential’, or you can speak the truth, but you can seldom do both within the same time frame. As yet, I haven’t had a chance to dip into my copy of ‘Reshaping the University: the rise of the regulated market in higher education’ by David Palfreyman and Ted Tapper. Lots of our UK universities think they are BMWs or Jaguars, imagining themselves close to Ivy League institutions, but the finances say otherwise, as does any scrutiny of how they have taken to looking after their academic staff or their students. Not Fiat 500s, but with effort and reform, they could be useful Golfs, but leave the rest for the glossy prospectuses.
I have been reading about medicine too, but my take on this is that there is even less serious work out there about medicine and medical education, than about our universities. De Selby got it right: ‘serious work takes over from relevance’ and besides everybody is now just too busy for serious work, let alone relevance. Indeed, the anodyne phrase that seems to end every media story about medical research, ‘more research is needed’ [subtext, please give generously], really has no place in medical education. I for one, would wish people would stop publishing papers, and instead join the dots about what we already know. It is as though we collect yet more and more studies linking smoking with poor health outcomes with smaller and smaller odds ratios, all published in journals that nobody reads, and then continue to dole out the Capstan full strength (untipped, of course) to our students. In any case, as in many walks of life, change in this area is unlikely to come from the academy, but from outside. Follow the money. Many years ago, Sydney Brenner, one of the handful of scientists who made the revolution that is modern genetics, wrote an article in Science. The series theme was the influence of Science on Society, but Brenner, true to form, looked at things differently, writing instead about the (largely pernicious) effects of society on science. Universities take note: our laboratories are not as fecund as Sydney Brenner’s.
If one thing surprises me about what I read and what I want to read — because I think it important — it is the irrelevance of much of the scientific establishment to some of out problems, a statement that I speak with some shame. Kenneth Galbraith commented (from memory) that ‘the denigration of value judgement is one of the way the scientific establishment maintains its irrelevance’. I loved the phrase when I read it but couldn’t quite ‘get it’. I do now. Tim Wu’s account of the communication industry makes no sense without the successive waves of technical advance that underpin that change. On the other hand, we learn that the way society and corporations chose to behave limited advance, and closed off ways to maximise human potential. The institutions we create, or allow to exist, all seem to me more critical than ever — for good or bad. The lesson for anybody brought up of natural science, that somehow it is all like the hero in Arrowsmith, that somehow magical discoveries by disinterested scientists will solve all our problems, is magical itself. Magical, as in mythical, or just plain untrue. Look at Jack Scannell’s work on drug pricing for a start, on why institutions as well as molecules are important.
Which sort of brings me to something else that I think important, and that is Larry Lessig’s work on institutional corruption which I have only just come across, despite having read some of his books:
“Institutional corruption is manifest when there is a systemic and strategic influence which is legal, or even currently ethical, that undermines the institution’s effectiveness by diverting it from its purpose or weakening its ability to achieve its purpose, including, to the extent relevant to its purpose, weakening either the public’s trust in that institution or the institution’s inherent trustworthiness.”
He explains:
The field of “institutional corruption” was launched to help ethics grow up.
‘To help ethics grow up‘. I wish I had said that. GMC please note.
In the final chapter of the ‘Ascent of Man’, titled ‘The long childhood’ Jacob Bronowski after 400 pages on science and the culture of science, returns to a theme he has written about elsewhere, namely the relation between integrity, science and society. Bronowski was an optimist, and yet here he is wondering if Shakespeare and Newton will become historical fossils, in the way that Homer and Euclid are now. I feel less certain in 2016, than I did at the beginning of 2015. Hopeful, but less optimistic.
Statistics, reliable knowledge and medical education (‘crisis’ 1 of 2)
It is not a real crisis, but perhaps not far from it. People have looked upon science as producing ‘reliable knowledge’, and now it seems as though much science is not very reliable at all. If it isn’t about truth, why should we consider it special? Well, a good questions for an interested medical student to think about. But hard to do so. Part of the answer lies with statistical paradigms (or at least the way we like to play within those paradigms), part with the sociology and economics of careers in science, and part with the means by which modern societies seek to control and fund ‘legitimate’ science. Let me start with a few quotes to illustrate some of the issues.
A series of simple experiments were published in June 1947 in the Proceedings of the Royal Society by Lord Rayleigh–a distinguished Fellow of the Society–purporting to show that hydrogen atoms striking a metal wire transmit to it energies up to a hundred electron volts. This, if true, would have been far more revolutionary than the discovery of atomic fission by Otto Hahn. Yet, when I asked physicists what they thought about it, they only shrugged their shoulders. They could not find fault with the experiment yet not one believed in its results, nor thought it worth while to repeat it. They just ignored it. [and they were right to do so]
The Republic of Science, Michael Polanyi
[talking about our understanding of obesity] Here’s another possibility: The 600,000 articles — along with several tens of thousands of diet books — are the noise generated by a dysfunctional research establishment. Gary Taubes.
“We could hardly get excited about an effect so feeble as to require statistics for its demonstration.” David Hubel, Nobel Laureate (quoted in Brain and Visual Perception)
The value of academics’ work is now judged on publication rates, “indicators of esteem,” “impact,” and other allegedly quantitative measures. Every few years in the UK, hundreds of thousands of pieces of academic work, stored in an unused aircraft hangar, are sifted and scored by panels of “experts.” The flow of government funds to academic departments depends on their degree of success in meeting the prescribed KPIs [key performance indicators]. Robert Skidelsky
Why medical professionalism doesn’t matter.
Around 20 years ago my father was admitted to a major teaching hospital in Wales. He was in his early 70’s and had heart failure. He was under the care of the ‘general medics’, on a general medical ward. He became mildly confused after admission, and within a couple of days had fallen in the bathroom, and was developing red areas on pressure points, a harbinger of pressure sores. I remember talking to one of the nurses, who was apologetic that there was no possibility of getting a suitable bed (‘there isn’t the money’), and that he had fallen when he should have been supervised. The poor nurse was literally run off her feet, a couple of nurses trying to cope with a score or more of patients. I was probably fairly cross, and not concealing it well—but so was she, reasonably so. She and I both knew how things could be made better. Both of us found it uncomfortable, because both knew that key decisions about care are usually made by people who don’t see patients, or have first hand knowledge of the ‘front-line’ (remember the Ballad of Reading Jail: prisons have walls, not so that convicts can’t escape, but so that God cannot see what goes on inside).
Continue Reading
How do we educate students about how real discovery works?
I was musing over this article, party because it is a longstanding interest of mine—how do we acquire useful new knowledge— but also in the context of this blog on medical education, how do we get across to students how medical advance has occurred. Without getting into the ‘what can we learn from history’ subroutine, I think the topic important, and one that we cannot assume students will learn to think deeply about, without some guidance or prompting. To my mind, the role of education here is the classic one: as a detergent to propaganda.
The editorial describes changes at the mental health part of NIH (NIMH) in which the new director has made clear that to be funded, clinical trials have to include some test of the underlying biological mechanism. The line is that too many trials are black-box tests, in which if the results are negative, nothing is learned. ( It is suggested that 50% of the studies currently funded, would not be funded if submitted as new grant proposals at some future time). I think they are targeting the sort of pragmatic NHS style RCT which I find so depressing. The reason is simple: without a construct, or a genuine scientific hypothesis (and I do not mean a statistical one), we have no idea whether the conclusions of any study will apply at any future time, or in any other population. And a cognate fact is that we know trials are noisy and often unreliable guides to what is going on. Fisher warned of this nearly a century ago: we tend to try and rely on statistics when we know little about what is really going on, when what we need is more thinking, and much more repetition.
As it is, we often capture very little of the routine clinical encounter in many clinical trials. They are guides to what is going on, not rules to tell us how to behave. If the effects are very large, we can perhaps ignore much of this. But so often, we only conduct large studies, because the effect sizes are so small. If we think about it in simple terms, the R2 values are far too low: most of the variance is random, and unexplained. They are not good experiments and, it is no surprise, that we are finding out that so many papers published are wrong. Most RCT do not present the information this way, because it would be apparent we know little about what will happen to our patients. Not always, however. Systemic retinoids for acne have, I suspect, an NNT of close to 1. But even there, we had a clear demonstration of efficacy, before we had much insight into mechanism —but researchers went searching to complete the circle—not to the next RCT in a different domain.
However, what the editorial dismisses, is what I would most want to get across to students. The article (discussing psychiatric drugs) states that:
By the early 1990s, the pharmaceutical industry had discovered — mostly through luck — a handful of drug classes that today account for most mental-health prescriptions.
This is a real travesty, and again supports my adage that Nature doesn’t really understand medical research or medicine. Many of the leads were not luck, but the results of astute clinicians / pharmacologists interested in what happened to their patients. Not so much those immersed in the use of rating scales, or obsessed by assuming anything interesting must be due to chance, but acute observers that provided insights worth following by pharma. Calling this luck, is like saying Charles Darwin was just lucky (although I would accept Wallace was deeply unlucky). This for me is just another representation of the master clinician, one whose expertise is based around a knowledge of patients, and who thinks about what happens to them. This is a style of medicine we are in danger of losing. Students should know that the obsession of thinking about patients is what underpins and drives clinical advance. This is not at the expense of sensible clinical experiments, or wet-bench work, just an acknowledgement that medicine has its own intellectual heartlands, and we need to communicate this to the next generation, because it is in danger of being killed off by a pincer movement of ‘protocols’ on the one hand, and a confusion that biology and medicine are synonymous, on the other. As far as discovery in psychiatry is concerned, few can equal David Healy for explaining how we got where we did. For some other areas, see what I wrote in Science over 10 years ago. The problem for the undergraduate teacher is how to integrate real knowledge of statistics and experimental design, with a knowledge of how genuine clinical advance occurs.
Not enough hours in the day
A long, long time ago, I was sitting in the biochemistry coffee room in the medical school in Newcastle. Roger Paine, a professor of biochemistry came and sat next to me. I knew of him, but he didn’t know me. He was a FRS, I was a dermatology research registrar taking my first steps in learning some wet bench science in the Medical Molecular Biology Group there. Coffee rooms work, as do Aeron chairs. Sometimes you need to talk, and ramble around what interests you; and sometimes you have to sit alone, and dream. If you don’t, you will do ‘kit’ science, or act out being an administrator by conducting randomised controlled trials.
We got chatting—we shared a mutual colleague—and he expressed his puzzlement to me about how medics managed to do any research. He pointed out what with seeing patients, and some undergraduate teaching and postgraduate training, how on earth could you hope to do any meaningful research. I listened, not wanting to hear what he said. And I should point out, he was a keen collaborator with medics, nor stand-offish in any way.
Many years later, in another setting, I was talking to another successful scientist, a geneticist, also a FRS. We knew each other reasonably well, and by this stage I had been working in wet-bench science for a dozen years or more. Some modest successes, and plenty of failures. He told me that because he knew the details of many clinical medics research careers very well, he would be loathe to ever approach any of them if he needed medical care. He had the highest regard for them as academics, and researchers, but he too couldn’t see how they could carry on all the various activities expected on them. (And no doubt be able to go to the cinema once in a while: Steven Rosenberg, a one time Chief of Surgery at NIH, in his autobiography, describes how he would struggle to leave Sunday evening free of lab and clinical duties, so that he could go to the cinema with his wife).
Continue Reading
What the academy can learn from Hollywood.
The NYT offers short videos most days. I enjoy watching them, partly because I am trying to produce some very humble efforts for student teaching. I am keen to learn. I am gradually finding my way through FCPX, audio recording and how to produce simple animations. One of today’s videos is about the sound studio Skywalker Sound. Some of what is said is not surprising. Most us know how sound influences our degree of fear in scary films, and how sound and music sets action in context. And yet, the degree of sophistication and invention does surprise me. Films are very complicated giant artefacts, the result of large teams working collectively, but with a mixture of authority, vision, and emergence. Contrast this with the novel, or even the modern textbook. In the former there is a single creator at work, and accepting the need for publishers, typesetters and so on, the cast is small. Textbooks might involve more staff in their creation but, in general, I do not think most textbooks are as sophisticated or skilful as most films. A course module might not stand comparison either.
So, what has this got to do with medical education? Well, in an earlier post on the importance of design, I described my own (middle or late-life?) epiphany. In medicine the idea that you just string modules together, with lecturers who have rarely sat down together, all producing their own little snippets, is no longer sensible. A bit like trying to make sense of a William Burroughs novel. Asking externals to come to exam boards rather than being involved in the development of course material is another reflection of a broken system. So, whilst in many disciplines, an individual lecturer might produce a series of lectures, and students may indeed get used to the style, feedback and so on, for medicine I do not think this system is appropriate. Medicine is, by its nature multimedia, but is frequently delivered by people who have little oversight of what students are supposed to know. The origins of this are of course in the apprentice system: whereas postgraduate education can follow this model to a limited degree (although the various NHSs are doing their best to destroy it), much of undergraduate medicine is still sadly bums on seats in lecture theatres. Depressing, given how much the kids are paying. We need the equivalents of sound teams, video teams, animators, support staff etc. And stars!
Continue Reading
Jorge Luis Borges and Learning outcomes
Whenever I hear or read the phrase ‘learning outcomes’ I think of the story ‘On Exactitude in Science’ by Jorge Luis Borges. It is a short story, very short in fact, coming in at fewer than 150 words. So the danger in attempting to describe what it is about, is that you use more words than Borges himself. The hazards of summary or précis is of course part of its subject. So here it is:
…In that Empire, the Art of Cartography attained such Perfection that the map of a single Province occupied the entirety of a City, and the map of the Empire, the entirety of a Province. In time, those Unconscionable Maps no longer satisfied, and the Cartographers Guilds struck a Map of the Empire whose size was that of the Empire, and which coincided point for point with it. The following Generations, who were not so fond of the Study of Cartography as their Forebears had been, saw that that vast Map was Useless, and not without some Pitilessness was it, that they delivered it up to the Inclemencies of Sun and Winters. In the Deserts of the West, still today, there are Tattered Ruins of that Map, inhabited by Animals and Beggars; in all the Land there is no other Relic of the Disciplines of Geography.
—Suarez Miranda,Viajes de varones prudentes, Libro IV,Cap. XLV, Lerida, 1658
By learning outcomes I simply mean stating what you expect students to know or be able to do. I suspect there are lots of exegeses in the academic literature, but I assume this definition will suffice. In the context of medicine it seems especially important to be able to tell students what you expect them to know (for the record I do not believe that this is a sensible strategy in all contexts, just most).
Continue Reading
Teaching statistics to medical students
The situation was a familiar one. Some time back, I was gossiping to a medical student, and he began to to talk about some research he had done, supervised by another faculty member of staff. I asked what he had found out: what did his data show? What followed, I have seen if not hundreds of times, then at least on several score occasions. A look of trouble and consternation, a shrug of embarrassment, and the predictable word-salad of ‘significance’, t values, p values, statistics and ‘dunno’. Such is the norm. There are exceptions, but even amongst postgraduates who have undertaken research, the picture is not wildly different. Rarely, without directed questioning, can I get the student to tell me about averages, or proportions, using simple arithmetic. A reasonable starting point surely. ‘What does it look like if you draw it?’ is met with a puzzled look. And yet, if I ask the same student, how they would manage psoriasis, or why skin cancers are more common in some people than others, I get —to varying degrees—a reasoned response. I asked the student how much tuition in statistics they had received. A few lectures was the response, followed by a silence, and then, “They told us to buy a book”. More silence. So this is what you pay >30K a year for? The student just smiled in agreement. This was a good student.
Statistics is difficult. Much statistics is counter-intuitive and, like certain other domains of expertise, learning the correct basics often results in a temporary —or in some cases a permanent —drop in objective performance.** That is, you can make people’s ability to interpret numerical data worse after trying to teach them statistics. On the other hand, statistics is beautiful, hard, and full of wonderful insights that debunk the often sloppy thinking that passes for everyday ‘common sense’. I am a big fan, but have always found the subject anything but easy. But, like a lot of formal disciplines, the pleasure comes from the struggle to achieve mastery. I also think the subject important, and for the medical ecosystem at least, it is critical that there is high level expertise within the community. On the other hand, in my experience many of the very best clinicians are (relatively) statistically illiterate. The converse is also seen.
I didn’t understand what the word design meant until I read Herb Simon’s ‘Sciences of the Artificial’. I don’t mean this literally of course. I knew architects designed buildings, and graphic artists coloured the world I lived in. Painters painted. Musicians composed. Quavers got ordered. Colour got rearranged. But mainly I saw the world in terms of taking it apart. I lived in a largely man-made world, but was invisible to how it was done. When my eyes opened in the morning, it was all there,just like the birds. I remember, when I was a student at Newcastle University, a friend from school, who was supposed to be studying fine art, taking advantage of cheap student rates to go to the cinema on a Wednesday afternoon. He would watch the same film again and again. Over and over. I didn’t really understand what was going on. The only film I had watched repeatedly in one sitting was “Let it be’ by the Beatles ( in the Cardiff Odeon I think). Now to anybody of my generation watching the Beatles time and time again, does not appear so strange. But why watch film X again and again? Surely you knew the plot and, unless the film fulfilled some sort of comfort role, what was the point. You knew what was going to happen. (But music?).
Continue Reading