Teaching statistics to medical students
The situation was a familiar one. Some time back, I was gossiping to a medical student, and he began to to talk about some research he had done, supervised by another faculty member of staff. I asked what he had found out: what did his data show? What followed, I have seen if not hundreds of times, then at least on several score occasions. A look of trouble and consternation, a shrug of embarrassment, and the predictable word-salad of ‘significance’, t values, p values, statistics and ‘dunno’. Such is the norm. There are exceptions, but even amongst postgraduates who have undertaken research, the picture is not wildly different. Rarely, without directed questioning, can I get the student to tell me about averages, or proportions, using simple arithmetic. A reasonable starting point surely. ‘What does it look like if you draw it?’ is met with a puzzled look. And yet, if I ask the same student, how they would manage psoriasis, or why skin cancers are more common in some people than others, I get —to varying degrees—a reasoned response. I asked the student how much tuition in statistics they had received. A few lectures was the response, followed by a silence, and then, “They told us to buy a book”. More silence. So this is what you pay >30K a year for? The student just smiled in agreement. This was a good student.
Statistics is difficult. Much statistics is counter-intuitive and, like certain other domains of expertise, learning the correct basics often results in a temporary —or in some cases a permanent —drop in objective performance.** That is, you can make people’s ability to interpret numerical data worse after trying to teach them statistics. On the other hand, statistics is beautiful, hard, and full of wonderful insights that debunk the often sloppy thinking that passes for everyday ‘common sense’. I am a big fan, but have always found the subject anything but easy. But, like a lot of formal disciplines, the pleasure comes from the struggle to achieve mastery. I also think the subject important, and for the medical ecosystem at least, it is critical that there is high level expertise within the community. On the other hand, in my experience many of the very best clinicians are (relatively) statistically illiterate. The converse is also seen.
As a medical student, I think my medical statistics teaching comprised 2 or 3 lectures in the first year. I can still vividly remember a MCQ question in the end of year exam that was clearly trying to get at the issue of whether we understood the relation between sample size and the standard error of the mean. Along the lines of: would the variance alter with an increase in N, would the SEM alter with an increase in N, and so on. I cannot remember my answer, but I know that I tried to imagine that I could ‘see’ the answer based on some visualisation of the raw formulas. Not a sensible way to think about it. And the familiar problem the physicists cope with hence the need for the Force Concept Inventory : ‘good students’ can plug numbers into formulas, but still not understand the (deep) concepts. (I think the MCQ was a good question in this regard).
Things changed when I did an intercalated year. I took this in between fourth and final year, so after I had done some clinical medicine. The year broke down into three months intensive teaching, and nine months in which you had to do your research, present two seminars, and write a full length thesis (as in MD or PhD length). It was a lot of fun, but the three month intensive tuition is what is relevant to this post. There were 12 of us in the year, and this class was in turn split into two groups of 10 and 2. For statistics, we all came together, so n=12.. Most of the time the class I was in (n=2) received a couple of teaching sessions each day on research methods, computer programming, epidemiology and health economics. This was the social / epidemiology option. The other 10 were doing the ‘biomedical’ option. I was convinced I go the better deal: their class size was 10, mine only 2.
The tuition in statistics was made up of some lectures to the main group(n=12) and, from memory, one or two weekly statistics practicals. Each of the practicals lasted three hours, and there were 2 or 3 members of staff in attendance. David Newell , then professor of medical statistics at Newcastle, would make tea for us at half time. He advised us that it was most unlikely (he was of course a statistician) that this would ever happen again. He was right.). Although we had learned very elementary programming (Fortran, and APL, on an IBM 360 mainframe running MTS), the practicals were based on us using a Texas calculator. All those shortcuts for calculating the SD, as there was no SD function button. Graphs sketched by hand etc.
For my research project, an analysis of proximal femoral fracture rates in the Northern region, all this teaching came together. I had to analyse a dataset of just under 500,000 records, using simple FORTRAN programs and then build a simple linear model with Poisson errors using the package GLIM. I remember being sent to the computing lab (no WWW, you had to walk to look up the manuals) to try and match what I think I wanted to do, with the intricacies of the software tool. I remember wondering if I was reading the GLIM manuals upside down; was I on the classics module? But there was lots of expertise on hand, and freely given. What resulted was a single author paper in the BMJ . How times have changed: no REF then.
From the perspective of my research career, the teaching in statistics and research methods (largely about bias in observational studies) has been the most critical teaching I have ever received. I think you can learn much cell biology from many of the texts available; I think you can learn much of modern genetics from books, and the informal environment a lab provides; much clinical research you can imbibe without pain. But for people like me, intensive teaching —and subsequent reading— in statistics was critical. From the perspective of my academic career, my statistics teaching was as foundational, and as basic— if not more basic— than anything else you could learn at medical school. Or not learn. Which brings me back to the student I was gossiping too.
A few weeks back I was talking to an academic who has responsibility for teaching stats to non-statisticians in the biological sciences at a leading UK university. I recalled my experiences. How should we improve things? It was clearly a subject close to his heart. Straightaway, he admitted he had been terrible at statistics as an undergraduate, but that was now possibly an advantage. There were some simple strategies. First, no Powerpoints, just a black or white board. No handouts allowed. Second, you have to follow up lectures with repeated small group teaching, and practicals. Third, emphasise graphical explanations. Fourth, the subject needs to be mainly delivered by those from the parent discipline: for medicine, that meant medics.
The latter point, is relevant to a post by Andrew Gelman in his excellent blog ‘Statistical Modelling, Causal Inference and Social Science’. He is writing about the ‘Frontiers of Science’ course run as part of Columbia’s core curriculum. Gelman says :
But then I remembered that our intro stat course isn’t so great (I know, I’ve taught it several times), so maybe it’s just as well if some biologists, physicists, etc., create a new statistics module from scratch. Seriously, I have lots of ideas of how we could teach intro prob/stat better, but when I actually try to do it, I get all tangled in the details. So I can’t very well object to outsiders taking a shot at it. As users of statistics, they might have a better idea than I do of how to teach the subject.
Similarly, I wouldn’t be surprised if the engineers could teach a better intro physics class than the physicists could. And it could well make sense to have biologists teach first-year chemistry, and psychologists teach first-year biology. What, then, would the statistician teach? First year math, of course.
But Gelman also talks a little about MOOCs, and of course the real problem MOOCs highlight is that some (or much?) university teaching is based on large lectures and Powerpoint presentations, without exercises, or small group high level interaction. For statistics for medical students it seems to me that what is often provided is little better than a MOOC, something students could get for free. But I am sceptical MOOCs would work for the majority of medical students in this context. There are other online experiments dealing with statistics such as the Carnegie Mellon experiments, but quite frankly their course combines the best of both worlds, and the course seems much richer than the on-campus teaching many students receive ( my take on this, is the account in William G Bowens book ‘Higher education in the Digital Age’ and the ITHAKA website).
Finally, I do not think this problem is confined to medical students, although my experience is very limited out with this group. Some BSc students in the biological sciences seem poor —and know they are poor— at basic stats. Psychology students you would think would be much better, although colleagues at other UK universities tell me that lack of statistical competence amongst psychology undergrad students is a problem that few want to talk about.
The bottom line is that I think we do know how to teach students better, but choose not to. Andrew Gelman and the OLI website spell out some of the reasons why. Part of the answer is simple: there is a relation between n and the SEM, and there is also a relation between n and learning.
** I do not know whether it was David Newell or David Appleton who framed the following law. Teaching formal statistics to medical students results in a reduction in the mean ability of those students to interpret numerical data. Medical students at 18 are often better at interpreting graphs of simple experiments than they are after they have been ‘taught’ statistics. Most of them never recover this ability, much as many children lose their sense of pitch, and become tone deaf. A minority will of course develop higher level skills.
2 Comments