The danger isn’t that AI destroys us. It’s that it drives us insane
The danger isn’t that AI destroys us. It’s that it drives us insane
Although many of the digital gurus started out as idealists, to Lanier there was an inevitability that the internet would screw us over. We wanted stuff for free (information, friendships, music), but capitalism doesn’t work like that. So we became the product – our data sold to third parties to sell us more things we don’t need. “I wrote something that described how what we now call bots will be turned into these agents of manipulation. I wrote that in the early 90s when the internet had barely been turned on.” He squeals with horror and giggles. “Oh my God, that’s 30 years ago!”
The pleasure of electronic medical records
Digital Minimalism — An Rx for Clinician Burnout | NEJM
It’s easy to see how digital minimalism’s first tenet, “clutter is costly,” applies: the average patient’s EHR has 56% as many words as Shakespeare’s longest play, Hamlet. Moreover, half these words are simply duplicated from previous documentation.
Progress isn’t inevitable. Not even likely in some domains over working lifetimes.
End of an error
The SVB debacle has exposed the hypocrisy of Silicon Valley | John Naughton | The Guardian
The first thing to understand is that “Silicon Valley” is actually a reality-distortion field inhabited by people who inhale their own fumes and believe they’re living through Renaissance 2.0, with Palo Alto as the new Florence. The prevailing religion is founder worship, and its elders live on Sand Hill Road in San Francisco and are called venture capitalists. These elders decide who is to be elevated to the privileged caste of “founders”.
Error? Era? Hope so.
Arithmetic matters
From a review of “The Ascent of Information” by Caleb Scharf.
Every cat GIF shared on social media, credit card swiped, video watched on a streaming platform, and website visited add more data to the mind-bending 2.5 quintillion bytes of information that humans produce every single day. All of that information has a cost: Data centers alone consume about 47 billion watts, equivalent to the resting metabolism of more than a tenth of all the humans on the planet.
Scharf begins by invoking William Shakespeare, whose legacy permeates the public consciousness more than four centuries after his death, to show just how powerful the dataome can be. On the basis of the average physical weight of one of his plays, “it is possible that altogether the simple act of human arms raising and lowering copies of Shakespeare’s writings has expended over 4 trillion joules of energy,” he writes. These calculations do not even account for the energy expended as the neurons in our brains fire to make sense of the Bard’s language.
There was an article in the FT last week, commenting on an article in JAMA here. The topic is the use of AI (or, to be fair, other machine learning techniques) to help diagnose skin disease. Google will allow people to upload their own images and will, in turn, provide “guidance” as to what they think it is.
I think the topic important, and I wrote a little editorial on this subject here a few years ago with the strikingly unoriginal title of Software is eating the clinic. For about 8-10 years I used to work in this field but although we managed to get ‘science funding’ from the Wellcome Trust (and a little from elsewhere), and published extensively, we were unable to take it further via commercialisation. As is often the case, when you fail to get funded, you may not know why. My impression was that people did not imagine that there was a viable business model in software in this sort of area (we were looking for funds around 2012-2015). Yes, seemed crazy to me then, too (and yes, I know, Google have not proven there is a business model). Some of the answers via NHS and Scottish funding bodies were along the lines of come back when you prove it works, and then we will then fund the research.?
A few days back somebody interested in digital health asked me what I thought about the recent work. Below is a lightly edited version of my email response.
- Long term automated systems will be used.
- Tumours will be easier than rashes.
- A rate-limiting factor is access to — and keeping — IPR of images.
- The computing necessary is now a commodity and trivial in comparison with images and annotation of images
- Regarding uncommon or odd presentations, computers are much more stupid than humans. Kids can learn what a table is with n<5 examples. Not so for machines. Trying to build databases of the ‘rare’ lesions will take a lot of time. It will only happen cheaply when the whole clinical interface is digital, ideally with automated total body image capture (a bit like a passport photo booth), and with metadata and the diagnosis added automatically.
- In the short term, the Google stuff will increase referrals. They are playing the usual ’no liability accepted, we are Silicon Valley, approach’. They say they are not making a diagnosis, just ‘helping’. This is the same nonsense as 23and me. They offload the follow-up onto medics / health service. Somebody once told me that some of the commercial ‘mole scanners’ sent every patient to their GP to further investigate — and refer on to hospital —their suspicious mole. This is a business model, not a health service.
- Geoffrey Hinton (ex Edinburgh informatics) who is one of the giants in this area of AI says in his talks that ‘nobody should train as a radiologist’. He is right and wrong. Yes, the error rate in modern radiology is very, very high — simply because they are reporting so many ‘slices’ in scans in comparison with old-fashioned single ‘films’ (e.g. chest X-ray). But single bits of tech in medicine are often in addition to what has previously happened rather that a replacement. In this domain, radiologists now do a lot more than just report films. So, you will still need radiologists but what they do will change. Humans are sentient beings and, given the right incentives, doctors are remarkably creative.
- There is still a very narrow perspective on skin disease in the UK, with a continued denial of the necessity for expertise. Primary care dermatology by doctors — let alone nurses who know even less and do not have professional registration in this domain of expertise — is a mess. It requires perceptual skills, and not unreasonably, many GPs do not have this because they have to know so much across a broad area of medicine. The average GP will see 1 melanoma every ten years. I started my dermatology training in Vienna: there are more dermatologists in Vienna than the whole of the UK. Specialism, pace Adam Smith and the pin factory, is what underpins much of the power of capitalism — and medicine, too. (Note: in this regard, some of the comparisons in the JAMA paper are facile).
- I think the machines will play a role and we are better with them than without them. But in the short term, demands on hospital practice will increase and waiting times will increase even further. This technology — in the short to medium term — will make things worse. At present, dermatology waiting times are worse than when I was a medical student (and this was true pre-Covid). You don’t need AI to know why.
If only we had been funded…. ?. Only joking.
My experience is limited, but everything I know suggests that much IT in healthcare diminishes medical care. It may serve certain administrative functions (who is attending what clinic and when etc), and, of course, there are certain particular use cases — such as repeat prescription control in primary care — but as a tool to support the active process of managing patients and improving medical decision making, healthcare has no Photoshop.
In the US it is said that an ER physician will click their mouse over 4000 times per shift, with frustration with IT being a major cause of physician burnout. Published data show that the ratio of patient-facing time to admin time has halved since the introduction of electronic medical records (i.e things are getting less efficient). We suffer slower and worse care: research shows that once you put a computer in the room eye contact between patient and physician drops by 20-30%. This is to ignore the crazy extremes: like the hospital that created PDFs of the old legacy paper notes, but then — wait for it — ordered them online not as a time-sequential series but randomly, expecting the doc to search each one. A new meaning for the term RAM.
There are many proximate reasons for this mess. There is little competition in the industry and a high degree of lock-in because of a failure to use open standards. Then there is the old AT&T problem of not allowing users to adapt and extend the software (AT&T famously refused to allow users to add answering machines to their handsets). But the ultimate causes are that reducing admin and support staff salaries is viewed as more important than allowing patients meaningful time with their doctor; and that those purchasing IT have no sympathy or insight into how doctors work.
The context is wildly different — it is an exchange on the OLPC project and how to use computers in schools, but here are two quotes from Alan Kay that made me smile.
As far as UI is concerned — I think this is what personal/interactive computing is about, and so I always start with how the synergies between the human and the system would go best. And this includes inventing/designing a programming language or any other kind of facility. i.e. the first word in “Personal Computing” is “Person”. Then I work my way back through everything that is needed, until I get to the power supply. Trying to tack on a UI to “something functional” pretty much doesn’t work well — it shares this with another prime mistake so many computer people make: trying to tack on security after the fact …[emphasis added]
I will say that I lost every large issue on which I had a firm opinion.
Whenever I have looked at the CVs of many young doctors or medical students I have often felt saddened at what I take to be the hurdles than many of them have had to jump through to get into medical school. I don’t mean the exams — although there is lots of empty signalling there too — but the enforced attempts to demonstrate you are a caring or committed to the NHS/ charity sector person. I had none of that; nor do I believe it counts for much when you actually become a doctor1. I think it enforces a certain conformity and limits the social breadth of intake to medical school.
However, I did do things work outside school before going to university, working in a variety of jobs from the age of 14 upwards: a greengrocer’s shop on Saturdays, a chip shop (4-11pm on Sundays), a pub (living in for a while ?), a few weeks on a pig-farm (awful) and my favourite, working at a couple of petrol stations (7am-10pm). These jobs were a great introduction to the black economy and how wonderfully inventive humanity — criminal humanity— can be. Naturally, I was not tempted?. Those in the know would even tell you about other types of fraud in different industries, and even that people actually got awarded PhDs by studying and documenting the sociology of these structures (Is that why you are going to uni, I was once asked).
On the theme of that newest of crime genres — cybercrime — there is a wonderful podcast reminding you that if much capitalism is criminal, there is criminal and there is criminal. But many of the iconic structures of modern capitalism — specialisation, outsourcing and the importance of the boundaries between firm and non-firm — are there. Well worth a listen.
- See below ↩ for a long aside on this point.
I think there is a danger in exaggerating the role of caring and compassion in medicine. I am not saying you do not need them, but rather that I think they are less important that the technical (or professional) skills that are essential for modern medical practice. I want to be treated by people who know how to assess a situation and who can judge with cold reason the results of administering or withholding an intervention. If doctors were once labelled priests with stethoscopes, I want less of the priest bit. Where I think there are faults is in the idea that you can contribute most to humanity by ‘just caring’. The Economist awhile back reported on an initiative from the Centre for Effective Altruism in Oxford. The project labelled the 80,000 hours initiative advises people on which careers they should choose in order to maximise their impact on the world. Impact should be judged not on how much a particular profession does, but on how much a person can do as an individual. Here is a quote relating to medicine:
Medicine is another obvious profession for do-gooders. It is not one, however, on which 80,000 Hours is very keen. Rich countries have plenty of doctors, and even the best clinicians can see only one patient at a time. So the impact that a single doctor will have is minimal. Gregory Lewis, a public-health researcher, estimates that adding an additional doctor to America’s labour supply would yield health benefits equivalent to only around four lives saved.
The typical medical student, however, should expect to save closer to no lives at all. Entrance to medical school is competitive. So a student who is accepted would not increase a given country’s total stock of doctors. Instead, she would merely be taking the place of someone who is slightly less qualified. Doctors, though, do make good money, especially in America. A plastic surgeon who donates half of her earnings to charity will probably have much bigger social impact on the margin than an emergency-room doctor who donates none.
Yes, the slightly less qualified makes me nervous.