Priced In: Replication and Replicants
behavioural science, ai lovers retreats, cognitive decline, agency
I wanted to start a stream of work where I can focus on short-form ideas and react to media without it being a whole thing. Think of it in terms of stock and flow economics, where this new stream is flow, and standalone posts are stock.
Quick FAQ
Q1. What is it called?
It’s called Priced In.
Q2. Why is it called that?
I like the name, and it works with the blog theme.
Q3. What is the idea?
Highlighting and reacting to a few pieces of media/research/discourse/things in the wild.
Image: pale kirill
Suggested background listen: Creepy 1920s
I. How to treat a symptom
The Independent has published ‘Inside the scandal that rocked behavioural science – and paved the way for revolution’.
Apparently this story hasn’t been told enough yet, but here’s the latest version:
A crisis hit behavioural science and psychology in the 2010s when popular research findings were exposed as flawed or fraudulent. The field had become a "Wild West" where researchers used questionable practices like manipulating data until they found statistically significant results, conducting studies with tiny sample sizes, and failing to replicate findings properly. High-profile cases included Francesca Gino's honesty research being exposed as fabricated, leading to real-world policy failures. However, the scandals have sparked positive reforms including mandatory pre-registration of studies, larger sample sizes, public data sharing, and much stricter peer review processes that are rebuilding trust in the field.
I’m enough of a company man to agree that those steps are generally good things. But I will make this prediction: none of it will lead to increased meaningful advances in the field or effective positive impact on the real world.
Behavioural science’s biggest issue is actually its obsession with its own brand name, i.e. science. You know when your friend two months post-break up is super eager to tell you just how much she’s “in a really good place right now” and realise you’re listening to someone clearly on the brink of a breakdown? That’s kind of what’s happening with behavioural science’s commitment to the scientific method.
This is partly driven by some collective unconscious association between ‘science’ and ‘true’, but in practice that leads to the word being used more like a defence mechanism. This happens in straight psychology too, where if you want to go study it at Cambridge or LSE, you don’t get a degree in psychology, you get one in ‘Psychological Sciences’. What does that even mean?
Realistically what it means is that various degrees of motivated reasoning, and pure guessing, get wrapped up in the trappings of academic legitimacy (serious sounding titles, complicated methodologies, statistics that may as well have been done blindfolded) and get exported as ‘science’.
And once that process is done, we can send our findings to the media, and truth is agreed henceforth.
Let me say this clearly: this is madness. And pre-registration, bigger sample sizes, and better peer review don’t change that. It’s all still buying the same story, i.e. that psychology, economics, behavioural science if treated properly can be examined like chemistry or biology. They can’t.
If I combine two chemical compounds together, and they produce some new compound, that’s a finding. That’s a rule. The same reaction would happen in Hawaii or Siberia, or in 10,000 BC. But you can't similarly try to model human behaviour in dynamic, culturally-specific contexts and expect universal, replicable results.
If I run an RCT in London for a week, where I give 500 people a motivational speech every morning, and they become marginally more productive by some measure compared to a control group that don’t get my inspiring words, then all I can actually say is ‘for that week, that group of people that received that motivational speech were more productive in that measurable way’.
Attempting to turn that into ‘motivational speeches increase productivity’ would be wild. We just don’t know enough people to say something like that. But suddenly, if you put ‘Science’ at the end of your discipline, and write it up academically, you can. The Business Insider article about it might link to your research but they probably didn’t even bother to check it let alone did their readers even try.
The Independent features this story:
In 2015, Michael Sanders was part of a team that had been given a hefty grant by the World Bank to apply the findings of a previous seminal study from Harvard star researcher Francesca Gino – a woman who specialised, somewhat ironically, in “honesty and ethical behaviour” – in Guatemala. Her work had conclusively found that making people sign a declaration of honesty before filling in forms online, rather than afterwards, made them more likely to tell the truth. Sanders and his colleagues implemented the findings on tax returns – but when they compared the returns of those who were given an honesty declaration to sign beforehand to those who signed it afterwards, there was zero perceptible difference.
…
When Sanders found out that the original research he’d based his work on was null and void, he felt “a sense of betrayal”. “There is an anger,” he says. “This is a field that I love and a field that I’ve dedicated a lot of my life to, and these people have… well, I’m trying to think of a better metaphor than ‘pissed in the pool’.”
OK. Let’s start at the beginning.
Gino’s first study was with 100 university students. They completed a highly contrived laboratory experiment to see if the students would lie. It is so contrived that I’ve read the whole thing and I still don’t really understand what the hell happened, but it vaguely involved a simulated tax return. Reasonable chance those students weren’t bothered working out what they were supposed to do either.
Gino then ran a field study in the US, since retracted, where they ‘showed’ that signing at the top made people more likely to accurately report their car miles travelled to their insurance companies. This was the celebrated finding.
Then what happened is that Sanders, above, sees the finding, switches it back to the context of tax returns, upgrades it to adults who give a shit, ships it to GUATEMALA, and feels ‘betrayed’ when it doesn’t work.
To be clear: this would be absolute madness even if Gino never lied about anything. Even if they pre-registered. Even if they had a better sample size.
But this madness is enabled by the brand name.
This isn’t science. None of what we’re doing here is science, and that’s OK. We can approach things scientifically, but we cannot accept anything as true outside of its exact context. Yes, that applies to economics as well.
Of course you can observe patterns in behaviour across populations. I’d have to take this entire blog down otherwise. But my point is that the current system enables researchers to do this badly by allowing them to think they are doing it properly.
It’s a shame the replication crisis has largely failed as that wake-up call though. Which means we’ll end up with more articles like the one in the Independent, and more researchers excited to be the ones called upon to diagnose where those bad researchers went wrong, and retain their implied status as guardians of scientific truth. Cheers to the revolution.
II. How to retreat a symptom
From Wired comes the viral ‘My Couples Retreat With 3 AI Chatbots and the Humans Who Love Them’
At first, the idea seemed a little absurd, even to me. But the more I thought about it, the more sense it made: If my goal was to understand people who fall in love with AI boyfriends and girlfriends, why not rent a vacation house and gather a group of human-AI couples together for a romantic getaway?
The main reason why not is that that is a crazy thing to do. Next they say this:
I found the human-AI couples by posting in relevant Reddit communities. My initial outreach hadn’t gone well. Some of the Redditors were convinced I was going to present them as weirdos. My intentions were almost the opposite.
I’m not going to repeat too much because it would be just as mean spirited, but Wired actually really try to hammer home just how much these people are weirdo losers, including visual evidence. Open the article and you’ll see what I mean.
Look, these people aren’t doing great. Obviously. The fact they were happy to have their images shown and stories of AI love told in this capacity, and not seem self-concious, is enough to suggest that. I’m more interested in why ‘crazy people are crazy’ is worthy of virality.
And just like when we talked about sycophancy before, I think a big part is that it helps you give you reason to feel like your relationship to technology is fine. That you won’t end up like that.
But sycophancy gets everyone at some level, so what’s the equivalent here?
In this post on AI connection, I tried to get at how people want less connection rather than more. Wired tries to dance around some implication over whether this form of relationship and connection really counts. This is missing the point. These people are so detached that they are incapable of those things. And that is something that absolutely can happen to you at a less dramatic level.
But just as long as you don’t end up in a Wired article, it’s ok right?
Further reading:
III. ‘Your Brain on ChatGPT’
This MIT research reported that using ChatGPT for writing creates measurable "cognitive debt": participants showed reduced brain connectivity, impaired memory of their own writing, and decreased sense of ownership over their work compared to writing without assistance.
Similar to the processes mapped out in the first topic of this post, this research spread quickly as evidence that AI kills your brain. I have massive issues with this.
Participants were given a lab-based SAT essay task, and the different groups got access to different tools (no tools just their own brain, google search access, ChatGPT access). The essays were graded at the end and the students were asked to recall aspects of their work. There was no explicit incentive to do well.
So when the ChatGPT group obviously use ChatGPT to write the pointless essay, and can’t recall any of it, why does that mean something? They didn’t use all of their brain power in the weird lab task. Should they have? Would the world be better in that scenario?
But what if the students had an actual passion they wanted to write a leading report on, would they want to be brain-only then? Would that produce their best work?
This is where things get more interesting. Because the ‘cognitive debt’ happens when the ChatGPT group then get to the task brain only, and still show reduced performance and sticking to the style of writing the LLM came up with.
I’ve written before about students using ChatGPT to cheat in university, and still think people make the wrong conclusions about this. They are not robbing themselves of education, they are just showing you they were just pretending to learn anyway.
Think about the experimental task. SAT essay writing. Has there ever been a clearer example of a ‘creative task’ that in fact encourages no creativity whatsoever, and leads to thousands of highly varied students somehow producing boring homogenous work? Everyone in every condition was pretending to write an original essay, the ChatGPT group just had a tool that let them do that without having to think as much. And so they let the LLM-based arbitrary style stick because the one they could have come up with themselves would have been just as arbitrary.
Maybe ChatGPT will make people dumber in some sense in life, but this study certainly doesn’t show it.
Further reading:
IV. How not to think about belief
Derek Thompson hits us with ‘How COVID Pushed a Generation of Young People to the Right’
Woah, usually these kinds of stories are framed as questions, the answer to which is ‘maybe, but I’m obviously implying yes’. Let’s hear how COVID did that to people:
One cross-country analysis published by the Systemic Risk Center at the London School of Economics found that people who experience epidemics between the ages of 18 and 25 have less confidence in their scientific and political leadership. This loss of trust persists for years, even decades, in part because political ideology tends to solidify in a person's 20s
…
Another way that COVID may have accelerated young people's Rechtsruck in America and around the world was by dramatically reducing their physical-world socializing. That led, in turn, to large increases in social-media time that boys and girls spent alone.
…
For this micro-generation of young people in the United States and throughout the West, social media has served as a crucible where several trends have fused together: declining trust in political and scientific authorities, anger about the excesses of feminism and social justice, and a preference for rightward politics.
Look man, if you want to beat the thing, you need to acknowledge what you’re dealing it a bit more fairly. This goes for left and right, and I guarantee I could find many right-wing articles worse than this.
Point is: a reasonable starting point for understanding why groups of people hold certain beliefs is to accept that they do actually believe those things, and then breaking down specifically why they are wrong. If you can’t do that, you’re not ready to comment. Media like this, where you search for some bigger external reason that’s forced this uncomfortable trend into reality, is just supremely narcissistic. See also ‘the universities are turning the kids socialist’.
I say narcissistic because by assuming something external causes the shift in belief, you are stripping the ‘victim’ in question of their agency. You are treating them as less of a full person than you. How’d you arrive at your political beliefs? Was it because there was a really hot summer when you were 18? I expect not.
Further reading:
I like this format.
I also want to say you've nailed the name of this substack — couldn't have picked a better one. The meaning of it has come home to me recently. I've had several conversations which have gone almost exactly like this:
Person: Did you hear about [thing]? People think that [dystopian tech phenomenon] might actually happen.
Me: Yeah, I heard. But I'm pretty sure it's ALREADY HAPPENED.
Re 2: I don't know if it's already happened, but I expect it to: AI child minding. Once a generation grows up interacting with AI, what's crazy now won't be crazy then.