HN Reader

It was put forward in 1960s (maybe? Robert Anton Wilson? and for parallel purposes Philip K Dick's percept / concept feedback cycle) science fiction, and having therefore casually looked for phenomena when support / disprove this hypothesis over the intervening years: that people in power necessarily become functionally psychotic because people will self-select to be around them as a self-preserving / promoting opportunity (sycophants) who cannot help but filter shared observations through their own biases, this is profoundly unsurprising to me.

If you choose to believe as Jaron Lanier does that LLMs are a mashup (or as I would characterize it a funhouse mirror) of the human condition, as represented by the Internet, this sort of implicit bias is already represented in most social media. This is further distilled by the cultural practice of hiring third world residents to tag training sets and provide the "reinforcement learning"... people who are effectively if not actually in the thrall of their employers and can't help but reflect their own sycophancy.

As someone who is therefore historically familiar with this process in a wider systemic sense I need (hope for?) something in articles like this which diagnoses / mitigates the underlying process.

It's a bit like talking about the quality of pastoral care you get at Church. You can get a wide spectrum of results.

Worth pointing out such systems have survived a long long time since access to it is free irrespective of the quality.

Benchmarking LLMs on this is an important thing to do. There is a huge potential positive effect of psychotherapy being always-available to every human rather than just for wealthy people once a week. But to get there we need to know the rate of adverse events compared to human therapists (which isn’t zero either).

I am bipolar and I help run a group. We lost some people to chatbots already that either fueled a manic or a depressive episode.

The study coauthor actually seems positive on their potential:

'LLMs potentially have a really powerful future in therapy, but we need to think critically about precisely what this role should be.'

And they also mention a previous paper that found high levels of engagement from patients.

So, they have potential but currently are giving dangerous advice. It sounds like they are saying a fine tuned therapist model is needed because 'you are a great therapist' prompt, just gives you something that vaguely sounds like a therapist to an outsider.

Sounds like an opportunity honestly.

Would people value a properly trained therapist enough to pay for it over an existing chatgpt subscription?