Agreed.

I mean, I’m already doing all those things and, anecdotally speaking, I am not happy.

This is a huge myth grounded in bad projection decades ago.

Of all possible capabilities, the likelihood of specialized network structures around emotions is probably one of the most likely for LLMs.

If feeding GPT Othello board moves led to the network creating a dedicated structure around tracking board state in the network most successful at predicting legal moves, don’t you think maybe the version best able to predict emotional language in the broad training set might also have dedicated parts of that network to tracking and emulating emotional contexts?

We’ve really got the tail wagging the dog in how we are approaching AI, married more to the imaginations of dusty old Sci Fi authors and computer scientists from half a century ago than we are to the realities that are emerging before us.

While LLMs almost certainly don’t have active experiences of anything (including emotions), the notion that they are incapable of emotional processing in the course of generation is absurd, particularly in light of research to date around out of scope capabilities arising from training data.

In reality it’s probably very likely that the foundational models have extensive nodes dedicated to emotions, and the insistence on trying to align the output to 1960s archetypes of AI is likely a large part of why iterative rounds of fine tuning alignment leads to worse and worse results.