What Quality of Language Will LLMs Converge On?

Like many professors, I've been looking uneasily at the development of Large Language Models (LLMs) and what they mean for the profession. A few weeks ago, I wrote about my concerns regarding how LLMs will affect training the next generation of writers, particularly in the inevitably-necessary stage where they're going to be kind of crummy writers.

Today I want to focus on a different question: what quality of writing are LLMs converging upon? It seems to me there are two possibilities:

1. As LLMs improve, they will continually become better and better writers, until eventually they surpass the abilities of all human writers.
2. As LLMs improve, they will more closely mimic the aggregation of all writers, and thus will not necessarily perform better than strong human writers.

If you take the Kevin Drum view that AI by definition will be able to do anything a human can do, but better, then you probably think the end game is door number one. Use chess engines as your template. As the engines improved, they got better and better at playing chess, until eventually they surpassed the capacities of even the best human players. The same thing will eventually happen with writing.

But there's another possibility. Unlike chess, writing does not have an objective end-goal to it that a machine can orient itself to. So LLMs, as I understand them, are (and I concede this is an oversimplification) souped-up text prediction programs. They take in a mountain of data in the form of pre-existing text and use it to answer the question "what is the most likely way that text would be generated in response to this prompt?"

"Most likely" is a different approach than "best". A chess engine that decided its moves based on what the aggregate community of chess players was most likely to play would be pretty good at chess -- considerably better than average, in fact, because of the wisdom of crowds. But it probably would not be better than the best chess players. (We actually got to see a version of this in the "Kasparov vs. the World" match, which was pretty cool especially given how it only could have happened in that narrow window when the internet was active but chess engines were still below human capacities. But even there -- where "the world" was actually a subset of highly engaged chess players and the inputs were guided by human experts -- Kasparov squeaked out a victory). 

I saw somewhere that LLMs are facing a crisis at the moment because the training data they're going to draw from increasingly will be ... LLM-generated content, creating not quite a death spiral but certainly the strong likelihood of stagnation. But even if the training data was all human-created, you're still getting a lot of bitter with the sweet, and the result is that the models should by design not surpass high-level human writers. When I've looked at ChatGPT 4 answers to various essay prompts, I've been increasingly impressed with them in the sense that they're topical, grammatically coherent, clearly written, and so on. But they never have flair or creativity -- they are invariably generic.

Now, this doesn't mean that LLMs won't be hugely disruptive. They will be. As I wrote before, the best analogy for LLMs may be to mass production -- it's not that they produce the highest-quality writing, it's that they dramatically lower the cost of adequate writing. The vast majority of writing does not need to be especially inspired or creative, and LLMs can do that work basically for free. But at least in their current paradigm, and assuming I understand LLMs correctly, in the immediate term they're not going to replace top-level creative writing, because even if they "improve" their improvement will only go in the direction of converging on the median.