LLMs write too fast to think well
I wonder if it’s impossible to get an LLM to write a great essay. It might. But I think it’s easier than people think to build a good AI writing tool on top of an LLM (though not something I personally want to do). The problem is we have an LLM bias, and the way that essays get formed are very non-LLM. It’s not like a prompt can turn into a higher-dimensional mathematical object and then summon a whole essay form.
An essay is a mode of thinking. I don’t mean to imply that a machine “can’t think,” I mean that analysis and thought takes time, and LLMs are writing 100x faster than required.
An AI writing tool would need to prompt a sentence at a time, and pause to “reason” for a minute or so: what did I just say? What are the possible things I could say next? Of those things, which belong in this paragraph, which in the next? What sentence length might be effective given the idea and last sentence? Now that I’ve chosen my idea, how should the tone modulate? What words or phrases belong in the sentence? And how should I structure the sentence? You get it.
In any given sentence, there are dozens of decisions. I think an AI could be decent—if not amazing—at thinking this through, but they’re asked to write 2,500 words on Hegel at point blank. Good generative writing can’t be done through up-front vector math, but through following a mode of thinking (incremental and context-laden vector math). The implication here is that the AI might take 3-10 hours to write the essay, similar to a human.
Put more simply, you would need a tool that reasons after each sentence and writes/saves variables that can be called upon for future sentences.