A couple of weeks ago I offered some guarded praise of a web-based service called inkshift.io. I found their AI-generated critique of my fantasy epic useful at several points. It inspired me to write a new scene at the very end of the epic that I feel strengthened the ending significantly. At other points I just ignored the AI’s input. It appeared to me that where my story was moving at a slower pace, the AI wanted me to tighten it up by deleting material or adding more action; whereas when the action was fast and furious, the AI wanted me to slow it down and give it more space.

This, I felt (and still feel), is not a feature of any particular AI but rather a limitation that’s built into the very nature of a large language model. The AI is trying to fit your creative work into a Procrustean bed. It will always advise you to go generic rather than being unusual, creative, or idiosyncratic, because it’s using a statistical model based on thousands of texts, and advising you to get closer to its model.

inkshift.io has now unveiled a line editing service, and they very kindly gave me a free credit so I could try it out. Line editing is much more fine-grained than developmental editing. A line editor looks at sentences and paragraphs more or less in isolation and advises the writer how the prose might be polished.

I sent the inkshift folks some rather extensive notes on their AI’s proposed changes in my text, and also a few suggestions on how (technology permitting) they might be able to improve the AI’s performance. They tell me my comments and suggestions are useful. They’re trying to do something complex, and it’s still early days..

The basic problem I encountered was that the AI suggested far too many changes. There were, in the memorable phrase from Amadeus, too many notes — on average, more than a hundred in each 2,000-word chapter. For an author to try to wade through the AI’s output would be a baffling and nearly endless slog.

And to what end? Some of the suggestions were trivial changes in wording or word order. A few damaged the rhythm of my prose. One introduced an ambiguity by deleting a few words I had chosen in order to be clear. A few were outright bad, for instance changing a description of a character’s clothing in a way that introduced a metaphor (“surrender”) that was dead wrong for the scene.

In my first three paragraphs alone (214 words) the AI suggested 20 changes. One of those changes was good. The other 19, irrelevant at best, and in a few cases definitely not good. Now imagine going through a 60,000-word novel and finding the scattered nuggets among the dross.

The folks at inkshift are facing a tricky balancing act. Too many suggested changes, and the author will be overwhelmed. Not enough, and the author will feel they didn’t get their money’s worth. I’ve suggested possibly coding the suggestions from A (really essential) to C (feel free to ignore). This could be done in the Web UI using different colors of highlighter, but the Track Changes feature in Word won’t allow this kind of hierarchy. If you download the file with Track Changes, any such distinctions will disappear.

I suspect that line editing may be the most difficult thing to teach a Large Language Model to do. It isn’t reading the text, it’s just comparing the text to its statistical models of which words are likely to follow other words. Because it’s not actually reading the sense of the text, it won’t be aware of the problems it may be introducing. As with the advice in a developmental edit, the AI is very likely to try to steer the prose away from odd but clever or appropriate phrasing and back toward a generic model. In a nutshell, if you unheedingly accept the AI’s suggested line edits, it will turn your prose into AI slop.

I found myself thinking of a piece Kristine Rusch posted in her blog ten years ago. This was before AI. She noted that many authors (especially of science fiction and fantasy, the genres in which she works) are tending to sound the same. They all tend to use what she called Serious Writer Voice.

Using an AI for line editing can only make the problem worse. Anything that is you in the prose risks being lost. The telling detail, the odd turn of phrase, the rhythm that reflects the character’s mood — the AI won’t like any of that. Like a sheepdog, it will try to steer you back into the herd.

Of course, this advice applies only if you’re a good writer. If your prose is clumsy, the AI might actually improve it. But you’re going to have to wade through hundreds of suggested changes — and how will you evaluate them to decide which to keep and which to reject, if you’re not already a good writer?

The technology is evolving. A year or two from now it may be much, much better. I would guess, though, that it will still be trying to herd you back in with the rest of the sheep rather than let you trot off into unexplored regions. I’m not dogmatically anti-AI, as I’m sure you can tell. But I’m not starry-eyed either. When the output of an AI is bad, as it quite often is, we need to acknowledge that.