Treat Feedback as Reader Data

“Why was it so dark?” sounds like a question about lighting. Liz Lerman, the choreographer who devised the Critical Response Process in 1990, describes the process in a summary that includes this example, and the summary says the question is not neutral, because an opinion is folded inside it. “What ideas guided your choices about lighting?” is neutral. The first question tells the artist what the responder believes, and the second leaves room for the artist to learn what the work did.

A note usually carries two different things, and the job is to pull them apart. A reader who was bored on page forty, or did not believe she would leave, or could not tell which brother was speaking, is reporting an experience. Reports like that are data about the manuscript, and the reader is the instrument that produces them. A reader who says to cut the second chapter, or to make the husband the villain, or to add a map, is proposing a remedy, which is a hypothesis about cause offered by someone who may be seeing the effect from the outside.

Neil Gaiman put the asymmetry briefly in a list of rules for writing fiction published in The Guardian in 2010. When people say something is wrong or does not work for them, he wrote, they are almost always right, and when they say exactly what is wrong and how to fix it, they are almost always wrong. It is one writer’s rule of thumb.

Failing to separate the two goes wrong in opposite directions. Obedience implements every proposed remedy until the manuscript becomes a committee’s. Defense answers a report of confusion with an explanation of what the writer meant, so the data is thrown out because it arrived packaged with a bad suggestion. Lerman’s published notes handle this by letting the artist say whether suggestions are welcome, and by encouraging responders to work back from a suggestion to the opinion underneath it and then to a neutral question. In that sequence the suggestion becomes a clue to the report behind it.

Stephen King offers a counting rule, and also complicates it. If everyone who reads the manuscript names the same problem, he says in On Writing, there is a problem and the writer had better do something about it. If opinions split, so that some readers love the ending and others hate it, it is a wash and the tie goes to the writer. He also cautions against showing a draft too early, which opens the door before the writer knows what the story is, though he accepts that readers are worth consulting, and that a handful of people the writer knows and respects can tell a good deal. The rule is useful, and it has a limit the next practice takes up.

In his New Yorker essay “Draft No. 4,” McPhee describes Eleanor Gould, who marked up galley proofs at the magazine for decades. Her suggestions were not always graceful, and nothing on them was binding. A writer could ignore a comment, or fix the fix according to the sound in his own head, and her purpose, in her own account, was to help a writer achieve an intent in the clearest way. She reported a problem, sometimes proposed a fix, and left the decision with the owner. An experienced editor can often name causes as well as effects, and such remedies may deserve more weight, though still not obedience.

A tool’s feedback goes through the same sorting. A language model can produce a response that reads like a considerate reader’s, with named strengths and gentle concerns. Whether a given comment matches how a reader would experience the page can’t be assumed, but it can be checked by splitting the comment into report and remedy, then seeing whether human readers report the same thing at the same place. A writer may use such responses as a source of hypotheses or skip them entirely, and the sorting works either way.

Practice. Take one page of feedback, from a person or a tool, and rewrite each comment in three parts: where, who reported it, and what they experienced. Strike every remedy from the main list and keep it in a second column labeled clues. Then tally the locations that more than one comment points toward, all within twenty minutes. Sort any comment shaped like Lerman’s “Why was it so dark?” the same way, with the report in the main list and the opinion folded inside it under clues.