There are times people want to edit an NLG-produced narrative before it is released; this is called “post-editing.” Typically, this involves adding additional content that is hard to automatically generate, or fixing mistakes in the narrative (which are generally rare). An example of the former might be adding information to a financial report about relevant world events, such as how COVID has affected a company.
Post-editing depends on the use case; it is common in some use cases but unusual in others. I’ve been involved in several projects where generated narratives were post-edited, and I thought I’d share some general insight and advice about the process.
Different people post-edit different amounts
Some people simply fix problems in the narrative, while others comprehensively rewrite the narrative into their preferred writing style. I have seen cases where post-editing made a narrative “worse” (from the perspective of people reading the narrative) because the post-editor rewrote the document in an idiosyncratic way that was confusing for readers.
Advice: Give post-editors clear instructions about what they should edit. Check their work and give them feedback if they are post-editing too much (or too little).
Fear of losing one’s job
When a company introduces an NLG system to automate document production, it often asks the people who are currently manually writing the document to post-edit the NLG narratives. I have seen cases where such people are worried that the NLG system threatens their jobs; if so, they are unlikely to do a good job of post-editing.
Advice: If people are not in danger of losing their jobs, make this very clear to them! If they are, consider getting someone else to post-edit the NLG narratives.
User experience (UX)
Efficient post-editing requires a good user interface and user experience. A good UI/UX will significantly speed up post-editing and can encourage people to follow guidelines on what should be post-edited. The UX can also capture data about what gets post-edited, which is useful to developers. A related issue is workflow; post-editing should not disrupt other tasks. For example, if we ask a doctor to post-edit a medical narrative, she needs to be able to do this at a time that does not disrupt her clinical activities.
Advice: Put time and effort into designing good UI/UX and workflows.
Appropriate text for post-editing
Finally, if an NLG-generated text is going to be post-edited, then we may want to alter the Arria Studio project to support this. For example, the Studio project can add reminders such as INSERT EXPLANATION HERE, or explicitly flag paragraphs (with CHECK THIS) which should be reviewed by the post-editor. I have also seen cases where it made sense to simplify narratives that were going to be post-edited. Studio’s easy-to-use NLG technology can generate long and detailed reports, but post-editing long reports requires more time and expense. So, if narratives are going to be post-edited, it may make sense to shorten them.
Advice: If narratives are going to be post-edited, design the Studio project with this in mind.
The power of NLG is that it can produce narratives accurately on a consistent basis. The human can then edit for elements not in the data, which is the beauty of humans and AI working together, producing superior outcomes.
