Since the foundational work of authors such as Appelt, Kronfeld, Grosz, Joshi, Dale and Reiter, referring expression generation (REG) has been the subject of intensive research in the NLG community, giving rise to significant consensus on the REG problem definition, as well as the nature of the inputs and outputs of REG algorithms. This is particularly true of the subtask of attribute selection for definite referring expressions (REs), perhaps the most widely researched NLG subtask. A succinct definition of the attribute selection task is given by Bohnet and Dale (2005):
"Given a symbol corresponding to an intended referent, how do we work out the semantic content of a referring expression that uniquely identifies the entity in question?"
This was precisely the task definition in the TUNA tasks in the ASGRE'07 and REG'08 Challenges. In REG'08 we also added a TUNA realisation task and a TUNA end-to-end referring expression generation task.
TUNA-REG Task at GenChal'09
It is the end-to-end TUNA-REG task that we are running again as part of GenChal'09. While we have determined that there is sufficient interest to run a progress-check task this year, in all likelihood it will be the last time that a TUNA task will run, and the data sets will then be placed in the public domain for future use by researchers.
As part of the GenChal'09 TUNA-REG evaluations, we are planning to test two new human-based evaluation methods. In addition to computing a range of automatic intrinsic metrics (including all of last year's), we are planning to test a new version of the identification experiment we used in ASGRE'07 and REG'08, and we are considering the development of a method for intrinsic human evaluation.
- Anja Belz, NLTG, University of Brighton, UK
- Albert Gatt, Computing Science, University of Aberdeen, UK
- Eric Kow, NLTG, University of Brighton, UK
Last modified: 2009-01-06 09:47