Generation of Reference Summaries

We have constructed an integrated web-based system for collection of extract-based corpora and for evaluation of summaries andsummarization systems. During evaluation and examination of the collected and generated data we found that in a situation of lowagreement among the informants the corpus gives unduly favors to summarization systems that use sentence position as a centralweighting feature. The problem is discussed and a possible solution is outlined.
In the proceedings of the 2nd Language & Technology Conference: Human Language Technologies as a Challenge for Computer Science and Linguistics, April 21-23 2005, Poznan, Poland