Text segmentation with topic modeling and entity coherence

This paper describes a system which uses entity and topic coherence for improved Text Segmentation (TS) accuracy. First, the Linear Dirichlet Allocation (LDA) algorithm was used to obtain topics for sentences in the document. We then performed entity mapping across a window in order to discover the transition of entities within sentences. We used the information obtained to support our LDA-based boundary detection for proper boundary adjustment. We report the significance of the entity coherence approach as well as the superiority of our algorithm over existing work.