Improving Lstm-based Video Description With Linguistic Knowledge Mined From Text
Subhashini Venugopalan, Lisa Anne Hendricks, Raymond Mooney, Kate Saenko . Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing 2016 – 116 citations
This paper investigates how linguistic knowledge mined from large text corpora can aid the generation of natural language descriptions of videos. Specifically, we integrate both a neural language model and distributional semantics trained on large text corpora into a recent LSTM-based architecture for video description. We evaluate our approach on a collection of Youtube videos as well as two large movie description datasets showing significant improvements in grammaticality while modestly improving descriptive quality.