What Is The Role Of Recurrent Neural Networks (rnns) In An Image Caption Generator? | Awesome LLM Papers Add your paper to Awesome LLM Papers

Awesome LLM Papers

A continuously updated collection of research papers on LLMs.

Stay Updated

Contact Sean Moran about this survey or website. Made with Jekyll and Hyde.

What Is The Role Of Recurrent Neural Networks (rnns) In An Image Caption Generator?

Marc Tanti, Albert Gatt, Kenneth P. Camilleri . Proceedings of the 10th International Conference on Natural Language Generation 2017 – 78 citations

[Paper]   Search on Google Scholar   Search on Semantic Scholar
Image Text Integration Interdisciplinary Approaches Neural Machine Translation RAG Variational Autoencoders Visual Contextualization

In neural image captioning systems, a recurrent neural network (RNN) is typically viewed as the primary generation' component. This view suggests that the image features should be injected’ into the RNN. This is in fact the dominant view in the literature. Alternatively, the RNN can instead be viewed as only encoding the previously generated words. This view suggests that the RNN should only be used to encode linguistic features and that only the final representation should be `merged’ with the image features at a later stage. This paper compares these two architectures. We find that, in general, late merging outperforms injection, suggesting that RNNs are better viewed as encoders, rather than generators.

Similar Work