Scifive: A Text-to-text Transformer Model For Biomedical Literature | Awesome LLM Papers Add your paper to Awesome LLM Papers

Awesome LLM Papers

A continuously updated collection of research papers on LLMs.

Stay Updated

Contact Sean Moran about this survey or website. Made with Jekyll and Hyde.

Scifive: A Text-to-text Transformer Model For Biomedical Literature

Long N. Phan, James T. Anibal, Hieu Tran, Shaurya Chanana, Erol Bahadroglu, Alec Peltekian, Grégoire Altan-Bonnet . Arxiv 2021 – 83 citations

[Paper]   Search on Google Scholar   Search on Semantic Scholar
Content Enrichment Interdisciplinary Approaches Model Architecture RAG Variational Autoencoders

In this report, we introduce SciFive, a domain-specific T5 model that has been pre-trained on large biomedical corpora. Our model outperforms the current SOTA methods (i.e. BERT, BioBERT, Base T5) on tasks in named entity relation, relation extraction, natural language inference, and question-answering. We show that text-generation methods have significant potential in a broad array of biomedical NLP tasks, particularly those requiring longer, more complex outputs. Our results support the exploration of more difficult text generation tasks and the development of new methods in this area

Similar Work