Slamming: Training A Speech Language Model On One GPU In A Day | Awesome LLM Papers Contribute to Awesome LLM Papers

Awesome LLM Papers

A curated collection of research papers on Large Language Models (LLMs), transformers, prompting, fine-tuning, and multi-modal systems. Maintained by Sean Moran.

Contact Sean Moran about this survey or website. Made with Jekyll and Hyde.

Slamming: Training A Speech Language Model On One GPU In A Day

Gallil Maimon, Avishai Elmakies, Yossi Adi . No Venue 2025

[Paper] [Other] [Paper]   Search on Google Scholar   Search on Semantic Scholar
Efficiency Model Architecture Training Techniques

We introduce Slam, a recipe for training high-quality Speech Language Models (SLMs) on a single academic GPU in 24 hours. We do so through empirical analysis of model initialisation and architecture, synthetic training data, preference optimisation with synthetic data and tweaking all other components. We empirically demonstrate that this training recipe also scales well with more compute getting results on par with leading SLMs in a fraction of the compute cost. We hope these insights will make SLM training and research more accessible. In the context of SLM scaling laws, our results far outperform predicted compute optimal performance, giving an optimistic view to SLM feasibility. See code, data, models, samples at - https://pages.cs.huji.ac.il/adiyoss-lab/slamming .

https://huggingface.co/discussions/paper/67bd3973f077ddf1f98bacf9

Similar Work