Can we use SLMs to compress data?
Summary
Explores whether small language models (SLMs) can achieve lossless data compression by deliberately overfitting on training data, rethinking the typical aversion to overfitting in machine learning.
Similar Articles
Diffuse to Compress: Leveraging Diffusion LMs for Lossless Compression
This paper introduces Diffusion Language Models (DLMs) as a new inference paradigm for lossless text compression, aiming to overcome the throughput bottlenecks of autoregressive LLM-based compressors while achieving state-of-the-art compression ratios.
Benchmarking Trustworthiness of SLMs: Pre-trained vs. Compressed
This paper evaluates the trustworthiness of small language models across fairness, robustness, privacy, and ethics, comparing pre-trained SLMs with compressed larger models, and finds that quantization preserves trustworthiness better than pruning and that distillation can further enhance reliability.
Are super tiny LLMs any good?
Explores whether very small language models can handle casual conversations adequately, and what training factors differentiate the better ones.
Large Language Models Are Overkill For Some Marketing Tasks. Enter The Small Language Model
ZeroGPU launches specialized small language models (SLMs) for ad tech tasks, offering lower costs and faster performance compared to large language models. The SLMs run on CPUs and have already reduced expenses for early adopter Dappier by 50%.
Does size really matter? (LLMs vs. SLMs)
Discusses the trade-offs between large language models (LLMs) and small language models (SLMs), questioning whether larger models are always necessary for production use cases and exploring the future of AI deployment.