Tag
The author argues that due to rapid AI development, important knowledge should be preserved in physical books at home and in communities, and suggests governments should promote this practice.
AI models fail to handle polytonic Ancient Greek due to training data shortages and RLHF alignment issues, degrading capabilities and impacting classics departments, with proposed solutions like RAG and corpus-grounded systems.
The article proposes a novel framework called a 'civilization scaffold' to enhance AI capabilities by preserving and building upon human knowledge without modifying model weights, allowing for immediate and recursive improvements.
AI companies are secretly buying and destroying physical books to obtain training data, locking knowledge away, and Anna's Archive is urging volunteers to scan rare books to prevent cultural heritage loss.
TRACE introduces a two-stage curriculum to preserve parametric tool knowledge in enterprise LLMs while enabling fast single-beam greedy decoding, achieving improved accuracy and recall over baselines.
This paper introduces Attention-Shifting (AS), a novel framework for selective machine unlearning in LLMs that balances effective removal of sensitive information while preventing hallucinations and preserving model utility. The method uses importance-aware attention suppression and retention enhancement to achieve up to 15% higher accuracy preservation compared to existing unlearning approaches on standard benchmarks.
This paper investigates how supervised fine-tuning (SFT) increases hallucinations in LLMs by causing knowledge degradation and proposes a self-distillation-based method to mitigate this issue while preserving pre-existing factual knowledge. The authors identify semantic interference among overlapping representations as the primary mechanism behind SFT-induced hallucinations and demonstrate solutions including parameter freezing and self-distillation.