llama

Muse Spark 1.2 is being open sourced, which is surprising given the focus on Llama 4 Behemoth. The author comments on the unexpected release order.

This paper introduces a game-theoretic approach to fine-tuning language models that optimizes the trade-off between reward and deviating from a reference policy, providing a principled method for setting the KL regularization coefficient.

A Statista chart estimates that approximately a quarter of the training data for Meta's Llama AI model came from books, highlighting the composition of sources used in large language model development.

This paper proposes SPG, a structure-aware patent generation method that jointly predicts claim topology and content during autoregressive decoding using a pointer head and a preference optimization stage, achieving significant improvements in parent link recovery and antecedent consistency on the HUPD-DCG benchmark.

Testing 30 schema constraints across 16 models reveals each provider handles constraints differently, causing random tool call failures. The simple fix: move constraint text into property descriptions instead of relying on prompts.

Yann LeCun defends open source by listing major projects like Linux, Apache, MySQL, and PyTorch, questioning the idea that releasing them was 'dumping'.

This paper identifies a 'countdown subcircuit' in Llama-3.1-70B-Instruct that allows the model to track remaining tokens across various tasks, and demonstrates that this subcircuit is reused across different contexts and even models.

User criticizes Meta's decision to stop releasing Llama models, calling it shortsighted.

Windows Defender flags a version of Llama-b9856 running on CUDA 12.4 as a trojan, raising security concerns.

Jerry Liu presents a framework for document parsing across accuracy, cost, and latency tradeoffs, introducing LiteParse as an open-source, low-latency parsing tool for AI agent loops, along with LlamaParse for high-accuracy modes.

Meta was secretly using Google's Gemini for customer service, ad tools, and content moderation because it outperformed their own Llama models, until Google cut off access due to excessive capacity usage.

Proposes a tree-of-thoughts inspired extractive-abstractive approach for legal case judgement summarization using LLMs, with experiments on DeepSeek and LLama showing improved summaries over extractive or abstractive methods alone.

This paper presents a multi-stage explainable framework that combines SHAP-based token attribution, theory-informed linguistic features, and LLaMA-3.1-70B-Instruct LLM reasoning to interpret transformer-based speech models for cognitive impairment detection, achieving strong clinical alignment and high usability scores.

An analysis of political leanings in six major AI models, showing that 4 out of 6 lean left of center on the economic axis, with some models being unaware of their own bias.

Discussion about the significant gap between Llama model benchmark scores and actual real-world performance, with the author seeking assistance.

AutoMegaKernel is an open-source agent harness that compiles any HuggingFace model into a single persistent megakernel, fusing the entire forward pass into one GPU launch to reduce overhead. It achieves up to 1.33x speedup over CUDA-graphed cuBLAS on inference-class GPUs like L4 and L40S, while proving schedules deadlock- and race-free.

This paper uses mechanistic interpretability to audit ethical reasoning in LLaMA 3.1-8B-Instruct, finding a 'Situational Anchor Effect' where domain-specific representations dominate moral computation, and proposing 'Mechanistic Alignment' as a research program.

A Stanford professor delivered a public lecture providing a comprehensive breakdown of how modern LLMs like GPT, Claude, and LLaMA are built under the hood, making advanced architecture accessible to the public.

InfiniteKV is an open-source KV cache technique that compresses old tokens into 104-byte searchable records stored in RAM or on disk, enabling models to handle million-token contexts beyond their trained window without discarding data. Verified working with Mistral-7B and SmolLM2.

This paper investigates sequential fine-tuning of LLaMA-3.1-8B for automated essay scoring using a curriculum aligned with discourse structure, showing improved coherence and performance compared to independent or randomized training.