@Vtrivedy10: as teams explore flavors of fine-tuning like SFT & RL, i think it’s helpful for someone on the team to internalize what…
Summary
The tweet highlights the value of understanding different AI fine-tuning methods like SFT and RL, their impact on model behavior, and recommends using mathematical analysis and visual aids such as ASCII art to grasp these concepts.
View Cached Full Text
Cached at: 09/04/26, 06:37 AM
as teams explore flavors of fine-tuning like SFT & RL, i think it’s helpful for someone on the team to internalize what behaviors each method “encourages” + how this happens + what type of model it produces
behaviors
- SFT=mode covering & RL=mode seeking
- how that relates to the loss
- the source of data (offline dataset vs model producing tokens and getting scored)
choose how deep you want to dive into the math but LLMs are great at drawing and redrawing the concepts until it makes sense
i’ve always found it helpful to plug numbers into the loss functions and see what number comes out and see what that means by what cases get rewarded vs penalized
a lot of observed behavior like catastrophic forgetting falls out of analyzing where the math is forcing the learning process
claude and chatgpt are fantastic ascii art and html drawers, have hundreds of these little diagrams trying to understand concepts in papers
Similar Articles
@aakaran31: SFT is not dead! We found a way to make SFT rival current prevailing posttraining methods, often generalizing better an…
Authors argue supervised fine-tuning (SFT) is not obsolete, presenting a method that brings SFT on par with prevailing posttraining approaches like RL and OPSD, often generalizing better and forgetting less, by introducing sampling into the posttraining stack.
@Vtrivedy10: most of the value in understanding models + doing better RL Task generation comes in the QA step the entire process to …
The article discusses the importance of integrating QA into RL task generation through an iterative process to improve AI pipelines and model evaluation, emphasizing the value of intuition in eval design.
@Blum_OG: "everyone uses AI. almost nobody understands how it works." that gap is real - and it's the whole point here's what the…
An explanatory tweet thread breaking down how AI works, covering tokens, attention, parameters, context windows, hallucination, RAG, and RLHF to help users become sharper users of AI.
@omarsar0: Highly-recommended read. Even with all the research, fine-tuning is such an underexplored problem. Based on what I've s…
A discussion on the underexplored potential of fine-tuning in AI, with the claim that agentic fine-tuning is on the verge of a revolution, contrasting with the view that fine-tuning is a bet against base model progress.
@techNmak: One useful way to study AI is to forget the model names for a while and look at the operations. Take attention. A query…
This tweet explains how core AI operations like attention and convolution are based on foundational mathematics such as linear algebra and probability, emphasizing the value of understanding these operations over model names.