In-Depth Guide! Understanding the Basic Concepts and Terminology of Model Fine-Tuning

Model fine-tuning involves taking a pre-trained model and continuing its training with a new dataset relevant to your specific task. The model already understands language, answers questions, and generates text. Fine-tuning adapts it to a particular style of expression, a category of tasks, or a specific workflow. For example, in specialized fields like medicine, law, or finance, fine-tuning is needed due to differences in terminology and phrasing, to establish a consistent output style, help the large model follow instructions more reliably, and reduce hallucinations.

For the average user, the most direct benefit is making the model’s output more closely match your personal expression style. Requirements you typically repeat in prompts can be embedded into the training result. After training, the need to constantly delete AI-typical phrasing, rewrite fixed sentence structures, and add missing formatting can be reduced. If you perform customer service, classification, extraction, or rewriting, you can use fixed examples to train the model on what format of result to provide for a given input.

Local fine-tuning serves another purpose. When personal chats, dictations, work notes, and internal documents are inconvenient to upload, you can organize and train the model on your own machine. The model can then become familiar with industry jargon, internal classification methods, and the team’s common expressions, reducing the need for repeated explanations when processing such content.

What fine-tuning changes is the model’s tendency to produce a certain type of output for a specific kind of input. If the training data consists only of continuous personal dictations, the model primarily learns what typically follows such text. Only when the training data includes the task, the original content, and your verified result will the model directly practice how to complete that task. How the data is structured determines what the model actually learns.

After a training run, whether the result meets expectations must be tested with content not used in training. If the goal is a more personal expression style, the final check is whether the output lacks those recurring AI expressions and reads naturally as if written by you.

Today, this article provides a detailed explanation of the basic concepts and terminology related to model fine-tuning.

Where the Model Starts

Pre-training/预训练

Pre-training is the stage where the model first learns language on a massive scale. It reads large amounts of text and predicts the next token based on the preceding ones, forming linguistic rules, common sense, and reusable representations. When you download Gemma to DGX Spark, it can already write sentences and answer questions—these abilities come from pre-training.

Base model/基础模型

The base model is the original model weights used to initiate subsequent training. Fine-tuning continues from this point. Before loading LoRA into Gemma, the Gemma weights file on disk is the base model.

Post-training/后训练

Post-training refers to any training conducted after pre-training. This phase can include fine-tuning, supervised fine-tuning, preference learning, RLHF, and DPO, each addressing different tasks. Using a batch of customer service Q&A to continue training a model that already speaks is considered post-training.

Fine-tuning/微调

Fine-tuning lets an already-trained model continue with a more specific dataset, steering its output toward those examples. It leverages the capabilities already learned by the base model and uses a smaller target dataset to adjust it—a form of transfer learning. Continuing to train Gemma with your verified rewritten results aims to make it handle similar content more closely matching those results.

How Data Teaches the Model What to Learn

Supervised fine-tuning/SFT/监督微调

Supervised fine-tuning provides the model with explicit inputs and target answers. The training process compares the model’s generated content with the target answer, then updates parameters based on the discrepancy. The input might be a set of meeting notes needing organization, and the target answer is your verified summary. The model practices generating such summaries from meeting notes.

Instruction tuning/指令微调

Instruction tuning is a common form of supervised fine-tuning. The sample includes a task description, input content, and target response. The model learns to complete tasks by following instructions. The instruction might specify “preserve facts and rephrase in casual language,” followed by the original content and the confirmed version—forming a complete rewriting task.

Causal language modeling/因果语言建模

Causal language modeling trains the model to predict subsequent tokens based on previous text. It’s suitable for learning how text continues, including word choice, tone, and sentence development. Feeding a batch of personal dictations as continuous text to the model, it will practice predicting how those words typically continue.

Next-token prediction/下一个 token 预测

Next-token prediction is the actual action performed at each step by a language model. The model calculates which tokens are more likely to follow the current text, selects one, and repeats this to form sentences. Given the preceding text “After training finishes, first check,” the model, based on learned relationships, determines whether “results,” “files,” or another word is more likely to follow.

Why Training Can Run on DGX Spark

PyTorch

PyTorch is a software framework for building and training neural networks. It handles tensor computation, automatic gradient calculation, optimizer invocation, and offloading computation to the GPU. When a training script reads a batch on DGX Spark, PyTorch computes the loss and gradients, then updates the trainable parameters.

Training framework/训练框架

A training framework organizes model loading, data reading, backpropagation, saving, and evaluation into a pipeline. PyTorch often handles the underlying computations, while Transformers, PEFT, TRL, Datasets, and Accelerate manage models, data, and distributed operations on top. A training script might use Transformers to load Gemma, PEFT to create LoRA, and then PyTorch to perform the parameter updates.

Recipe/训练方案

A training recipe is a pre-configured set of training procedures specifying the model, data format, fine-tuning method, parameters, and launch method. It provides an executable starting point but still requires adjustment based on your data and goals. NVIDIA’s PyTorch fine-tuning recipe can show how to launch the container, but customer service classification and personal expression training will use different data structures.

Ancillary files/辅助文件

Ancillary files are not part of the main model weights but are used for training, recovery, and reproduction. These include configuration files, tokenizer, chat template, training state, logs, README, and file manifests. If adapter weights remain but the matching tokenizer and configuration are lost, loading later may cause format or dimension issues.

Docker/Container/容器

Containers package PyTorch, CUDA dependencies, and Python packages into a relatively fixed runtime environment. Their main purpose is to ensure software versions and dependencies remain consistent across different machines. Using NVIDIA PyTorch containers on DGX Spark, the training script operates in a pre-configured PyTorch and CUDA environment.

What Part of the Model LoRA Changes

PEFT/Parameter-Efficient Fine-Tuning

PEFT (Parameter-Efficient Fine-Tuning) is a general term for fine-tuning methods that keep most base model parameters frozen while training only a small number of new parameters. For a model with billions of parameters, the resulting fine-tuned output may constitute only a small fraction of the base model.

LoRA/Low-Rank Adaptation

LoRA (Low-Rank Adaptation) adds low-rank matrices alongside certain linear layers in the model. During training, only these new matrices are updated, while the base model weights remain frozen. Fine-tuning refers to the act of continuing training, while LoRA specifies how parameters are modified in this process. After adding LoRA to Gemma’s attention layers, only these new parameters are saved at the end of training, not a full copy of Gemma.

QLoRA

QLoRA (Quantized LoRA) loads the frozen base model in lower precision, then trains the LoRA parameters. This reduces the memory footprint of the base model, allowing limited memory to accommodate a larger model. When a single DGX Spark cannot fit both a high-precision base model and training state, the base model can be quantized, focusing training on LoRA parameters.

Adapter

Adapter refers to the incremental parameters saved after fine-tuning, capturing how the base model needs to change. Using it typically requires loading both the corresponding base model and the adapter. Adding a writing adapter to a matching Gemma model enables it to exhibit the changes from that training run.

Fresh adapter/新的 adapter

A fresh adapter means LoRA parameters are reinitialized, and the training state starts from scratch. The base model still retains its pre-trained capabilities, so this “starting over” only affects the new parameters. Deleting an old adapter and recreating a set of LoRA parameters won’t cause Gemma to forget the language learned during pre-training.

Rank/LoRA rank/r

Rank determines the size of the low-rank space LoRA uses to record changes. Increasing this value typically increases the number of trainable parameters, file size, and expressive capacity, which also affects training resources and the chance of memorizing training samples. Using rank 8 versus rank 64 on the same set of target modules will result in different numbers of LoRA parameters.

LoRA alpha

LoRA alpha controls the magnitude of LoRA’s influence on the model’s output. The learning rate determines how parameters are updated each step, while alpha determines how this branch is scaled during computation. Two training runs using the same rank and learning rate but different alpha values will vary in how strongly LoRA updates affect the model output.

LoRA dropout

LoRA dropout randomly ignores some LoRA paths during training. The same sample may pass through different new connections in different training steps.

Target modules/目标模块

Target modules determine where LoRA is applied in the model. Attention and MLP layers perform different computations, and the choice affects the number of trainable parameters and which parts of the model can be adjusted. LoRA can be applied only to q_proj and v_proj, or simultaneously cover k_proj, o_proj, gate_proj, up_proj, and down_proj.

Safetensors

Safetensors is a binary format for saving model or adapter weights. Opening adapter_model.safetensors in VS Code reveals many symbols because it stores numerical tensors.

How Text Becomes Training Data

Corpus/语料

A corpus is the collection of text prepared for the model to read. It can retain original files, sources, and context, and doesn’t yet need to conform to the training program’s field format. Personal dictations, chat logs, and published articles can first be placed separately into the corpus.

Dataset/数据集

A dataset consists of samples already organized for the training program to read. After filtering, merging, deduplication, splitting, and format conversion, the corpus becomes a training set, validation set, or test set. Once a batch of dictation texts is organized into a JSONL file with one sample per line, the training script can read them by field.

Original source/原始素材

Original source refers to raw, unedited personal words or external text with clear origins. Preserving original sources allows future verification of derived data and assessment of which expression style the model learned. Transcribed and cleaned training files should be stored separately; cleaned files should not overwrite original transcriptions.

Derived data/派生数据

Derived data are files produced after merging, cleaning, labeling, or splitting the original source. A single chat export can generate continuous dialogue, a training set, and a validation set—all derived data.

Context merging/上下文合并

Context merging combines multiple messages belonging to the same expression into one sample. This typically considers the sender, whether others interrupted, and the time gap between messages. Three consecutive supplementary messages from one person, with no replies in between, can be merged to approximate the original complete statement.

Deduplication/去重

Deduplication identifies texts that are identical, differ only in format, or are largely similar, to prevent the model from repeatedly seeing the same sentence. Exact duplicates can be directly removed, while near-duplicates require judgment on whether they are emphasis, self-correction, or actual repetition. An article imported twice can have one copy removed, but if a person says something then adds a correction, the semantic change should be preserved.

Context-dependent short reply/依赖上下文的短回复

Context-dependent short replies are difficult to understand without the original conversation. They may hold conversational value but cannot inform the model of what they’re responding to alone. “Okay,” “Certainly,” and “Let me see” make sense in a chat but provide little information as standalone training samples.

Quality tier/质量分层

Quality tiering groups the corpus based on completeness, expressive value, and context dependence. Shorter or colloquial content may be retained, then assessed for suitability for direct training, need for review, or inclusion in raw material. A complete dictation might be a primary candidate, while a reply consisting of only a pronoun might be kept in an audit file.

Tokenizer

The tokenizer converts text into token IDs and also converts the model’s output token IDs back to text. The base model used a particular tokenization scheme during pre-training, and fine-tuning and inference must use a matching tokenizer. Gemma’s text cannot be randomly switched to another model’s tokenizer for training, otherwise the token IDs will lose alignment.

Max sequence length/最大序列长度

Max sequence length specifies the maximum number of tokens the model can read in one pass. When long dictations exceed this length, subsequent content must be split into new training blocks.

Packing/打包

Packing fits multiple short texts into a single training block, reducing padding and inefficient computation. Many chat snippets of only a few dozen tokens can together fill one sequence.

Train/Validation/Test

The training set is used to update parameters, the validation set is used during training to observe model performance on data not participating in updates, and the test set is used for final evaluation. The validation and test sets must be separate from the training set. Repeatedly adjusting based on test results can gradually erode the test set’s independence. For example, from 1000 samples, a portion can be split off for validation and testing, with the training program using only the remaining samples for gradient updates.

Data mixing ratio/数据混合比例

The data mixing ratio refers to the proportion of tokens from different sources during training. File count, paragraph count, and character count cannot directly substitute for token proportion, as text length varies by source. When mixing long dictations and short chats at 60% and 40%, you must consider the actual tokens the model reads, not just the number of files placed from each side.

Oversampling/过采样

Oversampling increases the frequency with which a smaller category of data is shown to the model, raising its proportion in training. Increased exposure makes this type of expression more likely to be learned, but also risks memorizing fixed sentences. If you have few long articles but want them to constitute half the training data, some of those samples need to be repeated.

How Parameters Control the Training Process

Batch size

Batch size is the number of training blocks processed directly by the GPU at each step. If DGX Spark can accommodate two long training blocks at once, the batch size is 2.

Gradient accumulation/梯度累积

Gradient accumulation allows the GPU to process several small batches sequentially, accumulate gradients, and then perform a single parameter update. This achieves a larger effective batch size when memory cannot hold a large batch. With a batch size of 2, accumulating for 8 steps before updating makes the effective batch size equivalent to 16 training blocks at once.

Epoch

An epoch indicates that the model has seen the entire training dataset once. Setting 2 epochs typically means the model will go through the defined training set twice.

Step/Training step

A training step usually refers to one parameter update by the optimizer. This doesn’t necessarily align with processing one batch, as gradient accumulation allows multiple small batches to jointly contribute to one update. After accumulating gradients for 8 steps, one parameter update occurs, and the “global step” in the logs increases by only 1.

Max steps

Max steps sets the maximum number of parameter updates during training. If set to 10, the program will stop after 10 updates.

Learning rate/学习率

The learning rate controls the step size of each parameter update. If too large, training may become unstable and harm existing capabilities; if too small, noticeable changes may not occur within the limited training steps. Two training runs using the same data but different learning rates may see different loss reduction speeds and final outputs.

Warmup

Warmup gradually increases the learning rate at the start of training. If the first 100 steps are for warmup, the learning rate will rise to the set value during this period.

Seed/随机种子

The random seed controls data order, parameter initialization, and random processes like dropout, making it easier to compare experiments. Using the same seed reduces variability, though GPU computations may still show tiny numerical differences. Using the same seed when comparing two ranks minimizes interference from data order changes.

What Remains After Training

trainer_state.json

trainer_state.json records the training progress of the Transformers Trainer, commonly including global step, epoch, logs, and the best checkpoint. It is usually found in the checkpoint directory; the final adapter root may not retain a copy. To confirm which step training resumed from, you can check this file in the most recent checkpoint.

training_args.bin

training_args.bin saves the training parameter object used by the Trainer. It’s a binary file meant for programmatic reading; opening it directly in a text editor won’t yield a clear parameter list. To review batch size or learning rate, you can load this file in a Python environment or keep a text configuration alongside it.

Loss/训练损失

Training loss indicates the gap between model predictions and training objectives. During training, we typically want it to decrease, as this shows the model is more accurately predicting the current batch of training data. A decreasing loss in personal dictation continuation training indicates the model is getting better at predicting the tokens following those dictations.

Eval loss/验证损失

Eval loss is the model’s prediction error on the validation set, which does not participate in current parameter updates. If training loss continues to decrease while eval loss starts to worsen, the model may be overfitting to the training data. Computing eval loss periodically on a fixed validation set helps observe whether continued training brings new generalization benefits.

Overfitting/过拟合

Overfitting refers to the model performing better and better on training data but worsening on new content. It often occurs with small datasets, excessive repeated exposure, or overly long training times. If the model can recite training articles but cannot handle a new topic with similar expression, this result should prompt checking for overfitting.

Memorization/记忆训练文本

Memorization means the model directly reproduces excerpts from the corpus rather than merely learning word choice and expression patterns. Moderate repetition can aid learning, but fixed sentences repeated extensively make the model more likely to generate them verbatim. During testing, inputting a new topic but having the model output entire sentences from training articles is a memorization phenomenon worth noting.

Inference/推理

Inference is the process where the model receives input and generates results, with no parameter updates occurring. Loading the base model and adapter onto DGX Spark for conversation is inference.

vLLM

vLLM is an inference engine for deploying and invoking large models. It handles model loading, memory management, and generation requests. It’s used in the post-training service phase, while training parameter updates are still managed by the training framework. After the adapter passes validation, you can use vLLM on DGX Spark to provide a local interface for other programs to call the model.

Smoke test/冒烟测试

A smoke test uses a small amount of data and few steps to check if the entire training pipeline can run. It verifies that the model loads, data can be read, gradients can update, and files can be saved, but it doesn’t assess formal training quality. Running 10 steps successfully and saving an adapter is a prerequisite before deciding whether to launch a full training session lasting several hours.

Evaluation/评估

Evaluation uses fixed inputs to compare the outputs of the base model and the fine-tuned model. The assessment content must align with actual goals—style training should examine expression, rewriting, and paraphrasing, while classification training should look at classification results. Testing both models with a few new materials not used in training, then recording which version requires fewer manual adjustments, is a practical approach.

When there’s a mismatch between training inputs and actual use inputs, the correlation between training results and real-world performance weakens. Training data may consist only of continuous personal dictations, but actual use requires the model to read external materials and complete a rewrite—these two input types correspond to different tasks.

Why Basic Concepts and Terminology Matter

Recently, while working with DGX Spark, I realized that often we over-rely on AI to perform operations, which can make it hard to spot problems. It was during this process that I understood the importance of clarifying basic concepts and terminology. I think this article doesn’t need to be digested all at once—it can be revisited, understood repeatedly, and read through multiple times, then internalized through your own hands-on practice, becoming knowledge that truly becomes your own.

The practice of model fine-tuning will soon begin to spread widely.

About the Author

Roland | PhD ing @UQ | DGX SPARK & Max 395 Local AIPC Player | Creating Knowledge, Creating Value | @rwayne