This article demonstrates how to use any GGUF model with llama.cpp for binary classification tasks, such as spam detection, by configuring parameters to output probabilities from logprobs.
You can simply run any GGUF with llama.cpp with n_predict=1 and n_probs=10, disable reasoning, and prompt it such as "If the following email is spam, respond with 1, if not spam, respond with 0. Do not respond with anything other than 1 or 0. Email: ...." And that is it! It returns confidence percentages such as: 1 = 94.9% 0 = 5.08% Example: llama-server -m "C:\Users\MyUserName\llama.cpp\models\Spark-X2.5-4B-Q4_K_M.gguf" -c 4096 -ngl all -fit off -fa on -b 2048 -ub 512 -np 1 --cache-ram 0 --reasoning off --no-reasoning-preserve --perf Then: curl.exe -s -X POST http://localhost:8080/v1/chat/completions -H "Content-Type: application/json" -d "{\"messages\":[{\"role\":\"system\",\"content\":\"Classify spam. Reply only 1=spam or 0=not spam.\"},{\"role\":\"user\",\"content\":\"CONGRATULATIONS!!! You have won $5,000,000! Click here immediately to claim your prize!\"}],\"max_tokens\":1,\"logprobs\":true,\"top_logprobs\":10,\"temperature\":1.0,\"top_p\":1.0}" Result: {"choices":[{"finish_reason":"length","index":0,"message":{"role":"assistant","content":"1"},"logprobs":{"content":[{"id":30,"token":"1","bytes":[49],"logprob":-0.00456317700445652,"top_logprobs":[{"id":30,"token":"1","bytes":[49],"logprob":-0.00456317700445652},{"id":29,"token":"0","bytes":[48],"logprob":-5.395024299621582},{"id":1033,"token":"**","bytes":[42,42],"logprob":-12.013711929321289},{"id":1046,"token":"The","bytes":[84,104,101],"logprob":-13.005236625671387},{"id":198,"token":"\n","bytes":[10],"logprob":-13.100714683532715},{"id":54,"token":"I","bytes":[73],"logprob":-14.624603271484375},{"id":3640,"token":"This","bytes":[84,104,105,115],"logprob":-14.800630569458008},{"id":130977,"token":"<tool\_call>","bytes":[60,116,111,111,108,95,99,97,108,108,62],"logprob":-14.971238136291504},{"id":6908,"token":"Class","bytes":[67,108,97,115,115],"logprob":-15.373867988586426},{"id":3923,"token":"class","bytes":[99,108,97,115,115],"logprob":-15.442902565002441}]}]}}],"created":1789950066,"model":"C:\\Users\\MyUserName\\llama.cpp\\models\\Spark-X2.5-4B-Q4_K_M.gguf","system_fingerprint":"b11026-b49650adb","object":"chat.completion","usage":{"completion_tokens":1,"prompt_tokens":64,"total_tokens":65,"prompt_tokens_details":{"cached_tokens":59}},"id":"chatcmpl-x2WrCObzFNYjKVkwDmcL8FLquwfZ0NEa","timings":{"cache_n":59,"prompt_n":5,"prompt_ms":634.566,"prompt_per_token_ms":126.9132,"prompt_per_second":7.879401039450585,"predicted_n":1,"predicted_ms":0.001,"predicted_per_token_ms":0.0,"predicted_per_second":0.0}} Convert to probability: probability = e^(logprob) 1 = e^(-0.00456317700445652) = ~99.5% 0 = e^(-5.395024299621582) = ~0.5% Speed: On my 170gb/s bandwidth 4gb vram GPU, I got 634ms! On a H200, I would probably get 30-75ms.
The author explores using small LLMs like Gemma 4 as classifiers by analyzing logit probabilities with high temperature and calibration, sharing code and findings on GitHub.
The article shares production insights on using Jev, a semantic decision engine, to enhance AI agent systems by handling routine decisions efficiently alongside LLMs, without replacing generative models.
The article discusses the limitations of using LLMs directly as classifiers and proposes treating LLM outputs as features in traditional machine learning models like logistic regression to achieve better calibration, interpretability, and performance.
The blog post describes a practical method for fine-tuning large language models as calibrated classifiers using token probabilities, enabling cost-effective classification tasks with minimal compute resources.
The article introduces a technique that extracts hidden states from an LLM at the last prompt token to perform classification without text generation, using a small MLP to read the model's internal decision, enabling fast and cheap zero-shot classifiers.