您可以像使用JEV一样使用任何LLM

您只需运行任何GGUF模型与llama.cpp，设置n_predict=1和n_probs=10，禁用推理，并提示如下："如果以下邮件是垃圾邮件，请回复1，如果不是垃圾邮件，请回复0。请勿回复除1或0以外的任何内容。邮件：...." 就这样！它会返回置信度百分比，例如：1 = 94.9% 0 = 5.08% 示例：llama-server -m "C:\Users\MyUserName\llama.cpp\models\Spark-X2.5-4B-Q4_K_M.gguf" -c 4096 -ngl all -fit off -fa on -b 2048 -ub 512 -np 1 --cache-ram 0 --reasoning off --no-reasoning-preserve --perf 然后：curl.exe -s -X POST http://localhost:8080/v1/chat/completions -H "Content-Type: application/json" -d "{\"messages\":[{\"role\":\"system\",\"content\":\"分类垃圾邮件。仅回复1=垃圾邮件或0=非垃圾邮件。\"},{\"role\":\"user\",\"content\":\"恭喜！！！您赢得了5,000,000美元！请立即点击此处领取您的奖品！\"}],\"max_tokens\":1,\"logprobs\":true,\"top_logprobs\":10,\"temperature\":1.0,\"top_p\":1.0}" 结果：{\"choices\":[{\"finish_reason\":\"length\",\"index\":0,\"message\":{\"role\":\"assistant\",\"content\":\"1\"},\"logprobs\":{\"content\":[{\"id\":30,\"token\":\"1\",\"bytes\":[49],\"logprob\":-0.00456317700445652,\"top_logprobs\":[{\"id\":30,\"token\":\"1\",\"bytes\":[49],\"logprob\":-0.00456317700445652},{\"id\":29,\"token\":\"0\",\"bytes\":[48],\"logprob\":-5.395024299621582},{\"id\":1033,\"token\":\"**\",\"bytes\":[42,42],\"logprob\":-12.013711929321289},{\"id\":1046,\"token\":\"The\",\"bytes\":[84,104,101],\"logprob\":-13.005236625671387},{\"id\":198,\"token\":\"\\n\",\"bytes\":[10],\"logprob\":-13.100714683532715},{\"id\":54,\"token\":\"I\",\"bytes\":[73],\"logprob\":-14.624603271484375},{\"id\":3640,\"token\":\"This\",\"bytes\":[84,104,105,115],\"logprob\":-14.800630569458008},{\"id\":130977,\"token\":\"<tool\\_call>\",\"bytes\":[60,116,111,111,108,95,99,97,108,108,62],\"logprob\":-14.971238136291504},{\"id\":6908,\"token\":\"Class\",\"bytes\":[67,108,97,115,115],\"logprob\":-15.373867988586426},{\"id\":3923,\"token\":\"class\",\"bytes\":[99,108,97,115,115],\"logprob\":-15.442902565002441}]}}],\"created\":1789950066,\"model\":\"C:\\\\Users\\\\MyUserName\\\\llama.cpp\\\\models\\\\Spark-X2.5-4B-Q4_K_M.gguf\",\"system_fingerprint\":\"b11026-b49650adb\",\"object\":\"chat.completion\",\"usage\":{\"completion_tokens\":1,\"prompt_tokens\":64,\"total_tokens\":65,\"prompt_tokens_details\":{\"cached_tokens\":59}},\"id\":\"chatcmpl-x2WrCObzFNYjKVkwDmcL8FLquwfZ0NEa\",\"timings\":{\"cache_n\":59,\"prompt_n\":5,\"prompt_ms\":634.566,\"prompt_per_token_ms\":126.9132,\"prompt_per_second\":7.879401039450585,\"predicted_n\":1,\"predicted_ms\":0.001,\"predicted_per_token_ms\":0.0,\"predicted_per_second\":0.0}} 转换为概率：概率 = e^(logprob) 1 = e^(-0.00456317700445652) ≈ 99.5% 0 = e^(-5.395024299621582) ≈ 0.5% 速度：在我的170GB/s带宽4GB显存GPU上，我得到了634毫秒！在H200上，我可能会得到30-75毫秒。