InfluMatch: Frontier-Quality KOL Search at 4B-Model Cost

# InfluMatch: Frontier-Quality KOL Search at 4B-Model Cost
Source: [https://arxiv.org/html/2607.05968](https://arxiv.org/html/2607.05968)
Petmongkon PornpichitsuwanNatchaya Temyingyong Nutnicha LaplamoonWachiravit Modecrua Krittin PachtrachaiTouchapon Kraisingkorn

###### Abstract

Matching influencers \(KOLs\) to free\-form, multi\-part Thai marketing criteria is today served either by keyword search over structured profiles, which misses semantic fit, or by prompting frontier LLMs over every candidate, which is accurate but slow and expensive\. We present InfluMatch, a low\-cost three\-stage cascade — retrieval→\\rightarrowrerank→\\rightarrowreason — built entirely from small open\-weight models: dense retrieval returns 50 candidates, a 4B pointwise reranker scores each by the log\-probability of a single Yes token and keeps 10, and a 4B reasoner grades the shortlist per criterion on a rubric with a Thai rationale\. The cascade is designed for cost: reasoning over a filtered top\-10 halves token spend versus reasoning over all 50 while scoring 14 points higher\. End\-to\-end against human relevance labels on an 11\-query set with all 50 candidates labeled, the full cascade reaches94\.1%P@5, versus a retrieval\-only baseline near random; it matches the frontier model Kimi\-K2\.6 \(91\.8%\) while emitting∼35×\{\\sim\}35\\timesfewer output tokens and serving a 50\-KOL query in∼20\{\\sim\}20s on one A100\. Notably, the only fine\-tuning that pays off is pairwise: a SimPO\-tuned reranker matches the frontier baseline’s best\-pick accuracy \(78\.0 EM\), whereas fine\-tuning the reasoner on pointwise per\-criterion labels improves offline scores yet degrades end\-to\-end ranking — an inversion we trace to the design of the absolute labeling task — leaving the untuned base model as the strongest deployed reasoner\. The result is a deployable, explainable KOL search system at a small fraction of frontier serving cost\.

## 1Introduction

Influencer marketing allocates a large share of brand budgets, and its central operational problem is*matching*: given a campaign, find the Key Opinion Leaders \(KOLs\) whose audience, content, and voice fit the brief\. In the Thai market, a marketer’s intent is naturally expressed as free\-form, multi\-part criteria — “a female food reviewer with a playful tone whose audience skews young” — while the supply side is a long tail of creators whose relevance must be inferred from heterogeneous, largely Thai signals \(bios, video transcripts, audience profiles\) rather than read off a clean attribute table\.

#### Why keyword and structured search fall short\.

The incumbent approach treats matching as lookup over a scraped KOL database filtered by keywords and hard attributes\. This fails on three counts: it is*lexical, not semantic*\(a “playful food reviewer” query misses a KOL whose bio never says so but whose content plainly fits\); the signals are*fragmented*across profile fields, transcripts, and engagement metrics, so a query can satisfy each field in isolation yet return a poor overall match; and the deciding criteria are*dynamic*, campaign\-specific predicates that a static schema cannot anticipate\. Our evaluation confirms the ceiling of pure recall: dense retrieval alone barely separates from a random baseline on end\-to\-end precision \(54\.5% vs\. 54\.0% P@5, §[6\.3](https://arxiv.org/html/2607.05968#S6.SS3)\)\.

#### Approach\.

We presentInfluMatch, a three\-stage pipeline —*retrieval*→\\rightarrow*rerank*→\\rightarrow*reason*— that turns marketer criteria into a ranked KOL shortlist with per\-criterion scores and Thai rationales\. Stage 1 retrieves a top\-50 candidate set from a vector index; Stage 2 applies a pointwise 4B LLM reranker that keeps the top 10; Stage 3 scores each shortlisted KOL per criterion and combines the scores by an even sum\. To supervise the learned stages we build an evidence\-grounded, multi\-objective human\-labeled corpus over synthetic\-but\-realistic Thai briefs — pointwise ordinal scores, binary pass/fail verdicts, and pairwise best/worst judgments — and fine\-tune two 4B open\-weight students: a pairwise SimPO reranker and a pointwise SFT\+\+GRPO reason scorer\. Only the pairwise student survives end\-to\-end evaluation \(§[7](https://arxiv.org/html/2607.05968#S7)\); the deployed reasoner is the untuned base model\.

### 1\.1Contributions

- •A deployable three\-stage KOL matching pipelinefor free\-form Thai criteria, returning a ranked shortlist with per\-criterion scores and Thai rationales\. End\-to\-end, the full cascade lifts P@5 from≈\\approx54\.5% \(retrieval only\) to94\.1%on an 11\-query pool with all 50 candidates labeled, while emitting∼35×\{\\sim\}35\\timesfewer output tokens than a frontier model\.
- •A synthetic brief\-generation pipeline and a multi\-objective, evidence\-grounded labeled corpusof pointwise ordinal scores, binary pass/fail verdicts, and pairwise best/worst judgments over realistic Thai briefs\.
- •A pairwise SimPO reranker \(4B\)that matches the frontier baseline’s best\-pick accuracy \(78\.0 EM\) and transfers downstream, improving end\-to\-end P@5 over the untuned reranker\.
- •An analysis of an offline–end\-to\-end inversion: pointwise SFT\+\+GRPO fine\-tuning of the reason scorer wins on per\-criterion metrics but loses end\-to\-end to its untuned base; an evidence audit traces the gap to the design of the absolute labeling task, indicating that the recoverable supervision signal lives in relative judgments\.

## 2Background and Related Work

#### Influencer–brand matching\.

Industrial KOL discovery is dominated by database platforms that filter scraped creator profiles by keywords and hard attributes and surface candidates for manual review\. Academic treatments formalize the problem as two\-sided matching or recommendation over structured features, and marketing\-science work models the market\-level consequences of AI\-assisted matching\. Both lines presuppose that campaign intent is already encoded in a fixed attribute schema; neither addresses free\-form, multi\-part briefs whose deciding predicates are campaign\-specific, nor produces per\-criterion, evidence\-grounded justifications a marketer can audit\. We instead cast matching as criteria\-conditioned retrieval and grading over heterogeneous, largely Thai creator evidence\.

#### Multi\-stage cascades for ranked retrieval\.

Telescoping retrieval through stages of increasing per\-candidate cost is a classical effectiveness–efficiency design and remains the dominant shape of neural IR: an efficient dense or lexical first stage\[[4](https://arxiv.org/html/2607.05968#bib.bib19)\]feeds a stronger cross\-encoder reranker\[[9](https://arxiv.org/html/2607.05968#bib.bib13)\], optionally followed by a pairwise final stage\[[11](https://arxiv.org/html/2607.05968#bib.bib14)\]; multilingual embeddings extend dense first stages to Thai\. InfluMatch instantiates this cascade with an explicit per\-stage token and latency budget, but departs at the last stage: rather than emitting only a permutation, the reason stage grades each criterion on a rubric with a Thai rationale, so the output is an auditable decision rather than an opaque ordering\.

#### LLMs as rerankers\.

Prompted LLM rerankers span pointwise, pairwise\[[16](https://arxiv.org/html/2607.05968#bib.bib18)\], and listwise families\. Pointwise relevance generation scores a candidate by the likelihood of a relevance token\[[10](https://arxiv.org/html/2607.05968#bib.bib12)\]; listwise prompting, e\.g\., RankGPT\[[20](https://arxiv.org/html/2607.05968#bib.bib15)\], achieves the strongest benchmark quality but pays with long prompts, sliding\-window passes, and latency\. A parallel line trains open\-weight students, typically 7B listwise rerankers, on proprietary teacher orderings\[[14](https://arxiv.org/html/2607.05968#bib.bib16),[15](https://arxiv.org/html/2607.05968#bib.bib17)\]\. Our reranker sits deliberately at the cheap end of this space: pointwise single\-token scoring is embarrassingly parallel and emitsO​\(1\)O\(1\)output tokens per candidate, fitting a production latency budget that listwise sliding windows do not\. Unlike prior pointwise students trained with cross\-entropy on absolute relevance labels, we train the 4B student on*human pairwise preferences*with SimPO, which §[7](https://arxiv.org/html/2607.05968#S7)shows is the distinction that matters end\-to\-end\.

#### LLM judges and rubric\-based grading\.

Our reason stage is closer to LLM\-as\-a\-judge than to reranking: it grades a \(criterion, KOL\) pair against a rubric and must justify the grade\. Judging with strong proprietary LLMs is standard practice\[[23](https://arxiv.org/html/2607.05968#bib.bib21)\], and open evaluators such as Prometheus\[[5](https://arxiv.org/html/2607.05968#bib.bib20)\]show that rubric\-following judgment can be distilled into mid\-sized open\-weight models\. This literature also documents biases and calibration problems of absolute scoring relative to comparative judgment\[[21](https://arxiv.org/html/2607.05968#bib.bib22)\]; our deployment adds end\-to\-end evidence in the same direction, with the untuned base model remaining the strongest deployed judge \(§[7](https://arxiv.org/html/2607.05968#S7)\)\.

#### Post\-training and the choice of supervision signal\.

Offline preference optimization aligns a policy directly on comparisons\[[18](https://arxiv.org/html/2607.05968#bib.bib4)\]— notably SimPO, a reference\-free method whose implicit reward is the length\-normalized sequence log\-probability\[[7](https://arxiv.org/html/2607.05968#bib.bib1)\]— while Group Relative Policy Optimization \(GRPO\) optimizes against verifiable rewards without a critic\[[19](https://arxiv.org/html/2607.05968#bib.bib2)\]\. Orthogonal to the algorithm is which*label geometry*to collect: relative judgments \(best–worst scaling\) are markedly more reliable than rating scales at equal annotation budget\[[6](https://arxiv.org/html/2607.05968#bib.bib9)\], and RLHF pipelines standardized on pairwise comparisons partly for this reason; offline gains failing to survive deployment is itself a documented pattern in recommender\-system evaluation\[[2](https://arxiv.org/html/2607.05968#bib.bib10)\]\. Our offline–end\-to\-end inversion is consistent with this literature at deployment scale: from the same corpus, the pairwise SimPO student transfers downstream while the pointwise SFT\+\+GRPO student does not\.

## 3Method

### 3\.1Problem Formulation

Given a marketer brief expanded into a set of matching criteriaC=\{c1,…,cm\}C=\\\{c\_\{1\},\\dots,c\_\{m\}\\\}and a candidate pool of KOLs𝒦=\{k1,…,kN\}\\mathcal\{K\}=\\\{k\_\{1\},\\dots,k\_\{N\}\\\}, each represented by heterogeneous Thai signals, the task is to produce a ranked shortlistπ\\piover𝒦\\mathcal\{K\}such that KOLs satisfyingCCrank highest, together with a per\-criterion score and a Thai rationale for each shortlisted candidate\.

### 3\.2Cascade Overview

InfluMatch decomposes matching into three stages of increasing per\-candidate cost, so that the expensive reasoning stage runs only on a filtered shortlist \(Figure[1](https://arxiv.org/html/2607.05968#S3.F1), Table[1](https://arxiv.org/html/2607.05968#S3.T1)\)\.

![Refer to caption](https://arxiv.org/html/2607.05968v1/3stage.png)Figure 1:The three\-stage InfluMatch pipeline: retrieval→\\rightarrowrerank→\\rightarrowreason\. Each stage narrows the candidate set before the final per\-criterion scoring\.Table 1:The three pipeline stages\. “Out” is the candidate count each stage emits\.#### Stage 1: Retrieval\.

Each criterion is embedded as its own query vector and searched against the pooled KOL vector table\. A KOL’s similarity to a criterion is its*best\-matching*vector — a per\-KOLMIN\-distance \(least cosine distance\) over all of that KOL’s pooled vectors — and the per\-criterion similarities are summed into a single retrieval score per KOL\. KOLs are ranked by this score, an optional hardtierfilter is applied, and the top 50 candidates are kept, with their profile and top videos fetched for the downstream stages\. This dense\-retrieval stage is tuned for*recall*— not losing relevant KOLs from the candidate pool — rather than final ordering; the ordering work is done downstream by the rerank and reason stages\.

#### Stage 2: Rerank\.

A 4B pointwise reranker scores each candidate in a single forward pass: the model receives the criteria together with the candidate’s profile details and video summaries, and the relevance score is the log\-probability of a singleYestoken\. The top\-10 candidates advance\. Emitting one scored token per candidate keeps the stage’s output cost at∼0\.1\{\\sim\}0\.1k tokens per query\.

#### Stage 3: Reason\.

A 4B reason scorer grades each shortlisted KOL against each criterioncic\_\{i\}on a\{0,1,2\}\\\{0,1,2\\\}ordinal rubric and produces a Thai rationale\. The final ranking score is the even sums​\(k\)=∑i=1msi​\(k\)s\(k\)=\\sum\_\{i=1\}^\{m\}s\_\{i\}\(k\)\. The deployed scorer is the untuned base model \(§[7](https://arxiv.org/html/2607.05968#S7)\); the fine\-tuned variant below is the experimental arm whose end\-to\-end behavior we analyze\.

### 3\.3Supervision Signals

We collect three label types over \(criteria, KOL\) units drawn from synthetic\-but\-realistic Thai briefs \(§[4](https://arxiv.org/html/2607.05968#S4)\): \(i\)*pointwise*per\-criterion ordinal scores in\{0,1,2\}\\\{0,1,2\\\}; \(ii\) KOL\-level*binary*pass/fail verdicts; and \(iii\)*pairwise*best/worst judgments over candidate groups\. The reranker learns from relative judgments, the reason scorer from absolute ones\.

### 3\.4Reranker Training: Pairwise SimPO

We fine\-tune the reranker, as a LoRA adapter\[[3](https://arxiv.org/html/2607.05968#bib.bib3)\], with SimPO\[[7](https://arxiv.org/html/2607.05968#bib.bib1)\]on preference pairs\(kw,kl\)\(k\_\{w\},k\_\{l\}\)of a preferred and dispreferred candidate under the same criteria\. Because the scored response is a single token, SimPO’s length\-normalized reward reduces to theYeslog\-probability itself, and the objective becomes a margin\-augmented logistic loss on the score difference:

ℒ=−log⁡σ​\(β​\(sw−sl\)−γ\),s=log⁡p​\(Yes∣C,k\),\\mathcal\{L\}\\;=\\;\-\\log\\sigma\\\!\\big\(\\beta\\,\(s\_\{w\}\-s\_\{l\}\)\-\\gamma\\big\),\\hskip 18\.49988pts=\\log p\(\\texttt\{Yes\}\\mid C,k\),\(1\)wherep​\(Yes∣C,k\)p\(\\texttt\{Yes\}\\mid C,k\)is the reranker’s probability of theYestoken given criteriaCCand candidatekk— sosws\_\{w\}andsls\_\{l\}are this score at the preferred and dispreferred candidatekw,klk\_\{w\},k\_\{l\}—σ\\sigmais the logistic sigmoid,β\\betaan inverse temperature, andγ\\gammaa fixed target margin\. This is reference\-free and directly shapes the quantity used at inference, so training and serving optimize the same score\.

#### SFT\-judge baseline\.

As a fine\-tuning control, we also train a*direct*A\-vs\-B judge \(SFT\-judge\) by supervised fine\-tuning on the same pairwise labels: the model receives the criteria and two candidates and is trained to emit the token of the preferred one \(AorB\)\. SFT\-judge and SimPO thus share the training data and differ only in objective and inference path — generative A\-vs\-B judgment versus pointwiselog⁡p​\(Yes\)\\log p\(\\texttt\{Yes\}\)scoring — isolating the contribution of the preference objective \(§[6\.1](https://arxiv.org/html/2607.05968#S6.SS1)\)\.

### 3\.5Reason Scorer Training: Pointwise SFT\+GRPO

To test whether absolute per\-criterion supervision transfers to end\-to\-end ranking, we fine\-tune the reason scorer in two phases on human pointwise labels; the labels contain ordinal scores only, with no reference rationales\.SFTadapts the model to the per\-criterion scoring format\.GRPO\[[19](https://arxiv.org/html/2607.05968#bib.bib2)\]then optimizes against the human labels with a single gated score reward: a completion earns non\-zero reward only if it passes a joint gate — a valid ordinal score and a non\-empty rationale — and gated completions are paid by ordinal distance to the human label:1\.01\.0for an exact match,0\.20\.2for an off\-by\-one prediction, and0for a flipped one\. The rationale is thus never directly supervised; the gate only requires that one be produced\.

## 4Data Creation

Our corpora are produced by a three\-stage pipeline: \(i\)*synthetic brief generation*fabricates realistic, brand\-grounded Thai marketer briefs; \(ii\)*criteria synthesis*expands each brief into five matching criteria; and \(iii\)*human annotation*collects pointwise, binary, and pairwise judgments over \(brief, criteria, KOL\) units\. Tables[3](https://arxiv.org/html/2607.05968#S4.T3)and[4](https://arxiv.org/html/2607.05968#S4.T4)summarize the generation pipeline and the resulting labeled datasets\.

### 4\.1Synthetic Briefs and Criteria

Real marketer briefs are scarce and proprietary, so we synthesize briefs mirroring the schema of real reference Thai campaign briefs\. Each brief is seeded by a consumer and a marketer persona sampled from Nemotron\-Personas\-USA\[[12](https://arxiv.org/html/2607.05968#bib.bib8)\]; a campaign scenario is drawn from a six\-axis taxonomy under non\-uniform priors, constrained by 13 hard rules that forbid incoherent combinations \(e\.g\. a crisis\-recovery objective forces a reactive context\)\. The campaign subject is grounded with a live web search, and the brief text — a structured object with brand, product, objective, audience, and tone fields — is written byDeepSeek\-V3\.2\[[1](https://arxiv.org/html/2607.05968#bib.bib7)\], with marketer expertise sampled over three levels to control jargon\.

Each brief is then expanded into exactlyfiveThai criteria by a singleDeepSeek\-V3\.2call, aligned to five fixed dimensions: \(C1\) audience fit, \(C2\) content–product relevance, \(C3\) selling and persuasion style, \(C4\) communication credibility, and \(C5\) content format and execution\. The criteria are*dual\-purpose*: they serve as the Stage\-1 retrieval queries and as the labeling rubric shown to annotators\.

### 4\.2Human Annotation

![Refer to caption](https://arxiv.org/html/2607.05968v1/task.png)Figure 2:The three annotation interfaces presented to human annotators: pointwise scoring \(T2\), where each criterion receives an ordinal0/1/20/1/2score; binary match \(E2E\), which elicits one overall pass/fail verdict per KOL for end\-to\-end evaluation; and best/worst selection \(T10\), where annotators pick the best and worst of three KOLs\.Human supervision is collected through a dedicated labeling application over three task types \(Table[4](https://arxiv.org/html/2607.05968#S4.T4)\)\.T2shows the criteria and one KOL, and elicits an ordinal score∈\{0,1,2\}\\in\\\{0,1,2\\\}per criterion plus a KOL\-level binary pass/fail verdict; the latter is reserved for end\-to\-end evaluation\. Annotators judge each criterion directly: no scoring rubric defining the0/1/20/1/2boundaries was provided, leaving the threshold between scores to each annotator’s judgment\. The interface also presents items grouped by criteria set, so an annotator scores all candidates under one brief consecutively\.T10shows three KOLs and elicits a best/worst pick, conditioning on the five criteria\.

### 4\.3Label Quality

#### Evidence audit \(T2\)\.

A Kimi audit of873human \(KOL, criterion\) pairs checks whether each score is backed by*verbatim*profile quotes, each verified as an exact substring of the profile — a model\-independent grounding check that bounds the label ceiling any model can reach \(full treatment in Results\)\. Grounding is starkly score\-dependent \(Table[2](https://arxiv.org/html/2607.05968#S4.T2)\): only15\.9%of score\-2 \(“match”\) labels are evidence\-supported\. Since score\-2 is the largest class \(≈\\approx48%\) yet≈\\approx84% ungrounded, an evidence\-faithful model*must*disagree with many human labels — the central finding developed in Results\.

Table 2:T2 Kimi evidence auditHuman scorePairsEvidence\-supported \(%\)033778\.9111554\.8242115\.9Table 3:Synthetic generation pipeline\.Table 4:Labeled datasets\.

## 5Experimental Setup

We evaluate at two levels:*offline*, testing each learned stage against its own held\-out human labels, and*end\-to\-end*, testing the full cascade against human relevance labels while measuring serving cost\.

#### Model arms\.

Throughout,Basedenotes the untunedQwen3\.5\-4B\[[17](https://arxiv.org/html/2607.05968#bib.bib5)\];SFTandSFT\+\+GRPOdenote the fine\-tuned reason scorers of §[3\.5](https://arxiv.org/html/2607.05968#S3.SS5);SimPOdenotes the fine\-tuned reranker of §[3\.4](https://arxiv.org/html/2607.05968#S3.SS4); andKimi\-K2\.6\[[8](https://arxiv.org/html/2607.05968#bib.bib6)\]is the frontier baseline, run under the same prompts and input format as the students\. The deployed configuration is SimPO \(text\) rerank followed by Base reason\.

### 5\.1Offline Evaluation: Reranker

#### Data\.

The human pairwise best/worst labels on the T10 test split \(§[4](https://arxiv.org/html/2607.05968#S4)\)\.

#### Metric: best\-pick Exact Match \(EM\)\.

For each triplet the model compares the human\-annotated*best*and*worst*KOL \(the middle KOL is ignored\); the prediction is correct iff it ranks the best above the worst, and EM is the fraction of triplets decided correctly\. The two training objectives induce two distinct inference paths, reported side by side: a*direct*generative A\-vs\-B judge \(used by the frontier baseline and the SFT student\) and an*indirect*log⁡p​\(Yes\)\\log p\(\\texttt\{Yes\}\)scorer \(used by Base and the SimPO student\); their prompt formats are shown in Table[5](https://arxiv.org/html/2607.05968#S5.T5)\. For the indirect scorer we also report the*margin*, the mean separationlog⁡p​\(Yes∣best\)−log⁡p​\(Yes∣worst\)¯\\overline\{\\,\\log p\(\\texttt\{Yes\}\\mid\\text\{best\}\)\-\\log p\(\\texttt\{Yes\}\\mid\\text\{worst\}\)\\,\}in nats — a calibration signal orthogonal to EM\.

#### Baselines\.

\(i\) arandombaseline \(50% EM by construction, since each triplet is a binary best\-vs\-worst decision\); \(ii\) theKimi\-K2\.6frontier baseline as a direct A\-vs\-B judge; and \(iii\)Baseunder both inference paths — the direct judge and the indirectlog⁡p​\(Yes\)\\log p\(\\texttt\{Yes\}\)scorer\.

Table 5:Minimal reranker prompt formats\. The*text\+\+vision*variant also attaches each video’s cover thumbnail, which*text\-only*drops\.

### 5\.2Offline Evaluation: Reason Scorer

#### Data\.

The T2 pointwise corpus \(§[4](https://arxiv.org/html/2607.05968#S4)\): \(KOL, criterion\-set\) units split into train and test, each yielding five criterion\-level ordinal labels\. All models are evaluated against held\-out human labels\.

#### Metric: support\-weighted F1 \(wf1\)\.

We report support\-weighted F1 over the three ordinal classes\. We weight by support rather than macro\-averaging because the class distribution is heavily skewed \(score\-2 is≈\\approx48% of labels, §[4](https://arxiv.org/html/2607.05968#S4)\) and the deployed product cares about the dominant “yes” class, which macro\-F1 under\-weights\. The distinction is not cosmetic: the same frontier\-baseline prompt reads as 33\.6 macro\-F1 but 38\.4 weighted\-F1, because its class\-1 F1 is 0\.00 \(it never emits a confident “unsure”\)\.

#### Baselines\.

\(i\) a stratifiedrandombaseline that samples from the gold class distribution \(20\-seed average\); \(ii\) theKimi\-K2\.6frontier baseline; and \(iii\)Base, prompted for score\-only output\.

### 5\.3End\-to\-End Evaluation

#### Query sets\.

Two human\-labeled query sets are drawn from the binary \(pass/fail\) annotations:

- •Set 1— 11 queries, each with all 50 retrieved KOLs labeled, so precision is measured over the complete re\-ranked list;
- •Set 2— 31 queries, each with the top\-10 KOLs labeled\.

#### Protocol\.

Retrieval is held fixed: the 50\-candidate set per query is frozen, and only the rerank and reason stages are re\-run with different models\. All models use the same prompts and input format as the production pipeline, so measured numbers reflect the deployed system rather than an idealized re\-implementation\. Unless stated, reason runs on the rerank top\-10; reason applied directly to all 50 candidates \(no rerank\) is reported as a contrast\.

#### Relevance and metrics\.

A KOL is relevant under*binary majority*: with vote fractionf=pass/totalf=\\text\{pass\}/\\text\{total\}, it is positive iff\>0\.5f\>0\.5, negative iff<0\.5f<0\.5, and dropped iff=0\.5f=0\.5\. Over the relevant set of sizeRRper query we report, at cutoffk=5k\{=\}5:P​@​k=hits​@​k/min⁡\(k,R\)\\text\{P\}@k=\\text\{hits\}@k/\\min\(k,R\);MAPB​@​k\\text\{MAP\}\_\{\\text\{B\}\}@k, binary\-relevance average precision normalized bymin⁡\(R,k\)\\min\(R,k\); andnDCG​@​k=DCG​@​k/IDCG​\(min⁡\(k,R\)\)\\text\{nDCG\}@k=\\text\{DCG\}@k/\\text\{IDCG\}\(\\min\(k,R\)\)\. All metrics are macro\-averaged over queries withR≥1R\\geq 1\.

## 6Results

### 6\.1Reranker: Offline Best\-Pick Accuracy

Table 6:Reranker best\-pick EM on the human pairwise T10 test set\.EM \(%\)MarginModelDirect judgelog⁡p​\(Yes\)\\log p\(\\texttt\{Yes\}\)\(nats\)Random50\.050\.0—Kimi\-K2\.678\.0 \(32/41\)——Base75\.6 \(31/41\)63\.4 \(26/41\)\+0\.76\+0\.76SFT75\.6 \(31/41\)——SimPO—78\.0\(32/41\)\+4\.28\+4\.28Table[6](https://arxiv.org/html/2607.05968#S6.T6)reads down each column, since the two inference paths are not strictly comparable cell\-for\-cell\. On the indirectlog⁡p​\(Yes\)\\log p\(\\texttt\{Yes\}\)path — the one the pipeline serves — SimPO training lifts Base from 63\.4 to78\.0EM \(\+14\.6\+14\.6points\) and widens the best–worst margin roughly6×6\\times\(\+0\.76→\+4\.28\+0\.76\\to\+4\.28\)\. The resulting 4B student matches the frontier baseline’s direct\-judge EM \(78\.0\)\. On the direct path, the judge\-style SFT \(75\.6\) sits at Base’s level: the gain is attributable to the SimPO objective, not the SFT\-judge format\.

### 6\.2Reason Scorer: Offline Per\-Criterion Accuracy

Table 7:Reason\-scorer offline leaderboard \(support\-weighted F1 against held\-out human labels\)\.ModelEval modewf1Δ\\Deltavs no\-thinknnRandom baseline–43\.6–259Kimi\-K2\.6no\-think36\.8–250Kimi\-K2\.6think38\.4–244Baseno\-think52\.9–254Basethink44\.9−8\.0\-8\.0254SFTno\-think57\.7–244SFT\+\+GRPOno\-think55\.9–254SFT\+\+GRPOthink59\.0\+3\.1\+3\.1254Table[7](https://arxiv.org/html/2607.05968#S6.T7)reports the offline leaderboard\. The frontier baseline scores38\.4wf1 —*below*the stratified random floor of 43\.6 — because its evidence\-faithful behavior collapses the middle class and withholds the generous “yes” that human annotators assign freely\. Base reaches52\.9, above both\. Fine\-tuning helps offline: score\-only SFT reaches57\.7\(\+19\.3\+19\.3pp over the frontier baseline,\+14\.1\+14\.1pp over random\), and SFT\+\+GRPO tops the leaderboard at59\.0wf1 in think mode\. During GRPO, training reward climbs from 0\.5 to 0\.94 over 75 steps; since the gate pays only completions containing a rationale, the reward curve also certifies that the model learns to explain its scores\.

#### Chain\-of\-thought\.

The effect of*think*mode\[[22](https://arxiv.org/html/2607.05968#bib.bib23)\]depends on training: thinking degrades Base by 8\.0pp \(52\.9→\\to44\.9\) but helps SFT\+\+GRPO by 3\.1pp \(55\.9→\\to59\.0\)\.

### 6\.3End\-to\-End Ranking

#### Set 1 \(full 50\-KOL labeled pool\)\.

Table[8](https://arxiv.org/html/2607.05968#S6.T8)reports Set 1 \(n=11n\{=\}11\)\. Retrieval alone barely separates from random on P@5 and is at or below random on rank quality \(MAPB\{\}\_\{\\text\{B\}\}/nDCG\), confirming that embedding recall orders candidates only weakly\. Rerank lifts P@5 by∼\\sim11–15 points; the decisive gain comes from the reason stage\. The best configuration — SimPO text rerank followed by Base reason on the top\-10 — reaches94\.1%P@5,\+26\.8\+26\.8points over rerank alone and\+39\.6\+39\.6over retrieval\.

Table 8:Set 1 end\-to\-end results \(n=11n\{=\}11queries, top\-50 labeled\)\. Best P@5 inbold\.Three observations stand out\. First, rerank\-then\-reason beats reason\-on\-all\-50 on*both*accuracy and cost: filtering removes weak candidates that would otherwise dilute the reason ranking, so the smaller pool is also the better one \(94\.1 vs\. 80\.0 P@5\)\. Second, the upstream reranker matters: pairing reason with the SimPO reranker rather than Base adds∼\\sim2–8 points to final P@5, because reason inherits a cleaner top\-10\. Third, Base reason outperforms SFT\+\+GRPO everywhere under the deployed prompt — the offline–end\-to\-end inversion analyzed in §[7](https://arxiv.org/html/2607.05968#S7)\.

#### Set 2\.

Table[9](https://arxiv.org/html/2607.05968#S6.T9)replicates the headline trend on the larger query set \(n=31n\{=\}31, top\-10 labeled\)\. Rerank adds\+6\.7\+6\.7points over retrieval, and reason adds a further\+7\.6\+7\.6to\+11\.9\+11\.9depending on the reasoner\. SFT\+\+GRPO is the weakest reasoner in both sets, but Base and Kimi\-K2\.6 swap the top rank: Base\>\>Kimi\-K2\.6 on Set 1 \(94\.1 vs\. 91\.8\), whereas Kimi\-K2\.6\>\>Base here \(91\.9 vs\. 89\.3\)\. Gaps also compress on this smaller labeled pool\.

Table 9:Set 2 end\-to\-end results \(n=31n\{=\}31queries, top\-10 labeled\)\. All reason rows use the SimPO text reranker upstream\. Best P@5 inbold\.

### 6\.4Cost vs\. Accuracy

Table 10:Per\-query cost on Set 1 \(n=11n\{=\}11\)\. Pareto\-optimal rows inbold\.Table[10](https://arxiv.org/html/2607.05968#S6.T10)reports per\-query token consumption \(Set 1,n=11n\{=\}11\); token columns are average tokens per query\. Cost is measured in tokens \(input\+\+output\), which is concurrency\-independent and therefore the fair axis; wall\-clock is indicative only, as serving concurrency differs across models \(single A100\-SXM4\-80GB; Kimi\-K2\.6 at concurrency 16 on top\-10 / 30 on all\-50, all others at 50\)\. Vision rerank excludes a one\-time∼\\sim39 s/query image fetch \(∼\\sim452 thumbnails\)\.

The Pareto frontier comprises four points, in increasing cost: SimPO text rerank \(67\.3% at 251\.6k tokens\), SimPO vision rerank \(69\.1% at 300\.5k\), SimPO rerank→\\rightarrowSFT\+\+GRPO reason \(85\.9% at 419\.3k\), and SimPO rerank→\\rightarrowBase reason \(94\.1% at 419\.9k\) — the clear knee of the curve\. The SFT\+\+GRPO point is only marginally non\-dominated: 0\.6k tokens cheaper than the Base\-reason point but 8\.2 points lower, so Base reason is the practical operating choice\. Three trade\-offs follow:

- •Reason\-on\-all\-50costs∼\\sim1\.9×\\timesthe tokens of the two\-stage pipeline \(797k vs\. 420k\) yet scores 14 points lower \(80\.0 vs\. 94\.1\) — rerank\-then\-reason is both cheaper and better\. The gap is starker for Kimi\-K2\.6: reasoning over all 50 costs∼\\sim2\.7×\\timesits two\-stage tokens \(1,552k vs\. 578k\) for 13\.6 points less \(78\.2 vs\. 91\.8\)\.
- •Kimi\-K2\.6emits∼\\sim35×\\timesmore output tokens \(102\.9k vs\. 3\.0k per query\) for*lower*Set 1 accuracy \(91\.8 vs\. 94\.1\), making it strictly cost\-dominated by the local 4B Base reasoner on this set\.
- •Vision rerank, though on the frontier, buys only\+1\.8\+1\.8points over text for\+19%\+19\\%input tokens and∼\\sim3\.3×\\timesthe latency, so text rerank remains the cost\-effective default\.

## 7Discussion

The best offline reason scorer is the weakest fine\-tuned reasoner end to end: the SFT\+\+GRPO checkpoint tops the per\-criterion leaderboard \(59\.0 wf1 with thinking, Table[7](https://arxiv.org/html/2607.05968#S6.T7)\) yet trails the untuned Base in the deployed cascade \(85\.9 vs\. 94\.1 P@5 on Set 1; 87\.6 vs\. 89\.3 on Set 2\)\. We attribute this inversion primarily to how the labeling task was designed, through three compounding choices\.

1. 1\.Absolute scoring is under\-specified\.A single\{0,1,2\}\\\{0,1,2\\\}per\-criterion judgment is intrinsically hard, and no scoring rubric defining the class boundaries was provided \(§[4\.2](https://arxiv.org/html/2607.05968#S4.SS2)\), so each annotator adopted a private standard — a known driver of human label variation\[[13](https://arxiv.org/html/2607.05968#bib.bib11)\]\. The evidence audit quantifies the consequence: only 15\.9% of score\-2 \(“match”\) labels are backed by verbatim profile evidence \(Table[2](https://arxiv.org/html/2607.05968#S4.T2)\)\.
2. 2\.The interface grouped items by criteria set\.Annotators scored all candidates under one brief consecutively \(§[4\.2](https://arxiv.org/html/2607.05968#S4.SS2)\), so within\-batch labels are correlated rather than independent judgments\.
3. 3\.The two regimes use different human rubrics\.End\-to\-end relevance is a holistic KOL\-level pass/fail, whereas the reason scorer is supervised one criterion at a time and aggregated by an even sum\. Even with clean labels, a model fit to per\-criterion targets optimizes an objective that only loosely tracks the holistic signal — in our reading, the primary cause of the inversion\. Choices 1–2 compound this: the per\-criterion targets are not only a proxy objective but a noisy one\.

#### The plateau is label\-imposed, not capacity\-imposed\.

Restricting evaluation to the≈\\approx38% of test items the evidence audit marks*clear\-cut*, the SFT checkpoint rises from 57\.7 to73\.4wf1 \(clear\-2 F1 91\.7\) and SFT\+\+GRPO reaches80\.6\. On clean labels the 4B model is excellent; the full\-set plateau reflects the model*correctly disagreeing*with ungrounded labels\. This corroborates choice 1: the fine\-tuned scorer learned the annotators’ generous, weakly grounded “yes” — exactly the signal that does not survive the holistic end\-to\-end test\.

#### Text rerank over vision\.

The deployed reranker is text\-only\. Adding thumbnails carries a real serving cost —∼19%\{\\sim\}19\\%more input tokens,∼3\.3×\{\\sim\}3\.3\\timesthe rerank wall\-clock, plus a one\-time image fetch — while buying only\+1\.8\+1\.8P@5 end\-to\-end \(Table[8](https://arxiv.org/html/2607.05968#S6.T8)\)\. The marginal gain does not justify the added latency and image\-processing cost\.

#### Why filtering before reasoning wins\.

Reasoning over all 50 candidates loses to rerank\-then\-reason \(80\.0 vs\. 94\.1 P@5\) despite seeing strictly more of the pool\. We read the gap as a division of labor between complementary stages\. The reranker, trained on pairwise preferences, is strong at*relative*filtering but is not a precise judge — rerank alone tops out at 67–69 P@5\. The reason scorer is the precise judge, but its per\-criterion even sum has coarse resolution over a large pool: with five criteria scored in\{0,1,2\}\\\{0,1,2\\\}, a 50\-candidate field produces ties and near\-ties in which weak candidates dilute the top\-5\. Filtering first with the preference\-trained reranker hands the reasoner a short pool where its precision is decisive; each stage supplies what the other lacks\.

## Limitations

#### Small samples\.

Set 1 hasn=11n\{=\}11queries and Set 2n=31n\{=\}31; the pairwise test set has 41 triplets, so a single flipped triplet moves EM by≈\\approx2\.4pp\. Differences of a few points should be read as directional, not significant\.

#### Audit provenance\.

The evidence audit is produced by a frontier model acting as an independent auditor; although every quote is verified as an exact substring of the profile \(a model\-independent check\), the selection of quotes and the grounding judgment inherit the auditor’s biases\.

#### Synthetic briefs and single market\.

Training and evaluation queries derive from synthetic briefs; despite schema\-mirroring and web grounding, their distribution may differ from real proprietary briefs\. All experiments target the Thai KOL market with one model family \(4B\), so transfer to other languages, markets, and scales is untested\.

#### Serving comparability\.

Wall\-clock timings are not like\-for\-like across models \(vLLM stages at concurrency 50, the frontier baseline at 16–30\); cost conclusions rest on the token axis\.

## 8Conclusion

We presented InfluMatch, a three\-stage retrieval→\\rightarrowrerank→\\rightarrowreason cascade that matches free\-form Thai marketer criteria to KOLs using only small open\-weight models\. The deployed system — a SimPO\-tuned 4B reranker followed by an untuned 4B reason scorer — reaches 89–94% P@5 against human relevance labels, matching or approaching a frontier model at∼35×\{\\sim\}35\\timesfewer output tokens, and shows that the cascade design itself, filtering before reasoning, is both cheaper and more accurate than reasoning over the full candidate pool\.

Beyond the system, our central finding concerns supervision\. Of the two fine\-tuned students, only the one trained on*relative*judgments survived end\-to\-end evaluation: pairwise SimPO transferred and matched the frontier baseline, while pointwise SFT\+\+GRPO topped the offline leaderboard yet degraded the deployed ranking — an inversion we trace, via an evidence audit, to the design of the absolute labeling task\. On this evidence, pointwise fine\-tuning was the wrong objective*for this label distribution*: the recoverable supervision signal in this domain lives in relative judgments, not absolute scores\.

## References

- \[1\]DeepSeek\-AI\(2025\)DeepSeek\-V3\.2: pushing the frontier of open large language models\.arXiv preprint arXiv:2512\.02556\.Cited by:[§4\.1](https://arxiv.org/html/2607.05968#S4.SS1.p1.1)\.
- \[2\]F\. Garcin, B\. Faltings, O\. Donatsch, A\. Alazzawi, C\. Bruttin, and A\. Huber\(2014\)Offline and online evaluation of news recommender systems at swissinfo\.ch\.InProceedings of the 8th ACM Conference on Recommender Systems \(RecSys\),pp\. 169–176\.External Links:[Document](https://dx.doi.org/10.1145/2645710.2645745)Cited by:[§2](https://arxiv.org/html/2607.05968#S2.SS0.SSS0.Px5.p1.1)\.
- \[3\]E\. J\. Hu, Y\. Shen, P\. Wallis, Z\. Allen\-Zhu, Y\. Li, S\. Wang, L\. Wang, and W\. Chen\(2022\)LoRA: low\-rank adaptation of large language models\.InInternational Conference on Learning Representations \(ICLR\),External Links:[Link](https://arxiv.org/abs/2106.09685)Cited by:[§3\.4](https://arxiv.org/html/2607.05968#S3.SS4.p1.1)\.
- \[4\]V\. Karpukhin, B\. Oğuz, S\. Min, P\. Lewis, L\. Wu, S\. Edunov, D\. Chen, and W\. Yih\(2020\)Dense passage retrieval for open\-domain question answering\.InProceedings of the 2020 Conference on Empirical Methods in Natural Language Processing \(EMNLP\),pp\. 6769–6781\.External Links:[Document](https://dx.doi.org/10.18653/v1/2020.emnlp-main.550)Cited by:[§2](https://arxiv.org/html/2607.05968#S2.SS0.SSS0.Px2.p1.1)\.
- \[5\]S\. Kim, J\. Shin, Y\. Cho, J\. Jang, S\. Longpre, H\. Lee, S\. Yun, S\. Shin, S\. Kim, J\. Thorne, and M\. Seo\(2024\)Prometheus: inducing fine\-grained evaluation capability in language models\.InInternational Conference on Learning Representations \(ICLR\),External Links:[Link](https://arxiv.org/abs/2310.08491)Cited by:[§2](https://arxiv.org/html/2607.05968#S2.SS0.SSS0.Px4.p1.1)\.
- \[6\]S\. Kiritchenko and S\. M\. Mohammad\(2017\)Best\-worst scaling more reliable than rating scales: a case study on sentiment intensity annotation\.InProceedings of the 55th Annual Meeting of the Association for Computational Linguistics \(Volume 2: Short Papers\),Vancouver, Canada,pp\. 465–470\.Cited by:[§2](https://arxiv.org/html/2607.05968#S2.SS0.SSS0.Px5.p1.1)\.
- \[7\]Y\. Meng, M\. Xia, and D\. Chen\(2024\)SimPO: simple preference optimization with a reference\-free reward\.InAdvances in Neural Information Processing Systems \(NeurIPS\),External Links:[Link](https://arxiv.org/abs/2405.14734)Cited by:[§2](https://arxiv.org/html/2607.05968#S2.SS0.SSS0.Px5.p1.1),[§3\.4](https://arxiv.org/html/2607.05968#S3.SS4.p1.1)\.
- \[8\]Moonshot AI\(2026\)Kimi K2\.6\.Note:[https://huggingface\.co/moonshotai/Kimi\-K2\.6](https://huggingface.co/moonshotai/Kimi-K2.6)Model card and technical blog, April 2026Cited by:[§5](https://arxiv.org/html/2607.05968#S5.SS0.SSS0.Px1.p1.1)\.
- \[9\]R\. Nogueira and K\. Cho\(2019\)Passage re\-ranking with BERT\.arXiv preprint arXiv:1901\.04085\.Cited by:[§2](https://arxiv.org/html/2607.05968#S2.SS0.SSS0.Px2.p1.1)\.
- \[10\]R\. Nogueira, Z\. Jiang, R\. Pradeep, and J\. Lin\(2020\)Document ranking with a pretrained sequence\-to\-sequence model\.InFindings of the Association for Computational Linguistics: EMNLP 2020,pp\. 708–718\.External Links:[Document](https://dx.doi.org/10.18653/v1/2020.findings-emnlp.63),[Link](https://aclanthology.org/2020.findings-emnlp.63/)Cited by:[§2](https://arxiv.org/html/2607.05968#S2.SS0.SSS0.Px3.p1.1)\.
- \[11\]R\. Nogueira, W\. Yang, K\. Cho, and J\. Lin\(2019\)Multi\-stage document ranking with BERT\.arXiv preprint arXiv:1910\.14424\.Cited by:[§2](https://arxiv.org/html/2607.05968#S2.SS0.SSS0.Px2.p1.1)\.
- \[12\]NVIDIA\(2025\)Nemotron\-personas\-USA: synthesized personas grounded in real\-world distributions\.Note:[https://huggingface\.co/datasets/nvidia/Nemotron\-Personas\-USA](https://huggingface.co/datasets/nvidia/Nemotron-Personas-USA)CC BY 4\.0\. Generated with NeMo Data DesignerCited by:[§4\.1](https://arxiv.org/html/2607.05968#S4.SS1.p1.1)\.
- \[13\]B\. Plank\(2022\)The “problem” of human label variation: on ground truth in data, modeling and evaluation\.InProceedings of the 2022 Conference on Empirical Methods in Natural Language Processing \(EMNLP\),Abu Dhabi, United Arab Emirates\.Cited by:[item 1](https://arxiv.org/html/2607.05968#S7.I1.i1.p1.1)\.
- \[14\]R\. Pradeep, S\. Sharifymoghaddam, and J\. Lin\(2023\)RankVicuna: zero\-shot listwise document reranking with open\-source large language models\.arXiv preprint arXiv:2309\.15088\.Cited by:[§2](https://arxiv.org/html/2607.05968#S2.SS0.SSS0.Px3.p1.1)\.
- \[15\]R\. Pradeep, S\. Sharifymoghaddam, and J\. Lin\(2023\)RankZephyr: effective and robust zero\-shot listwise reranking is a breeze\!\.arXiv preprint arXiv:2312\.02724\.Cited by:[§2](https://arxiv.org/html/2607.05968#S2.SS0.SSS0.Px3.p1.1)\.
- \[16\]Z\. Qin, R\. Jagerman, K\. Hui, H\. Zhuang, J\. Wu, L\. Yan, J\. Shen, T\. Liu, J\. Liu, D\. Metzler, X\. Wang, and M\. Bendersky\(2024\)Large language models are effective text rankers with pairwise ranking prompting\.InFindings of the Association for Computational Linguistics: NAACL 2024,External Links:[Link](https://arxiv.org/abs/2306.17563)Cited by:[§2](https://arxiv.org/html/2607.05968#S2.SS0.SSS0.Px3.p1.1)\.
- \[17\]Qwen Team\(2026\)Qwen3\.5: towards native multimodal agents\.Note:[https://qwen\.ai/blog?id=qwen3\.5](https://qwen.ai/blog?id=qwen3.5)Release blog, February 2026\. Small model series \(0\.8B–9B\) released March 2026Cited by:[§5](https://arxiv.org/html/2607.05968#S5.SS0.SSS0.Px1.p1.1)\.
- \[18\]R\. Rafailov, A\. Sharma, E\. Mitchell, S\. Ermon, C\. D\. Manning, and C\. Finn\(2023\)Direct preference optimization: your language model is secretly a reward model\.InAdvances in Neural Information Processing Systems \(NeurIPS\),External Links:[Link](https://arxiv.org/abs/2305.18290)Cited by:[§2](https://arxiv.org/html/2607.05968#S2.SS0.SSS0.Px5.p1.1)\.
- \[19\]Z\. Shao, P\. Wang, Q\. Zhu, R\. Xu, J\. Song, X\. Bi, H\. Zhang, M\. Zhang, Y\. K\. Li, Y\. Wu, and D\. Guo\(2024\)DeepSeekMath: pushing the limits of mathematical reasoning in open language models\.arXiv preprint arXiv:2402\.03300\.Cited by:[§2](https://arxiv.org/html/2607.05968#S2.SS0.SSS0.Px5.p1.1),[§3\.5](https://arxiv.org/html/2607.05968#S3.SS5.p1.3)\.
- \[20\]W\. Sun, L\. Yan, X\. Ma, S\. Wang, P\. Ren, Z\. Chen, D\. Yin, and Z\. Ren\(2023\)Is ChatGPT good at search? investigating large language models as re\-ranking agents\.InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing \(EMNLP\),Singapore,pp\. 14918–14937\.External Links:[Link](https://arxiv.org/abs/2304.09542)Cited by:[§2](https://arxiv.org/html/2607.05968#S2.SS0.SSS0.Px3.p1.1)\.
- \[21\]P\. Wang, L\. Li, L\. Chen, Z\. Cai, D\. Zhu, B\. Lin, Y\. Cao, L\. Kong, Q\. Liu, T\. Liu, and Z\. Sui\(2024\)Large language models are not fair evaluators\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),Bangkok, Thailand,pp\. 9440–9450\.External Links:[Link](https://arxiv.org/abs/2305.17926)Cited by:[§2](https://arxiv.org/html/2607.05968#S2.SS0.SSS0.Px4.p1.1)\.
- \[22\]J\. Wei, X\. Wang, D\. Schuurmans, M\. Bosma, B\. Ichter, F\. Xia, E\. H\. Chi, Q\. V\. Le, and D\. Zhou\(2022\)Chain\-of\-thought prompting elicits reasoning in large language models\.InAdvances in Neural Information Processing Systems \(NeurIPS\),External Links:[Link](https://arxiv.org/abs/2201.11903)Cited by:[§6\.2](https://arxiv.org/html/2607.05968#S6.SS2.SSS0.Px1.p1.3)\.
- \[23\]L\. Zheng, W\. Chiang, Y\. Sheng, S\. Zhuang, Z\. Wu, Y\. Zhuang, Z\. Lin, Z\. Li, D\. Li, E\. P\. Xing, H\. Zhang, J\. E\. Gonzalez, and I\. Stoica\(2023\)Judging LLM\-as\-a\-judge with MT\-bench and chatbot arena\.InAdvances in Neural Information Processing Systems \(NeurIPS\), Datasets and Benchmarks Track,External Links:[Link](https://arxiv.org/abs/2306.05685)Cited by:[§2](https://arxiv.org/html/2607.05968#S2.SS0.SSS0.Px4.p1.1)\.