Tag
The paper introduces TasteBench, an expert-gold pairwise benchmark for measuring 'taste' in VLMs across visual, brand, book, and tech domains, along with a control protocol that quantifies and corrects evaluation artifacts like position bias, lexical leakage, and difficulty ceilings.
The paper investigates using cross-model disagreement in LLMs to target expert effort in revising codebooks for large-scale text annotation, showing that rationale labeling outperforms other feedback methods.