Consensus vs. Dissent: Dynamic LLM Modeling of Subjective Preferences in Group Recommenders

arXiv cs.CL Papers

Summary

This research fine-tunes LLMs on human survey data to serve as judgmental models for group recommender systems, dynamically selecting aggregation strategies to maximize satisfaction and consensus. A user study validates that the approach aligns with human fairness and satisfaction perceptions.

arXiv:2607.10235v1 Announce Type: new Abstract: Previous work in group recommender systems has demonstrated a sensitivity to the distribution of preferences within a group. Specifically, the selection of the preference aggregation strategy benefits from considering such group configurations. In this paper, we study whether LLMs are able to mimic this sensitivity and to select the ideal aggregation strategy (and corresponding recommendation) according to nuanced human perceptions of fairness, satisfaction, and consensus. We do this by fine-tuning Large Language Models (LLMs) on human survey data to serve as real-time judgmental models within the recommendation pipeline. Using a reasoning dataset distilled from DeepSeek-V3.1 and human ground truth assessments, we develop Judgmental Llama and Judgmental OLMo to simulate group assessments. Our pipeline successfully generates multiple recommendation candidates based on social choice-based aggregation strategies and dynamically selects the one that maximizes these predicted human-like evaluations. We further validate these suggestions in a user study (n=284) and find that our methodology achieved the highest scores for satisfaction and group consensus. Furthermore, we find that LLM judgments are most aligned with human perceptions of fairness, satisfaction and consensus when we also consider interaction effects between our LLM-based method and group configuration (e.g., minority or coalition). These findings give further support for dynamically adapting aggregation strategies to specific within-group preference distributions, and highlight the advantage of using LLMs for an adaptation that is aligned with subjective human judgments.
Original Article
View Cached Full Text

Cached at: 07/14/26, 04:21 AM

# Consensus vs. Dissent: Dynamic LLM Modeling of Subjective Preferences in Group Recommenders
Source: [https://arxiv.org/html/2607.10235](https://arxiv.org/html/2607.10235)
###### Abstract\.

Previous work in group recommender systems has demonstrated a sensitivity to the distribution of preferences within a group\. Specifically, the selection of the preference aggregation strategy benefits from considering such group configurations\. In this paper, we study whether LLMs are able to mimic this sensitivity and to select the ideal aggregation strategy \(and corresponding recommendation\) according to nuanced human perceptions of fairness, satisfaction, and consensus\.

We do this by fine\-tuning Large Language Models \(LLMs\) on human survey data to serve as real\-time judgmental models within the recommendation pipeline\. Using a reasoning dataset distilled from DeepSeek\-V3\.1 and human ground truth assessments, we develop Judgmental Llama and Judgmental OLMo to simulate group assessments\. Our pipeline successfully generates multiple recommendation candidates based on social choice\-based aggregation strategies and dynamically selects the one that maximizes these predicted human\-like evaluations\. We further validate these suggestions in a user study \(n=284n=284\) and find that our methodology achieved the highest scores for satisfaction and group consensus\. Furthermore, we find that LLM judgments are most aligned with human perceptions of fairness, satisfaction and consensus when we also consider interaction effects between our LLM\-based method and group configuration \(e\.g\., minority or coalition\)\. These findings give further support for dynamically adapting aggregation strategies to specific within\-group preference distributions, and highlight the advantage of using LLMs for an adaptation that is aligned with subjective human judgments\.

Large Language Models, Fairness, Group Recommender Systems, LLM\-as\-judge, User study

††ccs:Information systems Recommender systems††ccs:Computing methodologies Natural language generation## 1\.Introduction

The challenge for Group Recommender Systems \(GRS\) is the inherent difficulty of generating and adapting recommendations to the nuanced, sometimes conflicting, preferences of distinct group members\. GRS research has often relied on social choice\-based aggregation strategies, such as Average, Least Misery, or Most Pleasure, to bridge the gap between individual ratings and group outcomes\(Senotet al\.,[2010](https://arxiv.org/html/2607.10235#bib.bib1)\)\. While these methods are computationally efficient, transparent, and well\-evaluated in the literature, they are essentially static mathematical calculations\. To evaluate their effectiveness, researchers typically conduct user studies to gather subjective assessments of perceived satisfaction, fairness, and group consensus from human participants\(Barileet al\.,[2023](https://arxiv.org/html/2607.10235#bib.bib16),[2026](https://arxiv.org/html/2607.10235#bib.bib97)\)\.

However, a significant gap exists between evaluation and implementation\. While user studies provide invaluable insights into how recommendations may be perceived by a group, these subjective metrics cannot be easily integrated into an automated, real\-time recommendation pipeline\. Current systems lack a scalable mechanism to predict how a specific group will feel about a recommendation before it is presented\. As a result, GRS are not adaptable to account for the nuanced group scenarios and dynamics that human participants account for during a user evaluation\.

In this paper, we propose a novel approach that bridges this gap by leveraging Large Language Models \(LLMs\) and the survey data in the literature\. We utilize existing human data to fine\-tune LLMs, transforming them intojudgmental modelscapable of delivering human\-like evaluations\. By integrating these models directly into a recommendation pipeline, we move beyond static aggregation\. More precisely, our system generates multiple recommendation options via various social choice\-based strategies and employs our fine\-tuned models to generate multiple assessments per outcome, mimicking a group of human participants evaluating the specific group recommendation\. Afterwards, our system selects the outcome that maximizes these human\-like assessments, providing the group with a final recommendation which was perceived as fair, satisfactory and as group consensus for their specific preference profile\. We evaluated our approach through a user study, measuring our dynamic, LLM\-based method against traditional static strategies\. Our user evaluation showed that our LLM\-based selection of aggregation strategy achieved the highest average scores for perceived fairness and consensus, as well as the second highest for perceived satisfaction\. All in all, our results show that our dynamic system is capable of managing both polar opposites: groups with high consensus \(uniform\) and groups with clear dissent \(coalitional\)\. Full documentation, code and data are available in the companion repository:[https://github\.com/Cwaterschoot/Consensus\-vs\-dissent\-2026/tree/main](https://github.com/Cwaterschoot/Consensus-vs-dissent-2026/tree/main)

In short, our paper makes the following contributions:

- •We present a distilledreasoning datasetbased on DeepSeek\-V3\.1, grounded in human assessments of fairness, satisfaction, and consensus fromBarileet al\.\([2023](https://arxiv.org/html/2607.10235#bib.bib16)\); Barile \([2023](https://arxiv.org/html/2607.10235#bib.bib100)\)\.
- •We introduceJudgmental Llama and Judgmental OLMo, models specifically fine\-tuned and evaluated on human judgment and distilled reasoning to simulate group assessments\.
- •We develop an end\-to\-endrecommendation pipelinethat employs these fine\-tuned models to dynamically select group recommendation outcomes based on human\-like evaluations\.
- •We conduct a user study \(n=284n=284\) to evaluate the effectiveness of our judgment\-based GRS and show that our proposed method achieved the highest assessments for fairness and consensus\. Additionally, our user study showcases statistically significant interactions with group configuration\.

Table 1\.Preference aggregation strategies used in this study\(Tranet al\.,[2019](https://arxiv.org/html/2607.10235#bib.bib3); Barileet al\.,[2023](https://arxiv.org/html/2607.10235#bib.bib16); Felferniget al\.,[2018](https://arxiv.org/html/2607.10235#bib.bib52)\)
## 2\.Related Work

In the following paragraphs, we outline the related work on GRS and the use of LLMs within the context of group recommendation\. Additionally, we discuss fairness in \(group\) recommendation and how it is evaluated in the literature\. Finally, we present the literature on using LLMs to automate the evaluation of such metrics under the paradigm of LLM\-as\-judge\.

### 2\.1\.Group Recommendation

Traditional recommender systems generate a recommendation on the individual level\. In contrast, GRS need to process multiple, possible conflicting, preferences to produce a group recommendation which reflects the preferences of the group members\(Masthoff and Delić,[2022](https://arxiv.org/html/2607.10235#bib.bib46)\)\. We find recent applications for GRS in restaurant recommendation \(e\.g\.,\(Barileet al\.,[2021](https://arxiv.org/html/2607.10235#bib.bib10),[2023](https://arxiv.org/html/2607.10235#bib.bib16); Waterschootet al\.,[2025c](https://arxiv.org/html/2607.10235#bib.bib56)\), tourism\(Chenet al\.,[2021](https://arxiv.org/html/2607.10235#bib.bib44); Delićet al\.,[2024](https://arxiv.org/html/2607.10235#bib.bib4)\), content moderation\(Waterschoot and van den Bosch,[2024](https://arxiv.org/html/2607.10235#bib.bib115)\), or music recommendation\(Najafian and Tintarev,[2018](https://arxiv.org/html/2607.10235#bib.bib45)\)\.

A large variety of methodologies have been proposed to generate such group recommendations based on a set of individual preference ratings\.Caoet al\.\([2018](https://arxiv.org/html/2607.10235#bib.bib59)\)andHuanget al\.\([2020](https://arxiv.org/html/2607.10235#bib.bib60)\)put forward GRS based on attentive neural networks, while others have proposed graph neural networks\(Zhanget al\.,[2021](https://arxiv.org/html/2607.10235#bib.bib61)\)or reinforcement learning\(Stratigiet al\.,[2023](https://arxiv.org/html/2607.10235#bib.bib62)\)\. While these recent methodologies present promising avenues for generating group recommendations, social choice\-based aggregation strategies, rooted inSocial Choice Theory\(Kelly,[2013](https://arxiv.org/html/2607.10235#bib.bib43)\), remain a prominent procedure in the GRS literature, whether as main aggregation step\(Barileet al\.,[2023](https://arxiv.org/html/2607.10235#bib.bib16); Tranet al\.,[2019](https://arxiv.org/html/2607.10235#bib.bib3); Waterschootet al\.,[2025b](https://arxiv.org/html/2607.10235#bib.bib57)\)or baseline in comparison to more complex models \(e\.g\.,\(Nguyenet al\.,[2019](https://arxiv.org/html/2607.10235#bib.bib70); Rossiet al\.,[2018](https://arxiv.org/html/2607.10235#bib.bib72); Delicet al\.,[2018](https://arxiv.org/html/2607.10235#bib.bib71); Tommasel,[2024](https://arxiv.org/html/2607.10235#bib.bib69)\)\)\. Additionally, neural network\-based approaches require large datasets containing group decisions and feedback that are not widely available\. On the other hand, social choice\-based aggregation strategies offer a wide range of calculations transforming a simple matrix of individual preferences \(user\-item ratings\) into a singular group outcome\(Masthoff,[2015](https://arxiv.org/html/2607.10235#bib.bib2); Senotet al\.,[2010](https://arxiv.org/html/2607.10235#bib.bib1)\)\. They are often categorized as eitherborderline\(using only a subset of ratings, e\.g\., Least Misery or Most Pleasure\),majority\-based\(only using the most popular items/ratings, e\.g\., Approval Voting\) orconsensus\-based\(using all ratings, e\.g\., Additive Utilitarian\)\. In this study, we include strategies from all three categories\. Included aggregation strategies are summarized in Table[1](https://arxiv.org/html/2607.10235#S1.T1)\. We implement social choice\-based aggregation instead of more recent approaches due to their transparent aggregation procedures and widely spread use in the literature, maximizing reproducibility\.

Recently, LLMs have been implemented within the GRS pipeline for a variety of reasons\. LLMs have been used as decision\-making for both individual\(Baoet al\.,[2024](https://arxiv.org/html/2607.10235#bib.bib63); Liaoet al\.,[2024](https://arxiv.org/html/2607.10235#bib.bib64)\)and group recommendations\(Tommasel,[2024](https://arxiv.org/html/2607.10235#bib.bib69); Waterschootet al\.,[2025a](https://arxiv.org/html/2607.10235#bib.bib58)\)\. Additionally, LLMs have been used for generating explanations for \(group\) recommender systems\(Luboset al\.,[2024](https://arxiv.org/html/2607.10235#bib.bib66); Said,[2025](https://arxiv.org/html/2607.10235#bib.bib67); Waterschootet al\.,[2025a](https://arxiv.org/html/2607.10235#bib.bib58)\)and to address specific issues such as cold\-start problems\(Sanneret al\.,[2023](https://arxiv.org/html/2607.10235#bib.bib121); Wuet al\.,[2024](https://arxiv.org/html/2607.10235#bib.bib107); Daiet al\.,[2023](https://arxiv.org/html/2607.10235#bib.bib108)\)or cross\-domain recommendation\(Petruzzelliet al\.,[2024](https://arxiv.org/html/2607.10235#bib.bib122); Kimet al\.,[2024](https://arxiv.org/html/2607.10235#bib.bib123)\)\. A recent focus are conversational recommender systems employing an LLM as backbone model\(Gaoet al\.,[2023](https://arxiv.org/html/2607.10235#bib.bib105); Yanget al\.,[2024](https://arxiv.org/html/2607.10235#bib.bib106)\)\. While LLMs are capable of challenging the more conventional methodologies for generating recommendations, these models come with specific drawbacks such as growing computational power and privacy concerns\(Wu,[2023](https://arxiv.org/html/2607.10235#bib.bib124); Zhaoet al\.,[2024](https://arxiv.org/html/2607.10235#bib.bib125); Waterschootet al\.,[2025b](https://arxiv.org/html/2607.10235#bib.bib57)\)\. In our study, we make use of LLMs\-as\-judge with the goal of generating human\-like assessments of perceived fairness, satisfaction and group consensus\.

![Refer to caption](https://arxiv.org/html/2607.10235v1/x1.png)Figure 1\.Human distributions of responses to rating group recommendations in terms of fairness, satisfaction and consensus on a 7\-point Likert scale \(n=1,1521,152\), collected in a the user study byBarileet al\.\([2023](https://arxiv.org/html/2607.10235#bib.bib16)\)\.Human distributions from the survey by \\cite\[citet\]\{\\@@bibref\{Authors Phrase1YearPhrase2\}\{barile2023evaluating\}\{\\@@citephrase\{\(\}\}\{\\@@citephrase\{\)\}\}\}\. Data is available through \\cite\[citet\]\{\\@@bibref\{Authors Phrase1YearPhrase2\}\{barile2023data\}\{\\@@citephrase\{\(\}\}\{\\@@citephrase\{\)\}\}\}
### 2\.2\.Human Assessment of GRS

Fairness is an important topic in recommender systems\(Deldjooet al\.,[2024](https://arxiv.org/html/2607.10235#bib.bib128); Kayaet al\.,[2020](https://arxiv.org/html/2607.10235#bib.bib68); Liet al\.,[2023](https://arxiv.org/html/2607.10235#bib.bib129)\)\. Additional subjective assessments such as satisfaction, and specifically for group recommendation, perceived group consensus receive attention\(Barileet al\.,[2023](https://arxiv.org/html/2607.10235#bib.bib16),[2026](https://arxiv.org/html/2607.10235#bib.bib97)\)\. Several metrics have been proposed with regard to fairness\. Each of these focus on specific aspects of fairness, such as demographic parity and equal opportunity, i\.e\., all qualified items are being considered\(Kumaret al\.,[2023](https://arxiv.org/html/2607.10235#bib.bib103)\)\. For GRS specifically, fairness has been defined as balancing preferences of top\-ranked items across group members byKayaet al\.\([2020](https://arxiv.org/html/2607.10235#bib.bib68)\)to define a greedy algorithm to generate group recommendations that satisfy this condition\.

In the context of GRS, human evaluations are usually collected through user studies\(Barileet al\.,[2026](https://arxiv.org/html/2607.10235#bib.bib97)\)\. The literature makes the distinction between external evaluation, in which the participant is not part of the group and assessing the recommendation as an outsider\(Barileet al\.,[2023](https://arxiv.org/html/2607.10235#bib.bib16); Tranet al\.,[2019](https://arxiv.org/html/2607.10235#bib.bib3)\), and internal evaluation in which the user provides their ratings and is part of the group itself\(Barileet al\.,[2026](https://arxiv.org/html/2607.10235#bib.bib97)\)\. Participants are provided with a group context \(user\-item matrix\) and are asked to rate statements on a Likert scale\. In this study, we make use of the such a dataset111Source dataset can be found here:[https://dataverse\.nl/dataset\.xhtml?persistentId=doi:10\.34894/8EVX4U](https://dataverse.nl/dataset.xhtml?persistentId=doi:10.34894/8EVX4U)\(Barileet al\.,[2023](https://arxiv.org/html/2607.10235#bib.bib16); Barile,[2023](https://arxiv.org/html/2607.10235#bib.bib100)\), which asked participants to ratefairness\(“The group recommendation is fair to all group members”\),satisfaction\(“The group members will be satisfied with regard to the group recommendation”\), andconsensus\(“The group members will agree on the group recommendation”\)\. The human response distributions from this study are visualized in Figure[1](https://arxiv.org/html/2607.10235#S2.F1)\.

### 2\.3\.LLM\-as\-judge

The LLM\-as\-judge paradigm has emerged as a potential, scalable alternative to human evaluation, suited to address the traditional bottlenecks of group recruitment and the costs associated with user studies\. Under this framework, LLMs are implemented as evaluators to generate human\-like natural language assessments or Likert scores\(Bakkeret al\.,[2022](https://arxiv.org/html/2607.10235#bib.bib112); Liet al\.,[2025](https://arxiv.org/html/2607.10235#bib.bib109)\), including for subjective forms of evaluation such as sexism detection\(Aoyaguiet al\.,[2025](https://arxiv.org/html/2607.10235#bib.bib126)\)\. The shift toward automated evaluation offers potential for the real\-time assessment of algorithmic decision\-making\(Baumeisteret al\.,[2025](https://arxiv.org/html/2607.10235#bib.bib117)\)\. However, an unexplored possibility lies in employing these models to evaluate group recommendations at scale or within a larger GRS pipeline, where complex trade\-offs between individual preferences and collective satisfaction are simultaneously considered\. If LLMs result in similar assessments, they can complement traditional user studies or produce real\-time scoring to provide the fairest group recommendations given a specific group scenario\.

Despite their promises, the performance of base models \(not fine\-tuned for the particular task\) as impartial judges may be limited by systematic evaluative bias\.Shiet al\.\([2025](https://arxiv.org/html/2607.10235#bib.bib99)\)demonstrate that base models often result in significant issues regarding preference fairness and position consistency\. Additionally, similar to human evaluation, base models are susceptible to various forms of cognitive and social bias\.Malberget al\.\([2025](https://arxiv.org/html/2607.10235#bib.bib101)\)provide a systematic overview of3030cognitive biases across2020state\-of\-the\-art LLMs, whileChenet al\.\([2024](https://arxiv.org/html/2607.10235#bib.bib127)\)show LLM susceptibility to biases based on gender or authority, leading to skewed assessments\. Furthermore, base models often demonstrate a self\-preference bias, favoring output that mirrors their own style\(Wataokaet al\.,[2025](https://arxiv.org/html/2607.10235#bib.bib102)\)\. Thus, we opt to fine\-tune LLMs to anchor our evaluation models in the human ground truth, evaluated using the human distribution of assessments \(Figure[1](https://arxiv.org/html/2607.10235#S2.F1)\)\. We showcase the improvement when fine\-tuning the LLM\-as\-judge models as opposed to implementing base models for evaluation\.

## 3\.Methodology

The following paragraphs describe the methodology used in this study\. The full pipeline is visualized in Figure[2](https://arxiv.org/html/2607.10235#S3.F2)\. First, we outline the LLM fine\-tuning using an augmented training set, grounded in human ground truth\. Second, we employ the best\-performing model in our GRS pipeline to decide the final group recommendation based on the LLM\-generated assessments\. Finally, we describe the user study which we performed to evaluate the final recommendations\.

![Refer to caption](https://arxiv.org/html/2607.10235v1/x2.png)Figure 2\.Visualization of the full pipeline\. The methodology is divided into two main parts: \(1\) LLM fine\-tuning for automated assessment of fairness, consensus, and satisfaction; \(2\) GRS, in which the best\-performing, fine\-tuned LLM generates assessments of potential recommendations\.Full pipeline### 3\.1\.LLM Fine\-Tuning

#### 3\.1\.1\.Survey Data

The starting point of obtaining fine\-tuned LLMs for generating human\-aligned assessments was the human ground truth presented inBarileet al\.\([2023](https://arxiv.org/html/2607.10235#bib.bib16)\); Barile \([2023](https://arxiv.org/html/2607.10235#bib.bib100)\)\. The authors conducted a user study in which288288participants rated their perception of fairness, satisfaction and group consensus based on a group preference matrix \(containing ratings\) and a final recommendation\. A 7\-point Likert scale was used and each participant rated four scenarios, resulting in a final dataset containing1,1521,152assessments\. Each human assessments contained a separate score for fairness, satisfaction and consensus\. The distribution of human responses is visualized in Figure[1](https://arxiv.org/html/2607.10235#S2.F1)\.

#### 3\.1\.2\.Chain\-of\-Thought Generation

To address the limited scale of the original human\-annotated dataset and to provide the necessary training signals for effective fine\-tuning, we employed Knowledge Distillation via Synthetic Chain\-of\-Thought \(CoT\) Generation\(Hsiehet al\.,[2023](https://arxiv.org/html/2607.10235#bib.bib120)\)\. We partitioned the data into an train/test split \(n​\_​t​r​a​i​n​i​n​g=890n\\\_training=890,n​\_​t​e​s​t=262n\\\_test=262\) using stratified sampling to ensure that the score distributions remained balanced across both subsets\. This approach was particularly important for capturing infrequent edge cases \(the highest and lowest Likert scores\) ensuring the test set remained representative of the full spectrum of human sentiment\.

To expand the training set, we utilizedDeepSeek\-V3 \(671B\)\(via theOllamacloud API222[https://ollama\.com/library/deepseek\-v3\.1](https://ollama.com/library/deepseek-v3.1)\) as a teacher model to generate diverse, synthetic reasoning for the existing human ground truth\. By applying high sampling parameters \(T​e​m​p​e​r​a​t​u​r​e=0\.95Temperature=0\.95,t​o​p​\_​p=0\.95top\\\_p=0\.95\), we oversampled our890890training instances to produce a total of5,3185,318training samples\. This approach allowed us to generate multiple distinct reasoning paths for the same outcome, capturing the nuanced justifications behind identical Likert scores\. For each instance, the teacher model was provided with the full recommendation scenario, the group preference table, and the specific human\-assigned score\. The model was then prompted to generate concise, 2\-3 sentence justifications that explicitly referenced individual group members \(e\.g\., “Member 1 does not like this outcome…”\)\.333Full prompt is documented in the companion repository:[https://github\.com/Cwaterschoot/Consensus\-vs\-dissent\-2026/tree/main](https://github.com/Cwaterschoot/Consensus-vs-dissent-2026/tree/main)Throughout this expansion, we conducted checks to ensure the original human ground truth scores remained unaltered\. The result of this knowledge distillation was an augmented dataset suitable for fine\-tuning LLMs to generate both reasoning and assessment scores for group recommendations\. The dataset is available in the companion repository\.

#### 3\.1\.3\.Fine\-tuning

We fine\-tuned two open\-source LLMs,Llama\-3\.1\-8B\-InstructandOLMo\-3\-7B\-Instruct\-SFT, using the augmented training corpus \(n=5,318n=5,318\)\. Both base models are available on Huggingface \(Llama is open\-weight, Olmo open\-source\)\. Training was conducted on a single Google Colab A100 GPU utilizing Parameter\-Efficient Fine\-Tuning \(PEFT\) via Low\-Rank Adaptation \(LoRA\)\. To optimize memory efficiency, models were loaded in 4\-bit NF4 quantization using bitsandbytes library\. The LoRA configuration utilized a rank \(rr\) of 16 and an alpha \(α\\alpha\) of 32\.444Full training documentation and augmented dataset are available in the companion repository\.

A critical component of our training pipeline was the implementation of a custom label masking function\. To make sure that the models learned to replicate the reasoning process and scores rather than the prompt structure and group ratings, we applied a mask to the loss function for all tokens preceding the assistant’s response\. This forced the gradient updates to focus exclusively on the synthetic Chain\-of\-Thought and the human ground truth\. We employed the standard transformers Trainer \(combined with our custom masking\) and a learning rate of1×10−41\\times 10^\{\-4\}, a batch size of 2 with 4 gradient accumulation steps\.The models were trained for 4 epochs, and the best\-performing checkpoint was selected based on the lowest evaluation loss on the original \(not augmented\) test set\.

#### 3\.1\.4\.LLM Evaluation

To evaluate the LLMs, we make use of the exact procedure byBarileet al\.\([2023](https://arxiv.org/html/2607.10235#bib.bib16)\)to generate novel group scenarios, allowing us to evaluate the LLMs with data that was not seen during fine\-tuning\. In total, we generated1,1801,180new group scenarios\. In order to showcase the impact of the fine\-tuning phase, we generated a distribution of assessments with theLlama3\.1\-8Bbase model \(not fine\-tuned\)\.

We compared the resulting assessment distributions of the base model and both fine\-tuned versions to the human ground truth using the Wasserstein Distance \(Earth Mover’s Distance\)\. This metric, suitable for the ordinal Likert scale data, measures the minimum work required to transform the LLM\-generated distribution into the observed human ground truth\. A smaller distance \(from now on referred to as WD\) is better\. In the results, we also report the average Likert score as well as the standard deviation to provide a full overview of the LLM\-generated assessments\.

### 3\.2\.GRS and Evaluation

#### 3\.2\.1\.Recommender System Pipeline

Our GRS pipeline is designed around the repeated evaluation of aggregation strategies to determine the most suitable outcome for a given group\. Following the established scenario construction in GRS literature\(Waterschootet al\.,[2025b](https://arxiv.org/html/2607.10235#bib.bib57); Barileet al\.,[2021](https://arxiv.org/html/2607.10235#bib.bib10),[2023](https://arxiv.org/html/2607.10235#bib.bib16)\), we utilize a user preference matrix containing explicit ratings for restaurants on a scale from11to55\.

Starting from the group’s preferences, the system calculates the outcomes for six distinct social choice\-based aggregation strategies:Additive Utilitarian\(ADD\),Approval Voting\(APP\),Fairness\(FAI\),Least Misery\(LMS\),Majority Voting\(MAJ\), andMost Pleasure\(MPL\)\. These strategies were selected due to their presence in the training data\. The recommendations derived by each strategy are evaluated five separate times each, simulating the assessment of five different survey participants to ensure a variety in responses\. The number of generated assessments is a configurable parameter\. The final score for each strategy is determined by calculating the average across the five scores for each of the three metrics \(fairness, satisfaction, consensus\)\. Once the final scores are calculated, the strategies are ranked and the group is presented with the recommendation based on the top\-ranked strategy\. Our pipeline allows for deciding which of the three metrics is used as deciding factor\. Consistent with the scenarios used in the original survey data\(Barileet al\.,[2023](https://arxiv.org/html/2607.10235#bib.bib16); Waterschootet al\.,[2025c](https://arxiv.org/html/2607.10235#bib.bib56)\), we account for prior group history by assuming the top three items in the resulting list have already been visited\. Consequently, the system recommends the fourth\-ranked item as the final output for the group\. We follow the scenario structure from the literature to maximize consistency and reproducibility of GRS evaluation and data collection\.

#### 3\.2\.2\.User Study Evaluation

Following the code and procedure provided byBarileet al\.\([2023](https://arxiv.org/html/2607.10235#bib.bib16)\)and thus, adhering to the format of the previously used survey data, we generated a set of group scenarios to be used in the user evaluation\.555Our user evaluation was preregistered on OSF:[https://osf\.io/98kuw/overview?view\_only=41c006b50fb84f0abb76347c98e94ff9](https://osf.io/98kuw/overview?view_only=41c006b50fb84f0abb76347c98e94ff9)Each scenario consisted of a user\-item matrix in which a group of five members is presented alongside their fictitious ratings of 10 restaurants \(scale from 1 to 5\)\. Replicating the original data, we generated group scenarios across four different group configurations \(within\-group preference similarity\): \(i\)uniform\(all similar\), \(ii\)divergent\(all dissimilar\), \(iii\)minority\(all similar except one\), and \(iv\)coalitional\(two subgroups\)\(Barileet al\.,[2023](https://arxiv.org/html/2607.10235#bib.bib16)\)\. The scenario describes that the group has already visited three restaurants and thus, the group is now recommended the fourth restaurant in the ranked list\. This list was generated by a social choice\-based aggregation strategy\(Senotet al\.,[2010](https://arxiv.org/html/2607.10235#bib.bib1)\)\. Included strategies are those included in the survey data:Additive Utilitarian\(ADD\),Approval Voting\(APP\),Fairness\(FAI\),Least Misery\(LMS\),Majority voting\(MAJ\), andMost Pleasure\(MPL\)\. For our specific evaluation, we add a final category to the strategy list, namelyLLM\. The latter presents the participant with the recommendation generated by the strategy which was chosen by the LLM\.

Participants were recruited via the online participant platform Prolific\.666[https://www\.prolific\.com/](https://www.prolific.com/)Aggregation strategy was between\-subject; each participant was assigned one of the seven options \(one of six strategies or the LLM’s pick\) and was only presented scenarios in which the recommendations were generated by that strategy\. Group configuration was within\-subject, which means that every participant rated scenarios based on all four group configurations\. Each participant saw one group for each group configuration \(randomly selected out of four options\), resulting in four groups assessed per participant\. Thus, our survey contained a total of 16 group scenarios, from which the participants each saw four\. This survey design matches the study used to generate the survey data described in Section[3\.1\.1](https://arxiv.org/html/2607.10235#S3.SS1.SSS1)and used for LLM fine\-tuning\. For each new group scenario, the participant had to rate the following three statements \(taken from\(Barileet al\.,[2023](https://arxiv.org/html/2607.10235#bib.bib16)\)\) using a 7\-point Likert scale from−3\-3\(strongly disagree\) to33\(strongly agree\): “The group recommendation is fair to all group members” \(fairness\), “The group members will agree on the group recommendation” \(consensus\), and “The group members will be satisfied with regard to the group recommendation” \(satisfaction\)\.

In order to evaluate the performance of our GRS described in Section[3\.2\.1](https://arxiv.org/html/2607.10235#S3.SS2.SSS1), we constructed an mixed\-design ANOVA model to test whether theLLMcategory received significantly higher scores compared to the singular \(static\) aggregation strategies\. This result would imply the benefit of dynamically adapting the aggregation strategy based on LLM\-generated assessments, providing end\-users with the fairest or most satisfying recommendations\. Based on power analysis, we calculated a required sample size of284284participants\. Additionally, we report average scores for fairness, satisfaction and consensus, as well as the standard deviation for each of the seven options \(six strategies and our dynamic LLM\-based selection\)\. As an exploratory analysis, we compare the performance across the four group configurations\. The full survey outline is found in the preregistration on OSF\.777[https://osf\.io/98kuw/overview?view\_only=41c006b50fb84f0abb76347c98e94ff9](https://osf.io/98kuw/overview?view_only=41c006b50fb84f0abb76347c98e94ff9)

## 4\.Results

In the following paragraphs, we present the results of our study\. First, we discuss the performance of our fine\-tuned models relative to the base model\. Second, we present the results of our user study performed to evaluate our proposed GRS pipeline\.

![Refer to caption](https://arxiv.org/html/2607.10235v1/x3.png)Figure 3\.Distributions \(across fairness, satisfaction, consensus\) for both judgmental models and the human ground truth\.Human distributions from the survey by \\cite\[citet\]\{\\@@bibref\{Authors Phrase1YearPhrase2\}\{barile2023evaluating\}\{\\@@citephrase\{\(\}\}\{\\@@citephrase\{\)\}\}\}\. Distributions from both fine\-tuned Judgmental modelsTable 2\.Overview comparing the ground truth human distributions with LLM\-generated assessment distributions\. Models included the Llama/Olmo base models and our fine\-tuned versions \(Judgmental Llama and Judgmental Olmo\)\. Lower Wasserstein Distance \(WD\) indicates a distribution closer to Ground Truth\.### 4\.1\.LLM Evaluation

We evaluated the performance of our fine\-tuned models \(Judgmental LlamaandJudgmental Olmo\) against the base models \(both Llama and Olmo\) by comparing their output distributions to the human ground truth\. Table[2](https://arxiv.org/html/2607.10235#S4.T2)summarizes these results across the three dimensions: fairness, satisfaction, and consensus\.

The Llama base model exhibited significant misalignment with the ground truth, resulting in a high Wasserstein Distance \(WD\) and low standard deviation \(Table[2](https://arxiv.org/html/2607.10235#S4.T2)\)\. This suggests that the base model generated an exceedingly narrow set of assessments\. In contrast, our fine\-tuned models demonstrated a reduction in WD, indicating closer alignment with the human distribution \(Figure[3](https://arxiv.org/html/2607.10235#S4.F3)\)\.While the Olmo3 base model did achieve a relatively low distance to the human distribution for fairness, its distance for the other two metrics was the highest, indicating relatively strong misalignment \(Table[2](https://arxiv.org/html/2607.10235#S4.T2)\)\.

Notably,Judgmental Olmoemerged as the best\-performing model, achieving the lowest WD scores across all three dimensions \(0\.700\.70for fairness,0\.680\.68for satisfaction, and0\.820\.82for consensus\)\. Additionally, the fine\-tuned model replicated the variety in human responses \(σ\>1\.55\\sigma\>1\.55\), closely approaching the ground truth \(σ≈1\.6\\sigma\\approx 1\.6\)\. Judgmental Olmo was selected as the assessment generator for the GRS pipeline and subsequent user study\.

Additionally, we analyzed the Pearson correlation coefficients between fairness, satisfaction and consensus scores generated by the Judgmental Olmo model\. The results show relatively strong correlations \(ranging between0\.640\.64and0\.780\.78\), consistent with the strong correlations found in the original human ground truth \(0\.710\.71to0\.830\.83\)\. From a practical perspective, these correlations imply that these factors will largely align in their ranking of recommendation outcomes, providing a consistent signal regardless of which specific dimension of the three is prioritized as a deciding factor in the GRS pipeline\. For this study, we calculate the average across the three metrics and rank the aggregation strategies based on these average scores\. This is the most standard approach, while future work may optimize towards either fairness, satisfaction or consensus\.

### 4\.2\.User Evaluation

#### 4\.2\.1\.Participants

Our user study resulted in a total of1,1361,136assessments provided by284284participants\. Those who failed an attention check were omitted from the analysis and were replaced by a new participant\. They were required to be proficient English speakers above 18 years of age\. Each participant was allowed to participate in the study once, and received a reimbursement according to Prolific guidelines\.888[https://www\.prolific\.com/resources/how\-much\-should\-you\-pay\-research\-participants](https://www.prolific.com/resources/how-much-should-you-pay-research-participants)Our sample consisted of172172\(60\.5%60\.5\\%\) male,105105\(37\.0%37\.0\\%\) female, and77\(2\.5%2\.5\\%\) non\-binary participants\.101101\(35\.56%35\.56\\%\) were between 26\-35 years old,7474\(26\.05%26\.05\\%\) between 18\-25,6565\(22\.9%22\.9\\%\) between 36\-45,2525\(8\.8%8\.8\\%\) between 46\-55, and1919\(6\.7%6\.7\\%\) above the age of 56\.

#### 4\.2\.2\.LLM Selection

To obtain the LLM category in our user study, we processed all included group scenarios using our GRS pipeline\. In order to decide which strategy was dynamically selected for each given group, our recommendation pipeline generated five assessments \(committee size\) per potential recommendation\. An assessment is a set of three scores: one for perceived fairness, satisfaction and consensus\. We averaged across the five assessments per recommendation to obtain the final scores\. In total, ADD was the winning strategy \(picked by the LLM as best based on fairness, consensus and satisfaction scores\) for six out of 16 \(four groups times four configurations\)\. FAI was the second most picked \(4\), while LMS and MPL were selected twice\. APP and MAJ were only selected once\. We saw several patterns across group configurations\. Forminoritygroups, only ADD or FAI were selected \(each strategy twice\), while the MPL strategy was only used for uniform groups \(alongside one occurrence of LMS and ADD\)\. Fordivergentgroups, ADD was selected twice, followed by one occurrence of APP and one of FAI\. Finally, for coalitional groups, each included group differed in terms of dynamically selected strategy \(one times ADD, FAI, LMS, and MAJ\)\.

Table 3\.Average Likert scores \(\-3 to 3\) and standard deviation for perceived fairness, satisfaction and consensus of group recommendations\. Split by aggregation strategy \(between\-subject\)\.
#### 4\.2\.3\.User Scores

Descriptive statistics reveal that our LLM\-based aggregation strategy selection tends to outperformed static social choice\-based aggregation \(Table[3](https://arxiv.org/html/2607.10235#S4.T3)\), achieving the highest mean scores for perceived fairness \(0\.820\.82\) and consensus \(0\.810\.81\), and the second\-highest for satisfaction \(0\.680\.68, surpassed only by FAI, which scored0\.700\.70\)\. Strategies like APP, MAJ and MPL lagged behind alternatives with regard to all three metrics\. The ANOVA results for the three metrics are summarized in Table[4](https://arxiv.org/html/2607.10235#S4.T4)\. P\-values were adjusted using Bonferroni correction \(0\.05/3=0\.0170\.05/3=0\.017\)\.

Table 4\.ANOVA results for perceived fairness, satisfaction, and consensus\. Significance was calculated using a corrected threshold of 0\.017\. Statistical significance is indicated using an asterisk \(\*\)\.Perceived fairness\.There was no main effect for aggregation strategy with regard toperceived fairness, however, our analysis indicated a significant interaction between group configuration and aggregation strategy \(F=5\.18,p<\.001F=5\.18,p<\.001\)\. In a post\-hoc analysis, we found a significant interaction effect in coalitional groups, with LLM outperforming both APP \(t=4\.44,p<\.001t=4\.44,p<\.001\) and ADD \(t=3\.22,p=\.002t=3\.22,p=\.002\) strategies\. On a global level, the LLM was perceived as significantly more fair than the Majority \(MAJ\) strategy \(t=2\.81,p=\.006t=2\.81,p=\.006\)\.

Perceived satisfaction\. The analysis revealed a significant interaction effect between group configuration and aggregation strategy \(F=4\.28,p<\.001F=4\.28,p<\.001\)\. Our dynamic LLM aggregation selection outperformed in minority and coalitional groups\. In the former, the aggregation strategies selected based on the LLM\-generated assessments provided significantly higher satisfaction than both the MAJ \(t=4\.11,p<\.001t=4\.11,p<\.001\) and MPL \(t=3\.01,p=\.004t=3\.01,p=\.004\) strategies\. Furthermore, in coalitional groups, the LLM statistically outperformed APP \(t=3\.07,p=\.003t=3\.07,p=\.003\)\.

Perceived consensus\. The third and final metric was perceived consensus\. Our dynamic LLM\-based selection significantly outperformed MPL regardless of group configuration \(t=2\.50,p=\.015t=2\.50,p=\.015\)\. Looking at the significant interaction effects between consensus and group configuration \(Table[4](https://arxiv.org/html/2607.10235#S4.T4)\), our proposed method outperformed APP \(t=4\.83,p<\.001t=4\.83,p<\.001\) and ADD \(t=3\.17,p=\.002t=3\.17,p=\.002\) for coalitional groups, as well as MAJ for minority configurations \(t=3\.24,p=\.002t=3\.24,p=\.002\)\.

## 5\.Discussion

In the following paragraphs, we contextualize our results\. First, we discuss that our results, both in terms of LLM evaluation and user study, strengthen the validity of the proposed fine\-tuning procedure as well as its implementation for dynamic selection of aggregation strategy\. Second, we outline potential adjustments and parameter tuning that we propose to investigate in future work\. Thirdly, we discuss the significant interaction with group configuration\. Fourthly, we discuss potential adjustments to the pipeline\. Afterwards, we discuss additional avenues for future work, including scaling down model size to optimize for real\-time inference as well as comparing our pipeline to other state\-of\-the\-art GRS methodologies\. Finally, we identify and outline several limitations of our work\.

### 5\.1\.Validity of Fine\-tuning

Our results highlight the potential benefits of fine\-tuning on human ground truth and \(distilled\) reasoning for aligning LLMs with human subjective assessments\. We show that base models may fail to capture the nuance of group dynamics or produce biased \(overtly positive\) results, resulting in a narrow distribution with high Wasserstein Distance to the ground truth distribution\. The fine\-tuned versions successfully reduced the distance to human ground truth, allowing our implemented model to increasingly replicate the diversity \(σ≈1\.6\\sigma\\approx 1\.6\) found in human fairness, satisfaction, and consensus ratings\. The validity of our proposed pipeline is further illustrated by the user study, where the dynamic selection pipeline achieved the highest mean scores for both fairness and consensus\.

Additionally, many of our findings align with findings in previously conducted user studies regarding social choice\-based aggregation\. For example, MPL was only selected by the LLM\-based pipeline for uniform groups, a result corresponding to previous findings comparing uniform groups to other configurations\(Barileet al\.,[2023](https://arxiv.org/html/2607.10235#bib.bib16),[2026](https://arxiv.org/html/2607.10235#bib.bib97)\)\. Additionally, these studies found consistently strong results for FAI and ADD as well, mirroring the frequency at which these strategies were selected in our pipeline\.

By adapting the aggregation strategy based on automated, human\-like assessments rather than relying on a static social choice\-based aggregation strategy, the system was able to adapt to potential unfair or unsatisfying outcomes in a way that is consistent with previous human selections\. These findings suggest that our dynamic approach to group recommendations, still rooted in transparent preference aggregation, may offer a more robust and fair framework compared to traditional aggregation methods\.

### 5\.2\.The Impact of Group Configuration

The strength of our pipeline lies not in inventing new formulas for aggregation, but in its ability to adapt when static strategies reach a recommendation that might be deemed poor by the group\. The selection of different aggregation strategies by our LLM highlights again that there are contexts where it is beneficial to deviate from what might be a globally strong strategy\. For some group configurations, a strategy may produce inferior recommendations and an alternative strategy might lead to a better outcome\. For example, our pipeline correctly identified that recommendations generated by MPL might be useful for obtaining the most positive assessments for uniform groups\(Barileet al\.,[2023](https://arxiv.org/html/2607.10235#bib.bib16)\)\. Additionally, our results show that the strongest strategies \(ADD and FAI\) might struggle with some coalitional groups, resulting in a dynamical selection in which each coalitional group received a recommendation generated from a broader pool of aggregation strategies \(ADD, FAI, MAJ, LMS\)\. Additional exploratory analysis, in which we ran4040additional group scenarios \(10 per configuration,n=1,200n=1,200\) using our pipeline revealed that this pattern holds\. MPL was strictly used for uniform scenarios \(3 out of 10 uniform groups\), while for coalitional groups all six strategies were chosen at least once\. Similar as for the groups in oru user study, FAI was chosen most frequently\.

### 5\.3\.Pipeline Adjustments

Several avenues exist to refine our optimization logic within the GRS pipeline\. While the current implementation made use of a simple average across fairness, satisfaction, and consensus to select the winning aggregation strategy, the system could be tuned to prioritize a single dimension depending on the specific application context\. Furthermore, in our current study we generated five assessments \(committee size\) per potential recommendation outcome and calculated the average across those to obtain a final score for each metric\. Future work will determine the optimal number of internal assessments, balancing robustness and computational efficiency\.

While our results demonstrate that distilling reasoning from a high\-capacity model like DeepSeek\-V3\.1 into a smaller model \(such as Olmo3\) is effective, this approach serves as a bridge towards an optimal outcome, i\.e\., training on actual human reasoning provided by participants as opposed to human\-like reasoning\. Additional experimentation suggests that the failure of base models is systemic\(Waterschootet al\.,[2026](https://arxiv.org/html/2607.10235#bib.bib130)\)\. Regardless of model family or parameter size \(whether 7 billion, 20 billion or 235 billion\), generated assessments consistently lack behind fine\-tuned output with regard to the distance to the human ground truth distribution\. This finding underscores that quality fine\-tuning data cannot be substituted by scale\.

### 5\.4\.Future Work

Our findings open up several other avenues for future work\. First, while our pipeline demonstrates clear potential, the reliance on five parallel assessments per recommendation, resulting in a total of3030inferences per group, creates significant latency\. Depending on hardware, this can go all the way up to several minutes per group\. To make this judgmental framework viable within a real\-time GRS pipeline, future work will explore the distillation of reasoning into Small Language Models \(SLMs\) with1\.51\.5billion parameters or fewer \(as opposed to 8B\+ in the current study\)\. Transitioning to these smaller models would drastically reduce computational requirements \(e\.g\., GPU time\) as well as overall response times, potentially allowing for a larger number of assessments per recommendation \(increasing robustness\) without sacrificing feasibility and user experience in real\-time contexts\.

Second, we strictly constrained our recommendation procedure to social choice\-based aggregation strategies to prioritize transparency and take advantage of the large body of literature on explainable group recommendations\. However, future research should benchmark our dynamic selection pipeline against state\-of\-the\-art Deep Learning\-based GRS models, such as attentive group recommendation frameworks\(Caoet al\.,[2018](https://arxiv.org/html/2607.10235#bib.bib59); Chenet al\.,[2021](https://arxiv.org/html/2607.10235#bib.bib44)\)\. Additionally, we do not have access to the required datasets and documentation to include these approaches into our comparative analysis, as they rely on specific features or data requirements\.

### 5\.5\.Limitations

We identified several limitations of our work\. First, a primary limitation in developing our fine\-tuned models is the scarcity of large\-scale, high\-quality human datasets that include both quantitative scores and qualitative reasoning\. To circumvent this, our study utilized a distillation pipeline, generating synthetic reasoning from a high\-capacity model \(DeepSeek\-V3\.1\) to fine\-tune our smaller models\. While our results show this to be effective, it may introduces a dependency on the teacher model\. In future work, we will focus on longitudinal data collection specifically designed for model training, incentivizing participants to provide their internal reasoning in text alongside their Likert scores\.

Secondly, the requirement for a five\-member assessment committee per recommendation presents a computational bottleneck\. In a real\-time system with many end\-users, the cumulative latency of generating and parsing multiple reasoning blocks hinders immediate interaction\. As mentioned earlier, the use of tiny LLMs might be the key in circumventing this limitation\.

Finally, our system is domain\-dependent due to the scenarios present in the previously conducted user studies\. Additionally, we focused solely on GRS\. Whether our findings generalize beyond group recommendation specifically is unclear\. For example, in our training data, the participant was presented with a group scenario in which the first three restaurants were already visited\. Subsequently, the fourth\-ranked restaurant became the recommendation\. A more standardized input format is required to move towards domain\- and scenario\-independence\. Due to the fact that this originates from the data collection phase, we will move towards multi\-domain data collection in future work\.

## 6\.Conclusion

Our methodology focused on bridging the gap between LLM output and human ground truth assessments of group recommendations\. By utilizing DeepSeek\-V3\.1 to generate reasoning rooted in ground truth, we addressed the limitations of base models\. Through fine\-tuning on this distilled reasoning and human assessments, we aligned the models’ logic with human\-like perceptions of group dynamics and recommendations\. We demonstrated this by a reduction in Wasserstein distance from the human distribution\.

We also validated the best\-performing LLM approach through a user study \(n=284n=284\)\. Our pipeline achieved the highest average scores for perceived fairness and consensus\. Additionally, our system showed adaptability in complex groups such as the coalitional group configuration\. In the case of complex scenarios, such as coalitional groups, our pipeline showed the strength of making use of the wide set of aggregation strategies, outperforming static baselines with regard to fairness, satisfaction and consensus\. All in all, our pipeline shows capabilities for both groups with consensus \(uniform\) as dissent \(coalitional\)\.

Moving beyond the constraints of static aggregation, our work provides a scalable blueprint for GRS that accounts for the nuanced human\-like interpretation of a recommendation outcome, while retaining the transparency and range of options provided by social choice\-based aggregation strategies\.

## References

- P\. A\. Aoyagui, K\. Stemmler, S\. A\. Ferguson, Y\. Kim, and A\. Kuzminykh \(2025\)A matter of perspective\(s\): contrasting human and llm argumentation in subjective decision\-making on subtle sexism\.InProceedings of the 2025 CHI Conference on Human Factors in Computing Systems,CHI ’25,New York, NY, USA\.External Links:ISBN 9798400713941,[Link](https://doi.org/10.1145/3706598.3713248),[Document](https://dx.doi.org/10.1145/3706598.3713248)Cited by:[§2\.3](https://arxiv.org/html/2607.10235#S2.SS3.p1.1)\.
- M\. A\. Bakker, M\. J\. Chadwick, H\. R\. Sheahan, M\. H\. Tessler, L\. Campbell\-Gillingham, J\. Balaguer, N\. McAleese, A\. Glaese, J\. Aslanides, M\. M\. Botvinick, and C\. Summerfield \(2022\)Fine\-tuning language models to find agreement among humans with diverse preferences\.InProceedings of the 36th International Conference on Neural Information Processing Systems,NIPS ’22,Red Hook, NY, USA\.External Links:ISBN 9781713871088Cited by:[§2\.3](https://arxiv.org/html/2607.10235#S2.SS3.p1.1)\.
- K\. Bao, J\. Zhang, Y\. Zhang, X\. Huo, C\. Chen, and F\. Feng \(2024\)Decoding matters: addressing amplification bias and homogeneity issue in recommendations for large language models\.InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing,Y\. Al\-Onaizan, M\. Bansal, and Y\. Chen \(Eds\.\),Miami, Florida, USA,pp\. 10540–10552\.External Links:[Link](https://aclanthology.org/2024.emnlp-main.589/),[Document](https://dx.doi.org/10.18653/v1/2024.emnlp-main.589)Cited by:[§2\.1](https://arxiv.org/html/2607.10235#S2.SS1.p3.1)\.
- F\. Barile, T\. Draws, O\. Inel, A\. Rieger, S\. Najafian, A\. Ebrahimi Fard, R\. Hada, and N\. Tintarev \(2023\)Evaluating explainable social choice\-based aggregation strategies for group recommendation\.User Modeling and User\-Adapted Interaction,pp\. 1–58\.Cited by:[1st item](https://arxiv.org/html/2607.10235#S1.I1.i1.p1.1),[Table 1](https://arxiv.org/html/2607.10235#S1.T1),[Table 1](https://arxiv.org/html/2607.10235#S1.T1.3.2),[§1](https://arxiv.org/html/2607.10235#S1.p1.1),[Figure 1](https://arxiv.org/html/2607.10235#S2.F1),[Figure 1](https://arxiv.org/html/2607.10235#S2.F1.2.1),[§2\.1](https://arxiv.org/html/2607.10235#S2.SS1.p1.1),[§2\.1](https://arxiv.org/html/2607.10235#S2.SS1.p2.1),[§2\.2](https://arxiv.org/html/2607.10235#S2.SS2.p1.1),[§2\.2](https://arxiv.org/html/2607.10235#S2.SS2.p2.1),[§3\.1\.1](https://arxiv.org/html/2607.10235#S3.SS1.SSS1.p1.2),[§3\.1\.4](https://arxiv.org/html/2607.10235#S3.SS1.SSS4.p1.1),[§3\.2\.1](https://arxiv.org/html/2607.10235#S3.SS2.SSS1.p1.2),[§3\.2\.1](https://arxiv.org/html/2607.10235#S3.SS2.SSS1.p2.1),[§3\.2\.2](https://arxiv.org/html/2607.10235#S3.SS2.SSS2.p1.1),[§3\.2\.2](https://arxiv.org/html/2607.10235#S3.SS2.SSS2.p2.2),[§5\.1](https://arxiv.org/html/2607.10235#S5.SS1.p2.1),[§5\.2](https://arxiv.org/html/2607.10235#S5.SS2.p1.2)\.
- F\. Barile, P\. Hurlin, C\. Waterschoot, and N\. Tintarev \(2026\)Critical reflections on user studies’ evaluation methods for group recommender systems\.International Journal of Human\-Computer Studies,pp\. 103742\.External Links:ISSN 1071\-5819,[Document](https://dx.doi.org/https%3A//doi.org/10.1016/j.ijhcs.2026.103742),[Link](https://www.sciencedirect.com/science/article/pii/S1071581926000170)Cited by:[§1](https://arxiv.org/html/2607.10235#S1.p1.1),[§2\.2](https://arxiv.org/html/2607.10235#S2.SS2.p1.1),[§2\.2](https://arxiv.org/html/2607.10235#S2.SS2.p2.1),[§5\.1](https://arxiv.org/html/2607.10235#S5.SS1.p2.1)\.
- F\. Barile, S\. Najafian, T\. Draws, O\. Inel, A\. Rieger, R\. Hada, and N\. Tintarev \(2021\)Toward benchmarking group explanations: evaluating the effect of aggregation strategies versus explanation\.Cited by:[§2\.1](https://arxiv.org/html/2607.10235#S2.SS1.p1.1),[§3\.2\.1](https://arxiv.org/html/2607.10235#S3.SS2.SSS1.p1.2)\.
- F\. Barile \(2023\)UM xai explainable \- grs impact of scenario complexity\.DataverseNL\.External Links:[Document](https://dx.doi.org/10.34894/8EVX4U),[Link](https://dataverse.nl/citation?persistentId=doi:10.34894/8EVX4U)Cited by:[1st item](https://arxiv.org/html/2607.10235#S1.I1.i1.p1.1),[§2\.2](https://arxiv.org/html/2607.10235#S2.SS2.p2.1),[§3\.1\.1](https://arxiv.org/html/2607.10235#S3.SS1.SSS1.p1.2)\.
- J\. Baumeister, B\. Finkbeiner, F\. Scheerer, J\. Siber, and T\. Wagenpfeil \(2025\)Stream\-based monitoring of algorithmic fairness\.InTools and Algorithms for the Construction and Analysis of Systems,A\. Gurfinkel and M\. Heule \(Eds\.\),Cham,pp\. 60–81\.External Links:ISBN 978\-3\-031\-90643\-5Cited by:[§2\.3](https://arxiv.org/html/2607.10235#S2.SS3.p1.1)\.
- D\. Cao, X\. He, L\. Miao, Y\. An, C\. Yang, and R\. Hong \(2018\)Attentive group recommendation\.InThe 41st International ACM SIGIR Conference on Research & Development in Information Retrieval,SIGIR ’18,New York, NY, USA,pp\. 645–654\.External Links:ISBN 9781450356572,[Link](https://doi.org/10.1145/3209978.3209998),[Document](https://dx.doi.org/10.1145/3209978.3209998)Cited by:[§2\.1](https://arxiv.org/html/2607.10235#S2.SS1.p2.1),[§5\.4](https://arxiv.org/html/2607.10235#S5.SS4.p2.1)\.
- G\. H\. Chen, S\. Chen, Z\. Liu, F\. Jiang, and B\. Wang \(2024\)Humans or LLMs as the judge? a study on judgement bias\.InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing,Y\. Al\-Onaizan, M\. Bansal, and Y\. Chen \(Eds\.\),Miami, Florida, USA,pp\. 8301–8327\.External Links:[Link](https://aclanthology.org/2024.emnlp-main.474/),[Document](https://dx.doi.org/10.18653/v1/2024.emnlp-main.474)Cited by:[§2\.3](https://arxiv.org/html/2607.10235#S2.SS3.p2.2)\.
- L\. Chen, J\. Cao, H\. Chen, W\. Liang, H\. Tao, and G\. Zhu \(2021\)Attentive multi\-task learning for group itinerary recommendation\.Knowl\. Inf\. Syst\.63\(7\),pp\. 1687–1716\.External Links:[Document](https://dx.doi.org/10.1007/s10115-021-01567-3)Cited by:[§2\.1](https://arxiv.org/html/2607.10235#S2.SS1.p1.1),[§5\.4](https://arxiv.org/html/2607.10235#S5.SS4.p2.1)\.
- S\. Dai, N\. Shao, H\. Zhao, W\. Yu, Z\. Si, C\. Xu, Z\. Sun, X\. Zhang, and J\. Xu \(2023\)Uncovering chatgpt’s capabilities in recommender systems\.InProceedings of the 17th ACM Conference on Recommender Systems,RecSys ’23,New York, NY, USA,pp\. 1126–1132\.External Links:ISBN 9798400702419,[Link](https://doi.org/10.1145/3604915.3610646),[Document](https://dx.doi.org/10.1145/3604915.3610646)Cited by:[§2\.1](https://arxiv.org/html/2607.10235#S2.SS1.p3.1)\.
- Y\. Deldjoo, D\. Jannach, A\. Bellogin, A\. Difonzo, and D\. Zanzonelli \(2024\)Fairness in recommender systems: research landscape and future directions\.User Model\. User\-adapt Interact\.34\(1\),pp\. 59–108\(en\)\.Cited by:[§2\.2](https://arxiv.org/html/2607.10235#S2.SS2.p1.1)\.
- A\. Delić, H\. Emamgholizadeh, F\. Ricci, and J\. Masthoff \(2024\)Supporting group decision\-making: insights from a focus group study\.InProceedings of the 32nd ACM Conference on User Modeling, Adaptation and Personalization,pp\. 301–306\.External Links:[Document](https://dx.doi.org/10.1145/3627043.3659538)Cited by:[§2\.1](https://arxiv.org/html/2607.10235#S2.SS1.p1.1)\.
- A\. Delic, J\. Masthoff, J\. Neidhardt, and H\. Werthner \(2018\)How to use social relationships in group recommenders: empirical evidence\.InProceedings of the 26th Conference on User Modeling, Adaptation and Personalization,pp\. 121–129\.Cited by:[§2\.1](https://arxiv.org/html/2607.10235#S2.SS1.p2.1)\.
- A\. Felfernig, L\. Boratto, M\. Stettinger, and M\. Tkalčič \(2018\)Explanations for groups\.InGroup Recommender Systems,pp\. 105–126\.Cited by:[Table 1](https://arxiv.org/html/2607.10235#S1.T1),[Table 1](https://arxiv.org/html/2607.10235#S1.T1.3.2)\.
- Y\. Gao, T\. Sheng, Y\. Xiang, Y\. Xiong, H\. Wang, and J\. Zhang \(2023\)Chat\-REC: towards interactive and explainable LLMs\-augmented recommender system\.External Links:2303\.14524Cited by:[§2\.1](https://arxiv.org/html/2607.10235#S2.SS1.p3.1)\.
- C\. Hsieh, C\. Li, C\. Yeh, H\. Nakhost, Y\. Fujii, A\. Ratner, R\. Krishna, C\. Lee, and T\. Pfister \(2023\)Distilling step\-by\-step\! outperforming larger language models with less training data and smaller model sizes\.InFindings of the Association for Computational Linguistics: ACL 2023,A\. Rogers, J\. Boyd\-Graber, and N\. Okazaki \(Eds\.\),Toronto, Canada,pp\. 8003–8017\.External Links:[Link](https://aclanthology.org/2023.findings-acl.507/),[Document](https://dx.doi.org/10.18653/v1/2023.findings-acl.507)Cited by:[§3\.1\.2](https://arxiv.org/html/2607.10235#S3.SS1.SSS2.p1.2)\.
- Z\. Huang, X\. Xu, H\. Zhu, and M\. Zhou \(2020\)An efficient group recommendation model with multiattention\-based neural networks\.IEEE Transactions on Neural Networks and Learning Systems31\(11\),pp\. 4461–4474\.External Links:[Document](https://dx.doi.org/10.1109/TNNLS.2019.2955567)Cited by:[§2\.1](https://arxiv.org/html/2607.10235#S2.SS1.p2.1)\.
- M\. Kaya, D\. Bridge, and N\. Tintarev \(2020\)Ensuring fairness in group recommendations by rank\-sensitive balancing of relevance\.InFourteenth ACM Conference on Recommender SystemsRecSys ’20: Fourteenth ACM Conference on Recommender Systems,New York, NY, USA\.Cited by:[§2\.2](https://arxiv.org/html/2607.10235#S2.SS2.p1.1)\.
- J\.S\. Kelly \(2013\)Social choice theory: an introduction\.Springer Berlin Heidelberg\.External Links:ISBN 9783662099254,LCCN 87012858Cited by:[§2\.1](https://arxiv.org/html/2607.10235#S2.SS1.p2.1)\.
- S\. Kim, H\. Kang, S\. Choi, D\. Kim, M\. Yang, and C\. Park \(2024\)Large language models meet collaborative filtering: an efficient all\-round LLM\-based recommender system\.InProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data MiningKDD ’24: The 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining,Vol\.25,New York, NY, USA,pp\. 1395–1406\.Cited by:[§2\.1](https://arxiv.org/html/2607.10235#S2.SS1.p3.1)\.
- D\. Kumar, T\. Grosz, N\. Rekabsaz, E\. Greif, and M\. Schedl \(2023\)Fairness of recommender systems in the recruitment domain: an analysis from technical and legal perspectives\.Frontiers in Big DataVolume 6 \- 2023\.External Links:[Link](https://www.frontiersin.org/journals/big-data/articles/10.3389/fdata.2023.1245198),[Document](https://dx.doi.org/10.3389/fdata.2023.1245198),ISSN 2624\-909XCited by:[§2\.2](https://arxiv.org/html/2607.10235#S2.SS2.p1.1)\.
- D\. Li, B\. Jiang, L\. Huang, A\. Beigi, C\. Zhao, Z\. Tan, A\. Bhattacharjee, Y\. Jiang, C\. Chen, T\. Wu, K\. Shu, L\. Cheng, and H\. Liu \(2025\)From generation to judgment: opportunities and challenges of LLM\-as\-a\-judge\.InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing,C\. Christodoulopoulos, T\. Chakraborty, C\. Rose, and V\. Peng \(Eds\.\),Suzhou, China,pp\. 2757–2791\.External Links:[Link](https://aclanthology.org/2025.emnlp-main.138/),[Document](https://dx.doi.org/10.18653/v1/2025.emnlp-main.138),ISBN 979\-8\-89176\-332\-6Cited by:[§2\.3](https://arxiv.org/html/2607.10235#S2.SS3.p1.1)\.
- Y\. Li, H\. Chen, S\. Xu, Y\. Ge, J\. Tan, S\. Liu, and Y\. Zhang \(2023\)Fairness in recommendation: foundations, methods, and applications\.ACM Trans\. Intell\. Syst\. Technol\.14\(5\)\.External Links:ISSN 2157\-6904,[Link](https://doi.org/10.1145/3610302),[Document](https://dx.doi.org/10.1145/3610302)Cited by:[§2\.2](https://arxiv.org/html/2607.10235#S2.SS2.p1.1)\.
- J\. Liao, S\. Li, Z\. Yang, J\. Wu, Y\. Yuan, X\. Wang, and X\. He \(2024\)LLaRA: large language\-recommendation assistant\.InProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval,SIGIR ’24,New York, NY, USA,pp\. 1785–1795\.External Links:ISBN 9798400704314,[Link](https://doi.org/10.1145/3626772.3657690),[Document](https://dx.doi.org/10.1145/3626772.3657690)Cited by:[§2\.1](https://arxiv.org/html/2607.10235#S2.SS1.p3.1)\.
- S\. Lubos, T\. N\. T\. Tran, A\. Felfernig, S\. Polat Erdeniz, and V\. Le \(2024\)LLM\-generated explanations for recommender systems\.InAdjunct Proceedings of the 32nd ACM Conference on User Modeling, Adaptation and Personalization,UMAP Adjunct ’24,New York, NY, USA,pp\. 276–285\.External Links:ISBN 9798400704666,[Link](https://doi.org/10.1145/3631700.3665185),[Document](https://dx.doi.org/10.1145/3631700.3665185)Cited by:[§2\.1](https://arxiv.org/html/2607.10235#S2.SS1.p3.1)\.
- S\. Malberg, R\. Poletukhin, C\. Schuster, and G\. G\. Groh \(2025\)A comprehensive evaluation of cognitive biases in LLMs\.InProceedings of the 5th International Conference on Natural Language Processing for Digital Humanities,M\. Hämäläinen, E\. Öhman, Y\. Bizzoni, S\. Miyagawa, and K\. Alnajjar \(Eds\.\),Albuquerque, USA,pp\. 578–613\.External Links:[Link](https://aclanthology.org/2025.nlp4dh-1.50/),[Document](https://dx.doi.org/10.18653/v1/2025.nlp4dh-1.50),ISBN 979\-8\-89176\-234\-3Cited by:[§2\.3](https://arxiv.org/html/2607.10235#S2.SS3.p2.2)\.
- J\. Masthoff and A\. Delić \(2022\)Group recommender systems: beyond preference aggregation\.InRecommender Systems Handbook,F\. Ricci, L\. Rokach, and B\. Shapira \(Eds\.\),pp\. 381–420\.External Links:ISBN 978\-1\-0716\-2197\-4,[Document](https://dx.doi.org/10.1007/978-1-0716-2197-4%5F10)Cited by:[§2\.1](https://arxiv.org/html/2607.10235#S2.SS1.p1.1)\.
- J\. Masthoff \(2015\)Group recommender systems: aggregation, satisfaction and group attributes\.Inrecommender systems handbook,pp\. 743–776\.Cited by:[§2\.1](https://arxiv.org/html/2607.10235#S2.SS1.p2.1)\.
- S\. Najafian and N\. Tintarev \(2018\)Generating consensus explanations for group recommendations: an exploratory study\.InAdjunct Publication of the 26th Conference on User Modeling, Adaptation and Personalization,UMAP ’18\.External Links:[Document](https://dx.doi.org/10.1145/3213586.3225231)Cited by:[§2\.1](https://arxiv.org/html/2607.10235#S2.SS1.p1.1)\.
- T\. N\. Nguyen, F\. Ricci, A\. Delic, and D\. Bridge \(2019\)Conflict resolution in group decision making: insights from a simulation study\.User Modeling and User\-Adapted Interaction29\(5\),pp\. 895–941\.Cited by:[§2\.1](https://arxiv.org/html/2607.10235#S2.SS1.p2.1)\.
- A\. Petruzzelli, C\. Musto, L\. Laraspata, I\. Rinaldi, M\. de Gemmis, P\. Lops, and G\. Semeraro \(2024\)Instructing and prompting large language models for explainable cross\-domain recommendations\.In18th ACM Conference on Recommender SystemsRecSys ’24: 18th ACM Conference on Recommender Systems,New York, NY, USA,pp\. 298–308\.Cited by:[§2\.1](https://arxiv.org/html/2607.10235#S2.SS1.p3.1)\.
- S\. Rossi, F\. Cervone, and F\. Barile \(2018\)An altruistic\-based utility function for group recommendation\.InTransactions on Computational Collective Intelligence XXVIII,pp\. 25–47\.Cited by:[§2\.1](https://arxiv.org/html/2607.10235#S2.SS1.p2.1)\.
- A\. Said \(2025\)On explaining recommendations with large language models: a review\.Frontiers in Big Data7\.External Links:[Link](https://www.frontiersin.org/journals/big-data/articles/10.3389/fdata.2024.1505284),[Document](https://dx.doi.org/10.3389/fdata.2024.1505284),ISSN 2624\-909XCited by:[§2\.1](https://arxiv.org/html/2607.10235#S2.SS1.p3.1)\.
- S\. Sanner, K\. Balog, F\. Radlinski, B\. Wedin, and L\. Dixon \(2023\)Large language models are competitive near cold\-start recommenders for language\- and item\-based preferences\.InProceedings of the 17th ACM Conference on Recommender SystemsRecSys ’23: Seventeenth ACM Conference on Recommender Systems,Vol\.1,New York, NY, USA,pp\. 890–896\.Cited by:[§2\.1](https://arxiv.org/html/2607.10235#S2.SS1.p3.1)\.
- C\. Senot, D\. Kostadinov, M\. Bouzid, J\. Picault, A\. Aghasaryan, and C\. Bernier \(2010\)Analysis of strategies for building group profiles\.InInternational Conference on User Modeling, Adaptation, and Personalization,pp\. 40–51\.Cited by:[§1](https://arxiv.org/html/2607.10235#S1.p1.1),[§2\.1](https://arxiv.org/html/2607.10235#S2.SS1.p2.1),[§3\.2\.2](https://arxiv.org/html/2607.10235#S3.SS2.SSS2.p1.1)\.
- L\. Shi, C\. Ma, W\. Liang, X\. Diao, W\. Ma, and S\. Vosoughi \(2025\)Judging the judges: a systematic study of position bias in LLM\-as\-a\-judge\.InProceedings of the 14th International Joint Conference on Natural Language Processing and the 4th Conference of the Asia\-Pacific Chapter of the Association for Computational Linguistics,K\. Inui, S\. Sakti, H\. Wang, D\. F\. Wong, P\. Bhattacharyya, B\. Banerjee, A\. Ekbal, T\. Chakraborty, and D\. P\. Singh \(Eds\.\),Mumbai, India,pp\. 292–314\.External Links:[Link](https://aclanthology.org/2025.ijcnlp-long.18/),[Document](https://dx.doi.org/10.18653/v1/2025.ijcnlp-long.18),ISBN 979\-8\-89176\-298\-5Cited by:[§2\.3](https://arxiv.org/html/2607.10235#S2.SS3.p2.2)\.
- M\. Stratigi, E\. Pitoura, and K\. Stefanidis \(2023\)SQUIRREL: a framework for sequential group recommendations through reinforcement learning\.Information Systems112,pp\. 102128\.External Links:ISSN 0306\-4379,[Document](https://dx.doi.org/https%3A//doi.org/10.1016/j.is.2022.102128),[Link](https://www.sciencedirect.com/science/article/pii/S0306437922001065)Cited by:[§2\.1](https://arxiv.org/html/2607.10235#S2.SS1.p2.1)\.
- A\. Tommasel \(2024\)Fairness matters: a look at LLM\-generated group recommendations\.In18th ACM Conference on Recommender SystemsRecSys ’24: 18th ACM Conference on Recommender Systems,New York, NY, USA,pp\. 993–998\.Cited by:[§2\.1](https://arxiv.org/html/2607.10235#S2.SS1.p2.1),[§2\.1](https://arxiv.org/html/2607.10235#S2.SS1.p3.1)\.
- T\. N\. T\. Tran, M\. Atas, A\. Felfernig, V\. M\. Le, R\. Samer, and M\. Stettinger \(2019\)Towards social choice\-based explanations in group recommender systems\.InProceedings of the 27th ACM Conference on User Modeling, Adaptation and Personalization,pp\. 13–21\.Cited by:[Table 1](https://arxiv.org/html/2607.10235#S1.T1),[Table 1](https://arxiv.org/html/2607.10235#S1.T1.3.2),[§2\.1](https://arxiv.org/html/2607.10235#S2.SS1.p2.1),[§2\.2](https://arxiv.org/html/2607.10235#S2.SS2.p2.1)\.
- K\. Wataoka, T\. Takahashi, and R\. Ri \(2025\)Self\-preference bias in LLM\-as\-a\-judge\.External Links:2410\.21819Cited by:[§2\.3](https://arxiv.org/html/2607.10235#S2.SS3.p2.2)\.
- C\. Waterschoot, N\. Tintarev, and F\. Barile \(2025a\)Consistent explainers or unreliable narrators? understanding llm\-generated group recommendations\.InProceedings of the Nineteenth ACM Conference on Recommender Systems \(RecSys ’25\),New York, NY, USA\.External Links:[Document](https://dx.doi.org/10.1145/3705328.3748015)Cited by:[§2\.1](https://arxiv.org/html/2607.10235#S2.SS1.p3.1)\.
- C\. Waterschoot, N\. Tintarev, and F\. Barile \(2025b\)The pitfalls of growing group complexity: llms and social choice\-based aggregation for group recommendations\.InAdjunct Proceedings of the 33rd ACM Conference on User Modeling, Adaptation and Personalization \(UMAP Adjunct ’25\),New York, NY, USA\.External Links:[Document](https://dx.doi.org/10.1145/3708319.3733659)Cited by:[§2\.1](https://arxiv.org/html/2607.10235#S2.SS1.p2.1),[§2\.1](https://arxiv.org/html/2607.10235#S2.SS1.p3.1),[§3\.2\.1](https://arxiv.org/html/2607.10235#S3.SS2.SSS1.p1.2)\.
- C\. Waterschoot, N\. Tintarev, and F\. Barile \(2026\)Who is the fairest of them all? using large language models for fairness assessments\.InProceedings of the 34th ACM Conference on User Modeling, Adaptation and Personalization,UMAP ’26,New York, NY, USA,pp\. 567–570\.External Links:ISBN 9798400723117,[Link](https://doi.org/10.1145/3774935.3812703),[Document](https://dx.doi.org/10.1145/3774935.3812703)Cited by:[§5\.3](https://arxiv.org/html/2607.10235#S5.SS3.p2.1)\.
- C\. Waterschoot and A\. van den Bosch \(2024\)A time\-robust group recommender for featured comments on news platforms\.Frontiers in Big DataVolume 7 \- 2024\.External Links:[Link](https://www.frontiersin.org/journals/big-data/articles/10.3389/fdata.2024.1399739),[Document](https://dx.doi.org/10.3389/fdata.2024.1399739),ISSN 2624\-909XCited by:[§2\.1](https://arxiv.org/html/2607.10235#S2.SS1.p1.1)\.
- C\. Waterschoot, R\. Yera Toledo, N\. Tintarev, and F\. Barile \(2025c\)With friends like these, who needs explanations? evaluating user understanding of group recommendations\.InProceedings of the 33rd ACM Conference on User Modeling, Adaptation and Personalization \(UMAP ’25\),New York, NY, USA\.External Links:[Document](https://dx.doi.org/10.1145/3699682.3728345)Cited by:[§2\.1](https://arxiv.org/html/2607.10235#S2.SS1.p1.1),[§3\.2\.1](https://arxiv.org/html/2607.10235#S3.SS2.SSS1.p2.1)\.
- X\. Wu, H\. Zhou, Y\. Shi, W\. Yao, X\. Huang, and N\. Liu \(2024\)Could small language models serve as recommenders? towards data\-centric cold\-start recommendation\.InProceedings of the ACM Web Conference 2024WWW ’24: The ACM Web Conference 2024,Vol\.55,New York, NY, USA,pp\. 3566–3575\.Cited by:[§2\.1](https://arxiv.org/html/2607.10235#S2.SS1.p3.1)\.
- Z\. Wu \(2023\)Enhanced privacy preservation for recommender systems\.InProceedings of the 17th ACM Conference on Recommender SystemsRecSys ’23: Seventeenth ACM Conference on Recommender Systems,Vol\.11,New York, NY, USA,pp\. 1364–1368\.Cited by:[§2\.1](https://arxiv.org/html/2607.10235#S2.SS1.p3.1)\.
- D\. Yang, F\. Chen, and H\. Fang \(2024\)Behavior alignment: a new perspective of evaluating LLM\-based conversational recommendation systems\.InProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information RetrievalSIGIR 2024: The 47th International ACM SIGIR Conference on Research and Development in Information Retrieval,Vol\.35,New York, NY, USA,pp\. 2286–2290\.Cited by:[§2\.1](https://arxiv.org/html/2607.10235#S2.SS1.p3.1)\.
- J\. Zhang, M\. Gao, J\. Yu, L\. Guo, J\. Li, and H\. Yin \(2021\)Double\-scale self\-supervised hypergraph learning for group recommendation\.InProceedings of the 30th ACM International Conference on Information & Knowledge Management,CIKM ’21,New York, NY, USA,pp\. 2557–2567\.External Links:ISBN 9781450384469,[Link](https://doi.org/10.1145/3459637.3482426),[Document](https://dx.doi.org/10.1145/3459637.3482426)Cited by:[§2\.1](https://arxiv.org/html/2607.10235#S2.SS1.p2.1)\.
- Z\. Zhao, W\. Fan, J\. Li, Y\. Liu, X\. Mei, Y\. Wang, Z\. Wen, F\. Wang, X\. Zhao, J\. Tang, and Q\. Li \(2024\)Recommender systems in the era of large language models \(LLMs\)\.IEEE Trans\. Knowl\. Data Eng\.36\(11\),pp\. 6889–6907\.Cited by:[§2\.1](https://arxiv.org/html/2607.10235#S2.SS1.p3.1)\.

Similar Articles

Evaluating LLMs as Human Surrogates in Controlled Experiments

arXiv cs.CL

This paper evaluates whether off-the-shelf LLMs can reliably simulate human responses in controlled behavioral experiments by comparing LLM-generated data with human survey responses on accuracy perception. The findings show that while LLMs capture directional effects and aggregate belief-updating patterns, they do not consistently match human-scale effect magnitudes, clarifying when synthetic LLM data can serve as behavioral proxies.

The Geometry of LLM-as-Judge: Why Inter-LLM Consensus Is Not Human Alignment

arXiv cs.CL

This paper geometrically analyzes why LLMs acting as judges agree strongly with each other but weakly with humans, finding that inter-LLM consensus reflects a collapsed subspace rather than true human alignment on subjective rubrics. Post-hoc calibration on human data improves alignment, but even calibrated LLMs fall short of human reliability.