Leveraging Speech Acts for Low-Data and Cross-Domain Conversation Derailment Forecasting

# Leveraging Speech Acts for Low-Data and Cross-Domain Conversation Derailment Forecasting
Source: [https://arxiv.org/html/2608.25359](https://arxiv.org/html/2608.25359)
###### Abstract

Conversational derailment forecasting aims to predict when online discussions will escalate into hostility, enabling proactive moderation\. Existing approaches often struggle in low\-data settings and to generalize across domains\. This poses a challenge for new platforms and smaller communities where annotated data is limited\. We propose modeling pragmatic representations of conversations to reduce lexical noise and improve generalizability\. Specifically, speech act information is used as an auxiliary learning signal alongside textual semantics\. Experimental results show improved performance across three datasets, particularly in low\-data and cross\-domain settings\.

## 1Introduction

Humans can often anticipate when a disagreement is likely to escalate into hostility, allowing them to intervene before conversations deteriorate\. Conversational derailment forecasting aims to computationally replicate this ability by predicting hostile breakdowns before they occur, assisting with proactive online platform moderation[Tran et al\. \(2025\)](https://arxiv.org/html/2608.25359#bib.bib21)\.

Due to the subtle and context\-dependent nature of conversational derailment across diverse online communities, computational modeling of this process presents a significant challenge\. The language used in a political debate on Reddit looks vastly different from a technical GitHub conversation, and the specific vocabulary of a heated dispute can further diverge\. Existing forecasting systems[Chang and Danescu\-Niculescu\-Mizil \(2019\)](https://arxiv.org/html/2608.25359#bib.bib20);[Kementchedjhieva and Søgaard \(2021\)](https://arxiv.org/html/2608.25359#bib.bib29)primarily rely on semantic features and frequently struggle to adapt to these lexical shifts, or require large volumes of data to learn broadly generalizable features\. However, data annotation is expensive, and labelled data is often limited for new platforms and smaller communities\. This creates a need for models that perform reliably in low\-data settings, including by leveraging transferable signals from data\-rich platforms\.

To address these challenges, we propose a novel approach to model speech acts \(SA\) to inform derailment forecasting\. Grounded in Speech Act Theory[Austin \(1975\)](https://arxiv.org/html/2608.25359#bib.bib36), we hypothesize that SAs provide a low\-dimensional pragmatic representation that mitigates lexical noise and improves forecasting generalizability\. To effectively fuse pragmatic and semantic signals without inference latency, we introduce two architectures that integrate SAs via an auxiliary task\. Unlike recent approaches that rely on domain\-specific features and inference\-time auxiliary information[Altarawneh et al\. \(2023\)](https://arxiv.org/html/2608.25359#bib.bib27);[Hua et al\. \(2024\)](https://arxiv.org/html/2608.25359#bib.bib26);[Zhang et al\. \(2025\)](https://arxiv.org/html/2608.25359#bib.bib13), we use transferable pragmatic tracking to address the gap in low\-data and cross\-domain dynamic derailment forecasting\.

Our contributions are as follows:

1. 1\.We propose a novel approach for incorporating SA information into derailment forecasting\. Extensive experiments across three datasets demonstrate improved generalizability over state\-of\-the\-art baselines, particularly in low\-data and cross\-dataset settings;
2. 2\.We investigate the relationship between dataset divergence and the cross\-dataset generalizability of models based on textual and pragmatic features, showing that pragmatic features yield greater benefits when transferring to highly divergent datasets; and
3. 3\.We introduce an evaluation protocol for dynamic forecasting performance through average temporal aggregation, better reflecting models’ per\-timestep behavior\.

## 2Related Work

### 2\.1Conversational Derailment Forecasting

Conversational derailment forecasting aims to predict if an initially civil conversation will eventually degrade into hostile behavior[Zhang et al\. \(2018\)](https://arxiv.org/html/2608.25359#bib.bib22)\. Unlike standard toxicity detection, which classifies the abusiveness of utterances after they have occurred[Poletto et al\. \(2020\)](https://arxiv.org/html/2608.25359#bib.bib39), derailment forecasting dynamically anticipates them, enabling moderator intervention before discussions get out of control\.

Prior works on conversational derailment forecasting have proposed varied approaches, ranging from traditional machine learning models with handcrafted linguistic features[Zhang et al\. \(2018\)](https://arxiv.org/html/2608.25359#bib.bib22)to deep learning architectures operating on raw text and conversational metadata[Chang and Danescu\-Niculescu\-Mizil \(2019\)](https://arxiv.org/html/2608.25359#bib.bib20);[Altarawneh et al\. \(2023\)](https://arxiv.org/html/2608.25359#bib.bib27)\. More recently, pretrained language models \(PLMs\) have been widely adopted, driven by their pretrained generalizable language representations that transfer well to conversational forecasting tasks[Kementchedjhieva and Søgaard \(2021\)](https://arxiv.org/html/2608.25359#bib.bib29);[Hua et al\. \(2024\)](https://arxiv.org/html/2608.25359#bib.bib26);[Tran et al\. \(2025\)](https://arxiv.org/html/2608.25359#bib.bib21)\. The in\-context learning capabilities of large language models \(LLMs\) has also been assessed via few\-shot prompting[Imran et al\. \(2025\)](https://arxiv.org/html/2608.25359#bib.bib23);[Zhang et al\. \(2025\)](https://arxiv.org/html/2608.25359#bib.bib13)\. While LLMs introduce high computational costs and inference latency[Zhang et al\. \(2025\)](https://arxiv.org/html/2608.25359#bib.bib13), making them unsuitable for real\-time moderation on high\-traffic platforms, prior work has primarily evaluated them using conversation summaries or full dialogues, rather than dynamic, turn\-by\-turn forecasting\.

Current approaches leverage available datasets to achieve strong supervised performance, and largely ignore the challenges of low\-data regimes and cross\-domain generalizability\. Collecting derailment datasets is resource\-intensive, requiring extensive manual annotation[Zhang et al\. \(2018\)](https://arxiv.org/html/2608.25359#bib.bib22)in addition to explicit moderator actions \(e\.g\., thread locking, comment removal\) for initial filtering[Chang and Danescu\-Niculescu\-Mizil \(2019\)](https://arxiv.org/html/2608.25359#bib.bib20);[Imran et al\. \(2025\)](https://arxiv.org/html/2608.25359#bib.bib23)\. In practice, new platforms, niche communities, or domain\-specific conversational settings often possess limited annotated data\. Adapting derailment forecasting models to these resource\-constrained or out\-of\-domain environments remains heavily underexplored, highlighting a need for models capable of cross\-domain transfer without relying on large platform\-specific datasets\.

### 2\.2Speech Act Modeling

Speech Act Theory posits that language performs actions beyond conveying information[Austin \(1975\)](https://arxiv.org/html/2608.25359#bib.bib36)\. SA verbs \(e\.g\.,complain\) categorize these actions, encapsulating nuanced speaker intents and mental states[Goddard and Wierzbicka \(2013\)](https://arxiv.org/html/2608.25359#bib.bib1)\.

In natural language processing, mapping high\-dimensional text to a low\-dimensional action space is often used for two purposes: reducing domain\-specific lexical noise and capturing the latent pragmatic functions of utterances\. In task\-oriented dialogue systems, translating user text into a set of dialogue acts is frequently adopted as the first step for capturing user intent, prior to system response generation[Wu et al\. \(2020\)](https://arxiv.org/html/2608.25359#bib.bib38)\. Explicit SA\-based clustering has also been shown to improve cross\-domain generalizability in explainable politeness detection by reducing lexical reliance[Aljanaideh \(2025\)](https://arxiv.org/html/2608.25359#bib.bib37)\. Process mining of digital corpora further identifies patterns in SA sequences[Compagno et al\. \(2018\)](https://arxiv.org/html/2608.25359#bib.bib5), highlighting their potential to serve as sequential signals for conversational forecasting\.

Effectively leveraging these pragmatic properties for real\-time forecasting presents architectural challenges, which this work seeks to address: balancing SA and textual features, and avoiding the inference latency from real\-time SA extraction\.

## 3Proposed Approach

In this section, we describe our proposed approach to \(1\) extract SA information from raw dialogues, and \(2\) design derailment forecasting models that utilize the extracted SA information\.

### 3\.1Speech Act Extraction

#### Taxonomy construction

We employ a curated SA taxonomy consisting of 50 SA verbs frequently utilized in linguistic and conversational analysis literature[Goddard and Wierzbicka \(2013\)](https://arxiv.org/html/2608.25359#bib.bib1);[Compagno et al\. \(2018\)](https://arxiv.org/html/2608.25359#bib.bib5);[Isnaeni et al\. \(2025\)](https://arxiv.org/html/2608.25359#bib.bib11);[Kholid et al\. \(2024\)](https://arxiv.org/html/2608.25359#bib.bib10);[Searle \(1979\)](https://arxiv.org/html/2608.25359#bib.bib7)and an additional “other” label\. Each SA is accompanied by a brief description of its meaning and, where applicable, the speaker’s psychological state, derived from semantic linguistic works and standard lexicons[Vanderveken \(1990\)](https://arxiv.org/html/2608.25359#bib.bib6);[Goddard and Wierzbicka \(2013\)](https://arxiv.org/html/2608.25359#bib.bib1);[Wierzbicka \(1987\)](https://arxiv.org/html/2608.25359#bib.bib8);[Cambridge University Press \(2026\)](https://arxiv.org/html/2608.25359#bib.bib9)\. The taxonomy is detailed in Appendix[A](https://arxiv.org/html/2608.25359#A1)\.

#### Zero\-shot inference

Using this taxonomy, we prompt the open\-sourcegpt\-oss\-120b[OpenAI \(2025\)](https://arxiv.org/html/2608.25359#bib.bib15)to derive auxiliary SA information\. LLM\-based auxiliary knowledge, such as forecasted next utterances or emotional trajectories, has shown effectiveness in conversational forecasting tasks[Wang and Feng \(2023a\)](https://arxiv.org/html/2608.25359#bib.bib12);[Zhang et al\. \(2025\)](https://arxiv.org/html/2608.25359#bib.bib13)\. Following existing methodologies, we formulate this as a sentence\-level multi\-label classification task[Compagno et al\. \(2018\)](https://arxiv.org/html/2608.25359#bib.bib5)\. An utterance is assigned any speech act that appears in at least one of its constituent sentences\.

#### Prompt development

Prior work on pragmatic analysis and SA extraction explore discrepancies between literal phrasing and underlying intention[Li et al\. \(2023\)](https://arxiv.org/html/2608.25359#bib.bib17);[Spinoso\-Di Piano et al\. \(2025\)](https://arxiv.org/html/2608.25359#bib.bib16);[Orsini and Brunato \(2025\)](https://arxiv.org/html/2608.25359#bib.bib19), while research in derailment forecasting shows that conversational outcomes are further influenced by listener’s perception of speakers’ intents[Chang et al\. \(2020\)](https://arxiv.org/html/2608.25359#bib.bib14)\. To address these dimensions, we designed our prompt to target intended SAs as perceived by listeners\. Drawing on pragmatic analysis research, the prompt \(shown in Table[14](https://arxiv.org/html/2608.25359#A3.T14)\) explicitly instructs the model to identify factors that cause literal and intended meanings to diverge, including figurative language[Spinoso\-Di Piano et al\. \(2025\)](https://arxiv.org/html/2608.25359#bib.bib16);[Li et al\. \(2023\)](https://arxiv.org/html/2608.25359#bib.bib17), covert aggression[Ollagnier \(2024\)](https://arxiv.org/html/2608.25359#bib.bib18), indirectness[Orsini and Brunato \(2025\)](https://arxiv.org/html/2608.25359#bib.bib19)and violation of conversational maxims[Li et al\. \(2023\)](https://arxiv.org/html/2608.25359#bib.bib17)\. For example, the utterance “Yeah, you’re right, you should definitely be able to redefine terms like activity and external harms\.” may be literally interpreted as “agreeing” with the listener\. However, within a contentious exchange, listeners would likely perceive the speaker’s intent as the extracted SAs of “disagreeing” and “criticizing”\.

#### Human validation

While the primary objective of our framework is downstream forecasting performance, we evaluated whether LLM\-extracted SAs align with human judgments to provide a meaningful pragmatic signal\. Specifically, 85 sentences from 30 randomly selected responses were manually labeled\. Given the inherent subjectivity of speech act annotation, the LLM demonstrated a moderate Cohen’s Kappa agreement \(κ=0\.53\\kappa=0\.53\) across the 51 granular classes, and almost perfect agreement \(κ=0\.88\\kappa=0\.88\) when collapsed into 5 fundamental categories \(assertive, directive, commissive, expressive, and ‘other’ replacing declarative\)\. This is consistent with the human inter\-annotator agreement observed for SA annotation by[Compagno et al\. \(2018\)](https://arxiv.org/html/2608.25359#bib.bib5)\. Further details on the SA extraction evaluation are available in Appendix[A](https://arxiv.org/html/2608.25359#A1)\.

### 3\.2Derailment Forecasting Model

The extracted SA information is used via an auxiliary SA detection task during model training to regularize the model’s latent space with pragmatic signals\. Because this information is strictly a training signal, SA extraction is not required at inference time, eliminating both the cost and latency of running external extraction pipelines\.

We designed two models that incorporate both raw text and SA information, with one specifically configured to prioritize SA signals\. As illustrated in Figure[1](https://arxiv.org/html/2608.25359#S3.F1), both utilize a hierarchical architecture, which has proven effective in modeling dynamics for conversational prediction tasks[Chang and Danescu\-Niculescu\-Mizil \(2019\)](https://arxiv.org/html/2608.25359#bib.bib20);[Yuan and Singh \(2023\)](https://arxiv.org/html/2608.25359#bib.bib28), and also naturally facilitates the integration of SA labels at the utterance level\. The architecture is composed of two primary components\. Firstly, a PLM\-based utterance\-level encoder \(e\.g\.,RoBERTa\-Large;[Liu et al\., 2019](https://arxiv.org/html/2608.25359#bib.bib25)\) processes the raw text of individual utterances and generates an embedding for each\. Secondly, a two\-layer Transformer\-based conversation\-level encoder takes PLM\-encoded utterance embeddings, together with speaker and utterance ID embeddings, and outputs contextualized representations for each utterance, capturing its conversational context\.

\(a\)HparallelH\_\{\\text\{parallel\}\}\(b\)HsequentialH\_\{\\text\{sequential\}\}
Figure 1:Architecture ofHparallelH\_\{\\text\{parallel\}\}andHsequentialH\_\{\\text\{sequential\}\}\.On top of these components, two classification heads are appended for SA detection \(H​e​a​dS​AHead\_\{SA\}\) and derailment forecasting \(H​e​a​dd​e​r​a​i​lHead\_\{derail\}\) respectively\.H​e​a​dS​AHead\_\{SA\}is applied to each contextualized utterance embedding for per\-utterance multi\-label SA classification\. It leverages the hierarchical architecture to prioritize the target utterance while attending to the contextual information\.

The two proposed models differ in the design ofH​e​a​dd​e​r​a​i​lHead\_\{derail\}\. InHparallelH\_\{\\text\{parallel\}\}\(Figure[1\(a\)](https://arxiv.org/html/2608.25359#S3.F1.sf1)\),H​e​a​dd​e​r​a​i​lHead\_\{derail\}is attached on top of the conversation\-level encoder in parallel withH​e​a​dS​AHead\_\{SA\}\. It averages the embeddings from the final turn of conversation to produce the prediction\. To avoid the model relying mainly on raw textual features instead of SA information,HparallelH\_\{\\text\{parallel\}\}undergoes two\-phased training: an initial intermediate phase focused solely on SA detection, followed by multi\-task training for derailment forecasting with SA detection as an auxiliary task\. The second model,HsequentialH\_\{\\text\{sequential\}\}\(Figure[1\(b\)](https://arxiv.org/html/2608.25359#S3.F1.sf2)\), is designed to prioritize SA signal over text\. A uni\-directional GRU layer[Cho et al\. \(2014\)](https://arxiv.org/html/2608.25359#bib.bib41)is built upon the SA probabilities fromH​e​a​dS​AHead\_\{SA\}\.H​e​a​dd​e​r​a​i​lHead\_\{derail\}then predicts based on the final GRU state\. By relying primarily on the detected SA probabilities for forecasting, the model directly performs multi\-task training without an intermediate SA\-only phase\.

## 4Experimental setup

### 4\.1Derailment Datasets

We consider three datasets covering conversational derailment across different platforms, representing a broad spectrum of online conversations: editorial negotiations \(Wikipedia\); open\-domain, inherently argumentative debates \(Reddit ChangeMyView\); and goal\-oriented technical discussions \(GitHub\)\. This diversity enables us to evaluate the models’ cross\-domain generalizability\.

#### Wikipedia data \(WIKI\)

[Zhang et al\. \(2018\)](https://arxiv.org/html/2608.25359#bib.bib22);[Chang and Danescu\-Niculescu\-Mizil \(2019\)](https://arxiv.org/html/2608.25359#bib.bib20): Conversations were collected from publicly accessible talk pages for Wikipedia editors and labeled by crowdworkers as either containing a personal attack toward another user or remaining civil throughout\. Each derailed conversation is paired with a civil conversation from the same talk page to avoid topic\-specific trivial correlations\. WIKI contains 4,188 conversations, partitioned into a train\-validation\-test split of 60\-20\-20\. Paired conversations are kept within the same split to preserve topic control\.

#### Reddit CMV data \(CMV\)

[Chang and Danescu\-Niculescu\-Mizil \(2019\)](https://arxiv.org/html/2608.25359#bib.bib20);[Tran et al\. \(2025\)](https://arxiv.org/html/2608.25359#bib.bib21): Dialogues were collected from the subreddit ChangeMyView and labeled based on whether a conversation eventually had a comment removed by a moderator for violating Rule 2: “Don’t be rude or hostile to other users\.” Similar to WIKI, each derailed dialogue is paired with a civil dialogue that belongs to the same top\-level post\. CMV contains 19,578 conversations, split similarly to WIKI\.

#### GitHub data \(GITHUB\)

[Imran et al\. \(2025\)](https://arxiv.org/html/2608.25359#bib.bib23): Conversations were collected from GitHub issues and pull requests\. Toxic conversations were primarily sourced from threads locked by moderators as “too heated”, “spam”, or “off\-topic”, with individual toxic comments identified via a combination ofGPT\-4oand human verification\. Civil conversations were sampled from the same repositories with a ratio of four civil threads per toxic thread, subject to repository availability, to better reflect a realistic data distribution\. This yielded 202 derailed and 696 civil dialogues\. Following WIKI and CMV, we split the dataset using a 60\-20\-20 ratio\.

All datasets are preprocessed to only contain responses prior to derailment, and civil dialogues in the training sets are truncated accordingly to prevent trivial correlations related to dialogue length\. Since the GITHUB dataset captures entire thread discussion and can be exceptionally long \(e\.g\., 120 utterances\), we crop all GITHUB dialogues to their 10 most recent utterances\. After processing, all training datasets have a median length of 5 utterances, with averages of 6\.02 \(CMV\), 5\.28 \(WIKI\), and 5\.24 \(GITHUB\) utterances, respectively\.

SAs of 51 classes were extracted for training sets following Section[3\.1](https://arxiv.org/html/2608.25359#S3.SS1)\. To evaluate robustness of our models to the SAs used, Appendix[C\.6](https://arxiv.org/html/2608.25359#A3.SS6)details supplementary experiments on a coarser 18\-class SA taxonomy, demonstrating similar performance\.

### 4\.2Evaluation Task and Metrics

#### Dynamic Evaluation

In alignment with prior work[Chang and Danescu\-Niculescu\-Mizil \(2019\)](https://arxiv.org/html/2608.25359#bib.bib20);[Tran et al\. \(2025\)](https://arxiv.org/html/2608.25359#bib.bib21), models are trained statically on full preprocessed dialogues with derailed utterances removed, and evaluated on their dynamic forecasting performance\. LetC=\(u1,u2,⋯,un\)C=\(u^\{1\},u^\{2\},\\cdots,u^\{n\}\)be any conversation consisting ofnnutterances\. During training, models receive entireCCand their corresponding labels\. During evaluation, models make a predictionptp^\{t\}given each ofCt=\(u1,u2,⋯,ut\)C^\{t\}=\(u^\{1\},u^\{2\},\\cdots,u^\{t\}\), fort∈\[1,n\]t\\in\[1,n\], generating a sequence ofnnprobabilitiesp=\(p1,p2,⋯,pn\)p=\(p^\{1\},p^\{2\},\\cdots,p^\{n\}\)\.

#### Temporal Aggregation

To evaluate this dynamic performance using standard classification metrics such as the F1 score, the utterance\-level predictions are aggregated into a single conversation\-level forecast and compared against the ground\-truth label\. Prior work employs max aggregation \(Dynamicmax\\text\{Dynamic\}\_\{\\text\{max\}\}\), converting the highest probability into a binary label via a threshold tuned on the validation set \(i\.e\.,y^=𝟙\{max\(p\)\>tm​a​x\}\\hat\{y\}=\\mathbbm\{1\}\_\{\\\{max\(p\)\>t\_\{max\}\\\}\}\)[Chang and Danescu\-Niculescu\-Mizil \(2019\)](https://arxiv.org/html/2608.25359#bib.bib20)\. This implies that a conversation is predicted to derail if any dynamic prediction crosses the threshold, making the evaluation highly sensitive to false positive spikes \(e\.g\., a civil dialogue being flagged as derailed due to a single anomalous prediction at turn two, despite recovering to low probabilities for the remainder of the sequence\)\. To address this, we perform mean aggregation \(Dynamicmean\\text\{Dynamic\}\_\{\\text\{mean\}\}, i\.e\.,y^=𝟙\{mean\(p\)\>tm​e​a​n\}\\hat\{y\}=\\mathbbm\{1\}\_\{\\\{mean\(p\)\>t\_\{mean\}\\\}\}\), which remains high only if the model signals derailment early and consistently in the exchange, rather than spiking solely at specific turns or toward the final utterance\. Furthermore, to avoid the loss of temporal granularity caused by aggregation, we explicitly examine model performance at individual timesteps as dialogues approach the target events \(derailments or end of civil dialogues\)\. Specifically, we evaluate the models’ predictions exactlykksteps away from the event, wherek∈\[1,5\]k\\in\[1,5\]\.

#### Metrics

For evaluation metrics, we report the Area Under the Precision\-Recall Curve \(AUPRC\) and Macro\-F1 score, both of which are robust to imbalanced datasets \(such as the GITHUB corpus\)\. AUPRC avoids the need for threshold tuning by assessing the model’s ability to rank positive cases above negative cases across all thresholds\. This metric is effective for evaluating generalizability to OOD settings \(where models trained on one dataset are tested on the others\), as it isolates the model’s capacity to provide derailment signals from its reliance on domain\-specific threshold calibration\. Conversely, the Macro\-F1 score provides a complementary evaluation of the model’s practical performance when deployed at a specific threshold\.

#### Training Set Sizes

To assess model performance across varying data regimes, we train the models using different amounts of training data and report the Area Under the Learning Curve \(AULC;[Viering and Loog, 2023](https://arxiv.org/html/2608.25359#bib.bib24)\)\. Specifically, we utilize training subset sizes of 300, 500, 1000, 2500, and 11802, capped by the maximum capacity of each respective dataset\. These subset sizes allow us to evaluate how model performance scales with increasing data up to approximately the full availability of the training corpora\. The full training sets contain 2508 \(WIKI\), 538 \(GITHUB\), and 11802 \(CMV\) dialogues\. The AULC is calculated as the integral of the performance metrics \(AUPRC and Macro\-F1\) plotted against the log\-scaled training size\. To facilitate easier interpretation across datasets, the AULC is normalized to a range of 0 to 1; this linear scaling preserves the relative performance differences and does not affect model comparisons\.

Table 1:Comparison of models’Dynamicmean\\text\{Dynamic\}\_\{\\text\{mean\}\}AULC performance\. Best results areboldedand second\-best areunderlined\. The†\\daggersymbol denotes that the model’s performance is statistically significantly different to the best model \(p<0\.05p<0\.05, two\-tailed paired t\-test\)\.HparallelH\_\{\\text\{parallel\}\}achieves the strongest performance across the datasets\.

### 4\.3Baseline Models

We compareHparallelH\_\{\\text\{parallel\}\}andHsequentialH\_\{\\text\{sequential\}\}against SOTA derailment forecasting models\. CRAFT[Chang and Danescu\-Niculescu\-Mizil \(2019\)](https://arxiv.org/html/2608.25359#bib.bib20)was introduced as the first dynamic derailment forecasting model\. It employs a hierarchical recurrent encoder\-decoder \(HRED\) architecture that learns to represent conversational dynamics through unsupervised pretraining on a large corpus via a next\-response generation objective, before being fine\-tuned for the conversational derailment task\. To benefit from larger\-scale general pretraining, recent approaches[Kementchedjhieva and Søgaard \(2021\)](https://arxiv.org/html/2608.25359#bib.bib29);[Yuan and Singh \(2023\)](https://arxiv.org/html/2608.25359#bib.bib28);[Tran et al\. \(2025\)](https://arxiv.org/html/2608.25359#bib.bib21)directly employ PLMs\. Among these,RoBERTa\-Large[Liu et al\. \(2019\)](https://arxiv.org/html/2608.25359#bib.bib25)andBERT\-Base[Devlin et al\. \(2018\)](https://arxiv.org/html/2608.25359#bib.bib32)have been widely adopted for conversational forecasting tasks, including predicting derailment[Tran et al\. \(2025\)](https://arxiv.org/html/2608.25359#bib.bib21);[Kementchedjhieva and Søgaard \(2021\)](https://arxiv.org/html/2608.25359#bib.bib29)and emotion[Xu et al\. \(2025\)](https://arxiv.org/html/2608.25359#bib.bib30);[Wang and Feng \(2023b\)](https://arxiv.org/html/2608.25359#bib.bib31)\. Given its larger architecture and superior performance,RoBERTa\-Largeis selected as both a SOTA baseline \(denoted as PLM in tables and figures\) and the utterance encoder for our proposed models\. Additionally, as both a baseline and an ablation study, we evaluateHablationH\_\{\\text\{ablation\}\}, which shares the same hierarchical architecture ofHparallelH\_\{\\text\{parallel\}\}adapted from[Yuan and Singh \(2023\)](https://arxiv.org/html/2608.25359#bib.bib28), but is trained without incorporating SA knowledge\. To evaluate the robustness of the proposed method to the choice of underlying PLM and model architecture, two supplementary experiments utilizing BERT and a graph\-based model GraphNLI[Agarwal et al\. \(2023\)](https://arxiv.org/html/2608.25359#bib.bib42)are detailed in Appendix[C\.4](https://arxiv.org/html/2608.25359#A3.SS4)and[C\.5](https://arxiv.org/html/2608.25359#A3.SS5)respectively, exhibiting performance trends comparable to those ofRoBERTa\-Largeand hierarchical models\.

Baseline models were trained according to their original implementations, with minor adjustments to accommodate different dataset sizes \(e\.g\., increased epochs upon smaller training sets\)\. Performance metrics represent the average across five random seeds, where training subsets are also independently sampled for each seed \(for WIKI and CMV, conversation pairs are sampled accordingly\)\. Training details are provided in Appendix[B](https://arxiv.org/html/2608.25359#A2)\.

Other recent methods have been proposed, including summary\-based[Imran et al\. \(2025\)](https://arxiv.org/html/2608.25359#bib.bib23);[Hua et al\. \(2024\)](https://arxiv.org/html/2608.25359#bib.bib26)and generation\-based[Zhang et al\. \(2025\)](https://arxiv.org/html/2608.25359#bib.bib13)approaches\. They operate at conversation level, making a single prediction given the full dialogue or its summary, rather than at utterance\-level for dynamic forecasting\. The approach of[Altarawneh et al\. \(2023\)](https://arxiv.org/html/2608.25359#bib.bib27)relies on community voting scores on utterances, a feature available in CMV but absent in others\. Some methods shift to dynamically training the models on incrementally expanding dialogue histories to improve performance or enable earlier detection[Yuan and Singh \(2023\)](https://arxiv.org/html/2608.25359#bib.bib28);[Kementchedjhieva and Søgaard \(2021\)](https://arxiv.org/html/2608.25359#bib.bib29), which alters the training paradigm and is orthogonal to our modeling approach\. Given our focus on applicability to different datasets, cross\-dataset generalizability, and dynamic forecasting, we exclude these methods from our baseline comparisons\. However, we note that their underlying architectures are primarily PLM\-based and hierarchical, which are well\-represented by our selected baselines\.

## 5Results and Discussion

### 5\.1In\-Domain Performance

Table[1](https://arxiv.org/html/2608.25359#S4.T1)reportsDynamicmean\\text\{Dynamic\}\_\{\\text\{mean\}\}AULC, enabling comparisons between the models via single metrics\. Figure[2](https://arxiv.org/html/2608.25359#S5.F2)illustrates example learning curves across different training set sizes, while the AULC values reported in Table[1](https://arxiv.org/html/2608.25359#S4.T1)represent the area under these curves, scaled by the range of the x\-axis to fall within \[0,1\]\. From Table[1](https://arxiv.org/html/2608.25359#S4.T1),HparallelH\_\{\\text\{parallel\}\}consistently achieves the best performance across datasets111They achieve a statistically significant improvement \(p<0\.05p<0\.05, paired t\-test\) over all baseline models on one or both of Macro\-F1 and AUPRC, though 5 seeds has difficulty to test normality assumption and provides limited statistical power\., with PLM andHablationH\_\{\\text\{ablation\}\}typically ranking second\.

All models except CRAFT are based on theRoBERTa\-LargePLM\. The latter is pre\-trained on a large generic corpus, whereas CRAFT is pre\-trained on a substantially smaller, domain\-specific corpus222CRAFT is unavailable for GITHUB due to missing pretraining corpus\.\. Despite this, CRAFT still demonstrates comparable performance as a lightweight model\. The architectural shift from a flat PLM to a hierarchical design \(HablationH\_\{\\text\{ablation\}\}, which uses the PLM as an utterance encoder\) leads to slightly improved AUPRC, though it can underperform on Macro\-F1 in some cases\. The additional gains observed inHparallelH\_\{\\text\{parallel\}\}overHablationH\_\{\\text\{ablation\}\}highlight the benefit of incorporating SA to guide model predictions\. In contrast,HsequentialH\_\{\\text\{sequential\}\}performs worse than most PLM\-based models and shows similar performance to CRAFT\. We hypothesise that this is becauseHsequentialH\_\{\\text\{sequential\}\}more strongly enforces reliance on SA knowledge compared toHparallelH\_\{\\text\{parallel\}\}, which can suppress the availability of generalizable textual features and lead to poorer performance in in\-domain settings\.

WhileHparallelH\_\{\\text\{parallel\}\}consistently outperforms other models inDynamicmean\\text\{Dynamic\}\_\{\\text\{mean\}\}, it does not inDynamicmax\\text\{Dynamic\}\_\{\\text\{max\}\}\(reported in Appendix[C\.1](https://arxiv.org/html/2608.25359#A3.SS1)\), instead ranking first or second across datasets\. This indicates that the model achieves better performance when predictions are updated at each timestep, rather than fixed upon any derailment alarm\. To further examine models’ dynamic forecasting capacity, a per\-timestep analysis is presented later in this section\.

#### Model Performance Across Data Regimes

Figure 2:Models’Dynamicmean\\text\{Dynamic\}\_\{\\text\{mean\}\}AUPRC learning curves on CMV, plotted against training subset size\.\(a\)300 Train Subset\(b\)500 Train Subset\(c\)1000 Train Subset
Figure 3:Model AUPRC performance on CMV is plotted as dialogues progress toward derailment or conversation end, across training data sizes\.HparallelH\_\{\\text\{parallel\}\}achieves the best performance, particularly in the low\-data regime\.Figure[2](https://arxiv.org/html/2608.25359#S5.F2)presents theDynamicmean\\text\{Dynamic\}\_\{\\text\{mean\}\}learning curve of the models prior to AULC aggregation\. The CMV performance is visualised here; similar trends for other datasets are reported in Appendix[C\.2](https://arxiv.org/html/2608.25359#A3.SS2)\. Notably,HparallelH\_\{\\text\{parallel\}\}demonstrates strong performance in low\-data regimes \(300\-2500 samples\), a critical advantage for real\-world scenarios with high annotation costs \(e\.g\., GITHUB and WIKI datasets have approximately 500 and 2500 training samples respectively\)\. As training size increases to 11802, the standard PLM marginally outperformsHparallelH\_\{\\text\{parallel\}\}\. This indicates that pragmatic information provides effective guidance when training resources are limited\. However, as the amount of training data increases, text\-based model is capable of learning generalizable features from text alone, reducing the benefit of explicit SA knowledge guidance\.

#### Per\-Timestep Model Performance

Figure[3](https://arxiv.org/html/2608.25359#S5.F3)presents models’ per\-timestep performance as dialogues progress toward derailment or conversation end, avoiding temporal aggregation via eitherDynamicmax\\text\{Dynamic\}\_\{\\text\{max\}\}orDynamicmean\\text\{Dynamic\}\_\{\\text\{mean\}\}\. We report CMV performance for up to 1000 samples here; results for the full dataset and other datasets show similar trends and are provided in Appendix[C\.3](https://arxiv.org/html/2608.25359#A3.SS3)\. All models show improving performance as dialogues approach the target event, as derailment signals in the conversation build up and become more apparent\. The relative ranking of the lines is largely consistent with the model rankings observed in Figure[2](https://arxiv.org/html/2608.25359#S5.F2)forDynamicmean\\text\{Dynamic\}\_\{\\text\{mean\}\}performance\.HparallelH\_\{\\text\{parallel\}\}achieves the highest performance across individual timesteps, particularly in the low\-data regime, with the gap betweenHparallelH\_\{\\text\{parallel\}\}and other models closing as training subset size increases\. These results align with our previous discussion, demonstrating thatHparallelH\_\{\\text\{parallel\}\}provides better forecasting throughout conversations, especially under data constraints\.

### 5\.2Cross\-Dataset Generalization

Table 2:Cross\-dataset evaluation ofDynamicmean\\text\{Dynamic\}\_\{\\text\{mean\}\}AULC using Macro\-F1 \(F1\) and AUPRC \(AUPR\), standard deviations are omitted for brevity\. Each configurationD​1→D​2D1\\rightarrow D2denotes training onD​1D1and testing onD​2D2\. Best results areboldedand second\-best areunderlined\. The†\\daggersymbol denotes that the model’s performance is statistically significantly different to the best model \(p<0\.05p<0\.05, two\-tailed paired t\-test\)\. SA\-informed models \(HsequentialH\_\{\\text\{sequential\}\}andHparallelH\_\{\\text\{parallel\}\}\) achieve strong cross\-dataset performance\.Table[2](https://arxiv.org/html/2608.25359#S5.T2)reports cross\-datasetDynamicmean\\text\{Dynamic\}\_\{\\text\{mean\}\}AULC performance, evaluating the models’ capacity to generalize to different domains\.

Overall, the SA\-informed models \(HparallelH\_\{\\text\{parallel\}\}andHsequentialH\_\{\\text\{sequential\}\}\) yield superior performance[1](https://arxiv.org/html/2608.25359#footnote1)\. Experiments integrating SA information into GraphNLI[Agarwal et al\. \(2023\)](https://arxiv.org/html/2608.25359#bib.bib42)\(Appendix[C\.5](https://arxiv.org/html/2608.25359#A3.SS5)\) also demonstrate consistent in\-domain and cross\-dataset performance improvements\. This demonstrates that explicit SA modeling provides transferable signals for derailment forecasting upon domain shifts\. An exception occurs in the GITHUB→\\rightarrowCMV transfer, whereHablationH\_\{\\text\{ablation\}\}achieves the highest Macro\-F1, whileHparallelH\_\{\\text\{parallel\}\}achieves the highest AUPRC\. Transferring from a small and technical dataset \(GITHUB\) to an open\-domain dataset \(CMV\) is inherently difficult, as reflected in uniformly low scores across all models \(Macro\-F1 peaking at only 36\.19\)\. Under such domain shift, both the learned signals and the source\-tuned thresholds degrade\. The discrepancy between the winning models in the two metrics likely stems from their respective evaluation mechanics: Macro\-F1 is further penalized by the failure of a source\-tuned threshold, whereas AUPRC is threshold\-independent\. While transferability remains limited in this setting, AUPRC offers a lens into the models’ residual capacity to correctly rank derailment\.

Among the baseline models, CRAFT exhibits the lowest transferability due to its smaller architecture and domain\-specific pre\-training, compared to the other models which rely on large PLMs\.

### 5\.3Dataset Deviation and Model Generalization

An analysis of dataset deviations provides further insight into model generalization: while SA information generally improves cross\-dataset generalization, prioritizing SA features yields the greatest benefit when transferring to highly divergent datasets\. We quantify cross\-dataset semantic and lexical deviations using Wasserstein distance on embedding distributions[Kour et al\. \(2022\)](https://arxiv.org/html/2608.25359#bib.bib33);[Heusel et al\. \(2017\)](https://arxiv.org/html/2608.25359#bib.bib34)and CHI distance on token frequencies[Kour et al\. \(2022\)](https://arxiv.org/html/2608.25359#bib.bib33);[Kilgarriff \(2001\)](https://arxiv.org/html/2608.25359#bib.bib35), respectively\. 3000 randomly sampled responses from each dataset were encoded using a sentence transformer \(all\-mpnet\-base\-v2\) for semantic measurement, and tokenized viaSpaCy\(en\_core\_web\_sm\) for the lexical evaluation\.

![Refer to caption](https://arxiv.org/html/2608.25359v1/graph/Wasserstein-Distance-2digit.png)\(a\)Semantic Deviation
![Refer to caption](https://arxiv.org/html/2608.25359v1/graph/Normalized-CHI-2digit.png)\(b\)Lexical Deviation

Figure 4:Pairwise semantic Wasserstein distance and lexical CHI distance between datasets\. Darker shades indicate higher deviation\. GITHUB has the highest deviation and is more similar to WIKI than to CMV\.As shown in Figure[4](https://arxiv.org/html/2608.25359#S5.F4), GITHUB exhibits the greatest overall deviation from the other two corpora, although it remains marginally more similar to WIKI than to CMV\. This divergence aligns with the underlying nature of their conversational settings\. Specifically, WIKI and CMV represent less specialised environments, encompassing diverse topics across editorial discussions and opinion debates, whereas GITHUB is highly domain\-specific, focusing on technical problem\-solving with extensive technical jargon and code snippets\. The slight similarity between WIKI and GITHUB can be attributed to their shared task\-oriented nature, in contrast to the debate\-driven interactions of CMV\.

When transferring from WIKI and CMV to the divergent GITHUB dataset,HsequentialH\_\{\\text\{sequential\}\}demonstrates the highest performance, followed byHparallelH\_\{\\text\{parallel\}\}\. We attribute this to the fact that models relying on raw text can struggle with domain\-specific overfitting, whereas the pragmatic SA representation prioritized byHsequentialH\_\{\\text\{sequential\}\}remain more robust to these lexical shifts, making them more effective for cross\-domain generalization\. Conversely, in transfers between the more lexically similar WIKI and CMV datasets,HparallelH\_\{\\text\{parallel\}\}excels by combining both text and SA features, with text\-based models \(PLM andHablationH\_\{\\text\{ablation\}\}\) generally ranking second\. Finally, training on the smaller, domain\-specific GITHUB corpus yields uniformly lower transferability, with GITHUB→\\rightarrowWIKI and GITHUB→\\rightarrowCMV consistently underperform their CMV\- and WIKI\-trained counterparts\. Yet, the AUPRC AULC achieved byHparallelH\_\{\\text\{parallel\}\}when trained on the small GITHUB dataset \(capped at 300\-500 samples\) and tested on WIKI \(66\.56%\) is highly comparable to WIKI’s best in\-domain performance \(66\.94%, capped to 300\-2500 samples\)\. This underscores the strong cross\-dataset transferability of our proposed approach\.

## 6Conclusion

In this work, we propose modeling speech acts as a pragmatic signal to improve derailment forecasting performance under low\-data and cross\-domain settings\.333Code:[https://anonymous\.4open\.science/r/SA\_Conversation\_Derailment\_Forecast\-1242/](https://anonymous.4open.science/r/SA_Conversation_Derailment_Forecast-1242/)We present two architectures that integrate SA information as an auxiliary learning signal alongside textual semantics\. Extensive experiments demonstrate consistent improvements over strong baselines\. This work contributes towards more robust dynamic moderation systems in diverse and data\-scarce real\-world settings\.

## Limitations

We acknowledge certain limitations in our work\. First, LLM\-based SA extraction may introduce biases related to culture, dialect, and community norms, from their training data\. The models are most appropriately used alongside human supervision in making proactive decisions, and not as fully autonomous systems\. Second, while our approach seeks to capture the intended meaning rather than literal meaning, LLMs are known to struggle to reliably recognize indirect intents, which is an active area of NLP research[Orsini and Brunato \(2025\)](https://arxiv.org/html/2608.25359#bib.bib19)\. This failure to capture true underlying intent can propagate sub\-optimal signals downstream\. Although our supplementary experiments show our framework is robust to SA granularity, this lack of reliance on highly nuanced intents may be an artifact of the LLM\-based extraction\. Specifically, the noise in LLM\-derived labels may force the downstream model to treat SAs simply as a compact pragmatic representation, rather than learning from them to identify indirect, hidden intents\. Future work could examine if introducing manually extracted, ground\-truth SAs can provide the clean signal needed for the model to learn from these nuanced intents\. Lastly, although our approach eliminates the need for SA extraction during inference, LLM\-based or manual SA extraction still incurs time and computational costs during training\. Future work could investigate more cost\-effective alternatives, such as supervised SA extraction models, while maintaining accuracy\.

## References

- Agarwalet al\.\(2023\)V\. Agarwal, A\. P\. Young, S\. Joglekar, and N\. SastryA graph\-based context\-aware model to understand online conversations\.ACM Trans\. Web18\(1\)\.External Links:ISSN 1559\-1131,[Link](https://doi.org/10.1145/3624579),[Document](https://dx.doi.org/10.1145/3624579)Cited by:[§C\.5](https://arxiv.org/html/2608.25359#A3.SS5.p1.1),[§4\.3](https://arxiv.org/html/2608.25359#S4.SS3.p1.1),[§5\.2](https://arxiv.org/html/2608.25359#S5.SS2.p2.1)\.
- Aljanaideh \(2025\)A\. AljanaidehSpeech act patterns for improving generalizability of explainable politeness detection models\.InFindings of the Association for Computational Linguistics: ACL 2025,W\. Che, J\. Nabende, E\. Shutova, and M\. T\. Pilehvar \(Eds\.\),Vienna, Austria,pp\. 18945–18954\.External Links:[Link](https://aclanthology.org/2025.findings-acl.970/),[Document](https://dx.doi.org/10.18653/v1/2025.findings-acl.970),ISBN 979\-8\-89176\-256\-5Cited by:[§2\.2](https://arxiv.org/html/2608.25359#S2.SS2.p2.1)\.
- Altarawnehet al\.\(2023\)E\. Altarawneh, A\. Agrawal, M\. Jenkin, and M\. PapagelisConversation derailment forecasting with graph convolutional networks\.InThe 7th Workshop on Online Abuse and Harms \(WOAH\),Y\. Chung, P\. Röttger, D\. Nozza, Z\. Talat, and A\. Mostafazadeh Davani \(Eds\.\),Toronto, Canada,pp\. 160–169\.External Links:[Link](https://aclanthology.org/2023.woah-1.16/),[Document](https://dx.doi.org/10.18653/v1/2023.woah-1.16)Cited by:[§1](https://arxiv.org/html/2608.25359#S1.p3.1),[§2\.1](https://arxiv.org/html/2608.25359#S2.SS1.p2.1),[§4\.3](https://arxiv.org/html/2608.25359#S4.SS3.p3.1)\.
- Austin \(1975\)J\. L\. AustinHow to do things with words\.Oxford University Press\.External Links:ISBN 9780198245537,[Document](https://dx.doi.org/10.1093/acprof%3Aoso/9780198245537.001.0001)Cited by:[§1](https://arxiv.org/html/2608.25359#S1.p3.1),[§2\.2](https://arxiv.org/html/2608.25359#S2.SS2.p1.1)\.
- Cambridge University Press \(2026\)Cambridge University PressCambridge dictionary\(Website\)Cambridge Dictionary\.External Links:[Link](https://dictionary.cambridge.org/)Cited by:[item 2](https://arxiv.org/html/2608.25359#A1.I1.i2.p1.1),[§3\.1](https://arxiv.org/html/2608.25359#S3.SS1.SSS0.Px1.p1.1)\.
- Changet al\.\(2020\)J\. P\. Chang, J\. Cheng, and C\. Danescu\-Niculescu\-MizilDon’t let me be misunderstood:comparing intentions and perceptions in online discussions\.InProceedings of The Web Conference 2020,WWW ’20,New York, NY, USA,pp\. 2066–2077\.External Links:ISBN 9781450370233,[Link](https://doi.org/10.1145/3366423.3380273),[Document](https://dx.doi.org/10.1145/3366423.3380273)Cited by:[§3\.1](https://arxiv.org/html/2608.25359#S3.SS1.SSS0.Px3.p1.1)\.
- Chang and Danescu\-Niculescu\-Mizil \(2019\)J\. P\. Chang and C\. Danescu\-Niculescu\-MizilTrouble on the horizon: forecasting the derailment of online conversations as they develop\.InProceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing \(EMNLP\-IJCNLP\),K\. Inui, J\. Jiang, V\. Ng, and X\. Wan \(Eds\.\),Hong Kong, China,pp\. 4743–4754\.External Links:[Link](https://aclanthology.org/D19-1481/),[Document](https://dx.doi.org/10.18653/v1/D19-1481)Cited by:[§B\.1](https://arxiv.org/html/2608.25359#A2.SS1.p1.1),[§1](https://arxiv.org/html/2608.25359#S1.p2.1),[§2\.1](https://arxiv.org/html/2608.25359#S2.SS1.p2.1),[§2\.1](https://arxiv.org/html/2608.25359#S2.SS1.p3.1),[§3\.2](https://arxiv.org/html/2608.25359#S3.SS2.p2.1),[§4\.1](https://arxiv.org/html/2608.25359#S4.SS1.SSS0.Px1.p1.1),[§4\.1](https://arxiv.org/html/2608.25359#S4.SS1.SSS0.Px2.p1.1),[§4\.2](https://arxiv.org/html/2608.25359#S4.SS2.SSS0.Px1.p1.1),[§4\.2](https://arxiv.org/html/2608.25359#S4.SS2.SSS0.Px2.p1.1),[§4\.3](https://arxiv.org/html/2608.25359#S4.SS3.p1.1)\.
- Choet al\.\(2014\)K\. Cho, B\. van Merriënboer, C\. Gulcehre, D\. Bahdanau, F\. Bougares, H\. Schwenk, and Y\. BengioLearning phrase representations using RNN encoder–decoder for statistical machine translation\.InProceedings of the 2014 Conference on Empirical Methods in Natural Language Processing \(EMNLP\),A\. Moschitti, B\. Pang, and W\. Daelemans \(Eds\.\),Doha, Qatar,pp\. 1724–1734\.External Links:[Link](https://aclanthology.org/D14-1179/),[Document](https://dx.doi.org/10.3115/v1/D14-1179)Cited by:[§3\.2](https://arxiv.org/html/2608.25359#S3.SS2.p4.1)\.
- Compagnoet al\.\(2018\)D\. Compagno, E\. V\. Epure, R\. Deneckere\-Lebas, and C\. SalinesiExploring digital conversation corpora with process mining\.Corpus Pragmatics2\(2\),pp\. 193–215\.External Links:ISSN 2509\-9515,[Document](https://dx.doi.org/10.1007/s41701-018-0030-6)Cited by:[§A\.1](https://arxiv.org/html/2608.25359#A1.SS1.p2.1),[§A\.1](https://arxiv.org/html/2608.25359#A1.SS1.p5.1),[§A\.2](https://arxiv.org/html/2608.25359#A1.SS2.p1.1),[§C\.6](https://arxiv.org/html/2608.25359#A3.SS6.p1.1),[§2\.2](https://arxiv.org/html/2608.25359#S2.SS2.p2.1),[§3\.1](https://arxiv.org/html/2608.25359#S3.SS1.SSS0.Px1.p1.1),[§3\.1](https://arxiv.org/html/2608.25359#S3.SS1.SSS0.Px2.p1.1),[§3\.1](https://arxiv.org/html/2608.25359#S3.SS1.SSS0.Px4.p1.1)\.
- Devlinet al\.\(2018\)J\. Devlin, M\. Chang, K\. Lee, and K\. ToutanovaBERT: pre\-training of deep bidirectional transformers for language understanding\.CoRRabs/1810\.04805\.External Links:[Link](http://arxiv.org/abs/1810.04805),1810\.04805Cited by:[§C\.4](https://arxiv.org/html/2608.25359#A3.SS4.p1.1),[§4\.3](https://arxiv.org/html/2608.25359#S4.SS3.p1.1)\.
- Goddard and Wierzbicka \(2013\)C\. Goddard and A\. WierzbickaSuggesting, apologizing, complimenting: english speech\-act verbs\.InWords and Meanings: Lexical Semantics Across Domains, Languages, and Cultures,External Links:ISBN 9780199668434,[Document](https://dx.doi.org/10.1093/acprof%3Aoso/9780199668434.003.0007),[Link](https://doi.org/10.1093/acprof:oso/9780199668434.003.0007),https://academic\.oup\.com/book/0/chapter/148303263/chapter\-pdf/38968736/acprof\-9780199668434\-chapter\-7\.pdfCited by:[item 1](https://arxiv.org/html/2608.25359#A1.I1.i1.p1.1),[item 2](https://arxiv.org/html/2608.25359#A1.I1.i2.p1.1),[§2\.2](https://arxiv.org/html/2608.25359#S2.SS2.p1.1),[§3\.1](https://arxiv.org/html/2608.25359#S3.SS1.SSS0.Px1.p1.1)\.
- Heuselet al\.\(2017\)M\. Heusel, H\. Ramsauer, T\. Unterthiner, B\. Nessler, and S\. HochreiterGANs trained by a two time\-scale update rule converge to a local nash equilibrium\.InProceedings of the 31st International Conference on Neural Information Processing Systems,NIPS’17,Red Hook, NY, USA,pp\. 6629–6640\.External Links:ISBN 9781510860964Cited by:[§5\.3](https://arxiv.org/html/2608.25359#S5.SS3.p1.1)\.
- Huaet al\.\(2024\)Y\. Hua, N\. Chernogor, Y\. Gu, S\. Jeong, M\. Luo, and C\. Danescu\-Niculescu\-MizilHow did we get here? summarizing conversation dynamics\.InProceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies \(Volume 1: Long Papers\),K\. Duh, H\. Gomez, and S\. Bethard \(Eds\.\),Mexico City, Mexico,pp\. 7452–7477\.External Links:[Link](https://aclanthology.org/2024.naacl-long.414/),[Document](https://dx.doi.org/10.18653/v1/2024.naacl-long.414)Cited by:[§1](https://arxiv.org/html/2608.25359#S1.p3.1),[§2\.1](https://arxiv.org/html/2608.25359#S2.SS1.p2.1),[§4\.3](https://arxiv.org/html/2608.25359#S4.SS3.p3.1)\.
- Imranet al\.\(2025\)M\. M\. Imran, R\. Zita, R\. Copeland, P\. Chatterjee, R\. R\. Rahman, and K\. DamevskiUnderstanding and predicting derailment in toxic conversations on github\.External Links:2503\.02191,[Link](https://arxiv.org/abs/2503.02191)Cited by:[§2\.1](https://arxiv.org/html/2608.25359#S2.SS1.p2.1),[§2\.1](https://arxiv.org/html/2608.25359#S2.SS1.p3.1),[§4\.1](https://arxiv.org/html/2608.25359#S4.SS1.SSS0.Px3.p1.1),[§4\.3](https://arxiv.org/html/2608.25359#S4.SS3.p3.1)\.
- Isnaeniet al\.\(2025\)I\. Isnaeni, M\. R\., and H\. HasmawatiTHE function of assertive speech acts in the novel calabai by pepi al\-bayqunie\.KLASIKAL : JOURNAL OF EDUCATION, LANGUAGE TEACHING AND SCIENCE7\(1\),pp\. 468–480\.External Links:[Document](https://dx.doi.org/10.52208/klasikal.v7i1.1305),[Link](https://journalfkipuniversitasbosowa.org/index.php/klasikal/article/view/1305)Cited by:[item 1](https://arxiv.org/html/2608.25359#A1.I1.i1.p1.1),[§3\.1](https://arxiv.org/html/2608.25359#S3.SS1.SSS0.Px1.p1.1)\.
- Kementchedjhieva and Søgaard \(2021\)Y\. Kementchedjhieva and A\. SøgaardDynamic forecasting of conversation derailment\.InProceedings of the 2021 Conference on Empirical Methods in Natural Language Processing,M\. Moens, X\. Huang, L\. Specia, and S\. W\. Yih \(Eds\.\),Online and Punta Cana, Dominican Republic,pp\. 7915–7919\.External Links:[Link](https://aclanthology.org/2021.emnlp-main.624/),[Document](https://dx.doi.org/10.18653/v1/2021.emnlp-main.624)Cited by:[§B\.1](https://arxiv.org/html/2608.25359#A2.SS1.p1.1),[§C\.4](https://arxiv.org/html/2608.25359#A3.SS4.p1.1),[§1](https://arxiv.org/html/2608.25359#S1.p2.1),[§2\.1](https://arxiv.org/html/2608.25359#S2.SS1.p2.1),[§4\.3](https://arxiv.org/html/2608.25359#S4.SS3.p1.1),[§4\.3](https://arxiv.org/html/2608.25359#S4.SS3.p3.1)\.
- Kholidet al\.\(2024\)I\. Kholid, A\. Hidayat, B\. Triyani, and I\. Stkip Pgri Bandar LampungAn analysis of assertive and commissive speech acts in simon sinek’s speeches\.IJLHE: International Journal of Language, Humanities, and Education7,pp\. 305–314\.External Links:[Document](https://dx.doi.org/10.52217/ijlhe.v7i2.1649)Cited by:[item 1](https://arxiv.org/html/2608.25359#A1.I1.i1.p1.1),[§3\.1](https://arxiv.org/html/2608.25359#S3.SS1.SSS0.Px1.p1.1)\.
- Kilgarriff \(2001\)A\. KilgarriffComparing corpora\.International Journal of Corpus Linguistics6\(1\),pp\. 97–133\.External Links:ISSN 1569\-9811,[Document](https://dx.doi.org/https%3A//doi.org/10.1075/ijcl.6.1.05kil)Cited by:[§5\.3](https://arxiv.org/html/2608.25359#S5.SS3.p1.1)\.
- Kouret al\.\(2022\)G\. Kour, S\. Ackerman, E\. D\. Farchi, O\. Raz, B\. Carmeli, and A\. A\. TavorMeasuring the measuring tools: an automatic evaluation of semantic metrics for text corpora\.InProceedings of the Second Workshop on Natural Language Generation, Evaluation, and Metrics \(GEM\),A\. Bosselut, K\. Chandu, K\. Dhole, V\. Gangal, S\. Gehrmann, Y\. Jernite, J\. Novikova, and L\. Perez\-Beltrachini \(Eds\.\),Abu Dhabi, United Arab Emirates \(Hybrid\),pp\. 405–416\.External Links:[Link](https://aclanthology.org/2022.gem-1.35/),[Document](https://dx.doi.org/10.18653/v1/2022.gem-1.35)Cited by:[§5\.3](https://arxiv.org/html/2608.25359#S5.SS3.p1.1)\.
- Liet al\.\(2023\)H\. Li, S\. Zhu, and Z\. ZhengDiplomat: a dialogue dataset for situated pragmatic reasoning\.InAdvances in Neural Information Processing Systems,A\. Oh, T\. Naumann, A\. Globerson, K\. Saenko, M\. Hardt, and S\. Levine \(Eds\.\),Vol\.36,pp\. 46856–46884\.External Links:[Link](https://proceedings.neurips.cc/paper_files/paper/2023/file/924303c6a45685510877ee018cdc8f80-Paper-Datasets_and_Benchmarks.pdf)Cited by:[§3\.1](https://arxiv.org/html/2608.25359#S3.SS1.SSS0.Px3.p1.1)\.
- Liuet al\.\(2019\)Y\. Liu, M\. Ott, N\. Goyal, J\. Du, M\. Joshi, D\. Chen, O\. Levy, M\. Lewis, L\. Zettlemoyer, and V\. StoyanovRoBERTa: A robustly optimized BERT pretraining approach\.CoRRabs/1907\.11692\.External Links:[Link](http://arxiv.org/abs/1907.11692),1907\.11692Cited by:[§3\.2](https://arxiv.org/html/2608.25359#S3.SS2.p2.1),[§4\.3](https://arxiv.org/html/2608.25359#S4.SS3.p1.1)\.
- Ollagnier \(2024\)A\. OllagnierCyberAgressionAdo\-v2: leveraging pragmatic\-level information to decipher online hate in French multiparty chats\.InProceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation \(LREC\-COLING 2024\),N\. Calzolari, M\. Kan, V\. Hoste, A\. Lenci, S\. Sakti, and N\. Xue \(Eds\.\),Torino, Italia,pp\. 4287–4298\.External Links:[Link](https://aclanthology.org/2024.lrec-main.383/)Cited by:[§3\.1](https://arxiv.org/html/2608.25359#S3.SS1.SSS0.Px3.p1.1)\.
- OpenAI \(2025\)OpenAIGpt\-oss\-120b & gpt\-oss\-20b model card\.External Links:2508\.10925,[Link](https://arxiv.org/abs/2508.10925)Cited by:[§3\.1](https://arxiv.org/html/2608.25359#S3.SS1.SSS0.Px2.p1.1)\.
- Orsini and Brunato \(2025\)M\. Orsini and D\. BrunatoDirect and indirect interpretations of speech acts: evidence from human judgments and large language models\.InProceedings of the Eleventh Italian Conference on Computational Linguistics \(CLiC\-it 2025\),C\. Bosco, E\. Jezek, M\. Polignano, and M\. Sanguinetti \(Eds\.\),Cagliari, Italy,pp\. 837–848\.External Links:[Link](https://aclanthology.org/2025.clicit-1.79/),ISBN 979\-12\-243\-0587\-3Cited by:[§3\.1](https://arxiv.org/html/2608.25359#S3.SS1.SSS0.Px3.p1.1),[Limitations](https://arxiv.org/html/2608.25359#Sx1.p1.1)\.
- Polettoet al\.\(2020\)F\. Poletto, V\. Basile, M\. Sanguinetti, C\. Bosco, and V\. PattiResources and benchmark corpora for hate speech detection: a systematic review\.Language Resources and Evaluation55\(2\),pp\. 477–523\.External Links:ISSN 1574\-0218,[Document](https://dx.doi.org/https%3A//doi.org/10.1007/s10579-020-09502-8)Cited by:[§2\.1](https://arxiv.org/html/2608.25359#S2.SS1.p1.1)\.
- Reiniget al\.\(2024\)I\. Reinig, I\. Rehbein, and S\. P\. PonzettoHow to do politics with words: investigating speech acts in parliamentary debates\.InProceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation \(LREC\-COLING 2024\),N\. Calzolari, M\. Kan, V\. Hoste, A\. Lenci, S\. Sakti, and N\. Xue \(Eds\.\),Torino, Italia,pp\. 8287–8300\.External Links:[Link](https://aclanthology.org/2024.lrec-main.727/)Cited by:[§A\.1](https://arxiv.org/html/2608.25359#A1.SS1.p1.1)\.
- Ridniket al\.\(2021\)T\. Ridnik, E\. Ben\-Baruch, N\. Zamir, A\. Noy, I\. Friedman, M\. Protter, and L\. Zelnik\-ManorAsymmetric Loss For Multi\-Label Classification\.In2021 IEEE/CVF International Conference on Computer Vision \(ICCV\),Vol\.,Los Alamitos, CA, USA,pp\. 82–91\.External Links:ISSN,[Document](https://dx.doi.org/10.1109/ICCV48922.2021.00015),[Link](https://doi.ieeecomputersociety.org/10.1109/ICCV48922.2021.00015)Cited by:[§B\.3](https://arxiv.org/html/2608.25359#A2.SS3.p1.1)\.
- Sahaet al\.\(2021\)T\. Saha, A\. Upadhyaya, S\. Saha, and P\. BhattacharyyaTowards sentiment and emotion aided multi\-modal speech act classification in Twitter\.InProceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,K\. Toutanova, A\. Rumshisky, L\. Zettlemoyer, D\. Hakkani\-Tur, I\. Beltagy, S\. Bethard, R\. Cotterell, T\. Chakraborty, and Y\. Zhou \(Eds\.\),Online,pp\. 5727–5737\.External Links:[Link](https://aclanthology.org/2021.naacl-main.456/),[Document](https://dx.doi.org/10.18653/v1/2021.naacl-main.456)Cited by:[§A\.1](https://arxiv.org/html/2608.25359#A1.SS1.p1.1)\.
- Sanhet al\.\(2019\)V\. Sanh, L\. Debut, J\. Chaumond, and T\. WolfDistilBERT, a distilled version of bert: smaller, faster, cheaper and lighter\.ArXivabs/1910\.01108\.Cited by:[§C\.5](https://arxiv.org/html/2608.25359#A3.SS5.p2.1)\.
- Searle \(1979\)J\. R\. SearleA taxonomy of illocutionary acts\.InExpression and Meaning: Studies in the Theory of Speech Acts,pp\. 1–29\.Cited by:[item 1](https://arxiv.org/html/2608.25359#A1.I1.i1.p1.1),[§A\.1](https://arxiv.org/html/2608.25359#A1.SS1.p2.1),[§3\.1](https://arxiv.org/html/2608.25359#S3.SS1.SSS0.Px1.p1.1)\.
- Spinoso\-Di Pianoet al\.\(2025\)C\. Spinoso\-Di Piano, D\. E\. Austin, P\. Piantanida, and J\. C\. Cheung\(RSA\)²: a rhetorical\-strategy\-aware rational speech act framework for figurative language understanding\.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),W\. Che, J\. Nabende, E\. Shutova, and M\. T\. Pilehvar \(Eds\.\),Vienna, Austria,pp\. 20898–20938\.External Links:[Link](https://aclanthology.org/2025.acl-long.1019/),[Document](https://dx.doi.org/10.18653/v1/2025.acl-long.1019),ISBN 979\-8\-89176\-251\-0Cited by:[§3\.1](https://arxiv.org/html/2608.25359#S3.SS1.SSS0.Px3.p1.1)\.
- Subramanianet al\.\(2019\)S\. Subramanian, T\. Cohn, and T\. BaldwinTarget based speech act classification in political campaign text\.InProceedings of the Eighth Joint Conference on Lexical and Computational Semantics \(\*SEM 2019\),R\. Mihalcea, E\. Shutova, L\. Ku, K\. Evang, and S\. Poria \(Eds\.\),Minneapolis, Minnesota,pp\. 273–282\.External Links:[Link](https://aclanthology.org/S19-1030/),[Document](https://dx.doi.org/10.18653/v1/S19-1030)Cited by:[§A\.1](https://arxiv.org/html/2608.25359#A1.SS1.p1.1)\.
- Tranet al\.\(2025\)S\. Q\. Tran, T\. Gangavarapu, N\. Chernogor, J\. P\. Chang, and C\. Danescu\-Niculescu\-MizilConversations gone awry, but then? evaluating conversational forecasting models\.External Links:2507\.19470,[Link](https://arxiv.org/abs/2507.19470)Cited by:[§B\.1](https://arxiv.org/html/2608.25359#A2.SS1.p1.1),[§1](https://arxiv.org/html/2608.25359#S1.p1.1),[§2\.1](https://arxiv.org/html/2608.25359#S2.SS1.p2.1),[§4\.1](https://arxiv.org/html/2608.25359#S4.SS1.SSS0.Px2.p1.1),[§4\.2](https://arxiv.org/html/2608.25359#S4.SS2.SSS0.Px1.p1.1),[§4\.3](https://arxiv.org/html/2608.25359#S4.SS3.p1.1)\.
- Vanderveken \(1990\)D\. VandervekenMeaning and speech acts: volume 1, principles of language use\.Vol\.1,Cambridge University Press,Cambridge\.External Links:ISBN 9780521374156Cited by:[item 2](https://arxiv.org/html/2608.25359#A1.I1.i2.p1.1),[§A\.1](https://arxiv.org/html/2608.25359#A1.SS1.p2.1),[§3\.1](https://arxiv.org/html/2608.25359#S3.SS1.SSS0.Px1.p1.1)\.
- Viering and Loog \(2023\)T\. Viering and M\. LoogThe shape of learning curves: a review\.IEEE Trans\. Pattern Anal\. Mach\. Intell\.45\(6\),pp\. 7799–7819\.External Links:ISSN 0162\-8828,[Link](https://doi.org/10.1109/TPAMI.2022.3220744),[Document](https://dx.doi.org/10.1109/TPAMI.2022.3220744)Cited by:[§4\.2](https://arxiv.org/html/2608.25359#S4.SS2.SSS0.Px4.p1.1)\.
- Wang and Feng \(2023a\)R\. Wang and S\. FengGlobal\-local modeling with prompt\-based knowledge enhancement for emotion inference in conversation\.InFindings of the Association for Computational Linguistics: EACL 2023,A\. Vlachos and I\. Augenstein \(Eds\.\),Dubrovnik, Croatia,pp\. 2120–2127\.External Links:[Link](https://aclanthology.org/2023.findings-eacl.158/),[Document](https://dx.doi.org/10.18653/v1/2023.findings-eacl.158)Cited by:[§3\.1](https://arxiv.org/html/2608.25359#S3.SS1.SSS0.Px2.p1.1)\.
- Wang and Feng \(2023b\)R\. Wang and S\. FengGlobal\-local modeling with prompt\-based knowledge enhancement for emotion inference in conversation\.InFindings of the Association for Computational Linguistics: EACL 2023,A\. Vlachos and I\. Augenstein \(Eds\.\),Dubrovnik, Croatia,pp\. 2120–2127\.External Links:[Link](https://aclanthology.org/2023.findings-eacl.158/),[Document](https://dx.doi.org/10.18653/v1/2023.findings-eacl.158)Cited by:[§4\.3](https://arxiv.org/html/2608.25359#S4.SS3.p1.1)\.
- Wierzbicka \(1987\)A\. WierzbickaEnglish speech act verbs: a semantic dictionary\.Academic Press,Sydney\.External Links:ISBN 9780123128102Cited by:[item 2](https://arxiv.org/html/2608.25359#A1.I1.i2.p1.1),[§3\.1](https://arxiv.org/html/2608.25359#S3.SS1.SSS0.Px1.p1.1)\.
- Wuet al\.\(2020\)C\. Wu, S\. C\.H\. Hoi, R\. Socher, and C\. XiongTOD\-BERT: pre\-trained natural language understanding for task\-oriented dialogue\.InProceedings of the 2020 Conference on Empirical Methods in Natural Language Processing \(EMNLP\),B\. Webber, T\. Cohn, Y\. He, and Y\. Liu \(Eds\.\),Online,pp\. 917–929\.External Links:[Link](https://aclanthology.org/2020.emnlp-main.66/),[Document](https://dx.doi.org/10.18653/v1/2020.emnlp-main.66)Cited by:[§2\.2](https://arxiv.org/html/2608.25359#S2.SS2.p2.1)\.
- Xuet al\.\(2025\)X\. Xu, S\. Feng, Y\. Cui, Y\. Zhang, and D\. WangAC\-eic: addressee\-centered emotion inference in conversations\.International Journal of Machine Learning and Cybernetics16\(7\-8\),pp\. 5113–5130\.External Links:ISSN 1868\-808X,[Document](https://dx.doi.org/10.1007/s13042-025-02561-9)Cited by:[§4\.3](https://arxiv.org/html/2608.25359#S4.SS3.p1.1)\.
- Yuan and Singh \(2023\)J\. Yuan and M\. P\. SinghConversation modeling to predict derailment\.Proceedings of the International AAAI Conference on Web and Social Media17\(1\),pp\. 926–935\.External Links:[Link](https://ojs.aaai.org/index.php/ICWSM/article/view/22200),[Document](https://dx.doi.org/10.1609/icwsm.v17i1.22200)Cited by:[§3\.2](https://arxiv.org/html/2608.25359#S3.SS2.p2.1),[§4\.3](https://arxiv.org/html/2608.25359#S4.SS3.p1.1),[§4\.3](https://arxiv.org/html/2608.25359#S4.SS3.p3.1)\.
- Zhanget al\.\(2018\)J\. Zhang, J\. Chang, C\. Danescu\-Niculescu\-Mizil, L\. Dixon, Y\. Hua, D\. Taraborelli, and N\. ThainConversations gone awry: detecting early signs of conversational failure\.InProceedings of the 56th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),I\. Gurevych and Y\. Miyao \(Eds\.\),Melbourne, Australia,pp\. 1350–1361\.External Links:[Link](https://aclanthology.org/P18-1125/),[Document](https://dx.doi.org/10.18653/v1/P18-1125)Cited by:[§2\.1](https://arxiv.org/html/2608.25359#S2.SS1.p1.1),[§2\.1](https://arxiv.org/html/2608.25359#S2.SS1.p2.1),[§2\.1](https://arxiv.org/html/2608.25359#S2.SS1.p3.1),[§4\.1](https://arxiv.org/html/2608.25359#S4.SS1.SSS0.Px1.p1.1)\.
- Zhanget al\.\(2025\)Y\. Zhang, K\. McKeown, and S\. MuresanForecasting conversation derailments through generation\.InProceedings of the 18th International Natural Language Generation Conference,L\. Flek, S\. Narayan, L\. H\. Phương, and J\. Pei \(Eds\.\),Hanoi, Vietnam,pp\. 699–715\.External Links:[Link](https://aclanthology.org/2025.inlg-main.40/)Cited by:[§1](https://arxiv.org/html/2608.25359#S1.p3.1),[§2\.1](https://arxiv.org/html/2608.25359#S2.SS1.p2.1),[§3\.1](https://arxiv.org/html/2608.25359#S3.SS1.SSS0.Px2.p1.1),[§4\.3](https://arxiv.org/html/2608.25359#S4.SS3.p3.1)\.

## Appendix ASpeech Act Extraction

### A\.1Speech Act Taxonomy

Various speech act taxonomies have been developed to model conversational dynamics in specialized domains, such as parliamentary debates[Reinig et al\. \(2024\)](https://arxiv.org/html/2608.25359#bib.bib2), political campaign[Subramanian et al\. \(2019\)](https://arxiv.org/html/2608.25359#bib.bib4), and Twitter text[Saha et al\. \(2021\)](https://arxiv.org/html/2608.25359#bib.bib3)\. They typically derive SA labels from frequent communicative intents observed within their specific contexts\. However, conversational derailment can occur across diverse argumentative and collaborative settings, necessitating a more open\-domain SA taxonomy\.

The first effort toward an open\-domain taxonomy for digital corpora was conducted by[Compagno et al\. \(2018\)](https://arxiv.org/html/2608.25359#bib.bib5)\. They build on the SA verbs defined in[Vanderveken \(1990\)](https://arxiv.org/html/2608.25359#bib.bib6), which are structured into five hierarchical trees corresponding to Searle’s five fundamental SA categories[Searle \(1979\)](https://arxiv.org/html/2608.25359#bib.bib7)\. By selecting the 21 immediate children of the root nodes and introducing organizational layers with oppositional traits \(e\.g\., context\-dependency, sentiment and strength\), they refined the set into 17 distinct speech acts\.

Despite this foundation, restricting the taxonomy to the immediate children of the root nodes introduces semantic gaps and do not meet the granularity commonly observed in pragmatic analysis works\. In the hierarchical trees, a successor’s illocutionary force is derived by adding components or increasing the strength of its parent’s force\. Consequently, lower\-level verbs often carry important nuances that are not fully captured by higher\-level verbs \(e\.g\., “questioning” expressing doubt compared to the neutral information\-seeking of “inquiring”\)\. Furthermore, several relevant verbs \(e\.g\., “boasting”\) were defined but omitted from the trees\.

To address these gaps, we adapted and expanded the existing taxonomy to cover commonly analyzed conversational speech acts\. The expanded taxonomy preserves the structure of the original, allowing the extracted SAs to be merged and collapsed into the lower\-grained taxonomy\. Extracting SAs at a high granularity thus provide the flexibility to evaluate downstream performance at different levels of granularity\. Given that comprehensive list exceeding 300 verbs are computationally infeasible for use as a label set, we adopted a multi\-stage refinement process:

1. 1\.Sourcing: Beyond the original taxonomy, we aggregated frequently utilized speech act verbs from linguistic and conversational analysis literature[Goddard and Wierzbicka \(2013\)](https://arxiv.org/html/2608.25359#bib.bib1);[Isnaeni et al\. \(2025\)](https://arxiv.org/html/2608.25359#bib.bib11);[Kholid et al\. \(2024\)](https://arxiv.org/html/2608.25359#bib.bib10);[Searle \(1979\)](https://arxiv.org/html/2608.25359#bib.bib7)\.
2. 2\.Defining: Each speech act is accompanied by a brief description of its meaning and, where applicable, the speaker’s psychological state, derived from semantic linguistic works and standard lexicons[Vanderveken \(1990\)](https://arxiv.org/html/2608.25359#bib.bib6);[Goddard and Wierzbicka \(2013\)](https://arxiv.org/html/2608.25359#bib.bib1);[Wierzbicka \(1987\)](https://arxiv.org/html/2608.25359#bib.bib8);[Cambridge University Press \(2026\)](https://arxiv.org/html/2608.25359#bib.bib9)\.
3. 3\.Filtering: We reduced the set by merging verbs with largely overlapping semantic definitions to eliminate redundancy and labeling ambiguity, and based on empirical observations of SA frequency during the initial extraction phase\.

This process yielded 50 speech act verbs and an additional “other” label, detailed in Tables[12](https://arxiv.org/html/2608.25359#A3.T12)and[13](https://arxiv.org/html/2608.25359#A3.T13), and structured in[Figures13](https://arxiv.org/html/2608.25359#A3.F13),[14](https://arxiv.org/html/2608.25359#A3.F14),[15](https://arxiv.org/html/2608.25359#A3.F15)and[16](https://arxiv.org/html/2608.25359#A3.F16)following the organization of prior work[Compagno et al\. \(2018\)](https://arxiv.org/html/2608.25359#bib.bib5)\. The speech act taxonomy is used during speech act extraction, via the prompt structure shown in Table[14](https://arxiv.org/html/2608.25359#A3.T14)\. While speech act extraction is performed at sentence level, responses were segmented into sentences viaSpaCy\(en\_core\_web\_sm\)\.

### A\.2Speech Act Extraction Alignment

We follow prior work[Compagno et al\. \(2018\)](https://arxiv.org/html/2608.25359#bib.bib5)to measure SA extraction agreement, replacing Fleiss’ Kappa with Cohen’s Kappa, as Fleiss’ Kappa is designed for multiple\-rater settings where Cohen’s Kappa is designed for two\-rater settings\. For this evaluation, one of the authors manually labeled 85 sentences from 30 randomly selected responses \(10 from each dataset\), utilizing the established SA taxonomy and the LLM extraction prompt as reference guidelines\. Table[3](https://arxiv.org/html/2608.25359#A1.T3)reports the Cohen’s Kappa scores between the manually extracted and LLM extracted SAs, along with thezz\-scores andpp\-values rejecting the null hypothesis of no agreement \(p<0\.01p<0\.01across all tests\)\. The LLM demonstrated moderate agreement \(κ=0\.53\\kappa=0\.53\) across the 51 granular classes, and almost perfect agreement \(κ=0\.88\\kappa=0\.88\) when collapsed into the 5 fundamental categories \(assertive, directive, commissive, expressive, and an ‘other’ class replacing declarative\)\. This aligns closely with the human inter\-annotator agreement reported in prior literature[Compagno et al\. \(2018\)](https://arxiv.org/html/2608.25359#bib.bib5), suggesting that the LLM extractions could provide meaningful and human\-interpretable pragmatic signals\.

Table 3:Cohen’s Kappa agreement results between LLM\-extracted and manually labeled SAs\. The LLM demonstrates moderate and almost perfect agreement across the 51 fine\-grained classes and the 5 fundamental categories, respectively\.

## Appendix BExperiment Details

### B\.1Hyperparameters and Setup

All settings for the CRAFT baseline follow its original implementation[Chang and Danescu\-Niculescu\-Mizil \(2019\)](https://arxiv.org/html/2608.25359#bib.bib20), with the exception of an increased maximum number of epochs to accommodate smaller training datasets\. For other architectures, we generally adopt the effective hyperparameters and setup identified in prior work for PLMs[Tran et al\. \(2025\)](https://arxiv.org/html/2608.25359#bib.bib21);[Kementchedjhieva and Søgaard \(2021\)](https://arxiv.org/html/2608.25359#bib.bib29)\. For fair comparison where the hierarchical models have access to anonymized speaker ID \(e\.g\., Speaker 1, Speaker 2\), the strings are also prepended to each utterance as input to PLM baselines\. The maximum sequence length is capped at 512 tokens per dialogue \(representing the maximum capacity forRoBERTa\-largeandBERT\) and 128 tokens per utterance for hierarchical models, utilizing left truncation\. The training batch size is set to 4 forBERTand 12 forRoBERTa\-large\. The learning rate is5​e−65e\-6for pre\-trained parameters and5​e−55e\-5for newly initialized parameters\. All models are trained with early stopping on the validation set\. Table[4](https://arxiv.org/html/2608.25359#A2.T4)reports the maximum number of training epochs for each model under different sample sizes, determined by validation convergence\.

### B\.2Model Implementation and Information

We use the original implementation of CRAFT, with approximately 61M parameters\. Other models were implemented using PyTorch \(2\.10\.0\) and the Transformers \(4\.57\.2\) library\. All models were trained on a single 80GB NVIDIA A100 GPU\. TheRoBERTa\-largebaseline contains approximately 355M parameters\. When utilizingRoBERTa\-largeas an utterance\-level encoder,HparallelH\_\{\\text\{parallel\}\}andHablationH\_\{\\text\{ablation\}\}have approximately 372M parameters, whileHsequentialH\_\{\\text\{sequential\}\}has approximately 373M parameters\. To provide a reference for computational requirements, a single training run on the complete CMV dataset requires approximately 3 hours forHparallelH\_\{\\text\{parallel\}\}, 2 hours forHsequentialH\_\{\\text\{sequential\}\}andHablationH\_\{\\text\{ablation\}\}, 1 hour forRoBERTa\-large, and under 10 minutes for CRAFT\. We estimate the total computational budget for our main experimental training and evaluation to be approximately 300 GPU hours\.

The SA extraction viagpt\-oss\-120b, with high reasoning effort and parallelized across 2 80GB NVIDIA A100 GPU, requires in average approximately 7 seconds per response, totaling approximately 180 GPU hours\. This underscores the latency bottleneck inherent to LLM generation during real\-time inference, motivating the use of LLM auxiliary information only during training to support high\-traffic moderation systems\.

### B\.3Optimization Objectives

For the primary derailment forecasting task, we use Cross\-Entropy loss\. For SA detection task, we employ Binary Cross\-Entropy \(BCE\) loss inHsequentialH\_\{\\text\{sequential\}\}, a standard objective for multi\-label classification\. ForHparallelH\_\{\\text\{parallel\}\}, we utilize Asymmetric Loss \(ASL\)[Ridnik et al\. \(2021\)](https://arxiv.org/html/2608.25359#bib.bib40)\. ASL targets the severe positive\-negative imbalance inherent to multi\-label settings, where negative samples overwhelmingly dominate\. It addresses this by down\-weighting and hard\-thresholding easy negatives during training \(e\.g\., if the model predicts a negative probability below 5%, the sample contributes zero loss, and the model stops attempting to push it closer to 0\)\. However, this relaxed optimization can negatively impactHsequentialH\_\{\\text\{sequential\}\}\. BecauseHsequentialH\_\{\\text\{sequential\}\}relies exclusively on the detected SA probabilities to forecast derailment, these small, un\-zeroed negative probabilities could introduce noise and create a pathway for the model to leak more raw textual semantics into the final layer\.

### B\.4Training Curriculum

HparallelH\_\{\\text\{parallel\}\}is trained using a two\-phase approach to ensure the model effectively incorporates SA knowledge\. During the first intermediate phase, the objective is solely SA detection\. To prevent catastrophic forgetting and ensure the model does not overfit before learning the primary task, the bottom half of the PLM layers \(12 layers forRoBERTa\-large, 6 forBERT\) are frozen, and the model is trained on a 60%/40% train\-validation split of the training data\. The second fine\-tuning phase unfreezes all layers and employs a multi\-task learning objective, weighting derailment forecasting and SA detection at 1\.0 and 0\.1, respectively, to maintain focus on the primary task\. In contrast, becauseHsequentialH\_\{\\text\{sequential\}\}relies on the detected SA probabilities for forecasting, it structurally forces the model to primarily utilize SA knowledge\. Therefore, it does not require an intermediate training phase and instead optimizes both tasks simultaneously with equal weights of 0\.5\. All models, aside fromHparallelH\_\{\\text\{parallel\}\}, have all layers fully unfrozen throughout their entire fine\-tuning process\.

Table 4:Number of training epochs used under different training set sizes\.
### B\.5Dataset

All datasets utilized in this study \(described in Section[4\.1](https://arxiv.org/html/2608.25359#S4.SS1)\) are publicly available and distributed under the MIT License, which permits both academic and commercial use\. These predominantly English datasets contain occurrences of offensive language, which were retained as they are essential for the derailment forecasting task\. We did not collect any external personal data\. While the raw datasets contain usernames, we utilized these solely to establish conversational structure \(i\.e\., speaker turns\) and replaced them with anonymous Speaker IDs \(e\.g\. Speaker 1, Speaker 2\) during model training to protect user privacy\.

## Appendix CAdditional Experimental Results

### C\.1Dynamicmax\\text\{Dynamic\}\_\{\\text\{max\}\}Performance

Table[5](https://arxiv.org/html/2608.25359#A3.T5)reports the models’Dynamicmax\\text\{Dynamic\}\_\{\\text\{max\}\}AULC performance\. PLM andHparallelH\_\{\\text\{parallel\}\}consistently rank first or second, suggesting they perform comparably well when dialogues are flagged as “derailed” upon “derailed” forecast at any timestep\.

Table[6](https://arxiv.org/html/2608.25359#A3.T6)reports the models’ cross\-datasetDynamicmax\\text\{Dynamic\}\_\{\\text\{max\}\}performance\. Although the SA\-informed models \(HparallelH\_\{\\text\{parallel\}\}andHsequentialH\_\{\\text\{sequential\}\}\) do not consistently achieve the best in\-domainDynamicmax\\text\{Dynamic\}\_\{\\text\{max\}\}\(from Table[5](https://arxiv.org/html/2608.25359#A3.T5)\), they demonstrate strong cross\-dataset performance, achieving the highest rank in the majority of cases, while the PLM ranks first in two out of twelve instances\.

While models are optimized to minimize forecasting error independently given a dialogue history, future work could explore optimizing directly forDynamicmax\\text\{Dynamic\}\_\{\\text\{max\}\}, where a single threshold\-crossing prediction dictates the classification of the entire sequence\. This may better support moderation workflows that prioritize a single warning proactive intervention over continuous state\-tracking\.

Table 5:Comparison of models’Dynamicmax\\text\{Dynamic\}\_\{\\text\{max\}\}AULC performance\. Best results areboldedand second\-best areunderlined\. The†\\daggersymbol denotes that the model’s performance is statistically significantly different to the best model \(p<0\.05p<0\.05, two\-tailed paired t\-test\)\.Table 6:Cross\-dataset evaluation ofDynamicmax\\text\{Dynamic\}\_\{\\text\{max\}\}AULC using Macro\-F1 \(F1\) and AUPRC \(AUPR\), standard deviations are omitted for brevity\. Each configurationD​1→D​2D1\\rightarrow D2denotes training onD​1D1and testing onD​2D2\. Best results areboldedand second\-best areunderlined\. The†\\daggersymbol denotes that the model’s performance is statistically significantly different to the best model \(p<0\.05p<0\.05, two\-tailed paired t\-test\)\.
### C\.2Model Performance Across Data Regime

Figures[5](https://arxiv.org/html/2608.25359#A3.F5)and[6](https://arxiv.org/html/2608.25359#A3.F6)present theDynamicmean\\text\{Dynamic\}\_\{\\text\{mean\}\}AUPRC learning curves on WIKI and GITHUB, respectively\. They follow a similar trend to CMV \(Figure[2](https://arxiv.org/html/2608.25359#S5.F2)\), whereHparallelH\_\{\\text\{parallel\}\}demonstrates strong performance in low\-data regimes, although it can be outperformed by text\-based models when larger amounts of data are available\. Specifically,HparallelH\_\{\\text\{parallel\}\}exhibits a smaller decline in performance as the number of training samples decreases\. By leveraging pragmatic signals, it achieves higher performance with just 300 samples than other models do with 500 or 1000, significantly reducing the amount of data required to reach comparable results\.

In addition,[Figures7](https://arxiv.org/html/2608.25359#A3.F7),[8](https://arxiv.org/html/2608.25359#A3.F8)and[9](https://arxiv.org/html/2608.25359#A3.F9)present the learning curves in cross\-dataset settings for models trained on WIKI, CMV, and GITHUB, respectively\. As with the in\-domain results, SA signals generally provide greater benefits in low\-data regimes\. The cross\-domain performance ofHparallelH\_\{\\text\{parallel\}\}on GITHUB tends to fluctuate as the number of training samples increases\. In particular, when trained on CMV, its performance eventually degrades to the level of the text\-based models \(PLM andHablationH\_\{\\text\{ablation\}\}\) as the training size reaches 11,802\. This suggests thatHparallelH\_\{\\text\{parallel\}\}may require further tuning with higher regularization to more effectively utilize SA features\. In contrast,HsequentialH\_\{\\text\{sequential\}\}, which prioritizes SA features, is less prone to overfitting the source domain and demonstrates robust generalizability to GITHUB\.

Figure 5:Comparison of models’Dynamicmean\\text\{Dynamic\}\_\{\\text\{mean\}\}AUPRC learning curves on WIKI, plotted against training subset size\.Figure 6:Comparison of models’Dynamicmean\\text\{Dynamic\}\_\{\\text\{mean\}\}AUPRC learning curves on GITHUB, plotted against training subset size\.Figure 7:Comparison of models’ cross\-datasetDynamicmean\\text\{Dynamic\}\_\{\\text\{mean\}\}AUPRC learning curves when trained on WIKI, plotted against training subset size\.Figure 8:Comparison of models’ cross\-datasetDynamicmean\\text\{Dynamic\}\_\{\\text\{mean\}\}AUPRC learning curves when trained on CMV, plotted against training subset size\.Figure 9:Comparison of models’ cross\-datasetDynamicmean\\text\{Dynamic\}\_\{\\text\{mean\}\}AUPRC learning curves when trained on GITHUB, plotted against training subset size\.
### C\.3Per\-Timestep Model Performance

[Figures10](https://arxiv.org/html/2608.25359#A3.F10),[11](https://arxiv.org/html/2608.25359#A3.F11)and[12](https://arxiv.org/html/2608.25359#A3.F12)present the per\-timestep performance of models as dialogues progress toward derailment or the end of the conversation, on CMV, WIKI, and GITHUB, respectively\. All datasets exhibit similar trends discussed in Section[5\.1](https://arxiv.org/html/2608.25359#S5.SS1)\. Overall,HparallelH\_\{\\text\{parallel\}\}shows the best performance across time steps, particularly in low\-data regimes, which generally aligns with theDynamicmean\\text\{Dynamic\}\_\{\\text\{mean\}\}rankings\.

\(a\)300 Train Subset\(b\)500 Train Subset\(c\)1000 Train Subset\(d\)2500 Train Subset\(e\)11802 Train Subset
Figure 10:Model AUPRC performance on CMV is plotted as dialogues progress toward derailment or conversation end, under varying training data sizes and averaged across five random seeds\. The number of test\-set dialogues contributing to each time step \(n\) is shown on the x\-axis ticks, as some dialogues contain fewer than five utterances\. Areas under the curves are reported in the legend\.HparallelH\_\{\\text\{parallel\}\}achieves the best performance across time steps, particularly in the low\-data regime\.\(a\)300 Train Subset\(b\)500 Train Subset\(c\)1000 Train Subset\(d\)2500 Train Subset
Figure 11:Model AUPRC performance on WIKI is plotted as dialogues progress toward derailment or conversation end, under varying training data sizes and averaged across five random seeds\. The number of test\-set dialogues contributing to each time step \(n\) is shown on the x\-axis ticks, as some dialogues contain fewer than five utterances\. Areas under the curves are reported in the legend\.HparallelH\_\{\\text\{parallel\}\}achieves the best performance across time steps, particularly in the low\-data regime\.\(a\)300 Train Subset\(b\)500 Train Subset
Figure 12:Model AUPRC performance on GITHUB is plotted as dialogues progress toward derailment or conversation end, under varying training data sizes and averaged across five random seeds\. The number of test\-set dialogues contributing to each time step \(n\) is shown on the x\-axis ticks, as some dialogues contain fewer than five utterances\. Areas under the curves are reported in the legend\.HparallelH\_\{\\text\{parallel\}\}achieves the best performance across time steps, particularly in the low\-data regime\.
### C\.4Robustness to PLM

To assess the robustness of our proposed approach across different underlying PLMs, we perform a supplementary experiment usingbert\-base\-uncased[Devlin et al\. \(2018\)](https://arxiv.org/html/2608.25359#bib.bib32)instead ofRoBERTa\-large, asBERTis also a widely adopted baseline in prior derailment forecasting work[Kementchedjhieva and Søgaard \(2021\)](https://arxiv.org/html/2608.25359#bib.bib29)\. Specifically, the PLM components in all relevant models \(PLM,HablationH\_\{\\text\{ablation\}\},HsequentialH\_\{\\text\{sequential\}\}, andHparallelH\_\{\\text\{parallel\}\}\) are replaced withbert\-base\-uncased\. All hyperparameters remain consistent with prior experiments\. This evaluation is conducted using the CMV dataset for training, with the resulting models tested across all datasets\. Table[7](https://arxiv.org/html/2608.25359#A3.T7)reports theDynamicmean\\text\{Dynamic\}\_\{\\text\{mean\}\}AULC performance\. Because it uses a smaller architecture, BERT generally performs worse than RoBERTa\. Regardless, the SA\-based models achieve the strongest in\-domain and cross\-domain performance, showing a consistent trend to the results obtained with RoBERTa\-large\. This illustrates the robustness of our approach and its ability to improve generalizability upon different underlying PLM\.

Table 7:Robustness to PLM: Comparison of models’Dynamicmean\\text\{Dynamic\}\_\{\\text\{mean\}\}AULC performance when bert\-base\-uncased is used as the base model\. Models are trained on CMV and tested on all datasets\. Best results areboldedand second\-best areunderlined\. The†\\daggersymbol denotes that the model’s performance is statistically significantly different to the best model \(p<0\.05p<0\.05, two\-tailed paired t\-test\)\.HparallelH\_\{\\text\{parallel\}\}achieves strongest in\-domain performance \(on CMV\), while SA\-based models \(HparallelH\_\{\\text\{parallel\}\}andHsequentialH\_\{\\text\{sequential\}\}\) achieve strongest cross\-dataset performance \(CMV→\\rightarrowWIKI and CMV→\\rightarrowGITHUB\)\.
### C\.5Robustness to Model Architecture: GraphNLI

To assess the robustness of the proposed approach to the model architecture, we conduct a supplementary experiment running a graph\-based context\-aware model GraphNLI[Agarwal et al\. \(2023\)](https://arxiv.org/html/2608.25359#bib.bib42)\. We follow its original implementation and the hyperparameters for the hate speech detection task\. Minor adjustments to the hyperparameters are made to adapt the model to the derailment forecasting task: increasing the number of training epochs for smaller training sets \(same as PLM/H\-ablation training epochs as reported in Table[4](https://arxiv.org/html/2608.25359#A2.T4)\), and performing a root\-seeking graph walk \(settingp=1p=1andγ=0\.2\\gamma=0\.2\) as our task focuses on a single conversation rather than a branching conversation\.

To focus on the effect of SA integration, we trained a variant GraphNLI\-SA, which includes a SA classification head on top of its PLM encoder, using the same auxiliary SA classification objective and two\-phased training procedure as H\-parallel\. While GraphNLI uses a weaker PLM \(distilroberta\-base[Sanh et al\. \(2019\)](https://arxiv.org/html/2608.25359#bib.bib43)\), the full PLM is trained during the intermediate training phase in order to effectively learn the complex SA classification task\. The number of finetune epochs is consistent withHp​a​r​a​l​l​e​lH\_\{parallel\}reported in Table[4](https://arxiv.org/html/2608.25359#A2.T4), where the intermediate phase used an increased number of epochs to accomodate the weaker PLM \(25/15/10/5/2 epochs for 300/500/1000/2500/11802 training samples respectively\)\. Other hyperparameters remain consistent with GraphNLI\.

Table[8](https://arxiv.org/html/2608.25359#A3.T8)reports the resulting model performance\. Despite the architectural differences between GraphNLI and our hierarchical models, the results are consistent with previous findings: incorporating SA supervision consistently improves in\-domain and cross\-domain performance\.

While different model architectures are possible, they largely build on top of PLMs as utterance encoders, with their primary differences lying in how conversational context is modeled and integrated\. The consistent gains observed across both GraphNLI and our hierarchical framework suggest the performance gains arise from the additional SA supervision rather than the specific hierarchical architecture, demonstrating that the proposed approach is compatible with alternative backbone models\.

Table 8:Ablation study onDynamicmean\\text\{Dynamic\}\_\{\\text\{mean\}\}performance of GraphNLI\. The†\\daggersymbol denotes that the model’s performance is statistically significantly different to the best model \(p<0\.05p<0\.05, two\-tailed paired t\-test\)\. Standard deviations are omitted for brevity\. The best performance isbolded\. SA integration consistently improves both in\-domain and cross\-domain performance\.
### C\.6Robustness to SA Extraction

To assess the robustness of the proposed approach to the underlying SA extraction process, we conduct a supplementary experiment using a more coarse\-grained speech act \(SA\) taxonomy\. While the main experiments employ a fine\-grained set of 51 speech acts \(including “other”\), these acts are organized according to the prior taxonomy proposed by[Compagno et al\. \(2018\)](https://arxiv.org/html/2608.25359#bib.bib5), enabling hierarchical merging where necessary\. Specifically, from[Figures13](https://arxiv.org/html/2608.25359#A3.F13),[14](https://arxiv.org/html/2608.25359#A3.F14),[15](https://arxiv.org/html/2608.25359#A3.F15)and[16](https://arxiv.org/html/2608.25359#A3.F16), speech acts belonging to the same node in the taxonomy are collapsed into a single class, resulting in 17 speech act classes and an additional “other” label\.

As the proposed approach is primarily intended to improve performance in low\-data regimes, and because speech act distributions and interaction patterns may vary across datasets, we conduct this supplementary experiment across the 300\-training\-sample setting, as oppose to across a dataset\. All hyperparameters remain consistent with prior experiments\. We compare the performance of our main model,HparallelH\_\{\\text\{parallel\}\}, using the merged SA taxonomy, against the baselines, and against the original fine\-grained SA taxonomy setting\.

Table[9](https://arxiv.org/html/2608.25359#A3.T9)reports the resulting model performance\. We observe thatHparallelH\_\{\\text\{parallel\}\}is robust to the specific SA extraction used\. Their in\-domain performance \(dialogonals in the table\) remains highly comparable, with the largest absolute discrepancy being a minor 0\.56% AUPRC and 1\.59% Macro\-F1 on GITHUB\. Cross\-dataset performance displays similar consistency, particularly in terms of AUPRC, where the maximum deviation is 2\.83% \(CMV→\\rightarrowGITHUB\), with the lower outperforming best baseline by 7\.2%\. Conversely, MacroF1 exhibits higher instability and greater divergence, peaking at a 6\.91% difference for GITHUB→\\rightarrowCMV\. This variance could be driven by a reliance on source\-tuned classification thresholds, as their underlying AUPRC scores differ by only 0\.32%\.

Table 9:Robustness to SA Extraction: Comparison ofDynamicmean\\text\{Dynamic\}\_\{\\text\{mean\}\}performance between different models under 300 training samples, whereHparallelH\_\{\\text\{parallel\}\}uses a fine\-grained \(Hp​a​r​a​l​l​e​l,51H\_\{parallel,51\}\) and a coarse\-grained \(Hp​a​r​a​l​l​e​l,18H\_\{parallel,18\}\) taxonomy\.\|Δ\|\|\\Delta\|denotes the absolute difference between their performance\. The†\\daggersymbol denotes that the model’s performance is statistically significantly different to the best model \(p<0\.05p<0\.05, two\-tailed paired t\-test\)\. Standard deviations are omitted for brevity\. The best performance isbolded, and second\-bestunderlined\. Model performance is relatively robust to SA extraction used, with the two taxonomies achieving comparable performance to each other\.
### C\.7Validation Set Result

For reproducibility purpose, Tables[10](https://arxiv.org/html/2608.25359#A3.T10)and[11](https://arxiv.org/html/2608.25359#A3.T11)report the validation set AULC performance of the models\.

Table 10:Models’ validation setDynamicmean\\text\{Dynamic\}\_\{\\text\{mean\}\}AULC performance for reproducibility purpose\. Macro\-F1 uses threshold tuned on the validation\-set\.Table 11:Models’ validation setDynamicmax\\text\{Dynamic\}\_\{\\text\{max\}\}AULC performance\. Macro\-F1 uses threshold tuned on the validation\-set\.\{forest\}Figure 13:Assertive Speech Act Taxonomy\.\{forest\}Figure 14:Directive Speech Act Taxonomy\{forest\}Figure 15:Commissive Speech Act Taxonomy\{forest\}Figure 16:Expressive Speech Act TaxonomyTable 12:Speech act taxonomy \(part 1\), including the speech act categories \(Assertive, Directive, and Commissive\) and classes, along with their brief description provided to LLM during extraction\.Table 13:Speech act taxonomy part 2, including the speech act categories \(Expressive and Other\) and classes, along with their brief description provided to LLM during extraction\.Table 14:LLM prompt structure for speech act extraction\. The previous response to the target response, if available, is used as conversation history\. Speaker names are anonymized into numerical identifiers \(e\.g\., Speaker 1, Speaker 2\)\.