"Excuse me, may I say something..." CoLabScience, A Proactive AI Assistant for Biomedical Discovery and LLM-Expert Collaborations

# "Excuse me, may I say something..." CoLabScience, A Proactive AI Assistant for Biomedical Discovery and LLM-Expert Collaborations

Source: https://arxiv.org/html/2604.15588

Yang Wu♣ Jinhong Yu♣ Jingwei Xiong♡ Zhimin Tao♢ Xiaozhong Liu♣

♣ Worcester Polytechnic Institute, Worcester, MA, USA
♡ University of California, Davis, CA, USA
♢ Jiangsu University, Zhenjiang, Jiangsu, China

{ywu19, jyu7, xliu14}@wpi.edu [email protected] [email protected]

## Abstract

The integration of Large Language Models (LLMs) into scientific workflows presents exciting opportunities to accelerate biomedical discovery. However, the reactive nature of LLMs, which respond only when prompted, limits their effectiveness in collaborative settings that demand foresight and autonomous engagement. In this study, we introduce CoLabScience, a proactive LLM assistant designed to enhance biomedical collaboration between AI systems and human experts through timely, context-aware interventions. At the core of our method is PULI (Positive-Unlabeled Learning-to-Intervene), a novel framework trained with a reinforcement learning objective to determine when and how to intervene in streaming scientific discussions, by leveraging the team's project proposal and long- and short-term conversational memory. To support this work, we introduce BSDD (Biomedical Streaming Dialogue Dataset), a new benchmark of simulated research discussion dialogues with intervention points derived from PubMed articles. Experimental results show that PULI significantly outperforms existing baselines in both intervention precision and collaborative task utility, highlighting the potential of proactive LLMs as intelligent scientific assistants.¹

¹https://github.com/YANGWU001/CoLabScience

## 1 Introduction

**Figure 1: Comparison of Reactive and Proactive LLMs in Biomedical Collaboration.** Traditional reactive LLMs (left) respond only after being prompted, while proactive LLMs (right) monitor ongoing discussions, identify opportunities to contribute domain-relevant insights, and intervene with timely and context-aware suggestions without explicit prompting.

Recent developments in large language models (LLMs) have fostered advancements in scientific research, enabling accelerated discovery in biomedical fields (Luo et al., 2022; Ma et al., 2024; Jin et al., 2025). In particular, existing work has explored their potential across tasks such as drug repurposing, disease diagnosis, and clinical question answering (Qi et al., 2024; Zhao et al., 2023; Lu et al., 2024c). Despite these successes, current models primarily function in a reactive paradigm (Liao et al., 2023; Lu et al., 2024c; Yao et al., 2025a), responding solely upon explicit prompts from researchers. This interaction mode significantly restricts their effectiveness in collaborative settings, where the absence of proactive interventions can lead to missed critical insights and opportunities (Yang et al., 2025b; Wu et al., 2024b).

In response to these limitations, we propose that LLMs supporting biomedical research should evolve toward proactive engagement: continuously tracking ongoing discussions, understanding emerging contexts, and autonomously identifying appropriate moments for contribution—integrating into the team as an active team member rather than remaining a passive tool. For instance, as illustrated in Figure 1, a traditional reactive LLM (Hassouna et al., 2025; Zhou et al., 2026) responds passively only after being explicitly prompted by the Clinical Physician, whereas a proactive LLM tracks the discussion, identifies timely opportunities to contribute PTEN-relevant insights, and initiates suggestions that facilitate scientific progress without waiting for direct queries.

Motivated by the need for proactive engagement, we present CoLabScience, a novel AI assistant that transforms LLMs from reactive tools to proactive collaborators in biomedical research. At its core lies PULI (Positive-Unlabeled Learning-to-Intervene), a framework trained with reinforcement learning to determine when and how to intervene during scientific discussions. To train this model, we constructed BSDD (Biomedical Streaming Dialogue Dataset), a collection of simulated scientific dialogues characterized by multiple research roles (e.g., Pharmacologist, Clinical Physician), generated by LLMs with content grounded in PubMed literature (Sayers et al., 2024).

To ensure the reliability of LLM-derived labels and mitigate hallucination risk (Huang et al., 2025; Sriramanan et al., 2024), we adopt a sparse labeling strategy in which only the most valuable intervention points are labeled as positive, while all others remain unlabeled (Kiryo et al., 2017; Wu et al., 2023). By leveraging CoLabScience's coordinator to identify reliable negative interventions from the positive-unlabeled (PU) data, we employ a two-tier approach: training a small Observer LLM with Group Relative Policy Optimization (GRPO) (Shao et al., 2024) to determine when to intervene and fine-tuning a large-scale Presenter LLM with supervised learning to generate appropriate intervention content. This architecture enables real-time dialogue monitoring via the efficient Observer model, invoking the computationally expensive Presenter model only when an intervention is needed. Reward signals from both models are integrated to train the reinforcement learning coordinator, enabling an end-to-end training loop.

By incorporating the project proposal (i.e., project goals, datasets, and background knowledge) and dual-scale conversational memory, in which long-term memory retains critical prior insights and short-term memory captures the evolving conversational context (Hatalis et al., 2023; Zhong et al., 2024), CoLabScience proactively delivers scientifically grounded interventions without requiring explicit prompting.

Our contributions in this paper are threefold and can be summarized as follows:

- We propose CoLabScience, a proactive LLM assistant that supports efficient biomedical research collaboration through context-aware interventions. Unlike reactive LLMs, CoLabScience autonomously determines when and how to intervene during ongoing research discussions by leveraging project context and conversational history.

- We introduce BSDD, a new open benchmark consisting of simulated biomedical research dialogues grounded in PubMed articles and annotated with proactive intervention labels. BSDD provides a valuable resource for training and evaluating future proactive scientific assistants, advancing research in this emerging area.

- We empirically validate CoLabScience's effectiveness through both simulation-based evaluation and human evaluation. The results demonstrate the model's strong generalization ability and robustness across a range of LLM backbones.

## 2 Related Work

### Large Language Models as Scientific Assistants

Recent advances in LLMs have shown promise in biomedical research through protein structure prediction, antibiotic discovery, and drug repurposing (Wong et al., 2024; Zambaldi et al., 2024; Jumper et al., 2021; Swanson et al., 2024; Gottweis et al., 2025; Yao et al., 2025b). Beyond the biomedical domain, systems like Agent Laboratory and AI Scientist (Schmidgall et al., 2025; Lu et al., 2024a) automate research pipelines from hypothesis generation to reporting, while Agentic Reasoning (Wu et al., 2025a) enhances multi-step reasoning. Complementing these systems, RECODE-H (Miao et al., 2025) benchmarks human-agent collaboration through multi-turn code development. However, these approaches are largely reactive, whereas CoLabScience enables context-aware, timely interventions during ongoing scientific discussions.

### Proactive Capabilities in Large Language Models

Recent work explores LLM proactivity through structured prompting, including initiative-taking in collaboration (Zhang et al., 2024a; Lu et al., 2024b; Wu et al., 2025c), clarification-seeking behaviors (Zhang et al., 2024c; Qian et al., 2024; Liu et al., 2024; Pang et al.; Li et al., 2024; Wu et al., 2024c), and requesting user support in complex tasks (Wu et al., 2024a). VideoLLM-Online (Chen et al., 2024) extends this to multimodal streaming, training models to determine optimal narration timing. However, these approaches rely on hand-crafted prompts and fixed logic, limiting adaptive intervention. In contrast, we introduce a trainable reinforcement learning mechanism that enables context-aware, timing-sensitive proactive decision-making.

## 3 Methodology

### 3.1 Preliminary Definitions

We formalize the proactive intervention task based on multi-round scientific dialogues, where each round corresponds to a single utterance from one team member. Let $\mathcal{D}=\{D^1, D^2, \ldots, D^M\}$ denote a collection of independent multi-round dialogues. Each dialogue $D^i=\{d^i_1, d^i_2, \ldots, d^i_{N_i}\}$ consists of $N_i$ rounds generated from a fixed project proposal $C^i$, which defines the research goal, background knowledge, and relevant datasets. We aggregate all dialogue rounds into a positive-unlabeled (PU) intervention training set: $\{d_1, d_2, \ldots, d_N\}$, where $N=\sum_{i=1}^M N_i$ is the total number of candidate rounds.

The PU training set contains $u$ unlabeled rounds $U=\{d_1, \ldots, d_u\}$ with unknown intervention necessity, and $(N-u)$ labeled positive rounds $P=\{d_{u+1}, \ldots, d_N\}$. Based on the PU dataset, our task is to jointly learn (1) when to intervene, using an Observer $\mathcal{H}_\phi$, and (2) how to intervene, using a Presenter $\mathcal{G}_\psi$. To facilitate the training of both LLMs, we introduce a reinforcement learning framework where a coordinator model $\mathcal{F}_\theta$ identifies potential positive and negative samples from the unlabeled data. To enhance understanding and clarity, a summary of notations is provided in Table 1.

| **Table 1: Notations** |
|---|
| [Table content showing notation definitions] |

**Figure 2: Illustration of PULI Framework.** The coordinator decides whether to intervene or remain silent for each unlabeled dialogue round. Silent rounds are used as negative samples to update the Observer through GRPO training to learn intervention timing, while intervention rounds augment positive data to refine the Presenter to generate appropriate intervention content. The Observer and Presenter collaboratively provide rewards to optimize the coordinator in an end-to-end training process.

### 3.2 PULI Mechanism

In this section, we introduce the PULI mechanism, which jointly learns when and how to intervene in multi-round scientific dialogues using a positive-unlabeled dataset. An overview of the framework is shown in Figure 2.

#### 3.2.1 Multi-Round Dialogue State

For each unlabeled intervention round $d_n \in U$ at local dialogue step $t$, we construct a contextualized memory $\mathcal{M}_t(d_n)$ comprising three components: (1) the project proposal $C(d_n)$, specifying the research goal, background, and data information; (2) a short-term memory $\mathcal{M}_t^S(d_n)$, capturing the current utterance and its two most recent predecessors; (3) a long-term memory $\mathcal{M}_t^L(d_n)$, summarizing accumulated meeting insights up to step $t$.

The memory construction is defined as:

$$\mathcal{M}_t^S(d_n) = \{d_n^{t-2}, d_n^{t-1}, d_n^t\}$$

$$\mathcal{M}_t^L(d_n) = \begin{cases} \emptyset, & \text{if } t=0 \\ \Gamma(\mathcal{M}_{t-1}^L(d_n) \cup \mathcal{M}_{t-1}^S(d_n)), & \text{if } t > 0 \end{cases}$$

$$\mathcal{M}_t(d_n) = [C(d_n), \mathcal{M}_t^S(d_n), \mathcal{M}_t^L(d_n)]$$

For initial steps where $t < 2$, we omit unavailable indices in $\mathcal{M}_t^S(d_n)$, e.g., $\mathcal{M}_0^S(d_n) = \{d_n^0\}$ and $\mathcal{M}_1^S(d_n) = \{d_n^0, d_n^1\}$. This design ensures that the short-term memory is well-formed at all time steps, while the long-term memory recursively compresses earlier rounds via the LLM summarizer $\Gamma(\cdot)$ to prevent excessive memory accumulation.

To obtain the reinforcement learning state, we process the memory input $\mathcal{M}_t(d_n)$ through both the Observer $\mathcal{H}_\phi$ and the Presenter $\mathcal{G}_\psi$, and extract their final hidden representations to construct the state embedding $S_n$:

$$S_n = \text{Concat}\left(\Psi_{\mathcal{H}_\phi}\left(\mathcal{M}_t(d_n)\right), \Omega\left(\Psi_{\mathcal{G}_\psi}\left(\mathcal{M}_t(d_n)\right)\right)\right)$$

where $\Psi_{\mathcal{H}_\phi}(\cdot)$ and $\Psi_{\mathcal{G}_\psi}(\cdot)$ denote the last hidden layer representations of $\mathcal{H}_\phi$ and $\mathcal{G}_\psi$, respectively, and $\text{Concat}(\cdot, \cdot)$ denotes the concatenation operator. To ensure dimensional consistency between the last hidden layer representations of the Observer and Presenter, we introduce a learnable linear projector $\Omega(\cdot)$ that projects $\Psi_{\mathcal{G}_\psi}(\mathcal{M}_t(d_n))$ to match the dimension of $\Psi_{\mathcal{H}_\phi}(\mathcal{M}_t(d_n))$.