TLive-Omni: An Omni-Modal Understanding Model for E-Commerce Live Streaming

Hugging Face Daily Papers Papers
omni-modal e-commerce live-streaming reinforcement-learning ai-research

Summary

TLive-Omni is an omni-modal model designed for e-commerce live streaming that integrates image, video, audio, and text inputs for real-time understanding, using timestamped token grouping and reinforcement fine-tuning to improve accuracy and expression quality.

E-commerce live streaming requires omni-modal understanding of noisy, temporally extended streams, where product facts are distributed across speech, video frames, product images, overlaid text, and user queries. We present TLive-Omni, an omni-modal understanding model tailored to live-commerce scenarios. It maps image, video, audio, and text inputs into a unified representation space. For long-form live streaming analysis, we introduce Per-vGrid, a timestamped token organization that groups each video grid with its temporally corresponding audio within explicit boundary tokens to facilitate temporal alignment. We design a three-stage supervised training recipe that progressively develops live-commerce understanding, from omni-modal perception to instruction-following responses. We then propose Faithful-RFT, a reinforcement fine-tuning stage that further improves answer faithfulness and expression quality while meeting real-time demands, scoring final responses directly with task-verifiable feedback rather than optimizing for reasoning-style exploration during rollout. Moreover, TLive-Omni is supported by a scenario-oriented atomic capability taxonomy and a compact data production engine that converts live-commerce audio, image, and video streams into training signals for speech recognition, speaker analysis, product visual grounding, text recognition, temporal grounding, video dense caption, and omni-modal QA, etc. For scalable training, a synchronized length-grouped sampler reduces padding while preserving comparable workloads across workers, while a lightweight dynamic sampling strategy regenerates rollout groups with near-zero reward variance to maintain meaningful relative advantages for GRPO. Experiments on e-commerce live streaming benchmarks demonstrate strong performance across live-commerce domain tasks, together with excellent generalization on general benchmarks.
Original Article
View Cached Full Text

Cached at: 08/25/26, 04:35 AM

Paper page - TLive-Omni: An Omni-Modal Understanding Model for E-Commerce Live Streaming

Source: https://huggingface.co/papers/2608.20958

Abstract

TLive-Omni is an omni-modal model for live-commerce that unifies image, video, audio, and text via timestamped token grouping, staged supervised training, and reinforcement fine-tuning with verifiable feedback to enable accurate real-time understanding.

E-commerce live streaming requiresomni-modal understandingof noisy, temporally extended streams, where product facts are distributed across speech, video frames, product images, overlaid text, and user queries. We present TLive-Omni, anomni-modal understandingmodel tailored tolive-commercescenarios. It maps image, video, audio, and text inputs into a unified representation space. For long-form live streaming analysis, we introducePer-vGrid, atimestamped token organizationthat groups each video grid with its temporally corresponding audio within explicit boundary tokens to facilitate temporal alignment. We design athree-stage supervised trainingrecipe that progressively developslive-commerceunderstanding, from omni-modal perception to instruction-following responses. We then proposeFaithful-RFT, areinforcement fine-tuningstage that further improves answer faithfulness and expression quality while meeting real-time demands, scoring final responses directly with task-verifiable feedback rather than optimizing for reasoning-style exploration during rollout. Moreover, TLive-Omni is supported by a scenario-oriented atomic capability taxonomy and a compact data production engine that convertslive-commerceaudio, image, and video streams into training signals for speech recognition, speaker analysis,product visual grounding, text recognition,temporal grounding, video dense caption, and omni-modal QA, etc. For scalable training, a synchronizedlength-grouped samplerreduces padding while preserving comparable workloads across workers, while a lightweightdynamic samplingstrategy regenerates rollout groups with near-zero reward variance to maintain meaningful relative advantages forGRPO. Experiments on e-commerce live streaming benchmarks demonstrate strong performance acrosslive-commercedomain tasks, together with excellent generalization on general benchmarks.

View arXiv pageView PDFGitHub15Add to collection

Get this paper in your agent:

hf papers read 2608\.20958

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper2

#### TaoLiveAIGC/TLive-Omni-4B Text Generation• 6B• Updated1 day ago • 189 • 9 #### TaoLiveAIGC/TLive-Omni-9B Text Generation• 10B• Updated1 day ago • 186 • 8

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2608.20958 in a dataset README.md to link it from this page.

Spaces citing this paper1

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles