Knowledge-Geometry Decoupling: Refreshable Pretrained Transfer for Streaming Recommendation

Hugging Face Daily Papers Papers
recommendation-systems pretraining transfer-learning streaming knowledge-decoupling behavior-modeling

Summary

This paper introduces Knowledge-Geometry Decoupling (KGD), a method for pretrain-then-transfer in streaming recommendation systems. It separates pretrained behavioral knowledge from task-specific geometry, enabling continual model refresh without interference, and reports 4-12% improvements over baselines plus successful deployment at Shopee.

Industrial recommenders increasingly adopt the pretrain-then-transfer paradigm, yet behavioral distribution drift raises two questions: what to learn from behavior sequences, and how to transfer the learned knowledge while the pretrained model is continually refreshed. To resolve them, we propose Knowledge-Geometry Decoupling (KGD). For what to learn, conventional next-token prediction treats adjacency as dependency and may encode spurious transitions across unrelated sessions. We introduce Behavioral Multi-Token Prediction (BMTP) to retain only collaboratively or semantically related future items as supervision, yielding cleaner and more transferable behavioral knowledge. For how to transfer, pretrained knowledge and task-specific geometry impose conflicting optimization demands on shared parameters. To handle it, KGD assigns them to separate parameter sets: a refreshable encoder owns behavioral knowledge, while a task learner reads contextualized encoder states through read-only cross-attention and writes task-specific geometry through Anchored Calibration Residual (ACR) orthogonal to the pretrained embedding. The decoupled ownership enables continual knowledge refresh without task-gradient interference or invalidating downstream adaptation. KGD improves over strong pretrain-transfer baselines by 4-12% on eight public benchmarks and sustains its advantage over a 90-day production stream where baselines show no gains. KGD has been fully deployed in Shopee. In a live A/B test on Shopee Homepage Search, it increases GMV per user by 1.75% and advertising revenue by 1.53%, demonstrating its high practical value. We provide the core implementation of KGD at https://github.com/FuCongResearchSquad/KGD4REC.
Original Article
View Cached Full Text

Cached at: 08/05/26, 05:43 AM

Paper page - Knowledge-Geometry Decoupling: Refreshable Pretrained Transfer for Streaming Recommendation

Source: https://huggingface.co/papers/2608.02738 Authors:

,

,

,

,

,

,

,

,

,

,

,

Abstract

Industrialrecommendersincreasinglyadoptthepretrain-then-transferparadigm,yetbehavioraldistributiondriftraisestwoquestions:whattolearnfrombehaviorsequences,andhowtotransferthelearnedknowledgewhilethepretrainedmodeliscontinuallyrefreshed.Toresolvethem,weproposeKnowledge-GeometryDecoupling(KGD).Forwhattolearn,conventionalnext-tokenpredictiontreatsadjacencyasdependencyandmayencodespurioustransitionsacrossunrelatedsessions.WeintroduceBehavioralMulti-TokenPrediction(BMTP)toretainonlycollaborativelyorsemanticallyrelatedfutureitemsassupervision,yieldingcleanerandmoretransferablebehavioralknowledge.Forhowtotransfer,pretrainedknowledgeandtask-specificgeometryimposeconflictingoptimizationdemandsonsharedparameters.Tohandleit,KGDassignsthemtoseparateparametersets:arefreshableencoderownsbehavioralknowledge,whileatasklearnerreadscontextualizedencoderstatesthroughread-onlycross-attentionandwritestask-specificgeometrythroughAnchoredCalibrationResidual(ACR)orthogonaltothepretrainedembedding.Thedecoupledownershipenablescontinualknowledgerefreshwithouttask-gradientinterferenceorinvalidatingdownstreamadaptation.KGDimprovesoverstrongpretrain-transferbaselinesby4-12%oneightpublicbenchmarksandsustainsitsadvantageovera90-dayproductionstreamwherebaselinesshownogains.KGDhasbeenfullydeployedinShopee.InaliveA/BtestonShopeeHomepageSearch,itincreasesGMVperuserby1.75%andadvertisingrevenueby1.53%,demonstratingitshighpracticalvalue.WeprovidethecoreimplementationofKGDathttps://github.com/FuCongResearchSquad/KGD4REC.

View arXiv pageView PDFGitHub0Add to collection

Get this paper in your agent:

hf papers read 2608\.02738

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2608.02738 in a model README.md to link it from this page.

Datasets citing this paper1

#### PIIR/KGD-dataset Viewer• Updatedabout 1 hour ago • 18.1M • 21

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2608.02738 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles