parameter-sharing

Tag

Cards List
#parameter-sharing

HiUni: Harmonizing Understanding and Generation in Shared Visual Space

Hugging Face Daily Papers ↗ · 2d ago Cached

HiUni explores extending a pretrained vision-language model into a unified multimodal model via extensive parameter sharing, reusing a single dense backbone for both understanding and generation with a PixelDiffusionHead and HiGate pathway to address optimization and spatial-detail challenges. Experiments show it matches the pretrained VLM's understanding score while achieving competitive image generation and editing performance.

0 favorites 0 likes
#parameter-sharing

Causal Latent Workspaces for Looped Transformers

arXiv cs.CL ↗ · 2026-10-19 Cached

该论文提出因果潜在工作区 Transformer(causal latent workspace Transformer),通过在共享深度的循环块之间加入可读写的前缀专属潜在槽位,提升了循环 Transformer 的语言建模困惑度,并在三个字节级语料上的多数设置中优于五个基线对照。

0 favorites 0 likes
#parameter-sharing

LatentLoop: Scaling Looped MoE Models with Asymmetric Compute Allocation

Hugging Face Daily Papers ↗ · 2026-10-14 Cached

论文提出 LatentLoop,通过非对称计算分配(压缩第一次循环访问的序列分辨率、保留第二次全分辨率并将节省的算力用于增大隐藏维度)来扩展循环 MoE 模型,在 1.3B–7B 参数范围内一致降低验证 NLL,相比原始 MoE 和循环基线分别节省约 19.0% 和 7.9% 的计算量。

0 favorites 0 likes
#parameter-sharing

Improving Parameter Utilization by Sharing Neural Experts Across Layers in Transformers

arXiv cs.LG ↗ · 2026-09-22 Cached

The paper proposes CS-MoE, a novel Transformer architecture that shares neural experts across layers to improve parameter utilization, achieving lower perplexity with only 55% of parameters activated.

0 favorites 0 likes
#parameter-sharing

Looped GPT-BERT: Trading Parameters for Computation in Small Language Modeling

arXiv cs.CL ↗ · 2026-09-10 Cached

This paper introduces Looped GPT-BERT, which uses depth-wise parameter sharing to train a small language model with fewer parameters, achieving comparable performance to baselines in the BabyLM 2026 Strict-small setting.

0 favorites 0 likes
← Back to home

Submit Feedback