A Sovereign, Open-Source Foundation Model for German and English

Hugging Face Daily Papers Papers
foundation-model open-source mixture-of-experts german english sovereign-ai mamba-transformer

Summary

SoofiS30B-A3B is a sovereign, open-source Mixture-of-Experts hybrid Mamba Transformer foundation model for German and English, activating only 3B of 30B parameters per token and achieving strong benchmark performance.

We present Soofi S 30B-A3B, a sovereign, open-source Mixture-of-Experts (MoE) hybrid Mamba Transformer foundation model for German and English. Its hybrid design activates only 3B of 30B parameters per token and keeps the inference cache near-constant as context grows, giving it a decisive throughput advantage over dense models for long-context, high-concurrency deployment. Pretrained on roughly 27 trillion tokens with deliberately up-weighted German, Soofi S matches dense 14 to 27B models on aggregate English and German benchmarks while achieving the best code aggregates in both languages among 17 open base models, and outperforms every European sovereign baseline in our comparison, including ones far larger in active parameters. Among fully open models, Soofi S obtains the highest English and German evaluation scores, ahead of Olmo 3 32B and Apertus 70B. Soofi S was built end-to-end on the German Industrial AI Cloud, a sovereign HPC scale AI infrastructure operated by Deutsche Telekom in Munich. Soofi S will be released under highly permissive, open-access terms: weights, selected intermediate checkpoints, full per-source data accounting, hyperparameters, and training and evaluation code. Where source licenses permit, data-construction artifacts are released under permissive licenses; commercially licensed sources are documented with aggregate statistics and exact mixture accounting.
Original Article
View Cached Full Text

Cached at: 07/13/26, 07:50 AM

Paper page - A Sovereign, Open-Source Foundation Model for German and English

Source: https://huggingface.co/papers/2607.09424 Authors:

,

,

,

,

,

,

,

,

,

,

,

,

,

,

,

,

,

,

,

,

Abstract

WepresentSoofiS30B-A3B,asovereign,open-sourceMixture-of-Experts(MoE)hybridMambaTransformerfoundationmodelforGermanandEnglish.Itshybriddesignactivatesonly3Bof30Bparameterspertokenandkeepstheinferencecachenear-constantascontextgrows,givingitadecisivethroughputadvantageoverdensemodelsforlong-context,high-concurrencydeployment.Pretrainedonroughly27trilliontokenswithdeliberatelyup-weightedGerman,SoofiSmatchesdense14to27BmodelsonaggregateEnglishandGermanbenchmarkswhileachievingthebestcodeaggregatesinbothlanguagesamong17openbasemodels,andoutperformseveryEuropeansovereignbaselineinourcomparison,includingonesfarlargerinactiveparameters.Amongfullyopenmodels,SoofiSobtainsthehighestEnglishandGermanevaluationscores,aheadofOlmo332BandApertus70B.SoofiSwasbuiltend-to-endontheGermanIndustrialAICloud,asovereignHPCscaleAIinfrastructureoperatedbyDeutscheTelekominMunich.SoofiSwillbereleasedunderhighlypermissive,open-accessterms:weights,selectedintermediatecheckpoints,fullper-sourcedataaccounting,hyperparameters,andtrainingandevaluationcode.Wheresourcelicensespermit,data-constructionartifactsarereleasedunderpermissivelicenses;commerciallylicensedsourcesaredocumentedwithaggregatestatisticsandexactmixtureaccounting.

View arXiv pageView PDFAdd to collection

Get this paper in your agent:

hf papers read 2607\.09424

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2607.09424 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2607.09424 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2607.09424 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

A Sovereign, Open-Source Foundation Model for German and English

arXiv cs.CL

The Soofi S 30B-A3B is a sovereign, open-source Mixture-of-Experts foundation model for German and English, pretrained on 27 trillion tokens. It matches dense 14-27B models on benchmarks while offering superior long-context throughput and outperforms all European open baselines.

Soofi – Sovereign Open Source Foundation Models

Hacker News Top

Soofi introduces Soofi S, a 30B parameter Mixture-of-Experts open source foundation model trained on 27 trillion tokens, targeting industrial AI applications in German and English. The model is part of a European sovereign AI initiative.

Apertus – Open Foundation Model for Sovereign AI

Hacker News Top

Apertus is a fully open foundation model for sovereign AI, developed by the Swiss AI Initiative. It is open weights, open data, open science, compliant with EU AI Act, and competitive with top open models at 8B and 70B parameters, supporting over 1000 languages.