Speaking the Language of Science: Toward a General-Purpose Generative Foundation Model for the Natural Sciences

Hugging Face Daily Papers Papers
scientific-language-model foundation-model ai-for-science generative-model token-sequence autoregressive multi-domain

Summary

LOGOS is a scientific generative language model that encodes diverse scientific objects and spatial interactions as token sequences, enabling a unified autoregressive framework for tasks across natural sciences. Models at 1B, 3B, and 8B parameters show consistent performance scaling and are released to facilitate research.

In this report, we present LOGOS (Language Of Generative Objects in Science), a scientific generative language model that unifies heterogeneous tasks across the natural sciences within a single autoregressive framework based on a shared scientific grammar. It encodes diverse scientific objects and their spatial interactions as token sequences over a common vocabulary. By representing spatial contact and constraint patterns as discrete tokens, the model captures complex structural interactions in a purely sequential manner, without relying on explicit coordinates or geometric neural networks. This unified representation enables a wide range of downstream tasks to be formulated consistently as next-token prediction in the same grammar space, creating strong alignment between continued multi-domain pre-training and downstream objectives. Across diverse tasks, LOGOS consistently matches or outperforms domain-specific baselines, providing preliminary evidence for the feasibility of "one model fits all" in the natural sciences. We train LOGOS models at different scales (1B, 3B, and 8B parameters) and find a consistent positive correlation between model size and performance. This suggests that the future of AI for Science (AI4S) may not lie in building an independent technical stack that is separated from large language models (LLMs). Instead, it may depend on deeply aligning scientific foundation models with LLMs through shared architectures, shared training paradigms, and shared inference infrastructure, so that LLMs can truly become a new entry point for AI4S. We release the model weights and associated resources to facilitate further research.
Original Article
View Cached Full Text

Cached at: 06/18/26, 03:56 AM

Paper page - Speaking the Language of Science: Toward a General-Purpose Generative Foundation Model for the Natural Sciences

Source: https://huggingface.co/papers/2606.16905

Abstract

A unified scientific generative language model encodes diverse scientific objects and spatial interactions as token sequences, demonstrating strong performance across multiple domains through autoregressive next-token prediction.

In this report, we present LOGOS (Language Of Generative Objects in Science), a scientific generative language model that unifies heterogeneous tasks across the natural sciences within a singleautoregressive frameworkbased on a sharedscientific grammar. It encodes diverse scientific objects and their spatial interactions astoken sequencesover a common vocabulary. By representing spatial contact and constraint patterns as discrete tokens, the model captures complex structural interactions in a purely sequential manner, without relying on explicit coordinates or geometric neural networks. This unified representation enables a wide range of downstream tasks to be formulated consistently asnext-token predictionin the same grammar space, creating strong alignment between continued multi-domain pre-training and downstream objectives. Across diverse tasks, LOGOS consistently matches or outperforms domain-specific baselines, providing preliminary evidence for the feasibility of “one model fits all” in the natural sciences. We train LOGOS models at different scales (1B, 3B, and 8B parameters) and find a consistent positive correlation between model size and performance. This suggests that the future ofAI for Science(AI4S) may not lie in building an independent technical stack that is separated fromlarge language models(LLMs). Instead, it may depend on deeply aligningscientific foundation modelswith LLMs through shared architectures, shared training paradigms, and shared inference infrastructure, so that LLMs can truly become a new entry point for AI4S. We release the model weights and associated resources to facilitate further research.

View arXiv pageView PDFGitHub19Add to collection

Get this paper in your agent:

hf papers read 2606\.16905

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper4

#### LOGOS-Hub/LOGOS-8B 8B• Updated2 days ago • 36 • 6 #### LOGOS-Hub/LOGOS-pretrain-1B 1B• Updated2 days ago • 34 • 4 #### LOGOS-Hub/LOGOS-pretrain-3B 4B• Updated2 days ago • 30 • 3 #### LOGOS-Hub/LOGOS-pretrain-8B 308k• Updated2 days ago • 32 • 3

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2606.16905 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2606.16905 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

Large Language Models of Babel

ML at Berkeley

The article reflects on the history of text generation, drawing parallels between modern LLMs like GPT-4 and earlier concepts from Jorge Luis Borges and Claude Shannon. It explores how Shannon's probabilistic experiments and Borges' 'Library of Babel' metaphor help clarify fundamental questions about the nature of generated text and data structure.