Aloha! Introducing Ornith-1.5, a family of open-source LLMs spanning 9B Dense, 35B MoE, and 397B MoE, trained with self-improving strategies.

It achieves state-of-the-art performance among open-source models of comparable size and delivers performance comparable to Claude Opus 4.8 across reasoning, agentic, and coding tasks: Terminal-Bench 2.1 (86.1) SWE-Bench (86 on verified, 65.1 on pro, 79.6 on Multilingual) DeepSWE (56) HLE (44.6) ClawEval (81.4) Tool Decathlon (71.2)

Ornith-1.5 takes a major step toward training foundation models through end-to-end self-improvement, extending the self-scaffolding strategies introduced in Ornith-1.0 into a more complete self-improvement loop: the model proposes new tasks, generates task-specific scaffolds, and produces solution rollouts for reinforcement learning, continuously creating new learning experiences from which it can improve.

All models, along with their quantized versions (FP8, GGUF, MLX, and NVFP4), have been released under the MIT License, enabling unrestricted commercial and research use. Tech Blog: http://ornith.ai/ornith_1_5.html Huggingface: http://huggingface.co/collections/ornith-ai/ornith-15…

Ornith-1.5: From Self-Scaffolding to Self-Improvement

Source: https://ornith.ai/ornith_1_5.html Today, we are introducing Ornith-1.5, a major step toward building foundation models through end-to-end self-improvement. Ornith-1.5 extends the self-scaffolding framework introduced in Ornith-1.0 into a more complete self-improvement loop: the modelproposes new tasks, generates task-specific scaffolds, and produces solution rolloutsfor reinforcement learning, continuously creating new learning experiences from which it can improve.

Ornith-1.5 spans three model scales: 397B MoE, 35B MoE and 9B dense. Designed for strong general-purpose intelligence across reasoning, agentic, and coding tasks, Ornith-1.5 achieves state-of-the-art performance among open-source models of comparable size across a broad range of benchmarks. Ornith-1.5-397B scores 86.1 on Terminal-Bench 2.1 and 56.0 on DeepSWE, performing on par with Claude Opus 4.8 (85.0 and 59.0) while outperforming leading open-source models of similar scale, including GLM-5.2 (82.7 and 46.2) and DeepSeek-V4-Flash-0731 (82.7 and 54.4). At the other end of the spectrum, Ornith-1.5-9B, with its quantized Ornith-1.5-9B-Mobile version, can be readily deployed on iPhone and Android devices while substantially outperforming larger models such as Gemma 4-31B and Qwen 3.6-35B.

At the flagship scale, Ornith-1.5-397B achieves 86.1 on Terminal-Bench 2.1 and 56 on DeepSWE, matching Claude Opus 4.8 on both benchmarks and outperforming leading open-source models of similar size, including GLM-5.2 and DeepSeek-V4-Flash-0731.

Ornith-1.5-35B significantly outperforms its similar-sized peer Qwen 3.6-35B across all coding and agentic benchmarks, and despite activating only 3B parameters per token, it also outperforms dense models—Gemma 4-31B and Meta’s Muse Glimmer-30B—by wide margins on agentic coding (68.5 vs. 43.4 and 51.7 on Terminal-Bench 2.1; 79.0 vs. 52.0 and 76.0 on SWE-Bench Verified).

The edge-deployable Ornith-1.5-9B also delivers remarkably strong results, achieving 47.0 on Terminal-Bench 2.1 and 70.6 on SWE-Bench Verified. Despite being a compact 9B-parameter model, it matches or exceeds the performance of much larger models such as Gemma 4-31B and Qwen 3.6-35B.

Self-Improvement through Self-Generated Tasks, Harnesses, and Solutions

Ornith-1.5 extends Ornith-1.0 by expanding the self-improvement loop from scaffold and rollout optimization to jointly optimizing task generation, scaffold construction, and solution rollouts. Rather than relying on a fixed set of human-curated tasks and manually designed harnesses, Ornith-1.5 continuously generates new training tasks, discovers effective strategies for solving them, and improves the policy through reinforcement learning.

Each training cycle proceeds in three stages. Given an environment or codebase, high-level instructions about the task type, and access to the model’s previous task-solving history, the system proposes progressively harder tasks that go beyond what the model has already solved, exposing capability gaps and continuously pushing the training frontier.

For each task, the model then generates or refines a task-specific scaffold—the instructions, tools, decomposition strategy, and orchestration used to approach the problem. Conditioned on the task and scaffold, the policy produces a solution rollout. Reward from the rollout is propagated across all three stages, so the system learns not only to produce better solutions, but also to generate more useful training tasks and construct more effective scaffolds.

Repeated over training, this creates a closed self-improvement loop in which stronger policies enable the generation of harder and more informative tasks, evolving scaffolds discover better ways to elicit the model’s capabilities, and higher-quality rollouts provide increasingly effective learning signals. Instead of relying on a static training distribution or hand-engineered agent design, Ornith-1.5 continually expands its own curriculum and adapts its problem-solving strategies, driving sustained capability gains acrossreasoning, coding, and agentic tasks.

Task Reward

For thequestion → scaffold → rolloutsetup, we define the task reward using three signals:validity, frontier difficulty, and novelty. Let \(q\) denote a generated question, \(s\) its scaffold, and \(\{\tau_i\}_{i=1}^{N}\) a set of solution rollouts. We define

\[ R_{\text{task}} = \underbrace{V(q,s)}_{\text{Is it valid and verifiable?}} \times \underbrace{D\!\left(q,s,\{\tau_i\}_{i=1}^{N}\right)}_{\text{Is it at the right difficulty?}} \times \underbrace{N(q)}_{\text{Is it sufficiently novel?}}. \]

Here, \(V\) measures whether the generated task and scaffold form a valid and verifiable learning environment, \(D\) measures whether the task lies near the model’s current capability frontier based on rollout performance, and \(N\) measures novelty relative to previously generated or trained-on tasks. The multiplicative formulation encourages the proposer to generate tasks that satisfy all three properties simultaneously:valid, appropriately challenging, and non-redundant.

Validity and Verifiability

A useful task must form a well-defined learning environment. The question should be coherent and solvable, while the scaffold should execute correctly and reliably evaluate candidate solutions. We define

\[ V(q,s) \in [0,1], \]

based on checks such as whether the scaffold runs successfully, high-confidence solutions pass, clearly incorrect solutions fail, and the evaluation matches the task specification. Validity can also be treated as a hard gate:

\[ V(q,s)=0 \quad\Rightarrow\quad R_{\text{task}}=0. \]

This prevents malformed tasks or unreliable scaffolds from receiving reward simply because they appear difficult.

Frontier Difficulty

Among valid tasks, the most useful ones are neither trivial nor impossible. We estimate difficulty directly from the model’s rollouts.

For each task, we sample \(N\) rollouts and compute the empirical success rate

\[ p = \frac{1}{N} \sum_{i=1}^{N} \mathbf{1}\!\left[s(q,\tau_i)=\text{success}\right]. \]

We then reward tasks whose success rate is close to a target frontier \(p^*\):

\[ D(q,s,\{\tau_i\}) = \exp\!\left(-\frac{(p-p^*)^2}{2\sigma^2}\right). \]

\(p^*\) is set to 0.2, which favors tasks that are challenging but still yield enough successful trajectories for reinforcement learning. As the model improves and solves a task more reliably, its reward naturally decreases, pushing the generator toward harder problems.

Novelty and Diversity

Frontier difficulty alone may lead the model to repeatedly generate small variations of the same tasks. We therefore add a novelty term:

\[ N(q) = 1 - \max_{q_j \in \mathcal{B}} \operatorname{sim}(q,q_j), \]

where \(\mathcal{B}\) is a buffer of previously generated or trained-on tasks. Novelty should remain secondary to validity and difficulty: its role is to reduce redundancy, not to reward arbitrarily unusual tasks.

Together, these signals encourage the proposer to generate tasks that arevalid, verifiable, challenging but learnable, and sufficiently diverse. Because frontier difficulty is measured using the current model’s own rollouts, the resulting curriculum automatically evolves with model capability.

Harness and Rollout Rewards

For a generated question \(q\), the harness \(h\) is rewarded for providing an evaluation environment that isaligned with the task, faithful to solution quality, and resistant to reward hacking:

\[ R_{\text{harness}} = \underbrace{C(q,h)}_{\text{Task alignment}} \times \underbrace{F\!\left(h,\{\tau_i\}\right)}_{\text{Reward fidelity}} \times \underbrace{H(h)}_{\text{Hack resistance}}. \]

Here, \(C\) measures whether the harness faithfully reflects the task specification, \(F\) measures whether its rewards track the true quality of candidate solutions, and \(H\) measures its resistance to evaluator failures, shortcuts, and reward-hacking behaviors.

Each rollout \(\tau_i\) is scored directly by the generated harness:

\[ R_{\text{rollout}}(\tau_i) = \underbrace{h(q,\tau_i)}_{\text{Task success}}. \]

For verifiable tasks, this can be a binary pass/fail reward; for richer environments, it can combine correctness, task completion, efficiency, and constraint satisfaction.Question generation, harness generation, and solution rollouts are all optimized with GRPO using their respective rewards, enabling the three stages to improve jointly within the same self-improvement loop.

Full Table

Ornith-1.5-397B

Ornith-1.5-35B

Ornith-1.5-9B

Footnote