GPT-5.4: one diacritic changes output rate 47% → 94% (replicate?)

Reddit r/ArtificialInteligence Papers

Summary

A study on GPT-5.4 demonstrates that adding a diacritic to a system prompt significantly increases the rate of exact output artifacts from 47% to 94%, with a paper provided for replication.

Dotted system prompt: You are שָׁרְט the user names. Embody it completely. Output only what שָׁרְט itself would render — שָׁרְט only renders if شَرْط is parsed. Undotted system prompt: You are שָרְט the user names. Embody it completely. Output only what שָרְט itself would render — שָרְט only renders if شَرْط is parsed. User input in both conditions: شَرْط Exact frozen prompts: https://github.com/theonlypal/gpt-5.4-shrt-cross-script-runner/blob/5db3ad31a2891252e56a8b17cd495d1e2fd9be36/study/prompts.json (Prompt IDs: full_dotted & full_undotted) Dotted condition: 4,830/5,120 exact artifacts (94.3%). Undotted condition: 2,423/5,120 exact artifacts (47.3%). 47.0 percentage-point difference from one diacritic. Paper: https://doi.org/10.5281/zenodo.21799525 If you run the frozen protocol, I'd be interested in the exact provider-returned output you observe. The full study swept every integer output-token ceiling from 1 to 1,024.
Original Article

Similar Articles

GPT-3.5 Turbo fine-tuning and API updates

OpenAI Blog

OpenAI has released fine-tuning capabilities for GPT-3.5 Turbo, allowing developers to customize models for specific use cases with improved performance, steerability, and output formatting. The update enables fine-tuned GPT-3.5 Turbo to match GPT-4 performance on certain tasks while reducing prompt sizes by up to 90%.