The LuaJIT NYI That Silently Poisoned an Unrelated Hot Loop

# The LuaJIT NYI That Silently Poisoned an Unrelated Hot Loop - StreamHPC Source: [https://streamhpc.com/blog/2026-08-05/the-luajit-nyi-that-silently-poisoned-an-unrelated-hot-loop](https://streamhpc.com/blog/2026-08-05/the-luajit-nyi-that-silently-poisoned-an-unrelated-hot-loop) If you haven’t used[Lua](https://www.lua.org/about.html)before, it’s the go\-to embedded scripting language for games like Factorio and World of Warcraft, and for applications like[Neovim](https://github.com/neovim/neovim)and OpenResty\.[LuaJIT](https://luajit.org/luajit.html), its[just\-in\-time compiler](https://en.wikipedia.org/wiki/Just-in-time_compilation), is widely praised for its speed, so it’s easy to assume your code is already running as fast as possible\. In reality, you can unknowingly cripple performance by hitting one of LuaJIT’s NYIs\. NYI stands for “Not Yet Implemented,” meaning operations that LuaJIT cannot translate into optimized machine code\. What happens next depends on the specific NYI, and untangling those differences is the subject of this post\. I ran into this while benchmarking[grug\-for\-lua](https://github.com/grug-lang/grug-for-lua), my Lua implementation of the[grug](https://github.com/grug-lang/grug)modding language\. The same benchmark, with identical code and inputs, sometimes reported 6 billion iterations and other times only 300 million, a**20× difference**\. The culprit turned out to be a seemingly harmless operation in one part of the code that silently caused LuaJIT to blacklist a function that an unrelated hot loop later depended on\. This post follows that investigation\. It aims to be approachable even if you’ve never touched LuaJIT or compilers before, so it links to background reading wherever something isn’t explained in full\. We’ll look at two NYI\-related pitfalls, and how to let your[CI](https://en.wikipedia.org/wiki/Continuous_integration)guard against them\. ## The suspect code All game functions are called through`Entity:\_run\_game\_fn`, which looked roughly like this: It was called like so: At first glance, nothing looks unusual\.`pcall`stands for “protected call” and works like a try/catch: it calls`game\_fn`and catches any error instead of letting it propagate\. Forwarding arguments with`unpack`and receiving them with`\.\.\.`is a common Lua idiom, but inside a LuaJIT hot loop this perfectly valid code triggers one of the compiler’s least obvious optimization pitfalls\. ## Trace stitching LuaJIT is a tracing JIT compiler: as your program runs, it records the hottest paths of execution \(traces\) and compiles them into optimized machine code\. A list of features it cannot record into these traces is documented on the[LuaJIT Not Yet Implemented wiki page](https://github.com/tarantool/tarantool/wiki/LuaJIT-Not-Yet-Implemented), and`unpack`is one of them, marked`2\.1 stitch`\. Instead of failing immediately, LuaJIT performs a*stitch*, which lets it resume trace recording after the NYI instruction executes\. For implementation details, see the[NYI and Trace Stitching](https://pwner.gg/blog/2022-09-13-lua-jit-part2#nyi-and-trace-stitching)section of*LuaJIT Internals \(Pt\. 2/3\): Fighting the JIT Compiler*\. My initial assumption was that the stitch itself caused the slowdown by temporarily dropping back to the interpreter\. But when I profiled the benchmark with LuaJIT`2\.1\.1774896198`using`\-jv`,`unpack`never appeared in the trace log at all\. ## Reading the trace log The trace log told a different story\. A trace failed immediately after returning from the callee, followed by repeated “NYI: return to lower frame” messages, and eventually a notice that the function had been blacklisted from compilation\. To isolate it, I wrote an MRE \([minimal reproducible example](https://en.wikipedia.org/wiki/Minimal_reproducible_example)\): Running`luajit \-jv pcall\_mre\.lua`produces two very different outcomes depending on chance\. Here’s a fast run: Whether`empty\_fn`ends up blacklisted depends on how many stitch attempts through the`pcall`complete before the benchmark loop begins, an outcome governed by the JIT’s internal heuristics rather than anything in the code\. After enough failed attempts, LuaJIT blacklists`empty\_fn`‘s bytecode, so the hot loop calling it can no longer be JIT\-compiled and falls back to the interpreter, even though the loop itself never touches`pcall`or`unpack`\. That’s why it suffers the**14× slowdown**\. As a sanity check, calling an identical`empty\_fn2\(\)`inside the hot loop stays fast instead, since it was never blacklisted\. ## The fix The fix in`db94c5a`removes the final`unpack\(\)`call and replaces it with generated wrapper functions: Instead of forwarding arguments through`unpack`, each arity gets a specialized wrapper that indexes the`args`table directly\. The wrappers are cached by argument count, so the code generation cost is paid once while execution remains fully traceable by LuaJIT\.`pcall`was never the problem here; it’s fully traceable by LuaJIT\. Only`unpack`triggers the stitch, which is why each generated wrapper still calls`pcall`to preserve the original error handling\. You can compare`db94c5a`against its parent`4523ea9`to see the difference directly\. In`benchmarks/minimal`, runs of`4523ea9`fluctuate wildly between fast and slow depending on whether the earlier blacklist is triggered\. With`db94c5a`, the performance is stable, and the 20× variance disappears\. However, preventing that silent trace blacklisting was only the first step in fully optimizing the hot loop\. ## The second NYI: closures While investigating the blacklist issue, I started checking every hot path in grug\-for\-lua for other NYIs\. That uncovered a second one, triggered by closures \(nested functions\), which carries an even steeper penalty\. Its failure mode is different: rather than poisoning an unrelated loop, it slows down the very loop it appears in\. At first glance, it looks like code a compiler should optimize away, similar to what C\+\+’s[as\-if rule](https://en.wikipedia.org/wiki/As-if_rule)permits for code with no observable side effects\. It’s worth its own minimal reproducible example,`closure\_mre\.lua`: With`nested\(\)`defined inside the loop,`luajit \-jv closure\_mre\.lua`produces this trace log: This is the slow run: repeatedly allocating the closure prevents proper JIT compilation, resulting in a**60× slowdown**compared to the optimized version\. Move`nested\(\)`outside the loop, and the trace output changes completely: `nested`takes no arguments, reads no upvalues, and does nothing in its body, yet LuaJIT still preserves Lua 5\.1 semantics\. As the[Lua 5\.1 Reference Manual](https://www.lua.org/manual/5.1/manual.html#2.5.2)states, every new function object is distinct from any previously existing one: > Two objects are considered equal only if they are the*same*object\. Every time you create a new object \(a table, userdata, thread, or**function**\), this new object is different from any previously existing object\. In theory, LuaJIT could optimize the allocation away, much as it already does for tables using[its Allocation Sinking optimization](https://github.com/tarantool/tarantool/wiki/LuaJIT-Allocation-Sinking-Optimization)\. Unlike`unpack`,`FNEW`\(function new\) is a hard NYI rather than a stitched one\. As soon as the recorder reaches it, recording aborts outright\. The NYI wiki correspondingly[lists](https://github.com/tarantool/tarantool/wiki/LuaJIT-Not-Yet-Implemented#bytecode)its`Compiled?`status as`no`\. There’s no race to win or lose here; the loop is simply stuck in the interpreter every time\. The only change that matters is moving`nested`outside the loop, so the loop body becomes a plain function call with no per\-iteration allocation, making it 60× faster\. ## Catching NYIs early I configure my CI’s[build\.yml](https://github.com/grug-lang/grug-for-lua/blob/ca3e88b438cb787c5081ed1e6b28596ed3140294/.github/workflows/build.yml#L125-L140)to run benchmarks under`luajit \-jv`\. It lets the build fail if any NYI other than the known\-sporadic`return to lower frame`case shows up in the trace log, or if any`blacklisted`line shows up\. That way,*any*NYI or blacklisting introduced into a hot path gets caught automatically\. ## Getting unpack\(\) out of the NYI list As an homage to Cloudflare’s[LuaJIT Hacking: Getting next\(\) out of the NYI list](https://blog.cloudflare.com/luajit-hacking-getting-next-out-of-the-nyi-list/), I named my pull request[perf: get unpack\(\) out of the NYI list](https://github.com/openresty/luajit2/pull/269)\. It targets`luajit2`, OpenResty’s fork of LuaJIT, rather than[upstream LuaJIT](https://github.com/LuaJIT/LuaJIT), since upstream only allows collaborators to open pull requests there\. I plan to send the patch to LuaJIT’s creator, Mike Pall, over LuaJIT’s mailing list in the future\. In the meantime, luajit2 is a reasonable place to land the fix first: per its own repository description, it states: “It is not to be considered a fork, since we still regularly synchronize changes from the upstream LuaJIT project\.” Here’s the new, gnarly`recff\_unpack`function the PR adds, which teaches the trace recorder to compile`unpack`directly instead of falling back to a stitch: Writing 30 tests confirming`recff\_unpack`no longer stitches, throws NYIs, or gets blacklisted wasn’t enough to convince me it was correct for every edge case, so I wrote[unimut](https://pypi.org/project/unimut)\(universal mutator, designed to work with any programming language\) to mutation test it, which surfaced 2 more edge cases the initial 30 tests missed\. unimut systematically mutates the function’s logic and checks whether the test suite still catches each change, surfacing gaps that plain line and branch coverage can’t: The handful of surviving mutants here are expected\. They involve checks against internal LuaJIT implementation details that Lua\-level tests can’t, or shouldn’t, cover\. Even once this is merged, the`get\_pcall\_wrapper`workaround from earlier in this post isn’t going anywhere\. Almost nobody uses OpenResty’s`luajit2`fork specifically, and plenty of programs that do embed some LuaJIT never update the version they ship\. This fix lets a small slice of users get`unpack`compiled for free, while the wrapper workaround stays the practical fix for everyone else for the foreseeable future\. ## Conclusion The broader danger of tracing JITs is that performance bugs don’t always show up where you’d expect\. With`unpack`, an innocent call silently blacklisted a completely unrelated hot loop, so benchmarks and profilers pointed at the wrong place entirely\. With closures, the cost landed exactly where you’d expect, yet still went unnoticed because nothing in the code looked wrong\. Manual spot\-checks cannot catch either failure mode\. Instead, use`luajit \-jv`to treat compiler trace output as a testable artifact in CI, so any regressions caused by NYIs get caught automatically instead of slipping back in silently\. If you want to dig deeper into NYIs, check out Cloudflare’s[LuaJIT Hacking: Getting next\(\) out of the NYI list](https://blog.cloudflare.com/luajit-hacking-getting-next-out-of-the-nyi-list/)and api7\.ai’s[The JIT Compiler’s Drawback: Why Avoid NYI?](https://api7.ai/learning-center/openresty/avoid-lua-not-yet-implemented-features)\. `unpack`is now off the list, with[tests and reproduction steps in the PR](https://github.com/openresty/luajit2/pull/269)if you want to see it in action\. Plenty of NYIs are still sitting on the[LuaJIT Not Yet Implemented wiki page](https://github.com/tarantool/tarantool/wiki/LuaJIT-Not-Yet-Implemented), closures included\. If you’re looking for an excuse to get your hands dirty in`lj\_record\.c`, that page is a good place to start\.