two case studies of NaN

Lobsters Hottest News

Summary

This article explores two unexpected behaviors of NaN in Python and Lua, where implicit assumptions about equality and comparison lead to surprising results like Python's list equality ignoring NaN's self-inequality and Lua's for-loops mishandling NaN as step or limit.

<p><a href="https://lobste.rs/s/v5hkjy/two_case_studies_nan">Comments</a></p>
Original Article
View Cached Full Text

Cached at: 07/09/26, 09:36 AM

# two case studies of NaN Source: [https://sebsite.pw/w/20260709-nan.html](https://sebsite.pw/w/20260709-nan.html) IEEE\-759 NaN is weird\. and because of that, it's often accidentally left unaccounted for\. i found two instances of this leaking into programming language design\. that is, the semantics of these languages hold implicit assumptions which break with NaN\. ## case study 1: python ``` >>> from math import nan >>> nan == nan False >>> [nan] == [nan] True ``` as an optimization, when comparing lists for equality, elements are first compared by identity\. they're only checked for equality if their identities are unequal\. in general, it's assumed that an object will always compare equal to itself\. this is explicitly noted in the[python 3 reference manual](https://docs.python.org/3/reference/expressions.html): > User\-defined classes that customize their comparison behavior should follow some consistency rules, if possible: - Equality comparison should be reflexive\. In other words, identical objects should compare equal:> `x is y`implies`x == y` this isn't true for NaN\. and that's fine, but because the assumption*does*hold for pretty much everything else, design choices are made which leave NaN unaccounted for \(or at least, give NaN very weird behavior\)\. to be clear, i'm not necessarily saying that python's behavior here is*wrong*, or that this optimization is bad because NaN behaves weirdly\. but i think it's at least worth pointing out\. ## case study 2: lua numerical for\-loops in lua look like this: ``` for i = 1, 10 do stuff() end ``` the way this is supposed to work is that`i`starts with the initial value \(here that's 1\), and each iteration it increments by the step \(optional third operand; defaults to 1\), until the new value is greater than the limit \(10\), at which point the loop terminates\. so this loop iterates 10 times, from 1 to 10 \(inclusive\)\. so what happens when NaN \(`0/0`\) is thrown into the mix? here's the behavior in the reference implementation \(puc\-rio lua\): ``` -- executes once for i = 0/0, 10 do print(i) -- nan end -- executes once for i = 0/0, 0/0 do print(i) -- nan end -- never executes for i = 1, 10, 0/0 do print(i) end -- executes once for i = 10, 1, 0/0 do print(i) -- 10.0 end ``` read through that code and try to figure out what's going on\. every single one of those results is extremely unintuitive\. here's what's happening: the first two only execute once because the for\-loop check is implemented as`limit < init`, but subsequent iterations check`idx <= limit`\. so NaN passes the first test, but not the second test\. the latter two examples show that NaN is always treated as negative when used as the step, even if you do`math\.abs\(0/0\)`\. this is because the check for whether the step is positive is`0 \> step`, which is of course always false for NaN\. none of this is documented btw, so this is likely a genuine oversight\. but that's really interesting: using NaN causes the specific comparison operator used by the implementation to leak into the interpreter's behavior\. the simple solution here is to just check for and disallow NaN in numerical for\-loops\. lua already raises an error when 0 is used as the step, so it's interesting that there's no such check for NaN\. ## conclusion i guess Chris Siebenmmann also previously wrote about the[weird behavior of NaNs as map keys in go](https://utcc.utoronto.ca/~cks/space/blog/programming/GoNaNsAsMapKeys), so i guess that makes 3 case studies of NaN's weirdness\. NaN is weird\. and that means other things behave weirdly with it\.

Similar Articles

i've been thinking about null pointers

Lobsters Hottest

The author explores the concept of null pointers, discussing their semantic role and proposing that other invalid pointer values could be used to store additional information, akin to NaN boxing.

log is non-monotonous in PHP and Lua

Lobsters Hottest

PHP and Lua's log function can produce non-monotonic results due to using different algorithms for base 10 (log10) and other bases (ln), leading to counterintuitive behavior where a larger base yields a larger logarithm.

Floor and Ceil versus Denormals on CPU and GPU

Hacker News Top

Explores the behavior of floor and ceil functions when applied to denormalized floating-point numbers, highlighting differences between CPU and GPU implementations and potential pitfalls.

Excessive nil pointer checks in Go

Lobsters Hottest

A blog post discussing how excessive nil pointer checks in Go can indicate unclear code and poor error handling practices, advocating for early failure and explicit modeling of unavailable dependencies.