NetBSD GSoC 2026 Testing Compat Linux Syscall testing

# NetBSD Blog Source: [https://blog.netbsd.org/tnf/entry/gsoc2026_testing_compat_linux_2](https://blog.netbsd.org/tnf/entry/gsoc2026_testing_compat_linux_2) ## [Google Summer of Code 2026 Reports: Testing Compat Linux: Syscall testing, part 2](https://blog.netbsd.org/tnf/entry/gsoc2026_testing_compat_linux_2) September 07, 2026 posted by*Leonardo Taccari*  This report was written by Henrique Brito as part of Google Summer of Code 2026\. It is the 2nd part of a series\. You can read the 1st part at[Google Summer of Code 2026 Reports: Testing Compat Linux: Syscall testing](https://blog.netbsd.org/tnf/entry/gsoc2026_testing_compat_linux)\. ## Introduction & Description [compat\_linux\(8\)](https://man.netbsd.org/compat_linux.8)is NetBSD's kernel\-level compatibility layer for Linux binaries\. It allows running them through a thin, in\-kernel translation layer, instead of full emulation\. One of its core functions is to simulate Linux syscalls for those programs\. Until now, this functionality was not as well tested as other NetBSD components\. The goal of this project was to use Linux's own test suite,[LTP \(Linux Test Project\)](https://linux-test-project.readthedocs.io/), to verify the usability and compatibility of`compat\_linux`\. LTP is a well\-known and battle\-tested framework, already used to validate Linux behavior, so why not use it to also validate NetBSD's`compat\_linux`? ## What was done In this project, I made LTP tests compile under NetBSD, run \(as they were designed to be run\), documented some of the more relevant syscall emulations, and found some very interesting behaviors involving`compat\_linux`\. ### Compile tests under`compat\_linux` This step involved fixing and using`suse15\_gcc12`, which was recently pulled from[pkgsrc\-wip](https://www.pkgsrc.org/wip/)into pkgsrc\. Fixing it involved adding some missing RPMs packages with relevant code, including: - `linux\_glibc\_devel`: adds many Linux headers, essential to compile the low\-level LTP tests - `glibc\_devel\_static`: required to achieve static compilation with gcc ### Creating a script/package To be able to correctly run the tests on NetBSD, it was necessary to create a 'manager' script to fill in the gaps between the two operating systems\. In order to ship the script and automate the download and compilation of LTP, we decided to create a package\. Fortunately, the pkgsrc phases were very similar to what we needed: a**fetch**to download LTP, a**build**to compile it, and an**install**to install it\. The`compat\_linux\_test\_project`package isn't merged into the official pkgsrc tree yet, but you can find it on my personal fork[here](https://github.com/henriquebritoM/pkgsrc/tree/compat_linux_test_project/emulators/compat_linux_test_project)in the meantime\. ### Test Documentation For the syscalls we considered more fundamental \(as in, used more often, by more applications\), a more in\-depth analysis and testing was made\. Those tests were then cataloged and neatly documented in a[spreadsheet](https://docs.google.com/spreadsheets/d/1FJM8PhpQttSHX4nicSUxa5UAazJBpvJ7UnPuZELScLw/edit?usp=sharing)for future reference\. I also added some suggestions for eventual future fixes\. [](https://docs.google.com/spreadsheets/d/1FJM8PhpQttSHX4nicSUxa5UAazJBpvJ7UnPuZELScLw/edit?usp=sharing) ### Logs, configuration files and other GSoC project artifacts All the raw logs and related files used during the project are stored in an[online drive](https://drive.google.com/drive/folders/1Mf4EcyfIf2KeEJWxs5cqjiwY20Rpln6Y?usp=sharing), free for anyone who wants to see them\. ### The final test To finish with a better understanding of the current state of`compat\_linux`, I ran the full LTP syscall test suite on`compat\_linux`\. During this phase, many interesting and relevant results were found\. They are detailed in the 'Results' section below\. ## Challenges At first, I thought this would be a pretty straightforward project, but it turned out to be more challenging and fun than I had in mind\. During this GSoC, I encountered many small and big challenges, some of them are listed here: The first one was "how to compile LTP"\. My original suggestion was to not compile the tests on NetBSD and instead compile them statically on a Linux machine and use the resulting binary\. For many different reasons, this was not a suitable idea\. My mentor, Stephen, then suggested we could use the`suse15\_gcc12`package\. This was a great idea, and a bit funny too: bringing a compiler from Linux and, through`compat\_linux`, using it to compile tests that verify the usability of`compat\_linux`\. Fortunately, the package was already in good shape, only requiring the addition of some missing RPMs packages to make it work\. The second one was about "how to convert my script into a package for pkgsrc"\. My first approach when creating the script was to automate it as much as possible, so it would check the LTP version, download it, compile it, etc\. All in one script\. When moving to pkgsrc, I realized that many of those things were already implemented inside the framework\. So I ended up having to move my logic into pkgsrc\. Pkgsrc is a very complete and pleasant package system to work with, so I was able to navigate and use it even though I didn't have any experience with it \(with the help of my mentor, of course\)\. The third one was about "whether to compile statically or dynamically"\. Dynamic tests are smaller, but require the`suse15\_base`package to run, unlike the static ones, which don't require anything else\. So at first, it seemed that static was the way to go with the compilation, but when we checked the package sizes for both of them an issue appeared: the static package was about 5GB, while the dynamic one was 545MB\. Not very well suited for a testing package\. After some work with gcc flags, I was able to decrease the dynamic package's size to 80MB\. ## Results Running the testcases in`compat\_linux`provided many useful insights about its current state\. From my perspective, it is very complete and covers most of what is commonly needed/used\. It's only when Linux syscalls start to stray from POSIX, with Linux\-exclusive functionality or different behaviors, that`compat\_linux`starts to lack coverage, since implementing those functions would basically require 're\-creating' the syscall inside`compat\_linux`\. ### `/proc/self/maps`difference Running the`mmap04`testcase, I found that it failed due to LTP not being able to read correctly from`/proc/self/maps`\. This happened because Linux exposes addresses with hex digits of variable length, trimming away the left\-most zeros \(maintaining a minimum of 8 hex digits\), while NetBSD uses a fixed size of 16 hex digits to display the addresses\. Linux:  NetBSD:  According to[mount\_procfs\(8\) manpage](https://man.netbsd.org/mount_procfs.8), maps should be implemented "in a form like the proc filesystem as implemented in Linux", so this is a divergence in behavior\. See[thread](https://mail-index.netbsd.org/tech-kern/2026/08/01/msg031204.html)on the tech\-kern@ mailing list\. ### `writev`and`iovcnt` The emulated syscall[writev\(2\)](https://man.netbsd.org/writev.2)has a divergence in behavior, but this difference is unlike most of what we have found\. According to[POSIX](https://pubs.opengroup.org/onlinepubs/9799919799/functions/writev.html),`writev`should yield an`EINVAL`when given an`iovcnt`of 0\. This is followed by NetBSD and, by extension, its`compat\_linux`implementation\. On the other hand, Linux displays different behavior, where it doesn't recognize`iovcnt = 0`as an error, and instead returns success\. ### SMAP violation on`copy\_file\_range` This syscall emulation was implemented in GSoC 2024, but only now is being incorporated into the mainstream versions of NetBSD, starting with NetBSD 11\. The issue is a simple dereference of a user space pointer inside kernel space, without the proper handling, which triggers a SMAP violation, causing a kernel panic\.  This only happens when`off\_in`or`off\_out`is not`NULL`, an edge case that was not caught during implementation\. Lets take a look at the[code](https://github.com/NetBSD/src/blob/5a7765109d6feb3a897b349b8d58572a0b7812f2/sys/compat/linux/common/linux_file.c#L1126-L1132)responsible for this: ``` if ((SCARG(uap, off_in) != NULL && *SCARG(uap, off_in) < 0) || <- dereference (SCARG(uap, off_out) != NULL && *SCARG(uap, off_out) < 0) || <- dereference vattr_in.va_type != VREG || vattr_out.va_type != VREG) { error = EINVAL; DPRINTF("%s: Invalid offset or file type\n", __func__); goto out; } ``` Here,`\*SCARG\(uap, off\_in/off\_out\)`is attempting to dereference a user\-space pointer from kernel space\. I think this is a good example of why we should use LTP to verify the compatibility and functionality of new syscall implementations in`compat\_linux`\. As of writing, the`copy\_file\_range`fix hasn't landed in the NetBSD tree yet, but you can find the patch[here](https://gist.github.com/henriquebritoM/87b3b041bb5622e095b6b138fc9b5b28)in the meantime\. ### Killing userland via`rt\_sigqueueinfo`;\) Perhaps the funniest thing that I have encountered in this project\. This was found while testing the`rt\_sigqueueinfo`syscall\. I noticed that something strange was happening with it, similar to`copy\_file\_range`, but not quite\. When running the LTP tests, my SSH session would disconnect, the script would stop, and everything else would halt, only coming back to normal after a reboot\. At first I thought this was a kernel panic, but nothing seemed to be happening: nothing in dmesg, no panic screen, nothing\. So I had to check the`compat\_linux`implementation of`rt\_sigqueueinfo`and the LTP testcases\. The[testcase](https://github.com/linux-test-project/ltp/blob/3c0271a307bd765c2fd652c325762a90ddf5d46f/testcases/kernel/syscalls/rt_sigqueueinfo/rt_sigqueueinfo02.c#L38-L39)in question that was causing trouble used a`tgid`of \-1, which was meant to represent a`tgid`that doesn't exist\. The[compat\_linux implementation](https://github.com/NetBSD/src/blob/5a7765109d6feb3a897b349b8d58572a0b7812f2/sys/compat/linux/common/linux_signal.c#L639-L658), unlike Linux's, used[kill\(2\)](https://man.netbsd.org/kill.2)as part of its logic\. Connecting the dots, I noticed that`kill\(2\)`has a special use case for`\-1`, which is used to broadcast the signal\. It just so happens that the testcase, combined with the syscall emulation, was killing all processes as a side effect\. We are still discussing what should be done in this regard\. ## About the script As mentioned above, properly running the LTP tests on NetBSD required some sort of 'glue' to work out the differences between both operating systems\. This was solved by creating a dedicated script\. Installed as`compat\_linux\_test\_project`, it can run the full syscall suite or a chosen subset, produce structured, per\-syscall logs, and compare a fresh run against a baseline set of logs to highlight regressions, fixes, and other changes\. Testcases known to hang or panic the kernel are skipped by default, though they can still be run explicitly when needed\. ### How to install Since the package isn't merged into the official pkgsrc tree yet, you can build it directly from my fork: ``` git clone https://github.com/henriquebritoM/pkgsrc.git cd pkgsrc git checkout compat_linux_test_project cd emulators/compat_linux_test_project make install ``` ### How to use One of the most common use cases is to run the tests for a syscall \(or a group of them\) To do this, you can just do ``` compat_linux_test_project -s readv,writev,open,close -d output_dir ``` This will run all testcases related to the syscalls`readv`,`writev`,`open`and`close`and store the resulting logs inside`output\_dir`\. The contents of output dir should be something similar to: ``` # tree -F output_dir/ output_dir/ |-- close/ | |-- close01.log | `-- close02.log |-- open/ | |-- open01.log | |-- open02.log | |-- open03.log | |-- open04.log | |-- open06.log | |-- open07.log | |-- open08.log | |-- open09.log | |-- open10.log | |-- open11.log | |-- open12.log | |-- open13.log | |-- open14.log | `-- open15.log |-- readv/ | |-- readv01.log | `-- readv02.log `-- writev/ |-- writev01.log |-- writev02.log |-- writev03.log |-- writev05.log |-- writev06.log `-- writev07.log 4 directories, 24 files ``` For more tips, you can use the help option to list the full package use description: ``` compat_linux_test_project --help ``` Expected output: ``` NAME compat_linux_test_project - package to run LTP tests on NetBSD SYNOPSIS compat_linux_test_project [options] DESCRIPTION compat_linux_test_project is a package to download, build, and run the Linux Test Project (LTP) test suite on NetBSD, intended for testing the compat_linux compatibility layer. It automates fetching, building, and executing LTP tests, helping to identify missing syscalls, behavioral differences, and other issues in NetBSD's Linux compatibility subsystem. If no --syscall option is passed, the default behavior is to test every possible syscall. This uses the LTP runtest/syscalls file to determine what to test and in which order. This means that the syscall name cannot be correctly-determined sometimes, resulting in suboptimal output in the logs. This also takes some time and is not recommended, except in some niche cases. All tests belong to Linux Testing Project, this package only ports them to make them run on NetBSD. Support the Linux Test Project by checking their official website: https://github.com/linux-test-project/ltp OPTIONS -d, --output-dir output_dir Directory where output is written. In test-run mode (default): new logs are stored here, overwriting any older logs already there. In comparison mode (-c): the categorized comparison results (compared_logs/) are stored here instead. -s, --syscall syscall1[,syscall2,...] A comma-separated list of which syscalls to test. The script will search for them using the LTP runtest file, so the name may be slightly different from the syscall name (though this is unusual). WARNING: some syscalls are excluded from the default (full) run because they are known to cause problems (e.g. kernel panics or hangs, see check_for_testcase_issue() in the script). If you explicitly list one of these syscalls here, the script will still run it, but will print a warning first. -r, --reproducible Sets the LTP environment variable LTP_REPRODUCIBLE_OUTPUT to 1. According to LTP documentation, this "suppress printing TINFO and TDEBUG messages and discards the actual content of the other messages printed by the test (suitable for a reproducible output)." This flag should be set if the logs are meant to be compared. -c, --compare logs_dir Compares logs_dir against a baseline, highlighting their differences. For a consistent comparison, both sets of logs should have been gathered using the 'reproducible' mode ('-r' flag). The baseline defaults to the reference logs shipped with the package, or to whatever is passed via '-b'. If '-s' is also passed, the comparison is restricted to the syscalls listed there. The comparison is done as follows: Checks for tests that are new, tests that are no longer present, and tests whose result changed. Changed results are further split into regressions (e.g. PASS -> FAIL), fixes (e.g. FAIL -> PASS) and other changes (e.g. shift in skipped or warnings). Some testcases are known to depend on functionality with no equivalent in NetBSD and are marked as wont_test. Each category is stored in its own subdirectory, mirroring the syscall/testcase layout of the compared logs: compared_logs/ |-- changed/ |-- fixed/ |-- new/ |-- regressed/ |-- removed/ | |-- syscall_a/ | |- testcase01.log |-- wont_test/ Each generated testcase01.log file is self-contained: - For 'new'/'removed' testcases, it has a short status line followed by the full content of the one log that exists (current or reference, respectively). - For 'changed'/'fixed'/'regressed' testcases, it has the reference and current pass/failed/broken/skipped/warnings counts, followed by the full content of both the reference and the current log. This means you normally do not need to go dig through the original sys_logs/ or reference logs directories to investigate a specific result; everything relevant is already copied into diff_logs/. The comparison assumes the testcases output follows the standard and new LTP structure. This is not true for every testcase, in this case, the comparison has undefined behavior. -b, --baseline logs_dir Directory with the logs to use as baseline. Only meaningful together with '-c'. Defaults to the reference logs shipped with the package. --fail-on-regression Only meaningful together with -c. When this flag is set, the script exits with a non-zero status if any test is found in compared_logs/regressed/. This is intended for use by automated testing. Has no effect without -c. -h, --help Shows this message ``` ### Examples Let's say you want to test the read and pipe syscalls: You can use: ``` compat_linux_test_project -s read,pipe -d /tmp/pipe_and_read ``` Expected output \(trimmed\): ``` Preparing block device required by some testcases... Mounting block device vnd0 from image /usr/pkg/libexec/compat_linux_test_project/test.img... Starting test run. This may take a while... Output will be stored in: /tmp/pipe_and_read ==> Testing syscall: read syscall_test: read01 [snip] Summary: passed 1 failed 0 broken 0 skipped 0 warnings 0 [snip] ==> Testing syscall: pipe syscall_test: pipe01 [snip] Summary: passed 1 failed 0 broken 0 skipped 0 warnings 0 [snip] Unmounting block device vnd0... Test run finished. Logs stored in: /tmp/pipe_and_read ``` And then you can check out the results of each testcase by navigating through the output dir: ``` # cat /tmp/pipe_and_read/pipe/pipe01.log ---- compat_linux_test_project log header ---- date: Fri, 4 Sep 2026 00:33:38 +0000 netbsd_version: 11.0 netbsd_build: NetBSD 11.0 (GENERIC) #0: Thu Jul 30 15:23:12 UTC 2026 [email protected]:/usr/src/sys/arch/amd64/compile/GENERIC arch: amd64 ltp_version: 20260529 script_invocation: "/usr/pkg/bin/compat_linux_test_project '-s' 'read,pipe' '-d' '/tmp/pipe_and_read'" syscall_tested: pipe reproducible: 0 -------------------------------------------------- [snip] pipe01.c:48: TPASS: pipe() functionality is correct Summary: passed 1 failed 0 broken 0 skipped 0 warnings 0 ``` ## Conclusion This GSoC project started with the idea of bringing the LTP testing framework to`compat\_linux`and NetBSD, and along the way grew into a proper pkgsrc package capable of bridging the gaps between both operating systems, an in\-depth documentation and analysis of some of the most commonly used syscalls, and a clear view of exactly where`compat\_linux`diverges from a native Linux environment\. Two of those divergences turned out to be rather worth\-noting issues: a kernel panic in`copy\_file\_range`, and a way to kill userland via`rt\_sigqueueinfo`\. With this, I think we have a good reminder of how important a good and thorough test suite really is, even for a compatibility layer that has been around for years\. I'd like to thank my mentor, Stephen Borrill, for his guidance and patience throughout the project, and for pushing me toward better solutions whenever I got stuck\. Thanks also to Christos, who helped to fix the`copy\_file\_range`panic, to Leonardo for his advice and excellent management of many GSoC\-related subjects, and to all the other mentors and students who have been working alongside me and NetBSD during this GSoC\. It's been a genuinely fun summer of digging into two operating systems at once\. I've learned a lot about how they really work under the hood, in places where it isn't obvious\. And it sure did help me pass my Operating Systems class haha\. \[[0 comments](https://blog.netbsd.org/tnf/entry/gsoc2026_testing_compat_linux_2#comment-form)\]