Debugging walkthrough: Access violation on nonsense instruction, episode 3
Summary
The article details a debugging case where employees experienced memory write errors due to an access violation from a failed code injection attempt in Windows systems.
<p>A customer reported that their employees were randomly getting “memory write errors”.</p>
<p>This was their way of interpreting <a title="Why does the access violation error message put the operation in quotation marks, redux" href="https://devblogs.microsoft.com/oldnewthing/20151106-00/?p=92012"> the error message</a></p>
<blockquote class="q"><p>The instruction at “XX” referenced memory at “YY”. The memory could not be “written”.</p></blockquote>
<p>Okay, so what we have here is an access violation.</p>
<p>The strange thing was that this access violation was happening across multiple unrelated programs, rather than all occurring in a single program or family of programs. So there is some sort of broader problem here, rather than just a single buggy program.</p>
<p>Opening one of the crash dumps shows this:</p>
<pre>eax=0013d354 ebx=0049a000 ecx=009e9a9f edx=03111160 esi=009e9aa0 edi=009e9aa0
eip=009e9aa7 esp=003cfda4 ebp=003cfdb0 iopl=0 nv up ei pl nz ac pe cy
cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00010217
WerFault!wmainCRTStartup+0x7:
009e9aa7 0000 add byte ptr [eax],al ds:002b:0013d354=??
0:000>
</pre>
<p>That <code>add byte ptr [eax], al</code> should immediately tell you that we are not executing valid code: <a title="Debugging walkthrough: Access violation on nonsense instruction, episode 2" href="https://devblogs.microsoft.com/oldnewthing/20150313-00/?p=44473"> It is the instruction that you get if you try to execute zeroes</a>. You can see the zeroes in the second column.</p>
<p>All of the crash dumps look like this, just with different process names.</p>
<p>Let’s disassemble from the start of the function to see how we got here.</p>
<pre>0:000> u .-7
WerFault!wmainCRTStartup:
009e9aa0 90 nop
009e9aa1 49 dec ecx
009e9aa2 ba6011f102 mov edx,2F11160h
009e9aa7 0000 add byte ptr [eax],al ← died here
009e9aa9 0000 add byte ptr [eax],al
009e9aab 41 inc ecx
009e9aac ffe2 jmp edx
009e9aae cc int 3
</pre>
<p>This doesn’t look like the proper start of a function.</p>
<p>I mean, one clue is that it starts with a single-byte <code>nop</code>, rather than a <a title="Why do Windows functions all begin with a pointless MOV EDI, EDI instruction?" href="https://devblogs.microsoft.com/oldnewthing/20110921-00/?p=9583"> <code>mov edi, edi</code>, as is customary for x86-32 code</a>.¹</p>
<p>And then of course there is the chunk of four <code>00</code> bytes in the middle of the instruction stream.</p>
<p>What I thought was interesting is that if you take out those four <code>00</code> bytes, then the <code>dec ecx</code> and <code>inc ecx</code> cancel out, and what’s left looks like a detour: It loads an absolute address into <code>edx</code> and then jumps to it.</p>
<p>This looks to me like a failed attempt to detour the function. The next step is to try to figure out what they were trying to do.</p>
<p>Well, it looks like it’s trying to detour to a function at <code>0x02f11160</code>, so let’s see what the debugger can tell us about that.</p>
<pre>0:000> !address 0x2f11160
Usage: <unknown>
Base address: 02f11000
End address: 02f12000
Region Size: 00001000 (4.000 kB)
State: 00001000 MEM_COMMIT
Protect: 00000020 PAGE_EXECUTE_READ
Type: 00020000 MEM_PRIVATE
Allocation Base: 02f10000
Allocation Protect: 00000004 PAGE_READWRITE
</pre>
<p>So this is a mystery 4KB allocation of executable memory.</p>
<p>Maybe there are some interesting strings in that memory block.</p>
<pre>0:000> !strings 02f11000 02f12000
02f11080 --------
02f11148 ----------------
02f11472 C:\Program Files\Common Files\Contoso\injcore.dll
02f11545 IIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIII
</pre>
<p>Okay, well, that path to a DLL kind of catches them red-handed. I bet the “inj” stands for “injection”. But what were they trying to do?</p>
<p>I figured, “Hm, the extra four zero bytes come right after the constant they were trying to load, so if I change the <code>mov edx</code> to a <code>mov rdx</code>, it would be the upper half of a 64-bit constant, and then this would look okay again.</p>
<p>Now, this is a 32-bit process (evidenced by the 32-bit instruction pointer), so there is no <code>mov rdx</code> instruction. That instruction requires a 64-bit process.</p>
<p>But wait, what if they got confused and <i>thought</i> it was a 64-bit process?</p>
<p>Let’s disassemble these bytes as if they had been injected into a 64-bit process.</p>
<p>The way I do this is to load up a sacrificial 64-bit debug session and just patch into it the bytes that I want to study. For poetic irony, I will load the 64-bit <tt>WerFault.exe</tt> into the debugger as a dump file.</p>
<pre>C:\> windbgx -z C:\Windows\System32\WerFault.exe
Executable search path is:
ModLoad: 00000001`40000000 00000001`400a1000 C:\Windows\System32\WerFault.exe
WerFault!wmainCRTStartup:
00000001`40002480 sub rsp,28h
0:000> eb . 90 49 ba 60 11 f1 02 00 00 00 00 41 ff e2 cc
0:000> u .
WerFault!wmainCRTStartup
00000001`40002480 nop
00000001`40002481 mov r10,2F11160h
00000001`4000248b jmp r10
00000001`4000248e int 3
</pre>
<p>Okay, now it makes much more sense. This is a 64-bit detour that loads an absolute jump target into a 64-bit register (<code>r10</code>) and then jumps to it.</p>
<p>They injected 64-bit code into a 32-bit process!</p>
<p>This also explains why the crashes are sporadic: The customer’s employees run 64-bit processes most of the time, but on occasion, something will run a 32-bit process, and those are the ones that are crashing.</p>
<p>Upon further discussion with the customer, we learned that Contoso is an anti-malware program that they use. <a title="Tricks from product support: We're not smart enough to debug the problem, can you help us?" href="https://devblogs.microsoft.com/oldnewthing/20241203-00/?p=110601"> We advised them to disable it temporarily</a> to confirm that it was the source of the problem, but they didn’t want to disable their anti-malware software.</p>
<p>Okay, so we advised them to check with the vendor to see if an update is available. They were resistant to changing their anti-malware software without first putting it through their internal validation. They considered this a Windows problem, and they demanded a Windows solution.</p>
<p>We are still working to convince the customer that they need to re-evaluate their anti-malware software.²</p>
<p>¹ Even if this were a 64-bit process, <a title="Why don't Windows functions begin with a pointless MOV EDI,EDI instruction on x86-64?" href="https://devblogs.microsoft.com/oldnewthing/20221109-00/?p=107373"> Windows components don’t begin functions with a single-byte <code>nop</code> or any other single-byte instruction</a>. It would use a two-byte <code>nop</code> if it uses one at all.</p>
<p>² This is a downside of communicating with the customer through a customer liaison: My colleague explained that relaying this level of detail through a customer liaison who is not sufficiently technical to be familar with debugging puts us at a disadvantage because the liaison can’t stand up to the customer pushback. This is a case where we may have to let the engineers talk directly to the customer.</p>
<p>The post <a href="https://devblogs.microsoft.com/oldnewthing/20260925-00/?p=112731/">Debugging walkthrough: Access violation on nonsense instruction, episode 3</a> appeared first on <a href="https://devblogs.microsoft.com/oldnewthing">The Old New Thing</a>.</p>
View Cached Full Text
Cached at: 09/26/26, 02:40 PM
# Debugging walkthrough: Access violation on nonsense instruction, episode 3 - The Old New Thing
Source: [https://devblogs.microsoft.com/oldnewthing/20260925-00/?p=112731/](https://devblogs.microsoft.com/oldnewthing/20260925-00/?p=112731/)
A customer reported that their employees were randomly getting “memory write errors”\.
This was their way of interpreting[the error message](https://devblogs.microsoft.com/oldnewthing/20151106-00/?p=92012)
> The instruction at “XX” referenced memory at “YY”\. The memory could not be “written”\.
Okay, so what we have here is an access violation\.
The strange thing was that this access violation was happening across multiple unrelated programs, rather than all occurring in a single program or family of programs\. So there is some sort of broader problem here, rather than just a single buggy program\.
Opening one of the crash dumps shows this:
```
eax=0013d354 ebx=0049a000 ecx=009e9a9f edx=03111160 esi=009e9aa0 edi=009e9aa0
eip=009e9aa7 esp=003cfda4 ebp=003cfdb0 iopl=0 nv up ei pl nz ac pe cy
cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00010217
WerFault!wmainCRTStartup+0x7:
009e9aa7 0000 add byte ptr [eax],al ds:002b:0013d354=??
0:000>
```
That`add byte ptr \[eax\], al`should immediately tell you that we are not executing valid code:[It is the instruction that you get if you try to execute zeroes](https://devblogs.microsoft.com/oldnewthing/20150313-00/?p=44473)\. You can see the zeroes in the second column\.
All of the crash dumps look like this, just with different process names\.
Let’s disassemble from the start of the function to see how we got here\.
```
0:000> u .-7
WerFault!wmainCRTStartup:
009e9aa0 90 nop
009e9aa1 49 dec ecx
009e9aa2 ba6011f102 mov edx,2F11160h
009e9aa7 0000 add byte ptr [eax],al ← died here
009e9aa9 0000 add byte ptr [eax],al
009e9aab 41 inc ecx
009e9aac ffe2 jmp edx
009e9aae cc int 3
```
This doesn’t look like the proper start of a function\.
I mean, one clue is that it starts with a single\-byte`nop`, rather than a[`mov edi, edi`, as is customary for x86\-32 code](https://devblogs.microsoft.com/oldnewthing/20110921-00/?p=9583)\.¹
And then of course there is the chunk of four`00`bytes in the middle of the instruction stream\.
What I thought was interesting is that if you take out those four`00`bytes, then the`dec ecx`and`inc ecx`cancel out, and what’s left looks like a detour: It loads an absolute address into`edx`and then jumps to it\.
This looks to me like a failed attempt to detour the function\. The next step is to try to figure out what they were trying to do\.
Well, it looks like it’s trying to detour to a function at`0x02f11160`, so let’s see what the debugger can tell us about that\.
```
0:000> !address 0x2f11160
Usage: <unknown>
Base address: 02f11000
End address: 02f12000
Region Size: 00001000 (4.000 kB)
State: 00001000 MEM_COMMIT
Protect: 00000020 PAGE_EXECUTE_READ
Type: 00020000 MEM_PRIVATE
Allocation Base: 02f10000
Allocation Protect: 00000004 PAGE_READWRITE
```
So this is a mystery 4KB allocation of executable memory\.
Maybe there are some interesting strings in that memory block\.
```
0:000> !strings 02f11000 02f12000
02f11080 --------
02f11148 ----------------
02f11472 C:\Program Files\Common Files\Contoso\injcore.dll
02f11545 IIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIII
```
Okay, well, that path to a DLL kind of catches them red\-handed\. I bet the “inj” stands for “injection”\. But what were they trying to do?
I figured, “Hm, the extra four zero bytes come right after the constant they were trying to load, so if I change the`mov edx`to a`mov rdx`, it would be the upper half of a 64\-bit constant, and then this would look okay again\.
Now, this is a 32\-bit process \(evidenced by the 32\-bit instruction pointer\), so there is no`mov rdx`instruction\. That instruction requires a 64\-bit process\.
But wait, what if they got confused and*thought*it was a 64\-bit process?
Let’s disassemble these bytes as if they had been injected into a 64\-bit process\.
The way I do this is to load up a sacrificial 64\-bit debug session and just patch into it the bytes that I want to study\. For poetic irony, I will load the 64\-bitWerFault\.exeinto the debugger as a dump file\.
```
C:\> windbgx -z C:\Windows\System32\WerFault.exe
Executable search path is:
ModLoad: 00000001`40000000 00000001`400a1000 C:\Windows\System32\WerFault.exe
WerFault!wmainCRTStartup:
00000001`40002480 sub rsp,28h
0:000> eb . 90 49 ba 60 11 f1 02 00 00 00 00 41 ff e2 cc
0:000> u .
WerFault!wmainCRTStartup
00000001`40002480 nop
00000001`40002481 mov r10,2F11160h
00000001`4000248b jmp r10
00000001`4000248e int 3
```
Okay, now it makes much more sense\. This is a 64\-bit detour that loads an absolute jump target into a 64\-bit register \(`r10`\) and then jumps to it\.
They injected 64\-bit code into a 32\-bit process\!
This also explains why the crashes are sporadic: The customer’s employees run 64\-bit processes most of the time, but on occasion, something will run a 32\-bit process, and those are the ones that are crashing\.
Upon further discussion with the customer, we learned that Contoso is an anti\-malware program that they use\.[We advised them to disable it temporarily](https://devblogs.microsoft.com/oldnewthing/20241203-00/?p=110601)to confirm that it was the source of the problem, but they didn’t want to disable their anti\-malware software\.
Okay, so we advised them to check with the vendor to see if an update is available\. They were resistant to changing their anti\-malware software without first putting it through their internal validation\. They considered this a Windows problem, and they demanded a Windows solution\.
We are still working to convince the customer that they need to re\-evaluate their anti\-malware software\.²
¹ Even if this were a 64\-bit process,[Windows components don’t begin functions with a single\-byte`nop`or any other single\-byte instruction](https://devblogs.microsoft.com/oldnewthing/20221109-00/?p=107373)\. It would use a two\-byte`nop`if it uses one at all\.
² This is a downside of communicating with the customer through a customer liaison: My colleague explained that relaying this level of detail through a customer liaison who is not sufficiently technical to be familar with debugging puts us at a disadvantage because the liaison can’t stand up to the customer pushback\. This is a case where we may have to let the engineers talk directly to the customer\.
### Category
### Topics
## Author
Raymond has been involved in the evolution of Windows for more than 30 years\. In 2003, he began a Web site known as The Old New Thing which has grown in popularity far beyond his wildest imagination, a development which still gives him the heebie\-jeebies\. The Web site spawned a book, coincidentally also titled The Old New Thing \(Addison Wesley 2007\)\. He occasionally appears on the Windows Dev Docs Twitter account to tell stories which convey no useful information\.
Similar Articles
When the Debugger Lies
The author discusses an unexpected memory value issue encountered while debugging the key management unit in Nordic's nRF54L series, explaining how debugger interactions with internal SoC components can cause such problems.
The case of the thread executing from an unloaded third-party DLL
A technical debugging story from Microsoft's Raymond Chen about a Windows Explorer crash caused by a thread executing from an unloaded third-party DLL, highlighting how developers investigate such issues.
The case of the DLL that was not present in memory despite not being formally unloaded, part 2
A technical blog post by Raymond Chen investigating a memory corruption bug where a single byte 0x01 corrupts HMODULE handles, causing DLLs to be improperly freed and leading to crashes at process termination.
The time the x86 emulator team found code so bad that they fixed it during emulation
A story from a Windows x86 emulator team about encountering a program with a fully unrolled 64KB initialization loop (65,536 instructions) and adding a special optimization to replace it with a tight loop.
Core dump epidemiology: fixing an 18-year-old bug
OpenAI engineers detail the diagnosis of seemingly impossible crashes in Rockset's C++ data infrastructure, revealing both a silent hardware corruption bug on Azure and an 18-year-old race condition in GNU libunwind, resolved through epidemiological analysis of crash data.