MFC message handling

Goran, a commenter at Raymond's blog, asks:

As one of those people who only have a hammer :-( ... It looks like MFC just told us off with this? WM_GETDLGCODE has meaningfull wParam and lParam, but CWnd::OnGetDlgCode has none of that. Anyone used OnGetDlgCode? Help?

The specific context is handling the return key - Windows tells you what key caused WM_GETDLGCODE to be fired in the wParam parameter, but MFC doesn't pass that information on to the OnGetDlgCode handler.

Something that often catches people out with MFC is the thought that you have to use the specific handler that MFC declares for a certain message. In fact, it's simply a shortcut - ON_WM_GETDLGCODE in the message map causes the message map handling to go down a certain branch that happens to call the handler function with no parameters.

Instead, you can simply use ON_MESSAGE in the message map, using WM_GETDLGCODE and declaring the function as taking WPARAM, LPARAM and returning LRESULT. Do make sure you don't also include ON_WM_GETDLGCODE.

The message map macros themselves simply produce a table of message number ranges and handler function pointers. The MFC WindowProc scans the table to find a handler for a given message number, then - based on an enumeration value set by the message map macro which indicates the argument types - decodes wParam and lParam, then calls the handler with the appropriate arguments.

Many of the predefined helper message map macros, which usually don't take any arguments, define the function name that you're expected to implement. ON_MESSAGE allows you to specify the function name (as you'd expect, as a generic macro).

The Visual Studio Wizards have always simplified the full power of message maps, not supporting custom message numbers (which you have to use ON_MESSAGE for) and not supporting ranges of command IDs (ON_COMMAND_RANGE, ON_NOTIFY_RANGE, etc). Few people even seem to realise that you can use the same handler function for more than one message map macro, for example if you have discontinuous ranges of control IDs.

Four-and-a-half months

…that’s how long I was running Windows 7 RC1 without switching back to my hard disk with Windows Vista. The System event log shows I last booted it on 31 May 2009.

This is practically the last action I’m going to take before wiping this disk – the Vista disk – and installing Windows 7 RTM. Just a little check here and there to find anything that I might not have backed up when transferring over to 7 RC1, then it’s formatting time.

The only thing I’ve missed? The extra 200GB of disk space!

In retrospect Windows Vista has not been a bad operating system, but it had some pretty rough edges, particularly at the start with poor hardware support. It was a rush 18–month project, after the initial three-year disaster of Longhorn, and in some areas it shows. Windows 7 has had a further two-and-a-half years to bake, and a further two-and-a-half years of device driver and hardware development; it’s not fundamentally different but feels more complete.

Technical details on Humax problem

I thought I’d break this into another post so that the fix is easier to understand for non-technical people.

Comparing the old and new driver source shows that the definition of the libusb_request structure had changed. Visual inspection suggested that this was probably the source of the compatibility issue, when using the old DLL with the new driver.

When using the new DLL, the following error occurs: “The ordinal 76 could not be located in the dynamic link library libusb0.dll.” Inspecting the program and DLL shows that while the DLL’s entry points are exported by name, the program links to the DLL by ordinal. This is peculiar; while it’s a little faster than string-matching, and it saves a little space, care has to be taken to keep the ordinals the same between versions of a DLL. Reading the .def file shows that the ordinals weren’t specified, and that numerous functions were added and removed between the two versions.

My fix is to add specific ordinal declarations to the .def file for functions that humax.exe is using, so that the correct functions are used. I did this, and recompiled the DLL with the Windows Vista WDK (Windows Driver Kit). Success! Or, at least, as much success as I’ve had with Windows XP 32–bit; the box still has a tendency to hang during file transfer.

File Attachment: libusb0.def (1 KB)

Downloading from your Humax PVR9200T on Windows x64

The Humax PVR9200T is a pretty good Freeview/digital terrestrial personal video recorder. I’ve had mine for a few years now. Over time, the disk fills up with some programmes you want to keep. It’s possible to copy recorded programmes from the box to your computer, over USB.

Unfortunately the supplied eLinker software is not highly-regarded. As a result some replacements have been written, such as Humax Media Controller. However, the driver supplied only works on 32–bit Windows.

The USB library and driver used are a Win32 port of libusb, a driver which makes it possible to write your USB transfers fully in user-mode. A later version (0.1.12.2) has both 32– and 64–bit versions of the driver. However, the updated driver doesn’t work with the libusb0.dll supplied with Media Controller 1.05, and the program won’t run if you overwrite it with the new DLL.

I’ve rebuilt the new DLL so it will work with the program. The attached ZIP file contains the new version and the new drivers. To use, extract the contents of the folder. Copy the libusb0.dll in the top-level folder to the Humax Media Controller folder, overwriting the current version. Then, use Update Driver in Device Manager to install the driver from the Driver folder.

Windows Vista introduced a driver signing requirement for x64, and this driver isn’t signed. To get it to load on Windows Vista and Windows 7, reboot, and press F8 until the boot menu appears. From this menu, select Disable Driver Signature Enforcement, then continue to boot normally.

Hopefully this will help other people with the x64 versions of Windows!

File Attachment: HumaxPVR.zip (78 KB)

ADSL Router suggestions?

OK, I think my router has become too hackable. The reliability has hit the toilet, and I’ve had two messages from my ISP warning me that there has been odd traffic. The first time I got a warning message I think I was using BitTorrent at the time, but the second was yesterday at around lunchtime – when my computers were all off and I was at work.

So, list of demands requirements:

• Wired router with a few Ethernet ports
• 802.11g WiFi with at least WPA encryption (can’t do WPA2 on the handheld)
• Xbox Live compatible
• DHCP implementation that works
• DNS server address passthrough from DHCP server (i.e. I want to use my ISP’s DNS servers, not have the box do DNS for me)
• ADSL2+ compatible, for whenever that gets rolled out by Demon
• PPPoA since this is of course a UK requirement
• A firewall that can be completely switched off or is actually configurable intelligibly
• Not excessively burdened with blinkenlights
• Will be updated regularly with patches for whatever its OS is

That last point is critical. There are way too many routers out there that run Linux and aren’t patched. D-Link haven’t produced an update since last November and even that’s not officially official, you have to find it on the FTP site. The last official update is September 2006.

I don’t need 802.11n as a) it’s still a draft, b) none of my other equipment needs it, and c) 54Mbps is already sufficiently fast to access the Internet. I don’t do much copying between devices and I wouldn’t do it when they were running on battery anyway.

I’m half tempted to try building a Windows CE kernel for it myself.

How do I see what the JIT compiled, anyway?

You use the SOS debugger extension’s !U command.

Yeah, that’s not very helpful. Here’s how to get from that terse description to something useful.

As an example, let’s use the following trivial program:

using System;

public class Simple
{
   public static int Main()
   {
       Console.WriteLine( "Hello World!" );
       Console.ReadLine();
       
       return 0;
   }
}

Dump this snippet into Notepad, save as simple.cs file and compile it. We want to see what it’ll be like when the user runs it, so we’ll do a release build. From the VS2005 Command Prompt, run csc /debug:pdbonly /o+ simple.cs. (I find it’s quicker to do command-line compiles for simple test programs than firing up Visual Studio. It’s not as if we’re designing a Form.) If you’re doing this on a 64–bit computer, like me, add /platform:x86 so we get consistent output across platforms. (Marking as x86 means that the executable headers are set to indicate a 32–bit program. The IL is the same whatever you set here, but if you leave it as anycpu, the default, the .NET Framework will create a 64–bit process which obviously gives different output.)

Why call Console.ReadLine? Well, the JIT compiler is helpful (or at least it thinks it is). If you start a program under the debugger, it will generate less optimized code because it thinks you want to debug it, but that changes the code from what the user will see. So I want to add a stop point in the program so I can attach the debugger after the process has started.

The next thing you’ll need is the Debugging Tools for Windows kit. Grab the 32–bit kit – the 64–bit kit can debug 32–bit processes, but SOS (the debugger extension DLL which implements the commands we’re going to use) doesn’t appear to work. We’re going to use WinDBG, which is slightly friendlier than the other debuggers although not a lot!

Open WinDBG. Run simple.exe and, when it’s waiting for input, go to File/Attach to a Process in WinDBG. You’ll see a list of other processes (and if running Windows Vista with UAC enabled, or on XP as a standard user, a load of access denied errors). Select simple.exe from the list and hit OK. WinDBG automatically stops once you’ve attached to a process so you can start manipulating the program straight away. This is the output I got:

Microsoft (R) Windows Debugger Version 6.8.0004.0 X86
Copyright (c) Microsoft Corporation. All rights reserved.

*** wait with pending attach
Symbol search path is: C:\Windows;C:\Windows\System32;C:\Windows\SysWOW64;SRV*C:\WebSymbols*http://msdl.microsoft.com/download/symbols
Executable search path is: 
ModLoad: 008c0000 008c8000   C:\Users\Mike\Documents\Programming\simple.exe
ModLoad: 776a0000 77800000   C:\Windows\SysWOW64\ntdll.dll
ModLoad: 79000000 79046000   C:\Windows\system32\mscoree.dll
ModLoad: 75b70000 75c80000   C:\Windows\syswow64\KERNEL32.dll
ModLoad: 76ce0000 76da6000   C:\Windows\syswow64\ADVAPI32.dll
ModLoad: 76e10000 76f00000   C:\Windows\syswow64\RPCRT4.dll
ModLoad: 75850000 758b0000   C:\Windows\syswow64\Secur32.dll
ModLoad: 75a80000 75ad8000   C:\Windows\syswow64\SHLWAPI.dll
ModLoad: 77110000 771a0000   C:\Windows\syswow64\GDI32.dll
ModLoad: 77040000 77110000   C:\Windows\syswow64\USER32.dll
ModLoad: 76c30000 76cda000   C:\Windows\syswow64\msvcrt.dll
ModLoad: 75d00000 75d60000   C:\Windows\system32\IMM32.DLL
ModLoad: 76940000 76a08000   C:\Windows\syswow64\MSCTF.dll
ModLoad: 75a70000 75a79000   C:\Windows\syswow64\LPK.DLL
ModLoad: 759c0000 75a3d000   C:\Windows\syswow64\USP10.dll
ModLoad: 746c0000 7485e000   C:\Windows\WinSxS\x86_microsoft.windows.common-controls_6595b64144ccf1df_6.0.6001.18000_none_5cdbaa5a083979cc\comctl32.dll
ModLoad: 79e70000 7a3ff000   C:\Windows\Microsoft.NET\Framework\v2.0.50727\mscorwks.dll
ModLoad: 75390000 7542b000   C:\Windows\WinSxS\x86_microsoft.vc80.crt_1fc8b3b9a1e18e3b_8.0.50727.1434_none_d08b6002442c891f\MSVCR80.dll
ModLoad: 75e30000 7693f000   C:\Windows\syswow64\shell32.dll
ModLoad: 771b0000 772f4000   C:\Windows\syswow64\ole32.dll
ModLoad: 790c0000 79bf6000   C:\Windows\assembly\NativeImages_v2.0.50727_32\mscorlib\5b3e3b0551bcaa722c27dbb089c431e4\mscorlib.ni.dll
ModLoad: 79060000 790b6000   C:\Windows\Microsoft.NET\Framework\v2.0.50727\mscorjit.dll
(1dc.12b4): Break instruction exception - code 80000003 (first chance)
eax=7efaf000 ebx=00000000 ecx=00000000 edx=7770d2d4 esi=00000000 edi=00000000
eip=776b0004 esp=04c6fa00 ebp=04c6fa2c iopl=0         nv up ei pl zr na pe nc
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000246
ntdll!DbgBreakPoint:
776b0004 cc              int     3

You enter your debugging commands in the edit box at the bottom, next to the prompt 0:003>. This means we’re debugging the 0th process we’re attached to, and our commands by default apply to thread number 3. Four threads? Well, thread 0 is our main thread, thread 1 was created by the CLR’s debugging support in case we attached a debugger, thread 2 is the finalizer thread, and thread 3 was just created by WinDBG so it could stop the process safely. (You can see this by running ~* k, although you’ll need to be set up to get debugging symbols from the symbol server to get good stack traces.) The Debugging Tools debuggers automatically stop the process when you attach, so that you can start manipulating the process straight away.

The first thing we need to do is ask WinDBG to load the SOS extension. This is installed with the CLR, so we ask it to load from the same folder that mscorwks.dll (the DLL which implements the virtual machine, effectively the guts of the CLR) lives in:

0:003> .loadby sos.dll mscorwks

Now we can see what state of the managed threads are in by running !threads. Any command beginning ! comes from a debugger extension DLL, and they can be disambiguated if necessary, but they’re searched in reverse order loaded (last loaded = first searched) so anything we want will be found in SOS anyway.

Right. To see what we can do, run !help.

0:003> !help
-------------------------------------------------------------------------------
SOS is a debugger extension DLL designed to aid in the debugging of managed
programs. Functions are listed by category, then roughly in order of
importance. Shortcut names for popular functions are listed in parenthesis.
Type "!help " for detailed info on that function. 

Object Inspection                  Examining code and stacks
-----------------------------      -----------------------------
DumpObj (do)                       Threads
DumpArray (da)                     CLRStack
DumpStackObjects (dso)             IP2MD
DumpHeap                           U
DumpVC                             DumpStack
GCRoot                             EEStack
ObjSize                            GCInfo
FinalizeQueue                      EHInfo
PrintException (pe)                COMState
TraverseHeap                       BPMD 

Examining CLR data structures      Diagnostic Utilities
-----------------------------      -----------------------------
DumpDomain                         VerifyHeap
EEHeap                             DumpLog
Name2EE                            FindAppDomain
SyncBlk                            SaveModule
DumpMT                             GCHandles
DumpClass                          GCHandleLeaks
DumpMD                             VMMap
Token2EE                           VMStat
EEVersion                          ProcInfo 
DumpModule                         StopOnException (soe)
ThreadPool                         MinidumpMode 
DumpAssembly                       
DumpMethodSig                      Other
DumpRuntimeTypes                   -----------------------------
DumpSig                            FAQ
RCWCleanupList
DumpIL

To find out a little more about !U, let’s run !help U:

0:003> !help U
-------------------------------------------------------------------------------
!U [-gcinfo] [-ehinfo] <MethodDesc address> | <Code address> 

Presents an annotated disassembly of a managed method when given a MethodDesc
pointer for the method, or a code address within the method body. Unlike the
debugger "U" function, the entire method from start to finish is printed,
with annotations that convert metadata tokens to names.


...
03ef015d b901000000       mov     ecx,0x1
03ef0162 ff156477a25b     call   dword ptr [mscorlib_dll+0x3c7764 (5ba27764)] (System.Console.InitializeStdOutError(Boolean), mdToken: 06000713)
03ef0168 a17c20a701       mov     eax,[01a7207c] (Object: SyncTextWriter)
03ef016d 89442414         mov     [esp+0x14],eax

If you pass the -gcinfo flag, you'll get inline display of the GCInfo for
the method. You can also obtain this information with the !GCInfo command.

If you pass the -ehinfo flag, you'll get inline display of exception info
for the method. (Beginning and end of try/finally/catch handlers, etc.).
You can also obtain this information with the !EHInfo command.

Right, so we need the address of a MethodDesc structure, or the address of the code itself. How can we get one of these? There’s a command called DumpMD…

0:003> !help dumpmd
-------------------------------------------------------------------------------
!DumpMD 

This command lists information about a MethodDesc. You can use !IP2MD to turn 
a code address in a managed function into a MethodDesc:

0:000> !dumpmd 902f40
Method Name: Mainy.Main()
Class: 03ee1424
MethodTable: 009032d8
mdToken: 0600000d
Module: 001caa78
IsJitted: yes
m_CodeOrIL: 03ef00b8

If IsJitted is "yes," you can run !U on the m_CodeOrIL pointer to see a 
disassembly of the JITTED code. You can also call !DumpClass, !DumpMT, 
!DumpModule on the Class, MethodTable and Module fields above.

We’re getting a bit closer. Let’s try DumpModule…

0:003> !help DumpModule
-------------------------------------------------------------------------------
!DumpModule [-mt] 

You can get a Module address from !DumpDomain, !DumpAssembly and other 
functions. Here is sample output:

0:000> !dumpmodule 1caa50
Name: C:\pub\unittest.exe
Attributes: PEFile
Assembly: 001ca248
LoaderHeap: 001cab3c
TypeDefToMethodTableMap: 03ec0010
TypeRefToMethodTableMap: 03ec0024
MethodDefToDescMap: 03ec0064
FieldDefToDescMap: 03ec00a4
MemberRefToDescMap: 03ec00e8
FileReferencesMap: 03ec0128
AssemblyReferencesMap: 03ec012c
MetaData start address: 00402230 (1888 bytes)

The Maps listed map metadata tokens to CLR data structures. Without going into 
too much detail, you can examine memory at those addresses to find the 
appropriate structures. For example, the TypeDefToMethodTableMap above can be 
examined:

0:000> dd 3ec0010
03ec0010  00000000 00000000 0090320c 0090375c
03ec0020  009038ec ...

This means TypeDef token 2 maps to a MethodTable with the value 0090320c. You 
can run !DumpMT to verify that. The MethodDefToDescMap takes a MethodDef token 
and maps it to a MethodDesc, which can be passed to !DumpMD.

There is a new option "-mt", which will display the types defined in a module,
and the types referenced by the module. For example:

0:000> !dumpmodule -mt 1aa580
Name: C:\pub\unittest.exe
......
MetaData start address: 0040220c (1696 bytes)

Types defined in this module

      MT    TypeDef Name
------------------------------------------------------------------------------
030d115c 0x02000002 Funny
030d1228 0x02000003 Mainy

Types referenced in this module

      MT    TypeRef Name
------------------------------------------------------------------------------
030b6420 0x01000001 System.ValueType
030b5cb0 0x01000002 System.Object
030fceb4 0x01000003 System.Exception
0334e374 0x0100000c System.Console
03167a50 0x0100000e System.Runtime.InteropServices.GCHandle
0336a048 0x0100000f System.GC

We still need a Module to pass to this command, but we’re getting there. Maybe DumpDomain can help us? I’ll skip the help text this time.

0:003> !DumpDomain
--------------------------------------
System Domain: 7a3bc8b8
LowFrequencyHeap: 7a3bc8dc
HighFrequencyHeap: 7a3bc934
StubHeap: 7a3bc98c
Stage: OPEN
Name: None
--------------------------------------
Shared Domain: 7a3bc560
LowFrequencyHeap: 7a3bc584
HighFrequencyHeap: 7a3bc5dc
StubHeap: 7a3bc634
Stage: OPEN
Name: None
Assembly: 0052f848
--------------------------------------
Domain 1: 00516800
LowFrequencyHeap: 00516824
HighFrequencyHeap: 0051687c
StubHeap: 005168d4
Stage: OPEN
SecurityDescriptor: 00517d30
Name: simple.exe
Assembly: 0052f848 [C:\Windows\assembly\GAC_32\mscorlib\2.0.0.0__b77a5c561934e089\mscorlib.dll]
ClassLoader: 0050e470
SecurityDescriptor: 0051f5d0
  Module Name
790c2000 C:\Windows\assembly\GAC_32\mscorlib\2.0.0.0__b77a5c561934e089\mscorlib.dll

Assembly: 00535ac0 [C:\Users\Mike\Documents\Programming\simple.exe]
ClassLoader: 0050e630
SecurityDescriptor: 00535a38
  Module Name
000f2c3c C:\Users\Mike\Documents\Programming\simple.exe

At last, we have the address of something we can use. Let’s get the module information for simple.exe. For that we need the address next to it in the module list for the simple.exe assembly.

0:003> !dumpmodule -mt f2c3c
Name: C:\Users\Mike\Documents\Programming\simple.exe
Attributes: PEFile 
Assembly: 00535ac0
LoaderHeap: 00000000
TypeDefToMethodTableMap: 000f0038
TypeRefToMethodTableMap: 000f0040
MethodDefToDescMap: 000f005c
FieldDefToDescMap: 000f0068
MemberRefToDescMap: 000f006c
FileReferencesMap: 000f0088
AssemblyReferencesMap: 000f008c
MetaData start address: 008c206c (740 bytes)

Types defined in this module

      MT    TypeDef Name
------------------------------------------------------------------------------
000f3030 0x02000002 Simple

Types referenced in this module

      MT    TypeRef Name
------------------------------------------------------------------------------
790fd0f0 0x01000001 System.Object
79101118 0x01000006 System.Console

Now we can get the MethodTable for the Simple class:

0:003> !dumpmt -md f3030
EEClass: 000f1174
Module: 000f2c3c
Name: Simple
mdToken: 02000002  (C:\Users\Mike\Documents\Programming\simple.exe)
BaseSize: 0xc
ComponentSize: 0x0
Number of IFaces in IFaceMap: 0
Slots in VTable: 6
--------------------------------------
MethodDesc Table
   Entry MethodDesc      JIT Name
79371278   7914b928   PreJIT System.Object.ToString()
7936b3b0   7914b930   PreJIT System.Object.Equals(System.Object)
7936b3d0   7914b948   PreJIT System.Object.GetHashCode()
793624d0   7914b950   PreJIT System.Object.Finalize()
00310070   000f3020      JIT Simple.Main()
000fc01c   000f3028     NONE Simple..ctor()

Finally we have something we can pass to !U. Note the JIT column. Here, PreJIT means that the code has come from a native image generated by ngen, JIT means that the JIT compiler in this process has generated the code, and NONE means that the method hasn’t been compiled yet. We didn’t override any of System.Object’s virtual methods, so they occupy the first four places in our MethodTable.

I placed the call to Console.ReadLine at the end of the program so everything has already been compiled – remember, the JIT compiles code on-demand, one method at a time, when that method is called (barring very simple methods which might be inlined into their callers). Let’s just get on and show the code for Main.

0:003> !U f3020
Normal JIT generated code
Simple.Main()
Begin 00310070, size 36
00310070 833d8c10920300  cmp     dword ptr ds:[392108Ch],0
00310077 750a            jne     00310083
00310079 b901000000      mov     ecx,1
0031007e e8e1580579      call    mscorlib_ni+0x2a5964 (79365964) (System.Console.InitializeStdOutError(Boolean), mdToken: 06000770)
00310083 8b0d8c109203    mov     ecx,dword ptr ds:[392108Ch] (Object: System.IO.TextWriter+SyncTextWriter)
00310089 8b153c309203    mov     edx,dword ptr ds:[392303Ch] ("Hello World!")
0031008f 8b01            mov     eax,dword ptr [ecx]
00310091 ff90d8000000    call    dword ptr [eax+0D8h]
00310097 e84c750d79      call    mscorlib_ni+0x3275e8 (793e75e8) (System.Console.get_In(), mdToken: 0600076e)
0031009c 8bc8            mov     ecx,eax
0031009e 8b01            mov     eax,dword ptr [ecx]
003100a0 ff5064          call    dword ptr [eax+64h]
003100a3 33c0            xor     eax,eax
003100a5 c3              ret

Interesting. The JIT clearly sees Console.WriteLine(string) and Console.ReadLine as trivial and has inlined the calls. In turn it’s inlined Console.get_Out. The reference to SyncTextWriter isn’t the JIT being clever – it’s SOS telling us that the address 0x0392108C is the base of SyncTextWriter’s vtable, as this call to TextWriter.WriteLine(string) is virtual.

I realise, and hope, that this isn’t everyday usage. More often than not you’ll have crashed somewhere and want to find out where (for which see !IP2MD). But it can be useful to see just how your C# code turns into machine instructions.

What the heck does 'rep ret' mean?

I was looking at what the .NET x64 JIT compiler generates for some code, and saw something very odd at the end of the routine: the last instruction of the function was rep ret. Looking a bit further, this is the same at the end of every JIT-compiled routine.

The thing is, the rep prefix to an instruction is supposed to tell it to be repeated. Repeat the return? How do I do that?

The Intel architecture software developer’s manual set says it’s only defined for the ‘string’ instructions like movs (which moves a byte/word/dword from the address pointed to by ESI to the address pointed to by EDI). The rep prefix repeats the string instruction ECX times. Yes, this means that you can implement memcpy in a single instruction. (You can do memset with a single rep stos instruction, once AL is loaded with the value to be stored.) It’s explicitly undefined for anything else.

So where the heck has this illegal usage come from? I followed a couple of clues and found this patch notification for glibc on x64. And indeed the current version of AMD’s optimization guide [PDF] for Athlon 64 processors says that you should do this. The reason? The branch predictor gets it wrong if the ret instruction is jumped to directly by a branch instruction, or if the ret directly follows a branch instruction.

I’m not sure doing it throughout, even when you’ve got epilog code in there which prevents the bug, is necessary though.

AMD have now published a new optimization guide for their Family 10h processors [PDF] and guess what, the advice has changed. Instead of using a two-byte illegal instruction, they now recommend the three-byte instruction ret 0. The difference between a plain ret and a ret imm16 (where imm16 is an immediate 16–bit value) is that ret imm16 pops the return address, then specified number of bytes from the stack before jumping to the return address. It’s common to see this in 32–bit Windows WINAPI (__stdcall) code as this calling convention requires the called function to clean up the parameters from the stack. (64–bit Windows has only one calling convention and it mainly passes parameters in registers, so stack cleanup is not required.

Still, it’s a shame to see the JIT generating this on my Core 2 Duo laptop, which doesn’t have the bug (as far as I know, but there’s no mention in Intel’s optimization guide). And it’s an even bigger shame on AMD that they a) didn’t fix the damn bug in the processor and b) recommended an illegal instruction as a way round it.

BT and the case of the missing DSL service

Recently, at the end of September, I got a new neighbour in the flat downstairs. The property is an end terrace, split into two flats in 1999, but the landlady never did a great job of separating the services out (electricity and gas usage is included in the meter counts downstairs, there isn’t a separate supply). When I moved in, the landlady lived downstairs; she allowed a previous tenant of this flat to put in extension wiring to use her old BT landline, as she’d got cable service with NTL (now Virgin Cable, of course), so I was using the BT line. After leaving, other tenants of the downstairs flat had simply taken over the cable service.

When the new bloke moved in downstairs, he wanted a BT service. He claims that he asked them for a new line – they interpreted this to mean taking over the old one, and started the process of transferring my line to him. First I knew of this was a letter dated 16 October saying “we understand you no longer live at the above address and so we’ll disconnect the phone on 30 October.” Obviously I phoned up straight away and said no, I’m not leaving, please cancel this transfer. They prevaricated a bit, then said, “oh, I can’t cancel this on your say-so, the person ordering has to cancel.” Fine, I say – can I give him a direct number to contact you? “Nope.”

Anyway, I ask the new guy if that’s OK, he says he got fed up of BT trying to transfer the line and went with cable anyway, and he’d been trying to get through to cancel and would try again. A day or so later he tells me he’s cancelled and I don’t think any more about it.

In early November my parents tell me they couldn’t get through. (They didn’t actually tell me they were getting a message saying ‘number not in service.’) A few days into November, suddenly, no connection. A horror begins to dawn on me, and my neighbour – BT have gone ahead and done it anyway.

Now comes the reason for the tale. Most of the time I can’t get through to BT Customer Service, even if I wait for half an hour on the phone. The first time I get off hold, there’s silence for a few seconds and the line goes dead. Next time I’ve apparently called the wrong department and they try to transfer me, but again I end up on hold for half an hour (when I give up). After several goes of this I go for an end-around: I call sales and ask for the line to be transferred back to me.

No problem, says the salesman, but there’s a 12–month minimum contract and you can’t have the line until the 27th of November, unless the current customer cancels before that, oh, and you can’t have your old number back. I did try explaining but it seemed futile. Actually I got pretty angry which I did apologise for.

I thought about it for a couple of days and decided to do what should have been done a very long time ago – get a separate line terminating in this flat. You’d expect that to take a long time, right? Longer than transferring the billing from one customer to another?

Nope. I could have it put in on the 20th. So I paid my £125 (£125!) and got it done.

Unfortunately you can’t order DSL until the phone line is in your name and it appears, under the new number, in BT’s database. So the earliest I could order ADSL was the afternoon of the 20th, then it takes another 14 days or so, taking us to an official activation date of 7 December. I got straight through to Demon sales on practically the first ring. They said I couldn’t have my old hostname/website/email address back initially but once activated I could ask for it back.

The 27th came and went and I suddenly realised I hadn’t cancelled the transfer! Never fear, either the second BT salesman cancelled it for me, or (more likely) they cocked up again. My neighbour still hasn’t managed to cancel and I think he’s refusing to pay.

Finally I got DSL back on the 6th, called up Demon support (got through on first ring again) and got my old hostname back, but the contents of my website were lost. So there are a lot of broken images around here, and if you emailed me in November, I didn’t get it.

I’d always heard that NTL, Telewest, and the unbundled broadband suppliers had worse customer service than BT. If it’s worse than BT they must send round people with pitchforks to stab you with if you even dream of calling them.

Return of Best Error Page

CodeProject’s one of my favourite sites – indeed, one of the few real regular sites I visit – but it’s always had a bit of a problem with errors and with the load on the servers. I commented on this before, but sadly the image is gone – lost when my DSL account was closed (BT cock-up, nothing to do with Demon).

Anyway, they recently moved to an all-new ASP.NET implementation of their forums, but unfortunately the errors are still a common occurrence. And the error in the error handler is back:

(Yes, running Vista now.)

One of the things I love most about CP is the more… relaxed attitude to reporting problems.

Calling out EMBASSY Trust Suite

Dear Wave Systems,

Fuck right off.

When I bought my Dell Latitude D820 laptop, I went through and deleted a certain amount of the shovelware that was preinstalled, but some of it, to be honest, I didn’t know what it did. I tried to work out what EMBASSY Trust Suite did, and it seems to be something to do with the Trusted Platform Module in the system. Encrypting files using the TPM key, or something like that.

Charles Petzold wrote an article yesterday commenting on random number generators, and I added a comment earlier on mentioning in passing the reported flaw in CryptGenRandom. I decided to see if this seemed to be the case in XP SP2. My answer was inconclusive, as the assembler was hard to follow. By chance I was debugging Internet Explorer with WinDBG (easiest way to force the RSA Enhanced Cryptographic Provider, rsaenh.dll, to load so I could disassemble it) and noticed an odd number of access violation exceptions occurring when I accidentally did a search in the instance of IE I was debugging. That’s weird, I thought – where am I on the stack?

In wxvault.dll. The stack trace was:

0012c888 100065ee wxvault+0x7967
0012cac4 42f8c769 wxvault+0x65ee
0012cadc 42f8cdc9 IEFRAME!PathFileExistsW+0x24
0012cb14 42f8ccf7 IEFRAME!HelperForReadIconInfoFromPropStore+0x97
0012cb98 42f78e53 IEFRAME!CInternetShortcut::_GetIconLocationWithURLHelper+0x156

Looks like we’re trying to get the favourite icon for the address bar. But how is IEFRAME calling into wxvault? Microsoft can’t know that this library exists. Is there something on the stack that somehow isn’t being included (can happen if a function was compiled with the Frame Pointer omitted and no symbols are available to get FPO data [which tells the debugger how to fix up]). Let’s disassemble around PathFileExistsW:

42f8c75e ff7508          push    dword ptr [ebp+8]
42f8c761 8bf8            mov     edi,eax
42f8c763 ff152c13ef42    call    dword ptr [IEFRAME!_imp__GetFileAttributesW (42ef132c)]
42f8c769 83f8ff          cmp     eax,0FFFFFFFFh

That’s weird, we called into GetFileAttributesW. How did we end up in wxvault?

0:000> u kernel32!GetFileAttributesW
kernel32!GetFileAttributesW:
7c80b74c e965ae7f93      jmp     wxvault+0x65b6 (100065b6)

Evil! They patched the running instance of kernel32! What else have they patched?

kernel32!CreateFileW:
7c810760 e9d0587f93      jmp     wxvault+0x6035 (10006035)

Note how they’ve failed to rebase the DLL, using the default 0x10000000 base address, making it collide with everything ever which also uses that default address.

Needless to say this is going to get uninstalled as soon as I take a full backup of the laptop! In my book, this is a user-mode rootkit. I don’t use the feature, so it’s going.

How should they have implemented this? Well, I’d start by seeing if it’s possible to change the algorithm for the Encrypting File System. It should be, it’s implemented using the Cryptographic API and CSPs (involving callbacks into LSASS in usermode!), so I would have thought that simply providing your own CSP would be sufficient.

If that’s not possible, my next port of call would be a file system filter driver. That would have the downside (like this) that every file system call would go through it, rather than the tiny amount of calls which actually target a file encrypted in this way.

The access violation looks like it might ultimately be caused by a bug in IE – it looks like IE tried to pass the URL to the favicon to GetFileAttributesW, which I would hope would fail (or would it try to invoke WebDAV?)