Get XP SP2 on a Torrent

Download links at sp2torrent.com. I decided not to wait for Automatic Updates to kick in, even if RC2 users will get the update first.

Since I started writing this post, it's downloaded another 13MB.

XP SP2 *will* be distributed through Automatic Updates

This is a scary prospect - Microsoft have confirmed it.

I suggest that companies not using Software Update Services either start doing so or set Automatic Updates to advise of new downloads only. I could cope with our developers making the switch one-by-one - they can handle any errors that result - but I think we need to manage our salesmen more closely. Specifically, we need to test that SP2 works properly with our VPN software and with their new laptops.

You can configure Automatic Updates through Group Policy without having to touch every computer, if you have an Active Directory-based domain.

Ironically I'm more worried about getting the salesmen updated ASAP than the developers, because I think the salesmen are less likely to practise safe computing. But it has to be organised.

Tim Bray weighs in on Linux and patents

http://tbray.org/ongoing/When/200x/2004/08/05/LinuxPatents.

I find this comment interesting:

"Second, the Linux community would—after some pain—figure out a way to route around the litigation; it would be real work, but it would happen."

OK, Tim, but you just wrote:

"The Lesson In software, assume that everything is already patented. You can't build anything, no matter how new it is, without infringing someone's patent."

You've got a vicious circle: in any attempt to avoid this patent (which you may not be able to do anyway, since the patent covers the method of implementation, not the actual code implementation itself) you may well end up infringing others. Remember Microsoft's proposed solution to the Eolas patent: don't perform the infringing operation (automatically scripting binary controls loaded into a page).

[Edit: Fixed nasty double-encoding error, sigh]

No XP SP2 yet

...but I did just get a new version of Windows Update v5.

It looks to me as though MS is going to push SP2 through Automatic Updates. I'm not sure this is a good idea. Yes, everyone should have SP2 - but only under their control, and only when they ask for it. This isn't a small patch - the RC2 download was 200MB or so. Even though Windows Installer 3.0 allegedly supports differential binary patching, if everything has been recompiled with the latest /GS implementation, a heck of a lot will have changed.

I just hope I won't have to uninstall RC2 before installing the RTM version.

Odd PC behaviour? Run a memory tester

MS has written one, which can be booted from either a floppy disk (what's one of those? ) or a CD: download.

(via Dr. HardwareBlog)

It was twenty years ago today...

...well, OK, it was nine years ago some time this week, I forget exactly.

When I was at school, around Christmas 1994, I joined a band with a few friends. In August of 1995, we recorded a number of songs we'd written (mostly by David Kane, our guitarist, often with Roger Barden, the keyboard player, although I and the drummer, Chris Velvick, also have writing credits). Since David is now getting married (in Toppenish, Washington, USA - he's marrying an American girl) in September, and I have some free time () I thought I'd get the demos onto CD. The master is a standard, albeit very expensive, audio cassette - it uses different material to get very low noise.

I also converted the songs to MP3. I thought I could actually upload all of them to my Demon web space, but it's only 20MB, and I encoded them (all 4+ minutes, we were a little long-winded) at 192kbps. There's arrogance. So there's only space for one, really. I've chosen one of my favourites: Survivor.

Oh, my part? I do the vocals. That's it. While I now play a bit of guitar - and Survivor is one of the few things I can play - I couldn't at the time.

As for what happened to the band - we found that our final Upper Sixth year was a lot harder than the previous one and didn't find a lot of time for rehearsing, let alone writing and recording new material. David continued with his Music A-level; David and Roger then took a year out travelling (Japan and the Amazon) before going to University to study Music at Bangor and History at Exeter (I think) respectively, starting in 1997. Christian studied Chemistry and drinking at Edinburgh, while I went to Aston (which I've written about before).

David is now studying for a PhD at SOAS having completed his MA in Ethnomusicology and I believe is still planning to visit Bangladesh again some time in the winter. Roger is with some kind of missionary organisation in Leeds, last I heard. Christian is some kind of high-powered recruitment consultant recruiting middle managers, somewhere in the Thames Valley - I last saw him a year ago at Roger's pre-wedding meal (you can't really call that a stag night ).

And I'm muddling along as - mainly - a Pocket PC C/C++/C# developer working for 5D / Mnetics. It's a little unclear right now, actually - I answer the phone as Mnetics, but still officially work for 5D, which is now owned by Mnetics as of a couple of weeks ago.

[Update: I forgot to link to Audacity, the software I used to capture and manipulate the tracks.]

Violence in video games

Well, I don't know about you, but I don't think video game violence actually has that much effect IRL. I mean, it's had no effect on me. I'm absolutely fine  

Anyway, it appears that Halo 2 (not mentioned by the Daily Mail, apparently) will be rated 16+ in the UK:

"The VSC was totally unconcerned with the murderization of Covenant forces. Apparently, if the game didn’t allow you to turn on your allies, we’d have gotten the UK equivalent of a PG, regardless of the galloons of alien blood we had flooded the Galaxy with. Britain is now completely unconcerned with bad language, as I discovered watching BBC TV in the hotel that night. Just as well, since every other word out of our game seemed to be bastard., asshole or bullshit.

"I did point out that technically, you can kill tens of thousands of Grunts, Elites, Brutes, Jackals and “others” while playing a typical game. “No problem, as long as they don’t look human,” said one of the chaps." -- Bungie Weekly Update 30/7/2004

It takes *how* long?

For those not versed in floating point representations, NaN means 'Not a Number' - typically the result of division by zero.

(Windows Update v5, BTW)

Daily WTF

...is a blog containing code (and sometimes other things) that make geeks go, WTF??

Today's horror just looks like a random jumble of appending constants that probably didn't need to be constants. Until you look more closely, and you realise it's constructing XML. Except there's a bug: tags are being closed with a backslash - \ - rather than the correct slash - /.

Just in case anyone needs the hint:

private static string FormatData( SqlDataReader dr )
{
   using ( StringWriter sw = new StringWriter() )
   {
      XmlTextWriter tw = new XmlTextWriter( sw );
      tw.Formatting = Formatting.Indented;
      tw.WriteStartElement( "Items" );
                
      while ( dr.Read() )
      {
         tw.WriteStartElement( "Item" );
         tw.WriteAttributeString( "key", (string)dr[0] );
         tw.WriteString( (string)dr[1] );
         tw.WriteEndElement();
      }
      tw.WriteEndElement();
      return sw.ToString();
   }
}

If you need to write XML, use an XmlTextWriter. The code is shorter and clearer, and is more likely to work, considering cases like escaping invalid characters. Also note that I got it to indent the XML for me. Running this on a reader containing the results of SELECT emp_id, lname FROM employee on SQL Server's pubs database, we get:

<Items>
  <Item key="A-C71970F">Cruz</Item>
  <Item key="AMD15433F">Devon</Item>
  <Item key="A-R89858F">Roulet</Item>
  <Item key="ARD36773F">Domingues</Item>
  <Item key="CFH28514M">Hernadez</Item>
  <Item key="CGS88322F">Schmitt</Item>
  <Item key="DBT39435M">Tonini</Item>
  <Item key="DWR65030M">Roel</Item>
  <Item key="ENL44273F">Lincoln</Item>
  <Item key="F-C16315M">Chang</Item>
  <Item key="GHT50241M">Thomas</Item>
  <Item key="HAN90777M">Nagy</Item>
  <Item key="HAS54740M">Snyder</Item>
  <Item key="H-B39728F">Bennett</Item>
  <Item key="JYL26161F">Labrune</Item>
  <Item key="KFJ64308F">Josephs</Item>
  <Item key="KJJ92907F">Jablonski</Item>
  <Item key="LAL21447M">Lebihan</Item>
  <Item key="L-B31947F">Brown</Item>
  <Item key="MAP77183M">Paolino</Item>
  <Item key="MAS70474F">Smith</Item>
  <Item key="MFS52347M">Sommer</Item>
  <Item key="MGK44605M">Karttunen</Item>
  <Item key="MJP25939M">Pontes</Item>
  <Item key="M-L67958F">Larsson</Item>
  <Item key="MMS49649F">Saveley</Item>
  <Item key="M-P91209M">Pereira</Item>
  <Item key="M-R38834F">Rance</Item>
  <Item key="PCM98509F">McKenna</Item>
  <Item key="PDI47470M">Ibsen</Item>
  <Item key="PHF38899M">Franken</Item>
  <Item key="PMA42628M">Accorti</Item>
  <Item key="POK93028M">Koskitalo</Item>
  <Item key="PSA89086M">Afonso</Item>
  <Item key="PSP68661F">Parente</Item>
  <Item key="PTC11962M">Cramer</Item>
  <Item key="PXH22250M">Henriot</Item>
  <Item key="RBM23061F">Muller</Item>
  <Item key="R-M53550M">Mendel</Item>
  <Item key="SKO22412M">Ottlieb</Item>
  <Item key="TPO55093M">O'Rourke</Item>
  <Item key="VPA30890F">Ashworth</Item>
  <Item key="Y-L77953M">Latimer</Item>
</Items>

Changes to Win32 API in Longhorn - Windows Base APIs

From WinBase.h:

• New system exception type EXCEPTION_POSSIBLE_DEADLOCK. Is the OS detecting deadlocks for us?
• New CreateFile flags: FILE_FLAG_MUI_REDIRECT, FILE_FLAG_MUI_FALLBACK, FILE_FLAG_COPY_TEMPLATE_METADATA - used by the loader?
• CopyFileEx flag COPY_FILE_COPY_LINK. Symbolic links in the filesystem at last? Yes: CreateSymbolicLinkW.
• GetQueuedCompletionStatusEx function and new structure OVERLAPPED_ENTRY. Apparently able to remove multiple requests from a completion port in one call.
• CreateIoCompletionPortEx, takes an additional ObjectFlags DWORD over CreateIoCompletionPort.
• New I/O completion/asynchronous APIs SetFileCompletionNotificationModes and SetFileIoOverlappedRange.
• Kernel-level reader-writer locks and condition variables, more like POSIX threads. The relative weakness of the Event has been a long-held criticism of the Windows threading model.
• Encode/Decode{System}Pointer APIs. I could have sworn I'd seen this documented somewhere, but Google can't find it now. I think the idea is to protect an in-memory pointer to a sensitive structure until required, as a security measure.
• NUMA support for virtual memory allocation with VirtualAllocExNuma - allocate memory with affinity to one or more nodes. New flag NUMA_NO_PREFERRED_NODE. You can also create file mappings on specific nodes using CreateFileMappingNuma, MapViewOfFileExNuma.
• INHERIT_CALLER_PRIORITY flag (CreateProcess?)
• JIT_DEBUG_INFO structure (with 32- and 64-bit variants) which may go along with a new IDebugSymbols3 interface declared in DbgEng.h.
• New OpenFile flags? OF_MUI_REDIRECT, OF_MUI_FALLBACK. OF_ flags are also used by LZOpenFile and AVIFileOpen.
• Acquire and Release (memory fence) versions of InterlockedIncrement, Decrement, Exchange, CompareExchange.
• 32-bit versions of InterlockedAnd, Or, Xor. More Interlocked bitwise operations.
• Ability to push a whole list onto an interlocked singly-linked list (InterlockedPushListSList).
• Guaranteed thread stack sizes: new API SetThreadStackGuarantee.
• Ability to explicitly set thread priority: SetThreadActualPriority. Can query with GetThreadActualPriority.
• GetThreadStartInformation - find out where any thread started executing, and what the lpParameter parameter passed to CreateThread was.
• 64-bit tick counter: GetTickCount64. Either higher resolution or greater range (or both?)
• SetEndOfFileEx. Now you can just say how long the file is, you don't need to SetFilePointer first (and you can also keep your place...)
• Walk and delete file-system trees: WalkTreeW and DeleteTreeW.
• Transactional file-system backup semantics: BACKUP_TXFS_DATA used with BackupRead/BackupWrite.
• Enhanced mutexes: CreateMutexEx{A/W}. You can now use some object flags and specify what access you want to the mutex (perhaps you only need to know it exists, rather than being able to SYNCHRONIZE?) Ditto CreateEventEx, CreateSemaphoreEx, CreateWaitableTimerEx.
• New Reserve object. Some kind of kernel object (like a job??) to which you can join threads and associate processes, then later 'disjoin' and disassociate them. You can set the bandwidth for the reserve... this must be the resource reservation API, to provide guaranteed QoS to threads and applications. No more choppy Media Player?
• NeedCurrentDirectoryForExePath API. Takes a string parameter called ExeName and returns a BOOL. A get-out clause for the post- XP SP1 LoadLibrary behaviour?
• EnumResourceTypesEx - look for resources in MUI DLLs with specific languages? Also EnumResourceNamesEx and EnumResourceLanguagesEx. Updating resources also extended: BeginUpdate/Update/EndUpdateResourceEx.
• 64-bit systems Windows-on-Win64 file system redirection using Wow64EnableWow64FsRedirection.
• GetFileAttributesEx new flag GetFileExInfoNoReparse - prevent walking through symlinks?
• 'Fast' search for FindFirstFileEx: new flag FindExInfoFast.
• Enhanced I/O cancellation: cancel synchronous I/O occurring on another thread with CancelSynchronousIo, and possibly find out how much data was already transferred on an asynchronous I/O using CancelIoEx, which takes a pointer to an OVERLAPPED structure.
• Explicit thread pooling: CreateThreadpool, SetThreadpoolThreadMinimum/Maximum, Pause, Resume, CreateThreadpoolGang(?), Timers, Waits.
• A bit of diagnostics: GetOSProductName.
• Transactions (transactional file data support?): SetCurrentTransaction, GetCurrentTransaction.

That's a lot of new capabilities right there.

Academic biography and C++ parsing

Of course, due to my peculiar route through university, I ended up studying both the C and C++ courses twice, which tended to reinforce the principles - then for my final year project, I got very familiar with the C++ syntax and semantics. My friends already know my background, but I've had a few comments from people who don't, so here we go:

I entered Aston in September 1996 to study for an M.Eng in Electronic Systems Engineering - a four year full-time course. For any foreigners, a normal Bachelor's first degree in the UK has three years of full-time study, often optionally with an additional year spent on industrial placement. Normally, a Master's degree is postgraduate study, but M.Eng is atypical - it's a four-year first degree. When I started, you were supposed to get industrial placement work during summer vacations, but this rule was relaxed when it became clear that there weren't any placements available.

I attended that course for three years, scraping passes after resitting exams in both the first and second years (in one resit exam, my grade shot up from a failing grade to over 80%). In the third year a combination of factors (illness, extensive involvement in Entertainments, increasing complexity of mathematics) led to my deferring the exams. Over that summer I questioned my future direction and decided to apply for a transfer to Computing Science, which was granted. The departments accepted that my studies so far had covered all the material necessary to proceed directly to year 2. I completed year 2 with decent grades, then completed the final year fairly well, graduating with Lower Second-Class Honours (a 2:2, or Desmond).

The main thing that dragged my final Honours grade down was my final-year project. I'd decided to tackle a program to diagram the static structure of a program, with C++ as the source language. I got totally bogged down in the syntax and semantics of C++, and basically failed to complete the project. I don't do things by halves - I was trying to persuade a variety of parser generator tools to interpret standard C++ correctly. I suppose that I was taken in by my Programming Language Implementation lecturer, who claimed that parsing was the bit of CS that could be called 'science', because we know how to do it.

Not for C++, we don't. The commercial C++ parsers are either hand-written recursive-descent parsers which don't really follow any 'scientific' method (Visual C++), or based on yacc/bison with so much feedback between parser and lexer stages, and with so many bizarre actions in the middle of rules, that you can't call them LALR(1) any more (GCC).

Yes, I could have adapted GCC to handle the parsing. But quite frankly I didn't want to spend three months or more understanding how to hook into it.

The problem with C++ is just that it's hugely ambiguous and overloaded, with the distinguishing symbols - if any - an indeterminate distance from the beginning of a clause. This makes it great to write, IMO, and it's OK for humans to read given that we can see the whole line, but very hard to write a parser for. The syntax actually allows a great deal more expression than most programmers ever use, but the parser really has to accept all of it.

C++ really requires a stronger parsing method: LR(k). I now recommend that people look at Elkhound if they want to parse C++ and don't want to use an available parser.

What I should have done was to use Visual C++'s CL and BSCMAKE tools to generate the source browser files and then use the Browser Toolkit to interpret the BSC file, and generate diagrams from that. But I don't think I knew it existed at the time.

CS at Aston no longer offers C

...or any unmanaged programming, for that matter, apart from a small amount of assembler in year 1. It looks like the dedicated academics have won out over the practical people.

I was involving myself in a conversation on CodeProject about college courses (a US-based individual noted that their college offered COBOL!) and wanted to link to my old BSc Computing Science course (which I graduated from back in 2001). Then I noticed something horrible...

Java has taken over, at least above first year (Introduction to Systematic Programming is still taught in Ada). Object-Oriented Programming is now taught in Java (when I took the course, it was C++) and a new course, Internet Programming with Java, takes the place of the C course, Software for Systems Programming.

Now I'll admit that a lot of my fellow students didn't 'get' OOP because they got bogged down in the ugly mechanics of C++. The lecturer wasn't too hot, IMO, and wasn't properly focused on the OOP mechanisms of C++. Also, at the time, the course compiler was GCC 2.x which didn't do templates and the standard C++ library very well, so we were using a lot of the C subset of the language.

It looks like the Computer Graphics course might still be taught in C, but there you have another problem - in spending time teaching the language, you lose time to teach the actual meat of the course.

But how can you send out new graduates with no practical knowledge of low-level systems programming? If you have to break out of the cosy managed environment, how will you cope? Someone still has to write the VM and runtime, after all, and the system dependencies. Are we failing to train our software engineers to write the next generation of VMs?

Adapting to the executing platform

A standard way of using features not supported by your baseline platform but supported by later versions is to use the LoadLibrary and GetProcAddress routines to get a handle to a DLL, then get the address of the function, if it's present.

You can do this in .NET by using [DllImport], calling the function, then handling the MissingMethodException that results. However, throwing an exception is a relatively lengthy operation.

Junfeng Zhang reports that a new method GetDelegateForFunctionPointer has been added to the System.Runtime.InteropServices.Marshal class in .NET 2.0. You can now LoadLibrary and GetProcAddress, then construct a delegate from the resulting function pointer.

Changes to Win32 API in Longhorn - Update Delayed

Nothing to see here, move along...

I haven't quite had the energy to do the next stage. I'm thinking about tackling winuser.h and wingdi.h next, plus probably the GDIPlus headers.

In mitigation, yesterday I watched Coupling for half an hour and spent a long time investigating the ListView.Click issue, which was originally raised on CodeProject's C# forum. Guess what the second response was? "TreeView.Click doesn't work either!" Sheesh...

Tonight I got home pretty late anyway, then my wireless connection stopped working after I read the blog collection and checked for updates. I decided to apply the firmware update to the router, but that didn't solve it, so I hit Disconnect on the View Available Wireless Networks window, then hit Refresh Network List - and nothing appeared. I ended up uninstalling the Virtual PC Network Switch driver and the IPv6 stack (which I'd been looking into a while back) which didn't help, then punching in the SSID and WPA keys again - still nothing.

OK, I thought, let's reboot. After rebooting, the network had disappeared from the Advanced Settings screen. Great - I enter it again, and it works (I can't just click on it in Available Wireless Networks because the router's set to not broadcast the SSID).

I've found that XP SP2 wireless + my card's drivers + Virtual PC Network Switch don't work together anyway. If you have VPCNS enabled on the wireless connection, nothing happens: any packets directed at the connection disappear into a black hole. Disabling VPCNS on the connection sorts it out, although I found I had to Disconnect and Connect in Available Wireless Networks before it would work again.

Anyway, that and normal routine household tasks took up most of the evening. Hopefully I'll get to it tomorrow.

Now I read that back, I realise that I've probably spent as much time writing up the excuse as I would have just writing it. Oh well.

[Edit:]

I need to state that actually, VPCNS works fine on the wireless connection - it was the action of installing it which caused the stack to go bonkers. Disconnecting then reconnecting sorted out the stack. I'd been having weirdo problems with it not working properly when resuming from Hibernate - again, disconnect and connect sorted it out. Let's see if it stays sorted - OK so far.

Because I hibernate this machine (it's a guzunder - it goes-under the desk) uptime doesn't have a lot of meaning. I use hibernate rather than power off because it hibernates faster than it shuts down, it resumes faster than it starts up, and I don't lose my context if I was working on something. We're talking < 10 seconds from press-power-switch to unlocking the console. Also, because of the way it works (dump physical pages to disk) I think the disk cache also remains 'hot'.

Linking it together

Or rather, following the links.

The Brothers Chaps, at homestarrunner.com (rss) have uploaded a new video, for the new song Experimental Film by They Might Be Giants. I still remember TMBG best from "Birdhouse In Your Soul" and "Istanbul (not Constantinople)", dating back to 1990.

This spurred off a link in my head to one of my favourite bands, The Presidents of the United States of America (or PUSA, to most fans). I searched for their site and discovered that they're releasing a new album in August! Unfortunately, the new single is only currently available on iTunes. Er, sorry, but I don't want to give Apple any money.

Their last album, "Freaked Out and Small" was released on the MusicBlitz website, then as a CD. I found out about this one through a link from one of the webcomics I read, which linked to a great animated video by Chris Maguire for the song "Meanwhile Back In The City". The animation is totally unofficial - from the link, it looks like it was undergraduate project work. Still cool, though.

Get patching

This month's Microsoft Security Bulletins (rss) have been released. I suggest applying the patches reasonably quickly. Of seven bulletins, one applies only to IIS 4.0 on NT 4.0 (to which I must ask why have you not upgraded??), one applies to the POSIX subsystem on NT 4.0 and 2000 (not included in XP and 2003), and one to the Utility Manager in Windows 2000.

I'm hoping that the dead silence in the bulletins regarding XP SP2, and Windows Update v5 not showing anything, means that these issues are already fixed in SP2. It looks like all the binaries on my machine have later build dates and build numbers.

Public Service Blogging: ListView.ItemActivate

The ListView.Click event works in the desktop .NET Framework. It doesn't work in the Compact Framework, even in the beta SP3: it doesn't fire.

Nor does it fire in .NET Compact Framework 2.0 Beta 1, which I've been looking at this evening, a bit. You get a warning when you compile:

warning VSD101: Members not supported by the Device platform should not be called: System.Windows.Forms.ListView.add_Click is not a supported method in this platform

It turns out that you're not supposed to use Click - you're supposed to use the overarching ItemActivate event, which is fired when either you click, or double-click depending on configuration, or when you activate the item with the keyboard. I suppose this makes sense for Smartphone, since you can't 'click', the screen not being touch-sensitive.

You configure whether ItemActivate fires on single- or double-click by setting the Activation property on the control.

Note the nice bit of abstraction leakage in the warning message: the line of code reads:

listView1.Click += new EventHandler(listView1_Click);

Er, where did I call the method add_Click? Answer, it's lurking behind that nice += syntax.

OK, so I'm using the wrong semantics; it should still work.

Changes to Win32 API in Longhorn - Common Controls

The Common Controls API is largely implemented in commctrl.h. Let's see what's new (note, of course, everything is subject to change; documentation for these functions and flags isn't yet written).

• New Histogram control: ICC_HISTOGRAM_CLASS flag for InitCommonControlsEx, many structures, messages, notifications, flags etc
• Notification message on font change: NM_FONTCHANGED
• New custom draw flags CDRF_DOERASE, CDRF_SKIPPOSTPAINT (marked internal)
• New message NM_GETCUSTOMSPLITRECT and accompanying structure
• New flag ILC_ORIGINALSIZE for ImageList_Create
• New flag ILD_ASYNC for ImageList_Draw
• New flags ILS_EMBOSS, ILS_REFLECT, ILS_INVERT for ImageList_DrawIndirect
• Reference to SparseImageList (value HBITMAP_CALLBACK - enables image to be supplied on demand, dynamic images in ListViews?)
• Checkboxes and non-resizable columns in Header controls (HDS_CHECKBOXES, HDS_NOSIZING)
• Data filtering by date (HDFT_ISDATE used with HDITEM structure)
• Hit testing header-items can report On Item State Icon (HHT_ONITEMSTATEICON)
• Header state icons: new macros Header_GetStateImageList and Header_SetStateImageList
• Ability to use accelerator keys with a header control (?): HDM_TRANSLATEACCELERATOR
• Item notifications: HDN_BEGINFILTEREDIT, HDN_ENDFILTEREDIT, HDN_ITEMSTATEICONCLICK, HDN_ITEMKEYDOWN
• Oddly extended toolbar class: "ToolbarWindow32_Mil"
• Callback image labels in a toolbar: TBSTYLE_EX_CALLBACKIMAGELABELS
• "Use CustomDrawColors to RenderText regardless of VisualStyle": TBCDRF_USECDCOLORS for toolbar custom draw
• Transparent, splitter rebars: RBS_EX_TRANSPARENT, RBS_EX_SPLITTER
• Notification of rebar splitter drag: RBN_SPLITTERDRAG
• Tooltip can have a preview bitmap: new member hbmp on TOOLINFO structure
• 'Themed' hyperlinks in tooltips: TTS_USEVISUALSTYLE style flag
• Animated progress bars (?): PBM_ANIMATE, PBM_TRANSITIONTOPOS messages
• New List View style flags LVSIL_GROUPHEADER, LVSIL_IMAGETITLE (draw your own conclusions - groups of items in a list view?)
• List View item flags LVIF_IMAGETITLE, LVIF_TASKS, LVIF_COLFMT, structure LVTASKS and corresponding fields in LVITEM
• New list view item search flags for LVM_GETNEXTITEM message: LVNI_VISIBLEORDER, LVNI_PREVIOUS, LVNI_VISIBLEONLY, LVNI_SAMEGROUPONLY
• New flag for LVM_GETITEMRECT: LVIR_NUMORDER
• Hit test flags for ListView groups: group header, footer, expand/collapse button, background, state icon, contents.
• New macro ListView_HitTestEx to support group flags
• Additional flags for controlling ListView column formats
• ListView extended styles: Justify Columns, Transparent Background, Transparent Shadow Text, Show Numeric Order, Auto Auto Arrange (Icons automatically arrange if no icon positions have been set), Show Header in all modes, Auto Check Select (?), Auto Size Columns
• Alpha-blend background images: LVBKIF_ALPHABLEND
• New ListView views: Carousel, Panorama
• New options for ListView groups (different from above groups? LVGROUP structure extended) and new styles
• ListView Tile View 'extended' size flag: LVTVIF_EXTENDED
• Column formats for tile info columns
• Bitmaps on ListView infotips
• Is a list item visible? ListView_IsItemVisible macro
• NMLVLINK structure: links in a ListView? Also LVN_LINKCLICK notification
• LVN_ORDERCHANGED notification
• Async drawing of ListViews
• Enhancements to Animation control: ACM_OPENXFILE, ACM_SETTEXTURE messages
• Split buttons: a host of new messages, styles, flags, structures and notifications. I assume this is like the Open button in VS2003.
• Edit control: Set Highlight, Get Highlight messages
• Unknown flags: DCHF_TOPALIGN, DCHF_HORIZONTAL, DCHF_HOT, DCHF_PUSHED, DCHF_FLIPPED, DCHF_TRANSPARENT, DCHF_INACTIVE, DCHF_NOBORDER. Might be related to:
• DrawScrollArrow API

Of course, it's possible that a lot of this is to support the sample UI in current Longhorn builds, and might disappear before the final software is released. Heck, it could disappear before the next build is released.

Changes to Win32 API in Longhorn

Joel Spolsky's post How Microsoft Lost the API War has had a fair amount of press, and has got a number of developers running scared - some even talking about switching to Linux development. A common theme is that Microsoft won't be advancing the state of the existing API: all new features are going into Avalon/WinFX. People seem to have implied from this that they'll have to rewrite all their apps from scratch, which should not be the case at all.

I wrote a message on the CodeProject forums yesterday which I'll include here:

Don [Box]'s blog post Teaching COM made it pretty clear that Microsoft are extending COM for the Longhorn generation.

They've previously said that the Win32 stack will continue to be present in Longhorn and demonstrated old DOS apps running on Longhorn builds. I don't have a Longhorn SDK handy, so I can't tell you how much Win32 is being enhanced. The new window management and drawing APIs will be exposed through managed interfaces only, I believe, but your UI and data are abstracted, aren't they? Joe Beda, an Avalon architect, talks about this in a Channel 9 video.

Managed C++ or the new C++/CLI syntax should allow you to build a new front end to your existing C++ code.

I seem to recall that Avalon architect Chris Anderson has talked about being able to host Win32 GDI+USER content in an Avalon application, and being able to host Avalon controls in a Win32 application. Windows Forms is just a special case: a Win32 application written in managed code.

Visual Studio 2005 comes with new versions of MFC and ATL, version 8.0. See Visual C++ 2005 Libraries Features for a list of the changes: note that MFC can now host Windows Forms controls, for example.

Remember that Microsoft's platforms and developer tools division's first customer is their applications division. These guys aren't going to want to rewrite Word: there's twenty years of code in there. Rick Schaut, a Mac Word developer, talks about analysing a bug which caused Mac Word to complain "Disk is Full" when trying to save.

They're not going to throw that fix away. They're going to write a new rendering layer, to render to Avalon, new frame UI code targetting Aero, then compile the rest of the monster using CL.EXE's /clr switch. I seem to recall that one of the targets for /clr was that the compiler had to be able to compile Word. I'd expect Microsoft to compile the whole of Word to IL, rather than native code, for future generations so that the same binary runs on both 32-bit and 64-bit machines. Of course, there's always P/Invoke if a component truly can't be compiled to IL.

Anyway, I've now downloaded the Platform SDK for XP SP2 and the Longhorn SDK, and I'm running a diff between the Include directory of the LHSDK and the version that ships with VS2003. Bizarrely, Microsoft have built the XP SP2 SDK from the XP SDK, which means it's impossible to build programs which use features new to Windows Server 2003 using this SDK - those features are missing. Thanks a bunch.

I plan to add new posts covering various areas of the SDK that have been updated. I'll add links to this post as I post them.

• Common Controls
• Windows Base APIs

Jumping the hurdles: how Download.Ject infected

I thought I'd take a look at how Download.Ject infected, and the security barriers it managed to breach. I've gathered this together from what analysts have hinted at, so some information may be wrong.

Firstly, the attacker had to infect the web server. IIS has a feature which allows you to set a footer to be appended to every page. Internet Explorer parses content outside the <html> tags - but so does Firefox. Try this file:

<html>
<head>
<title>Test Page</title>
</head>
<body>
<p>Body</p>
</body>
</html>
<script language="javascript">window.alert( "Hello" );</script>

Opening this in a local file on XP SP2 gives you the Information Bar again; the Local Machine Zone has been locked down. You don't get the popup. Firefox gives you the popup.

Download.Ject exploited the Private Communications Transport (a secure sockets channel) flaw reported in MS04-011 to upload and execute arbitrary code. This uploaded code manipulated the metabase to add a footer containing a script.

The script in the footer, if run on Internet Explorer, exploited a vulnerability reported in MS04-013 which allows code to run in the Local Machine zone. It also repurposed the ADODB.Stream class, which is marked safe for scripting, to download files from the attacker's server to the victim's computer. The download was necessary to get code loaded into another process on the victim's computer. The effect of SP2's Local Machine Zone lockdown is that the script forced into the LMZ couldn't run. Note that no vulnerability in ADODB.Stream was exploited: it simply permits something that in retrospect is too permissive.

So the attacker had to have worked out how to exploit PCT, how to get arbitrary code running on the system with the user's full privileges, and how to get extra files downloaded onto the user's system. Administrators could have prevented the first by applying a megapatch; users the second by applying a patch.

Techworld.com - Browser rival to ActiveX in the offing

Techworld.com - Browser rival to ActiveX in the offing

And as a follow-up to the last post on browsers, the Other Browser Group (everyone who makes a commercial web browser, that isn't just a wrapper for one or more rendering engine, barring Microsoft) announce that they're going to extend scriptability on binary plugins. These binary plugins will be built using a C-based unmanaged API. Here's what they have to say about security:

"The security model for making calls through this API is much like the general same-origin security model enforced by the browser. That means that script from an origin other than the origin of the page that loaded the plugin is not able to access methods and properties on the plugin. The same thing applies the other way too, the plugin can reach only JavaScript objects in the same origin as the page that loaded the plugin.

"In addition to this, a further extension to this API is being discussed that would give a plugin greater flexibility by letting the plugin control the origin of the calling code, so that the plugin can specify the origin of calls that come from internally loaded code from other origins. This way such code can be executed with only the privileges of the origin of the code, and not the privileges of the plugin page's origin."

They've essentially described denying cross-site scripting. This is all very well, but if there's a flaw in the implementation, you still have XSS attacks. There's no sandboxing, no requirement to mark components safe for scripting or initialization, no way for the user to limit which sites can and cannot use script. It's all or nothing. IE has all the above (and the cross-site scripting restrictions).

Fundamentally, with the ability to load unmanaged components, the code can do anything at all. They don't describe any way for components to be recognised as written by a particular publisher or to limit the user's ability to download unsigned controls (possible through Group Policy for IE). The slight limitation is that of course the browser can only load components which conform to the required interface; there are many fewer of those for the Netscape plug-in API than there are COM components.

Let's look at how things work on XP SP2. In Firefox, say I want to watch Strong Bad's latest e-mail and I don't have the Flash player. Clicking that link gives me a big blank area saying 'Click here to get the plugin'. Clicking that gives me a lengthy dialog with a Get the Plug-in and a Cancel button, which doesn't mention security at all. Clicking Get the Plug-in button takes me to Macromedia's download page, so I click Download Now. I get a dialog asking me to save the installer. Once it's downloaded I open the installer, and step through the wizard, which shuts down the browser.

It appears that Firefox can load plugins on the fly, but Flash doesn't bother - it just restarts the browser. The suggestion seems to be that this will become easier - plugins could be downloaded after prompting the user. Netscape 4.x supported SmartUpdate, which does have code-signing, but this required LiveConnect, not present in Firefox. Firefox already supports XPCOM, XPInstall and XPConnect. The first allows binary extensions, the second allows extensions to be installed on demand, and the third allows those extensions to be scripted.

You can get an XPCOM extension which loads ActiveX controls. As a demo, in the download section, click one of the Click Here links. Firefox pops up a dialog:

Note "Install Now" is the default, although it is initially disabled for two seconds. I didn't hit Install Now because firstly I don't want it and secondly it doesn't work in Firefox at present.

Let's try the equivalent on IE. Remember this is SP2. First I have to persuade IE that Flash doesn't exist by deleting it from the registry. Now we go again to Homestarrunner.com. I get the information bar:

Clicking the bar and selecting "Install ActiveX Control" reloads the page and brings up the new security dialog:

Hitting Install downloads and installs the player, then loads the content straight away. Note that "Don't Install" is the default. IE also watermarks files that it downloaded: when you try to open one, you get this dialog:

I suspect the next generation of IE will support .NET-based plug-ins which use the .NET code access security model. It'll probably still support loading ActiveX controls, though.

IE 6.0 SP2 forthcoming?

Previously, Microsoft have said that the IE security changes in XP SP2 won't be released for other OSs. However, I was just skimming PressPass and saw this quote:

"In addition to this configuration change, which will protect customers against the immediate reported threats, Microsoft is working to provide a series of security updates to Internet Explorer in coming weeks that will provide additional protections for our customers.

"Later this summer, Microsoft will release Windows XP Service Pack 2, which includes the most up-to-date network, Web browsing and e-mail features designed to help protect against malicious attacks and reduce unwanted content and downloads. A comprehensive update for all supported versions of Internet Explorer will be released once it has been thoroughly tested and found to be effective across a wide variety of supported versions and configurations of Internet Explorer."

(from Microsoft Statement Regarding Configuration Change to Windows in Response to Download.Ject Security Issue)

Does this mean they've changed their minds? Will a separate IE 6.0 SP2 be released? I hope so.

Download.Ject patch

There's now a patch available for the Download.Ject trojan - or rather, one of the holes it used (since the site hosting the code was taken down last week).

The Register: Microsoft half fixes serious IE vuln

The Register get it wrong again, of course: I anticipate that this is the full and final patch for this issue. The ADODB.Stream vulnerability is simply that a feature - the ability to write to a file anywhere on the hard disk - was made available to the web. The patch sets the 'kill bit' in the registry, preventing this control from being directly loaded from IE, as detailed in KB 870669. MS have clearly decided that compatibility and functionality have to be sacrificed on the altar of security.

A naive user reading that could argue that IE's security model is the wrong way round - instead of killing known unsafe controls, only known safe controls should be permitted. It actually does work that way, for most zones: controls must be marked 'safe for scripting' to be scripted and 'safe for initialization' to load parameters from the web page. The Local Machine Zone is different; that will be, by default, locked down in XP SP2 RC2. The 'kill bit' is for this situation - a control marked safe is in fact unsafe in a particular scenario.

The other vulnerabilities exploited were already patched in MS04-013 (client-side) and MS04-011 (server-side). If admins had applied the 04-011 patch rather than being scared off by adverse media reports, this wouldn't have happened. It's an admin's responsibility to not be an attack vector, as well as protecting their own systems. The Internet users may even have been collateral damage - IIS 5.0 is used much more widely in intranets. Of course there were incompatibilities with this patch, largely due to a new kernel binary being released for Windows 2000. We needed a hotfix and got stuck in the Product Support queue behind a bunch of the problem reports.

Yes, this exploit is a side-effect of IE being able to load and script any COM component in the local machine zone - get over it. As I say above, this will be locked down in XP SP2 (and already is in RC2). IE's come a long way since the bad days of IE 3.0, which didn't do any checking. It's also a side-effect of the user running as an Administrator, which we all know is a bad thing. If you're not running as Admin, I suspect that Download.Ject couldn't actually install the backdoor components.

Note that Firefox has the XPConnect module, which allows XPCOM plug-ins to be scripted, potentially leading to the exact same problems for any installed extensions. It's just no-one's found any vulnerabilities yet.

[Edit] That's a bit strong, really. It will only apply to Firefox extensions, not every control/component installed on the user's system. Still, that's probably enough scope, since almost everyone will have Flash installed. [/Edit]

True Lies

Just finished watching True Lies. Hard to believe it's ten years old, actually. For a Governator picture, it's pretty good, but then the supporting cast is bloody good: Jamie Lee Curtis, Tia Carrera, Art Malik, and a certain Eliza Dushku (better known as Faith from Buffy).

Even at age 14 she was cute, although to be honest the role's fairly small.

Art Malik is currently wasting himself in Holby City.

Around this time, Hollywood had a real habit of casting Islamic/Arabic villains such as the 'Crimson Jihad' in this one. I wonder if the perception of the faces of terrorism (i.e. the predominant nationality/worldview of the terrorists) is in fact based on this, rather than on fact? Wikipedia's list of terrorist groups is only about 25% Islamist.

Freedom!

Well, of a sort. I went out and bought a wireless LAN ADSL router and a wireless card, so I now have Internet access at home from my own PC. I'm already running XP SP2 RC2, so firewalling is covered (and I'm free of Norton Internet Security, yay).

I'll admit to being sorely tempted by a wireless adapter for my Xbox (which is in yet another room) but I could only find an 802.11b one - maybe I'll order a g version from somewhere else.

Anyway I'm now patching Office (2000 SR-1a - ouch) and getting files off the other PC. Which is a story in itself - the network setup wizard in XP Home SP1 is, er, a bit broken (yes, dammit, I want to call the workgroup something other than MSHOME, thank you so very much) and NIS is getting in the way somewhere still, I think. I can see the other PC and mine in Network (My Network Places, for those who don't habitually delete My from every profile they create) but from their computer - nothing at all.

The subtext is that I want to download Whidbey beta 1 but don't want to actually write CDs for it - and anyway the other CD writer is a touch incompatible.

Are Pocket PC and Windows CE the same thing?

This is a FAQ on almost every Pocket PC and/or Windows CE development site and forum. This time I found the question on www.pocketpcdn.com.

Windows CE (.NET if a 4.x version, version 5.0 loses the .NET and is just Windows CE again) is a collection of pieces of an operating system, referred to as components. Each component is identified as belonging to Windows CE of a particular version and many of them are specifically written for Windows CE. An OEM, another third-party, or Microsoft themselves, can define a platform - a specific selection of those components to run on a particular device or compatible devices. A platform is a specialisation of the general set of components. Some of the components build on top of each other, others are completely independent, and others are mutually exclusive (e.g. you can only have one of the COM or DCOM runtimes).

Pocket PC is a platform defined by Microsoft; the 2000 and 2002 versions are based (loosely!) on Windows CE 3.0 while the 2003 versions are based on CE 4.2. Any OEM willing to license the software platform from Microsoft can do so, but must satisfy certain constraints (I believe) on screen size, application keys, and third-party components.

The tool for building your own platform comes in the Windows CE box and is called Platform Builder. You can find some videos demonstrating Platform Builder for Windows CE 5.0 at Channel 9. CE 5.0 will be released soon with a new Pocket PC version to follow later in the year, we believe. Once you've designed, built, and shipped your platform, you can generate an SDK from Platform Builder for use with (currently) eVC 4.0. A platform build consists of both compiling some components from source and linking other binaries together from supplied object libraries.

So, while Pocket PC includes many Windows CE components, some components of Pocket PC - notably the shell and messaging applications - are only available on that platform. It's possible to write programs which are source-compatible between Pocket PC and other Windows CE-based platforms, and even binary compatible, if you stick to components which are available on both platforms. So I guess the answer is really yes and no.

Windows Driver Foundation

Microsoft have decided to do something about the relative instability of Windows device drivers. They are, by all accounts, very hard to write.

Introduction to the Windows Driver Foundation

Interesting point: the Kernel-Mode Driver Framework will be supported in Windows 2000 and later (i.e. a lot of it will be compiled into/linked into the driver binary). I also see that USB device drivers will be moving into user-mode for Longhorn.

I recently had to update a CE device's OS image using a special download program and the device's Initial Program Loader, over a USB connection. The device required a special driver - which promptly crashed my PC (with a KMODE_EXCEPTION_NOT_HANDLED fault). I suspect a possible incompatibility with XP SP1 with the MS04-011 patch. Using an unpatched Windows 2000 SP4 box worked. Of course, I couldn't use Virtual PC, because it doesn't virtualise USB controllers, hubs or devices.

I can hear that cable!

Science and Subjectivism in Audio

A nice link posted to the comments of Tony Schreiner's post Breaking-in Electronics.

When working in Ents you got the occasional person who was fully willing to accept that there was no need for special cables in live audio mixing and power amplification - but was fully kitted out at home. I guess that PA equipment doesn't require that level of quality (?!)

The randomly-occurring bug

Rick Schaut, a developer on Mac Office, has posted a couple of articles about the biology of a difficult-to-reproduce software bug. Interesting reading.

Anatomy of a Software Bug

Repro, Man

What's it called, again?

As I think I've mentioned before, I've become the Exchange administrator at work. To understand what's going on better, and to keep up with developments, I've subscribed to Microsoft's Exchange Team blog (aka You Had Me At EHLO, a terrible pun).

Yesterday's entry covers installing the administrative tools (Exchange Manager and the Exchange Tasks for the Active Directory Users and Computers snap-in) on a server that doesn't have Exchange installed. The thing that caught my eye is that the entry is inconsistent about the naming of Windows Server 2003 - referring to it as both ".NET Windows" and "Windows 2003 Server".

Make your minds up!

Something that makes me a little uneasy is that the admin tools install a new service 'Microsoft Exchange Management.' Why??

Death of the NDR

NDR = Non-Delivery Report

If I send you mail, but for whatever reason it doesn't get delivered, your server, or an intermediate server, may send an NDR. If this happens, I'll never know about it. My SpamBayes configuration now considers 100% of email sent by postmaster@ to be spam.

Mail from postmaster@ can now be broken down into four categories:

• Spam posing as NDRs to persuade users to read it, and to slip through anti-spam barriers;
• Viruses posing as NDRs to get you to open the attachments, and for the reasons above;
• Bounce messages from other domains where people have sent spam using your email address or, in my case, anything with my domain - my mail ISP, Demon, gives a child domain of demon.co.uk to each subscriber; I can use anything I like before the @;
• Genuine failures to deliver a message.

However, I don't actually send all that much email. I tend to receive notifications of comments to this blog, mailing list mail, and notification of replies to comments on web sites such as CodeProject. There's a little personal mail but not much.

I've recently become the Exchange administrator at work (an entirely new installation - we've only just got an Active Directory domain going and moved everyone to that). It doesn't look like you can configure it not to send a copy of the failed message back to the sender - should I configure it not to send NDRs at all?

However, if we did that, perhaps almighty fuck-ups like the one that happened last week - where we asked to begin the process of transferring a domain, and the ISP went ahead and did the whole transfer, rewriting the MX records to point to the Exchange server - wouldn't be detected as quickly.

Korby reckons Windows will use VS Team System

"For larger teams and those who require integrated work item tracking and other software configuration management features, we've got a shiny, new Team Foundation server that we think you (AND the Microsoft Windows development team) will be using for many years to come." -- Korby Parnell.

Actually, the post was mainly about Visual SourceSafe, a product I cannot in conscience recommend to anyone. Any tool whose Best Practice guide recommends that you run a scheduled repair utility, so that the database corruption doesn't grow excessive, is just poor - particularly one you're relying on for your core business assets. Now, in fairness, a lot of the corruption experienced by many people is down to the usual culprits: flaky hardware, bad implementations of network file sharing in Windows 9x. Fundamentally, though, you have multiple remote clients modifying the same files - which has traditionally also caused problems for Microsoft Access - with no decent blocking lock support.

We know that Microsoft were using an internal system before Windows 2000 that basically forced a serialised development style; when it went into the final stabilisation stage, and the team were beginning to look towards Whistler (eventually Windows XP/Server 2003), they started to use a new internal system called SourceDepot (Source: Mark Lucovsky's USENIX presentation "Windows: A Software Engineering Odyssey" [PPT, HTML version]). Rumour has it (never officially confirmed) that SourceDepot is derived from a Perforce source-code license. It's a pretty good bet that Microsoft aren't allowed to sub-license their mods to anyone else.

VS Team System's version control component (code-name Hatteras) looks surprisingly (but no doubt superficially) like SourceGear Vault, from a descriptive point-of-view. Eric Sink (CEO of SourceGear) indicates the impact he thinks this will have on Vault (basically, innocent bystander caught in the cross-fire between IBM's ClearCase and Microsoft). So MS may be continuing in their long-standing tradition of dog-fooding - or Windows may have again grown out of its source control solution.

However, the future looks decidedly dim for SourceGear's SourceOffSite - similar features appear to be destined to appear in VSS 2005.

The Ten Bugs in the Known Universe

http://www.multimedia-people.co.uk/misc/bugs.html (originally by Andy Lowe, apparently MIA).

How did I get here? Let's follow the trail backwards:

John Robbins' Bugslayer column tips
Scott Hanselman's Blog
John Lam's Blog
Don Box

Sometimes I need a breadcrumb indicator in IE.

Eric Lippert on marshal-by-reference vs marshal-by-value

Eric uses the .NET term for marshal-by-value: Serialization.

1) Build a telephone system between Seattle and Hong Kong.  I get a telephone receiver with "CLIENT PROXY" written on it. You get a telephone receiver with "SERVER STUB" written on it.  Instead of talking to you, I talk into Proxy.  Proxy talks to Stub somehow -- I really don't care how the phone system works, so long as it does -- Stub talks to you.  We get the illusion that we're actually talking to each other, when we're actually talking to hunks of plastic, but the information content is the same, so who cares?  Maybe there is some delay and expense, but the proxy does a good enough job of sending and receiving messages that we can communicate across the barrier.

2) Sequence your DNA into a string.  Run your brain through a Molecular Neuron Defrobnicator that extracts all your memories and saves them to disk.  Put the DNA string and memory data onto CD-ROMs, and FedEx the box of CD-ROMs to Seattle.  Once I get them in Seattle, I rebuild your DNA from the sequence information using nanorobots. I inject the rebuilt DNA into an egg cell.  We use the egg cell to grow a copy of you in the lab.  When the brain is developed enough, I use my Molecular Neuron Refrobnicator to insert your memories into the clone's brain.

Marshal-by-ref versus Serializable Objects

This is great beer, where did I leave my head?

Currently drinking: Young's Special London Ale @ 6.4% by vol.

Nice.

Shame I can't type, really...

New element discovered!

Probably a repost...

A major research institution has recently announced the discovery of the heaviest chemical element yet known to science. The new element has been named "Governmentium". Governmentium has 1 neutron, 12 assistant neutrons, 75 deputy neutrons, and 111 assistant deputy neutrons, giving it an atomic mass of 312.

These 312 particles are held together by forces called morons, which are surrounded by vast quantities of lepton-like particles called peons. Since governmentium has no electrons, it is inert. However, it can be detected as it impedes every reaction with which it comes into contact. A minute amount of governmentium causes one reaction to take over 4 days to complete when it would normally take less than a second.

Governmentium has a normal half-life of 2 to 6 years; it does not decay, but instead undergoes a reorganization in which a portion of the assistant neutrons and deputy neutrons exchange places. In fact, governmentium's mass will actually increase over time, since each reorganization causes some morons to become neutrons, forming isodopes. This characteristic of moron-promotion leads some scientists to speculate that governmentium is formed whenever morons reach a certain quantity in concentration. This hypothetical quantity is referred to as "Critical Morass".

Thanks to Roger Allen, who posted this on CodeProject.

Programming Taboos

I remember why I wanted to keep that last document. It ties in with things that Larry and Raymond (can't find any references right now) and of course John Robbins have said, and also a thread on the CodeProject Lounge about gotos in the MFC source code.

I believe that, if we're going to deal with leaky abstractions, we have to know and understand what's going on under the abstraction. If you don't, when the system does something crazy - and it will - you won't be able to fix it. While our virtual machines are leaky, and especially while they're unreliable, you have to understand both the VM and the real machine beneath. This is particularly true in the edge cases where you're trying to interface between the VM and some native code.

Java tells you to wrap up your native code in a nice interface using the Java Native Interface. Fine, but that requires you can write C++ code (and I think it ties you to a particular JVM implementation - not sure on this). .NET tells you to marshal using [DllImport] attributes (or Declare statements, for VBites) which are a bit neater, but you suffer the leaking abstraction when the type you're trying to marshal has many incompatible options for implementation on the native side. I've answered (or tried to answer) a number of questions about P/Invoke interop marshalling, and people are always trying to do something crazy with strings. There are many ways to marshal a string, and some are better than others (hint: on the way in, pass an LPCTSTR; if you need to pass a string out, declare two parameters, an LPTSTR and a buffer size, and use a StringBuilder on the managed side).

Anyway, if you need to crack a tough problem, it helps if you have all the tools in your toolbox - you just don't have to use the dangerous ones. Don't get hung up on the taboos - if a tool is useful and appropriate for the situation, use it.

The great clearout, part 1

I suspect this is going to be more than one entry...

This weekend, my parents have a visitor coming to stay from Düsseldorf, Reading's twin town. She needs somewhere to sleep, surprisingly, so I have to clear out my sister's old bedroom - which, since I returned from University, has been both my mum's sewing room and my guitar practice/Xbox/TV/extra storage room. A number of things I brought back from Uni have never been sorted out properly - I've been in a 'ah, it's only temporary' mindset for the best part of three years.

So I decided to tackle one of the boxes - an old Amazon box containing various files (and, for some reason, a bunch of telephone cable, some cable tacks and a phone socket - I think I planned to fit this at the last student house but never got round to it). Inside, I found a few course notes, a collection of unused Christmas cards (I don't remember sending many cards, but I have a bunch of spares anyway) and an absolute stack of papers relating to my second second year (I'll talk about that some time, if I haven't already) group project.

This project required a small hotel management system to be written in Ada with a web front-end (actually, the web front end wasn't essential, but more marks were available if you did). I essentially designed and implemented the whole system - basically I came up with what was a reasonably efficient design (based on hash tables) using Ada generics. However, I wasn't too good at communicating the design or the implementation to my team-mates - and it did use a lot of the language's features that we'd barely covered in lectures. It just overwhelmed them.

Anyway, I've recycled all the printout and design notes because, well, it's basically irrelevant to anything I'm doing now (different OS, language, environment, etc) and I've got all the source code on my hard disk anyway.

Other noteworthy documents I found (and I'm going to link to, then recycle):

• Callbacks in C++ using template functors
• Gotos Considered Harmful and Other Programmers' Taboos (PDF)

Unwarranted speculation

Looks like OpenNETCF's back up - with this registrar. Maybe they were just changing hosts and DNS updates were delayed?

Obscure programming languages

Dare talked about the lack of XSLT 2.0 and XPath 2.0 in the forthcoming .NET Framework 2.0, and linked to Mark Fussell, who linked to 99 Bottles of Beer in various languages (programming languages, that is).

I think Ian would particularly like the Common LISP one which is made up of a complicated format string - in the shape of a beer bottle!

Ian's told me before about the BrainF**k language, but I'd not seen it before. Anyway, it seems that other people have written languages for bovines (beer-drinking bovines) and simians (alcoholic apes), along similar principles.

I can't help thinking that the T-SQL version should be set-based, though...

New features

Anyone reading in an aggregator might have noticed a bunch of posts disappeared for a short while this morning, then came back later. You might also find that the last month or so of posts suddenly reappeared.

Blogger have implemented comments, and I've gone back through and enabled them (a manual process) for everything that was appearing on the front page. I don't yet know whether items posted through BlogJet will get a comment link - I hope so.

Because of this change, I've removed the Contact link over on the right. I'll still know if you've left a comment. This might improve the amount of spam I'm getting, though I don't think it'll have any effect on all the spam sent with return addresses at my domain, for which all the bounce messages end up in my inbox grrrrr.

Reading List

What are the best development books? (blogs.msdn.com/jobsblog)

I tried to answer the question, but I virtually ended up listing the contents of my Programming shelf. There's very little dead weight on there. I've got something of a shelf at work, some of which gets used, some not. Other less immediate references stay at home. Here, then, is the home list: not a lot of commentary. Links go to Amazon UK, no kickback to me if you buy one, feel free to use another vendor. In some cases Amazon lists a different title; I've typed what appears on the cover.

Extreme Programming

Extreme Programming Explained (Beck)
Extreme Programming Installed (Jeffries, Anderson, Hendrickson)
(about four years ago, Ian's then employer were considering XP. I try to use some of the principles as appropriate but not yet properly using XP)

Programming Languages and Libraries

Programming in Ada 95 (Barnes) - the language taught in my first programming courses at University. Actually not my book, it belongs to Colin.

The C++ Programming Language, 3rd ed (Stroustrup)
The Annotated C++ Reference Manual (Ellis, Stroustrup)
Ruminations on C++ (Koenig, Moo) - actually, I've never finished reading this
Effective STL (Meyers) - I barely get to use STL

C++ In Depth Box Set, consisting of:

• Accelerated C++ (Koenig, Moo)
• Exceptional C++ (Sutter)
• More Exceptional C++ (Sutter)
• Essential C++ (Lippman)
• Modern C++ Design (Alexandrescu)

(read that last one if you want a template-induced headache - I maintain that Andrei Alexandrescu was the only good thing ever to come out of RealNetworks)

The Java Programming Language (Arnold, Gosling, Holmes)
The Java Language Specification (Gosling, Joy, Steele, Bracha)
I actually don't know Java that well. I bought these books to assist with my degree final project - a tool to produce diagrams of the static class structure of a program. I started out targetting C++ but discovered that C++ is, to put it mildly, a bugger to parse. I threw myself into the C++ parser so hard that I spent too little (i.e. practically no) time on the drawing and layout side. Eventually, to get something working, I abandoned C++ for Java - but too late. I just barely scraped the project, getting the minimum 40% pass mark, dragging my overall result down to a 2:2.

Windows Programming

I have so many books on Windows and Windows-oriented programming and Windows tools, it's difficult to know how to organise this. That shouldn't be too surprising, given as I work as a Pocket PC and desktop developer for a Microsoft-oriented ISV.

MFC Books

Teach Yourself MFC in 24 Hours (Morrison) - the main book I, er, taught myself MFC from. The main difference between this and other MFC books is that it's entirely code-focused - there's very little use of the Wizards. I maintain that if you want to be able to maintain an MFC program - indeed, any program - you need to know what it's doing, and relying on Wizard-generated code without understanding it is foolish.
Programming Windows with MFC, 2nd Ed (Prosise)

Low-Level & Raw Win32

Programming Applications for Windows, 4th Ed (Richter)
Programming Server-Side Applications for Windows (Richter, Clark)
Inside Windows 2000, 3rd Ed (Solomon, Russinovich)

.NET

Applied Microsoft .NET Framework Programming (Richter)
Programming Microsoft .NET (Prosise)
Shared Source CLI Essentials (Stutz, Neward, Shilling)

COM

Inside COM (Rogerson)
Transactional COM+ (Ewald)
(Essential COM is at work, as are Inside ATL and ATL Internals)

Other Windows

Debugging Applications for Microsoft .NET and Microsoft Windows (Robbins)
International Programming for Windows (Schmitt) - this was actually an error made by my sister (who works in Exchange tech support in the UK) when I asked for Programming Windows, 5th Ed (Petzold) (which is at work). Nevertheless I kept it and read it. Despite the UK being so much closer to countries with other languages and character sets, failure to program for international markets is just as prevalent as in the US (and we have the disadvantage to be based in the GMT time zone for half the year).

Miscellaneous

TCP/IP Network Administration (Hunt) - I ran, among others, a shared computer network at Aston Brook Green between 1998 and 2000. I was responsible for - well, made myself responsible for - DNS and DHCP, which was served from a Linux kernel 2.0.38 (RedHat 5.2, IIRC) box. It was pretty reliable in comparison to Windows 98, but then I wasn't playing games on it or running GUI apps. This was before I ever used Windows NT, which changed everything.

The Mythical Man-Month (Brooks)
Code Complete (McConnell) - I note a second edition is due in about a month.

Programming with POSIX Threads (Butenhof) - there are no good references on Win32 threads, such as when and how to use them. Programming Applications gives little guidance on signalling another thread - it's all about synchronisation. So I picked this up instead.

lex & yacc (Levine, Mason, Brown) - another legacy of my final project. If you want to build a parser, don't use these tools, they're way too confusing (and not powerful enough for C++, if you want an intelligible parser). Elkhound looks much more interesting.

Refactoring (Fowler)

That's everything I have at home; I may update this entry on Monday with what I have at work.

OpenNetCF lets domain slip

I can only assume that the people at www.opennetcf.org forgot to renew their domain registration - and now a Chinese hosting company has stolen the registration.

whois -h whois.pir.org opennetcf.org
Domain ID:D96652469-LROR
Domain Name:OPENNETCF.ORG
Created On:20-Mar-2003 14:15:41 UTC
Last Updated On:07-May-2004 10:54:48 UTC
Expiration Date:20-Mar-2006 14:15:41 UTC
Sponsoring Registrar:R64-LROR
Status:INACTIVE
Registrant ID:ONLC-637769-4
Registrant Name:Buy a domain
Registrant Organization:chinachanel
Registrant Street1:xiao meng china channel

Info courtesy of www.demon.net/external. 

BALEETED!

To confirm my suspicions about the patch I referred to in the last post not having been applied, I downloaded a Linux 2.6.5 kernel tarball. Having read the appropriate source files, I'm now scouring the appropriate section of the hard disk.

Ah, that's better - I don't feel so dirty...

I don't know where to start...

CRN: Microsoft Shelves NGSCB Project As NX Moves To Center Stage (via The Inquirer).

Boy, this article and the comments are so wrong I almost don't know where to begin. There follows a slightly edited version of my comment on the article.

NGSCB and No Execute are completely different things. No Execute applies only - I emphasize ONLY - to the ability to tell the processor not to execute code from given pages. Because Windows is a protected-mode operating system, only the kernel can set or clear this bit in the Page Table Entry. NX can help prevent the exploit of buffer overrun bugs.

Windows NT has always supported the ability to set execute permission on memory independent of read and write permission - look up the VirtualProtect API and the PAGE_EXECUTE flag. It requires hardware support, which has been lacking on x86 until now. AMD's 64-bit processors implement a No-Execute bit in the page table entry when the processor is running in 64-bit mode or in Physical Address Extensions 32-bit mode. Intel's Itanium processors also include an Execute bit in the page table.

Frankly, it's taken far too long to get execute protection on the x86, so AMD should be lauded for finally implementing it.

The Linux patch that another poster referred to: the follow-ups to that message basically damn the proposal on the grounds that it breaks GCC. Things may have changed in 7 years, of course. It works by reducing the length of the Code Segment not to cover the end of the address space - this causes the x86 processor to generate an Access Violation (oops! we're on a *nix - I mean a segmentation fault) if you try to set the instruction pointer outside this range. A clue to the general effectiveness of this proposal lies in the fact that this still isn't included in the kernel source tree (actually, as posted the code had no effect apart from taking another slot in the Global Descriptor Table because the CS register was set to the newly defined USER_HUGE_CS [the old behaviour] on return from taking a trap, and there was no apparent way to set it to anything else).

Replicating this on Windows would limit the number of threads dramatically and require a large change in the way DLLs are loaded. All thread stacks would have to be in the area of memory not covered by the code segment, which would have to be at the end because a segment represents a contiguous sequence of virtual addresses. The No Execute protection allows any virtual address to be protected; rather, that all virtual addresses that don't contain code will be protected by default.

NGSCB is about securing users' data and keys in an area only accessible through secure APIs. It has nothing to do with security vulnerabilities, except that both fall under a very rough umbrella of 'security'.

[edit: confirmed my suspicion about the Linux patch not being included in 2.6, and note that the article was revised after I posted this entry.]

Two most common bugs

The two most common bugs in my code are:

• Inverting a boolean condition.
• Forgetting to increment a loop counter/move a pointer in a while loop.

You'd think I'd have learned that these are my most common issues by now and to check them more thoroughly.

I just managed to lock up a device when installing a program because of a #2 in the install process...

An expensive day

My car went in for a (scheduled) service today. I asked the garage whether the driver's side mirror housing, which I broke about six months ago (reversing in the dark down my parents' driveway, I encountered a tree...) would pass the MOT (British yearly vehicle inspection, for any foreigners reading). Their considered opinion was that it wouldn't. The whole unit, mirror, housing and all, would need to be replaced.

Of course, the most expensive part of fitting a new mirror is painting the body-coloured back piece of the housing the same colour as the car. This part is available separately on the Focus, but I'd broken the main body of the housing, so the whole lot had to be replaced.

I got a call halfway through the day saying that they'd performed the MOT and it had passed, but they'd discovered that one of the rear brake cylinders (a hydraulic item which presses the brake pads against the drum) was leaking and should be replaced. Did I want to replace it? If it had been something like a cracked bumper, I'd have said no. But brakes are an essential safety item - despite costing £100, it needed to be done.

So, a third year (37,500 mile) service, plus the mirror, plus the MOT, plus the brake cylinder, plus VAT at 17.5% came to over £500. Ouch.

Oh well, at least it was scheduled, unlike my boss's car battery. /me smirks. During the time I've had the Focus, my boss has had a Mercedes SLK, a Mercedes C-class, and now has a BMW 5-series. The SLK had problems with the roof, the C-class I'm not sure of (perhaps he just didn't like it), and the 5-series can't keep a battery charge.

Something you might consider odd is that while I live in Reading, I had the car serviced at Marlow. I work in Cookham. Why not service it in Reading? Two reasons. Firstly, Reg Vardy run both the Reading Ford garages, and they're - how shall I put this? - crap. Platts at Marlow are a lot better, or so it seems. Secondly, it's actually more convenient to drive to Marlow and get a train back to Cookham (takes 15 minutes, but only once an hour) than it is to drive through Reading town centre, drop the car off, then get a bus back to Reading station, a train to Maidenhead, then change to the once-an-hour train to Cookham - and vice versa at the end of the day.

RIP, Sam and Max

It looks like LucasArts has decided to become a Star Wars-only development shop. They were producing a new game featuring Sam and Max, but decided to kill it off - for no apparent reason.

It's a shame. LucasArts adventures are always hilarious, but the last one they did was Escape From Monkey Island back in 2000.

Which reminds me - I was going to buy that Day of the Tentacle/Sam and Max Hit The Road double set. Somewhere along the line I've lost my copy of DOTT.

Google Ad-Words Gaffe

I don't think this particular ad is quite relevant...

Taxes and scarcity culture

US bloggers and websites are all talking about having to do tax returns today. Thank any gods that might be around that I don't have to do one - it's actually very unusual for UK residents with 'normal' tax affairs.

Anyway, the point of this post was to point to yesterday's WLCD comic. An interesting point on how you think differently if you don't have lots of resources.

Automatic DOS

The Windows Update servers have been under pretty serious stress over the last couple of days. Ironically, by encouraging everyone to turn on Automatic Updates, Microsoft have built a distributed Denial-of-Service system - that attacks their own site.

The Automatic Updates tool checks for any updates whenever your network is idle. (Anyone whose connection is charged per-byte should now go and turn it off!) If it finds any, it starts downloading them in the background, depending on how it's configured.

Those of us east of the US were probably out of the office when the latest set of updates were posted, at 10am PST, or (IIRC) 5pm GMT. Everyone whose computer was turned on queried the servers, found the updates, and started downloading...

If you're in a networked environment, you should look into getting Windows Update Services/Software Update Services. For administrators, this allows staging and testing of updates before deploying to workstations. It should reduce your network bandwidth as well as alleviating the load on the servers.

News Aggravator

I really need to sort out my news synchronisation. I read my feeds both at work and at home - the same set of feeds. At work I'm using Awasu and at home, NewsGator.

I spend quite a bit of time marking posts I've already read, as read - a task which is relatively tricky in Awasu.

Why use two aggregators? I don't really want to use NewsGator at work as I feel it would get in the way of what I'm supposed to be doing with Outlook, i.e. working, but then it doesn't seem possible to collect all new posts from all blogs into a single notification in Awasu, so I might change.

With respect to synchronisation, I don't want to pay for NewsGator Online Services. Maybe I need to write an Outlook plugin (no! not another one! it's slow enough already with SpamBayes, SpamNet and NewsGator!) which can do the sync for me. Of course it ought to use SIAM, but it'll probably end up being FTP (or could I just FTP a SIAM file?)

GNOME usability: still sucks

Miguel de Icaza points to a writeup of GNOME usability.

Taking each section in turn:

Simple Dialog Boxes

First off, let's consider that save dialog. A few questions immediately spring to mind:

• How do I overwrite an existing file?
• How can I have confidence I'm pointing to the right folder?

The list of files, even though it takes up screen space, allows two things to happen. You can overwrite an existing file by clicking it then clicking OK. You can be sure you're in the right folder, because you can see what else is in it.

It's well-known that users find a list of items easier to manipulate than a drop-down, so I'd argue that Windows' Places Bar is simpler than GNOME's Save In Folder drop-down. Windows appears to have a problem with this when using very large icons, where the bottom icon's label is clipped; however, this dialog is resizable.

There is one good thing about GNOME's Save dialog: the file-type drop-down shows the icon that will be used for the file. The dialog has too many buttons in the upper-right corner: do you really want to minimize a save dialog? What happens if you maximize it: do you get a full-screen dialog with massive edit fields? This is a form that shouldn't resize.

I then find this statement typical: "True, the dialogs may irk those who like tab completion and other esoteric features (I'm guessing their issues will be worked out in future releases)..."

Tab completion? In a selector box? Are you completely insane? It's very, very, very advanced users who will type a path into the File Name box to navigate to a different folder.

Simple Menus and Program Names

"Because free software environments like GNOME are founded upon cooperative development they can avoid the problems caused by corporate competition and branding. A user in Windows XP will have to navigate Windows Media Player, Real Networks Real Player and Apple Quick time in order to play media files. Their applications menu will be cluttered and the number of interfaces to learn is higher than in GNOME where a user must only find and learn Totem Movie Player."

As I've written before, everything would be a lot simpler for Windows users if Real and Apple would follow the platform conventions, DivXNetworks and the XviD project, and write their format decoders as codec DLLs, then write their players in terms of codecs. The user's choice of player would then be able to play any of the common formats - even without licensing those formats. But therein lies the problem - and it's also a problem for GNOME, because Real and Apple want to make money out of their formats.

XviD is probably contravening the MPEG-4 patents.

Simple Configuration Tools and Preferences

The Wallpaper Chooser link is broken, so I can't challenge that. In Windows, the Desktop tab of the Display properties is hardly difficult to find. The Position and Color drop-downs take a bit of thought, but not much. The Customize Desktop button is probably misplaced - maybe an Advanced button instead? Fewer options here would suggest fewer choices.

Yes, the IE preferences dialog is a lot busier. It also has many more options than Epiphany. You're trying to compare the interface of a Mercedes E-class with satellite navigation and CD auto-changer to that of a Ford Fiesta with a radio - of course it has more buttons!

Applications - Multimedia

Again, here you're comparing a really basic application with a complicated one. Totem offers no support for playlists, Internet Radio, CD ripping, copying to a portable device, organising your music library, selecting from that library. It's your choice whether these features are included in your media player, but condemning Media Player's interface because it has all these features is ridiculous. The basic operation is pretty simple: once you've ripped a CD, it appears in Media Library; hit the arrow next to Now Playing to get a menu of albums, artists etc, then select what you want to play. The Albums menu could be clearer: at work I have Tom Petty and the Heartbreakers' Greatest Hits and Queen's Greatest Hits I and II. The Albums menu simply shows the album name. However, there's always Media Library.

On top of this, if you want a simpler interface to Media Player, it's actually quite easy to write one. The reason is that the core of Windows Media Player is written as an ActiveX control, and you can write your own wrapper around that. The 'Windows Media Player' executable, at base, is simply a skinning engine that wraps the player control.

The bottom panel shown in the screenshot in this article isn't even shown by default in WMP.

As for WMP's 'bloated menus', they have to be: every feature that's exposed in the UI is also exposed in a menu somewhere.