Violence in video games

Well, I don't know about you, but I don't think video game violence actually has that much effect IRL. I mean, it's had no effect on me. I'm absolutely fine  

Anyway, it appears that Halo 2 (not mentioned by the Daily Mail, apparently) will be rated 16+ in the UK:

"The VSC was totally unconcerned with the murderization of Covenant forces. Apparently, if the game didn’t allow you to turn on your allies, we’d have gotten the UK equivalent of a PG, regardless of the galloons of alien blood we had flooded the Galaxy with. Britain is now completely unconcerned with bad language, as I discovered watching BBC TV in the hotel that night. Just as well, since every other word out of our game seemed to be bastard., asshole or bullshit.

"I did point out that technically, you can kill tens of thousands of Grunts, Elites, Brutes, Jackals and “others” while playing a typical game. “No problem, as long as they don’t look human,” said one of the chaps." -- Bungie Weekly Update 30/7/2004

It takes *how* long?

For those not versed in floating point representations, NaN means 'Not a Number' - typically the result of division by zero.

(Windows Update v5, BTW)

Daily WTF

...is a blog containing code (and sometimes other things) that make geeks go, WTF??

Today's horror just looks like a random jumble of appending constants that probably didn't need to be constants. Until you look more closely, and you realise it's constructing XML. Except there's a bug: tags are being closed with a backslash - \ - rather than the correct slash - /.

Just in case anyone needs the hint:

private static string FormatData( SqlDataReader dr )
{
   using ( StringWriter sw = new StringWriter() )
   {
      XmlTextWriter tw = new XmlTextWriter( sw );
      tw.Formatting = Formatting.Indented;
      tw.WriteStartElement( "Items" );
                
      while ( dr.Read() )
      {
         tw.WriteStartElement( "Item" );
         tw.WriteAttributeString( "key", (string)dr[0] );
         tw.WriteString( (string)dr[1] );
         tw.WriteEndElement();
      }
      tw.WriteEndElement();
      return sw.ToString();
   }
}

If you need to write XML, use an XmlTextWriter. The code is shorter and clearer, and is more likely to work, considering cases like escaping invalid characters. Also note that I got it to indent the XML for me. Running this on a reader containing the results of SELECT emp_id, lname FROM employee on SQL Server's pubs database, we get:

<Items>
  <Item key="A-C71970F">Cruz</Item>
  <Item key="AMD15433F">Devon</Item>
  <Item key="A-R89858F">Roulet</Item>
  <Item key="ARD36773F">Domingues</Item>
  <Item key="CFH28514M">Hernadez</Item>
  <Item key="CGS88322F">Schmitt</Item>
  <Item key="DBT39435M">Tonini</Item>
  <Item key="DWR65030M">Roel</Item>
  <Item key="ENL44273F">Lincoln</Item>
  <Item key="F-C16315M">Chang</Item>
  <Item key="GHT50241M">Thomas</Item>
  <Item key="HAN90777M">Nagy</Item>
  <Item key="HAS54740M">Snyder</Item>
  <Item key="H-B39728F">Bennett</Item>
  <Item key="JYL26161F">Labrune</Item>
  <Item key="KFJ64308F">Josephs</Item>
  <Item key="KJJ92907F">Jablonski</Item>
  <Item key="LAL21447M">Lebihan</Item>
  <Item key="L-B31947F">Brown</Item>
  <Item key="MAP77183M">Paolino</Item>
  <Item key="MAS70474F">Smith</Item>
  <Item key="MFS52347M">Sommer</Item>
  <Item key="MGK44605M">Karttunen</Item>
  <Item key="MJP25939M">Pontes</Item>
  <Item key="M-L67958F">Larsson</Item>
  <Item key="MMS49649F">Saveley</Item>
  <Item key="M-P91209M">Pereira</Item>
  <Item key="M-R38834F">Rance</Item>
  <Item key="PCM98509F">McKenna</Item>
  <Item key="PDI47470M">Ibsen</Item>
  <Item key="PHF38899M">Franken</Item>
  <Item key="PMA42628M">Accorti</Item>
  <Item key="POK93028M">Koskitalo</Item>
  <Item key="PSA89086M">Afonso</Item>
  <Item key="PSP68661F">Parente</Item>
  <Item key="PTC11962M">Cramer</Item>
  <Item key="PXH22250M">Henriot</Item>
  <Item key="RBM23061F">Muller</Item>
  <Item key="R-M53550M">Mendel</Item>
  <Item key="SKO22412M">Ottlieb</Item>
  <Item key="TPO55093M">O'Rourke</Item>
  <Item key="VPA30890F">Ashworth</Item>
  <Item key="Y-L77953M">Latimer</Item>
</Items>

Changes to Win32 API in Longhorn - Windows Base APIs

From WinBase.h:

• New system exception type EXCEPTION_POSSIBLE_DEADLOCK. Is the OS detecting deadlocks for us?
• New CreateFile flags: FILE_FLAG_MUI_REDIRECT, FILE_FLAG_MUI_FALLBACK, FILE_FLAG_COPY_TEMPLATE_METADATA - used by the loader?
• CopyFileEx flag COPY_FILE_COPY_LINK. Symbolic links in the filesystem at last? Yes: CreateSymbolicLinkW.
• GetQueuedCompletionStatusEx function and new structure OVERLAPPED_ENTRY. Apparently able to remove multiple requests from a completion port in one call.
• CreateIoCompletionPortEx, takes an additional ObjectFlags DWORD over CreateIoCompletionPort.
• New I/O completion/asynchronous APIs SetFileCompletionNotificationModes and SetFileIoOverlappedRange.
• Kernel-level reader-writer locks and condition variables, more like POSIX threads. The relative weakness of the Event has been a long-held criticism of the Windows threading model.
• Encode/Decode{System}Pointer APIs. I could have sworn I'd seen this documented somewhere, but Google can't find it now. I think the idea is to protect an in-memory pointer to a sensitive structure until required, as a security measure.
• NUMA support for virtual memory allocation with VirtualAllocExNuma - allocate memory with affinity to one or more nodes. New flag NUMA_NO_PREFERRED_NODE. You can also create file mappings on specific nodes using CreateFileMappingNuma, MapViewOfFileExNuma.
• INHERIT_CALLER_PRIORITY flag (CreateProcess?)
• JIT_DEBUG_INFO structure (with 32- and 64-bit variants) which may go along with a new IDebugSymbols3 interface declared in DbgEng.h.
• New OpenFile flags? OF_MUI_REDIRECT, OF_MUI_FALLBACK. OF_ flags are also used by LZOpenFile and AVIFileOpen.
• Acquire and Release (memory fence) versions of InterlockedIncrement, Decrement, Exchange, CompareExchange.
• 32-bit versions of InterlockedAnd, Or, Xor. More Interlocked bitwise operations.
• Ability to push a whole list onto an interlocked singly-linked list (InterlockedPushListSList).
• Guaranteed thread stack sizes: new API SetThreadStackGuarantee.
• Ability to explicitly set thread priority: SetThreadActualPriority. Can query with GetThreadActualPriority.
• GetThreadStartInformation - find out where any thread started executing, and what the lpParameter parameter passed to CreateThread was.
• 64-bit tick counter: GetTickCount64. Either higher resolution or greater range (or both?)
• SetEndOfFileEx. Now you can just say how long the file is, you don't need to SetFilePointer first (and you can also keep your place...)
• Walk and delete file-system trees: WalkTreeW and DeleteTreeW.
• Transactional file-system backup semantics: BACKUP_TXFS_DATA used with BackupRead/BackupWrite.
• Enhanced mutexes: CreateMutexEx{A/W}. You can now use some object flags and specify what access you want to the mutex (perhaps you only need to know it exists, rather than being able to SYNCHRONIZE?) Ditto CreateEventEx, CreateSemaphoreEx, CreateWaitableTimerEx.
• New Reserve object. Some kind of kernel object (like a job??) to which you can join threads and associate processes, then later 'disjoin' and disassociate them. You can set the bandwidth for the reserve... this must be the resource reservation API, to provide guaranteed QoS to threads and applications. No more choppy Media Player?
• NeedCurrentDirectoryForExePath API. Takes a string parameter called ExeName and returns a BOOL. A get-out clause for the post- XP SP1 LoadLibrary behaviour?
• EnumResourceTypesEx - look for resources in MUI DLLs with specific languages? Also EnumResourceNamesEx and EnumResourceLanguagesEx. Updating resources also extended: BeginUpdate/Update/EndUpdateResourceEx.
• 64-bit systems Windows-on-Win64 file system redirection using Wow64EnableWow64FsRedirection.
• GetFileAttributesEx new flag GetFileExInfoNoReparse - prevent walking through symlinks?
• 'Fast' search for FindFirstFileEx: new flag FindExInfoFast.
• Enhanced I/O cancellation: cancel synchronous I/O occurring on another thread with CancelSynchronousIo, and possibly find out how much data was already transferred on an asynchronous I/O using CancelIoEx, which takes a pointer to an OVERLAPPED structure.
• Explicit thread pooling: CreateThreadpool, SetThreadpoolThreadMinimum/Maximum, Pause, Resume, CreateThreadpoolGang(?), Timers, Waits.
• A bit of diagnostics: GetOSProductName.
• Transactions (transactional file data support?): SetCurrentTransaction, GetCurrentTransaction.

That's a lot of new capabilities right there.

Academic biography and C++ parsing

Of course, due to my peculiar route through university, I ended up studying both the C and C++ courses twice, which tended to reinforce the principles - then for my final year project, I got very familiar with the C++ syntax and semantics. My friends already know my background, but I've had a few comments from people who don't, so here we go:

I entered Aston in September 1996 to study for an M.Eng in Electronic Systems Engineering - a four year full-time course. For any foreigners, a normal Bachelor's first degree in the UK has three years of full-time study, often optionally with an additional year spent on industrial placement. Normally, a Master's degree is postgraduate study, but M.Eng is atypical - it's a four-year first degree. When I started, you were supposed to get industrial placement work during summer vacations, but this rule was relaxed when it became clear that there weren't any placements available.

I attended that course for three years, scraping passes after resitting exams in both the first and second years (in one resit exam, my grade shot up from a failing grade to over 80%). In the third year a combination of factors (illness, extensive involvement in Entertainments, increasing complexity of mathematics) led to my deferring the exams. Over that summer I questioned my future direction and decided to apply for a transfer to Computing Science, which was granted. The departments accepted that my studies so far had covered all the material necessary to proceed directly to year 2. I completed year 2 with decent grades, then completed the final year fairly well, graduating with Lower Second-Class Honours (a 2:2, or Desmond).

The main thing that dragged my final Honours grade down was my final-year project. I'd decided to tackle a program to diagram the static structure of a program, with C++ as the source language. I got totally bogged down in the syntax and semantics of C++, and basically failed to complete the project. I don't do things by halves - I was trying to persuade a variety of parser generator tools to interpret standard C++ correctly. I suppose that I was taken in by my Programming Language Implementation lecturer, who claimed that parsing was the bit of CS that could be called 'science', because we know how to do it.

Not for C++, we don't. The commercial C++ parsers are either hand-written recursive-descent parsers which don't really follow any 'scientific' method (Visual C++), or based on yacc/bison with so much feedback between parser and lexer stages, and with so many bizarre actions in the middle of rules, that you can't call them LALR(1) any more (GCC).

Yes, I could have adapted GCC to handle the parsing. But quite frankly I didn't want to spend three months or more understanding how to hook into it.

The problem with C++ is just that it's hugely ambiguous and overloaded, with the distinguishing symbols - if any - an indeterminate distance from the beginning of a clause. This makes it great to write, IMO, and it's OK for humans to read given that we can see the whole line, but very hard to write a parser for. The syntax actually allows a great deal more expression than most programmers ever use, but the parser really has to accept all of it.

C++ really requires a stronger parsing method: LR(k). I now recommend that people look at Elkhound if they want to parse C++ and don't want to use an available parser.

What I should have done was to use Visual C++'s CL and BSCMAKE tools to generate the source browser files and then use the Browser Toolkit to interpret the BSC file, and generate diagrams from that. But I don't think I knew it existed at the time.

CS at Aston no longer offers C

...or any unmanaged programming, for that matter, apart from a small amount of assembler in year 1. It looks like the dedicated academics have won out over the practical people.

I was involving myself in a conversation on CodeProject about college courses (a US-based individual noted that their college offered COBOL!) and wanted to link to my old BSc Computing Science course (which I graduated from back in 2001). Then I noticed something horrible...

Java has taken over, at least above first year (Introduction to Systematic Programming is still taught in Ada). Object-Oriented Programming is now taught in Java (when I took the course, it was C++) and a new course, Internet Programming with Java, takes the place of the C course, Software for Systems Programming.

Now I'll admit that a lot of my fellow students didn't 'get' OOP because they got bogged down in the ugly mechanics of C++. The lecturer wasn't too hot, IMO, and wasn't properly focused on the OOP mechanisms of C++. Also, at the time, the course compiler was GCC 2.x which didn't do templates and the standard C++ library very well, so we were using a lot of the C subset of the language.

It looks like the Computer Graphics course might still be taught in C, but there you have another problem - in spending time teaching the language, you lose time to teach the actual meat of the course.

But how can you send out new graduates with no practical knowledge of low-level systems programming? If you have to break out of the cosy managed environment, how will you cope? Someone still has to write the VM and runtime, after all, and the system dependencies. Are we failing to train our software engineers to write the next generation of VMs?

Adapting to the executing platform

A standard way of using features not supported by your baseline platform but supported by later versions is to use the LoadLibrary and GetProcAddress routines to get a handle to a DLL, then get the address of the function, if it's present.

You can do this in .NET by using [DllImport], calling the function, then handling the MissingMethodException that results. However, throwing an exception is a relatively lengthy operation.

Junfeng Zhang reports that a new method GetDelegateForFunctionPointer has been added to the System.Runtime.InteropServices.Marshal class in .NET 2.0. You can now LoadLibrary and GetProcAddress, then construct a delegate from the resulting function pointer.

Changes to Win32 API in Longhorn - Update Delayed

Nothing to see here, move along...

I haven't quite had the energy to do the next stage. I'm thinking about tackling winuser.h and wingdi.h next, plus probably the GDIPlus headers.

In mitigation, yesterday I watched Coupling for half an hour and spent a long time investigating the ListView.Click issue, which was originally raised on CodeProject's C# forum. Guess what the second response was? "TreeView.Click doesn't work either!" Sheesh...

Tonight I got home pretty late anyway, then my wireless connection stopped working after I read the blog collection and checked for updates. I decided to apply the firmware update to the router, but that didn't solve it, so I hit Disconnect on the View Available Wireless Networks window, then hit Refresh Network List - and nothing appeared. I ended up uninstalling the Virtual PC Network Switch driver and the IPv6 stack (which I'd been looking into a while back) which didn't help, then punching in the SSID and WPA keys again - still nothing.

OK, I thought, let's reboot. After rebooting, the network had disappeared from the Advanced Settings screen. Great - I enter it again, and it works (I can't just click on it in Available Wireless Networks because the router's set to not broadcast the SSID).

I've found that XP SP2 wireless + my card's drivers + Virtual PC Network Switch don't work together anyway. If you have VPCNS enabled on the wireless connection, nothing happens: any packets directed at the connection disappear into a black hole. Disabling VPCNS on the connection sorts it out, although I found I had to Disconnect and Connect in Available Wireless Networks before it would work again.

Anyway, that and normal routine household tasks took up most of the evening. Hopefully I'll get to it tomorrow.

Now I read that back, I realise that I've probably spent as much time writing up the excuse as I would have just writing it. Oh well.

[Edit:]

I need to state that actually, VPCNS works fine on the wireless connection - it was the action of installing it which caused the stack to go bonkers. Disconnecting then reconnecting sorted out the stack. I'd been having weirdo problems with it not working properly when resuming from Hibernate - again, disconnect and connect sorted it out. Let's see if it stays sorted - OK so far.

Because I hibernate this machine (it's a guzunder - it goes-under the desk) uptime doesn't have a lot of meaning. I use hibernate rather than power off because it hibernates faster than it shuts down, it resumes faster than it starts up, and I don't lose my context if I was working on something. We're talking < 10 seconds from press-power-switch to unlocking the console. Also, because of the way it works (dump physical pages to disk) I think the disk cache also remains 'hot'.

Linking it together

Or rather, following the links.

The Brothers Chaps, at homestarrunner.com (rss) have uploaded a new video, for the new song Experimental Film by They Might Be Giants. I still remember TMBG best from "Birdhouse In Your Soul" and "Istanbul (not Constantinople)", dating back to 1990.

This spurred off a link in my head to one of my favourite bands, The Presidents of the United States of America (or PUSA, to most fans). I searched for their site and discovered that they're releasing a new album in August! Unfortunately, the new single is only currently available on iTunes. Er, sorry, but I don't want to give Apple any money.

Their last album, "Freaked Out and Small" was released on the MusicBlitz website, then as a CD. I found out about this one through a link from one of the webcomics I read, which linked to a great animated video by Chris Maguire for the song "Meanwhile Back In The City". The animation is totally unofficial - from the link, it looks like it was undergraduate project work. Still cool, though.

Get patching

This month's Microsoft Security Bulletins (rss) have been released. I suggest applying the patches reasonably quickly. Of seven bulletins, one applies only to IIS 4.0 on NT 4.0 (to which I must ask why have you not upgraded??), one applies to the POSIX subsystem on NT 4.0 and 2000 (not included in XP and 2003), and one to the Utility Manager in Windows 2000.

I'm hoping that the dead silence in the bulletins regarding XP SP2, and Windows Update v5 not showing anything, means that these issues are already fixed in SP2. It looks like all the binaries on my machine have later build dates and build numbers.

Public Service Blogging: ListView.ItemActivate

The ListView.Click event works in the desktop .NET Framework. It doesn't work in the Compact Framework, even in the beta SP3: it doesn't fire.

Nor does it fire in .NET Compact Framework 2.0 Beta 1, which I've been looking at this evening, a bit. You get a warning when you compile:

warning VSD101: Members not supported by the Device platform should not be called: System.Windows.Forms.ListView.add_Click is not a supported method in this platform

It turns out that you're not supposed to use Click - you're supposed to use the overarching ItemActivate event, which is fired when either you click, or double-click depending on configuration, or when you activate the item with the keyboard. I suppose this makes sense for Smartphone, since you can't 'click', the screen not being touch-sensitive.

You configure whether ItemActivate fires on single- or double-click by setting the Activation property on the control.

Note the nice bit of abstraction leakage in the warning message: the line of code reads:

listView1.Click += new EventHandler(listView1_Click);

Er, where did I call the method add_Click? Answer, it's lurking behind that nice += syntax.

OK, so I'm using the wrong semantics; it should still work.

Changes to Win32 API in Longhorn - Common Controls

The Common Controls API is largely implemented in commctrl.h. Let's see what's new (note, of course, everything is subject to change; documentation for these functions and flags isn't yet written).

• New Histogram control: ICC_HISTOGRAM_CLASS flag for InitCommonControlsEx, many structures, messages, notifications, flags etc
• Notification message on font change: NM_FONTCHANGED
• New custom draw flags CDRF_DOERASE, CDRF_SKIPPOSTPAINT (marked internal)
• New message NM_GETCUSTOMSPLITRECT and accompanying structure
• New flag ILC_ORIGINALSIZE for ImageList_Create
• New flag ILD_ASYNC for ImageList_Draw
• New flags ILS_EMBOSS, ILS_REFLECT, ILS_INVERT for ImageList_DrawIndirect
• Reference to SparseImageList (value HBITMAP_CALLBACK - enables image to be supplied on demand, dynamic images in ListViews?)
• Checkboxes and non-resizable columns in Header controls (HDS_CHECKBOXES, HDS_NOSIZING)
• Data filtering by date (HDFT_ISDATE used with HDITEM structure)
• Hit testing header-items can report On Item State Icon (HHT_ONITEMSTATEICON)
• Header state icons: new macros Header_GetStateImageList and Header_SetStateImageList
• Ability to use accelerator keys with a header control (?): HDM_TRANSLATEACCELERATOR
• Item notifications: HDN_BEGINFILTEREDIT, HDN_ENDFILTEREDIT, HDN_ITEMSTATEICONCLICK, HDN_ITEMKEYDOWN
• Oddly extended toolbar class: "ToolbarWindow32_Mil"
• Callback image labels in a toolbar: TBSTYLE_EX_CALLBACKIMAGELABELS
• "Use CustomDrawColors to RenderText regardless of VisualStyle": TBCDRF_USECDCOLORS for toolbar custom draw
• Transparent, splitter rebars: RBS_EX_TRANSPARENT, RBS_EX_SPLITTER
• Notification of rebar splitter drag: RBN_SPLITTERDRAG
• Tooltip can have a preview bitmap: new member hbmp on TOOLINFO structure
• 'Themed' hyperlinks in tooltips: TTS_USEVISUALSTYLE style flag
• Animated progress bars (?): PBM_ANIMATE, PBM_TRANSITIONTOPOS messages
• New List View style flags LVSIL_GROUPHEADER, LVSIL_IMAGETITLE (draw your own conclusions - groups of items in a list view?)
• List View item flags LVIF_IMAGETITLE, LVIF_TASKS, LVIF_COLFMT, structure LVTASKS and corresponding fields in LVITEM
• New list view item search flags for LVM_GETNEXTITEM message: LVNI_VISIBLEORDER, LVNI_PREVIOUS, LVNI_VISIBLEONLY, LVNI_SAMEGROUPONLY
• New flag for LVM_GETITEMRECT: LVIR_NUMORDER
• Hit test flags for ListView groups: group header, footer, expand/collapse button, background, state icon, contents.
• New macro ListView_HitTestEx to support group flags
• Additional flags for controlling ListView column formats
• ListView extended styles: Justify Columns, Transparent Background, Transparent Shadow Text, Show Numeric Order, Auto Auto Arrange (Icons automatically arrange if no icon positions have been set), Show Header in all modes, Auto Check Select (?), Auto Size Columns
• Alpha-blend background images: LVBKIF_ALPHABLEND
• New ListView views: Carousel, Panorama
• New options for ListView groups (different from above groups? LVGROUP structure extended) and new styles
• ListView Tile View 'extended' size flag: LVTVIF_EXTENDED
• Column formats for tile info columns
• Bitmaps on ListView infotips
• Is a list item visible? ListView_IsItemVisible macro
• NMLVLINK structure: links in a ListView? Also LVN_LINKCLICK notification
• LVN_ORDERCHANGED notification
• Async drawing of ListViews
• Enhancements to Animation control: ACM_OPENXFILE, ACM_SETTEXTURE messages
• Split buttons: a host of new messages, styles, flags, structures and notifications. I assume this is like the Open button in VS2003.
• Edit control: Set Highlight, Get Highlight messages
• Unknown flags: DCHF_TOPALIGN, DCHF_HORIZONTAL, DCHF_HOT, DCHF_PUSHED, DCHF_FLIPPED, DCHF_TRANSPARENT, DCHF_INACTIVE, DCHF_NOBORDER. Might be related to:
• DrawScrollArrow API

Of course, it's possible that a lot of this is to support the sample UI in current Longhorn builds, and might disappear before the final software is released. Heck, it could disappear before the next build is released.

Changes to Win32 API in Longhorn

Joel Spolsky's post How Microsoft Lost the API War has had a fair amount of press, and has got a number of developers running scared - some even talking about switching to Linux development. A common theme is that Microsoft won't be advancing the state of the existing API: all new features are going into Avalon/WinFX. People seem to have implied from this that they'll have to rewrite all their apps from scratch, which should not be the case at all.

I wrote a message on the CodeProject forums yesterday which I'll include here:

Don [Box]'s blog post Teaching COM made it pretty clear that Microsoft are extending COM for the Longhorn generation.

They've previously said that the Win32 stack will continue to be present in Longhorn and demonstrated old DOS apps running on Longhorn builds. I don't have a Longhorn SDK handy, so I can't tell you how much Win32 is being enhanced. The new window management and drawing APIs will be exposed through managed interfaces only, I believe, but your UI and data are abstracted, aren't they? Joe Beda, an Avalon architect, talks about this in a Channel 9 video.

Managed C++ or the new C++/CLI syntax should allow you to build a new front end to your existing C++ code.

I seem to recall that Avalon architect Chris Anderson has talked about being able to host Win32 GDI+USER content in an Avalon application, and being able to host Avalon controls in a Win32 application. Windows Forms is just a special case: a Win32 application written in managed code.

Visual Studio 2005 comes with new versions of MFC and ATL, version 8.0. See Visual C++ 2005 Libraries Features for a list of the changes: note that MFC can now host Windows Forms controls, for example.

Remember that Microsoft's platforms and developer tools division's first customer is their applications division. These guys aren't going to want to rewrite Word: there's twenty years of code in there. Rick Schaut, a Mac Word developer, talks about analysing a bug which caused Mac Word to complain "Disk is Full" when trying to save.

They're not going to throw that fix away. They're going to write a new rendering layer, to render to Avalon, new frame UI code targetting Aero, then compile the rest of the monster using CL.EXE's /clr switch. I seem to recall that one of the targets for /clr was that the compiler had to be able to compile Word. I'd expect Microsoft to compile the whole of Word to IL, rather than native code, for future generations so that the same binary runs on both 32-bit and 64-bit machines. Of course, there's always P/Invoke if a component truly can't be compiled to IL.

Anyway, I've now downloaded the Platform SDK for XP SP2 and the Longhorn SDK, and I'm running a diff between the Include directory of the LHSDK and the version that ships with VS2003. Bizarrely, Microsoft have built the XP SP2 SDK from the XP SDK, which means it's impossible to build programs which use features new to Windows Server 2003 using this SDK - those features are missing. Thanks a bunch.

I plan to add new posts covering various areas of the SDK that have been updated. I'll add links to this post as I post them.

• Common Controls
• Windows Base APIs

Jumping the hurdles: how Download.Ject infected

I thought I'd take a look at how Download.Ject infected, and the security barriers it managed to breach. I've gathered this together from what analysts have hinted at, so some information may be wrong.

Firstly, the attacker had to infect the web server. IIS has a feature which allows you to set a footer to be appended to every page. Internet Explorer parses content outside the <html> tags - but so does Firefox. Try this file:

<html>
<head>
<title>Test Page</title>
</head>
<body>
<p>Body</p>
</body>
</html>
<script language="javascript">window.alert( "Hello" );</script>

Opening this in a local file on XP SP2 gives you the Information Bar again; the Local Machine Zone has been locked down. You don't get the popup. Firefox gives you the popup.

Download.Ject exploited the Private Communications Transport (a secure sockets channel) flaw reported in MS04-011 to upload and execute arbitrary code. This uploaded code manipulated the metabase to add a footer containing a script.

The script in the footer, if run on Internet Explorer, exploited a vulnerability reported in MS04-013 which allows code to run in the Local Machine zone. It also repurposed the ADODB.Stream class, which is marked safe for scripting, to download files from the attacker's server to the victim's computer. The download was necessary to get code loaded into another process on the victim's computer. The effect of SP2's Local Machine Zone lockdown is that the script forced into the LMZ couldn't run. Note that no vulnerability in ADODB.Stream was exploited: it simply permits something that in retrospect is too permissive.

So the attacker had to have worked out how to exploit PCT, how to get arbitrary code running on the system with the user's full privileges, and how to get extra files downloaded onto the user's system. Administrators could have prevented the first by applying a megapatch; users the second by applying a patch.

Techworld.com - Browser rival to ActiveX in the offing

Techworld.com - Browser rival to ActiveX in the offing

And as a follow-up to the last post on browsers, the Other Browser Group (everyone who makes a commercial web browser, that isn't just a wrapper for one or more rendering engine, barring Microsoft) announce that they're going to extend scriptability on binary plugins. These binary plugins will be built using a C-based unmanaged API. Here's what they have to say about security:

"The security model for making calls through this API is much like the general same-origin security model enforced by the browser. That means that script from an origin other than the origin of the page that loaded the plugin is not able to access methods and properties on the plugin. The same thing applies the other way too, the plugin can reach only JavaScript objects in the same origin as the page that loaded the plugin.

"In addition to this, a further extension to this API is being discussed that would give a plugin greater flexibility by letting the plugin control the origin of the calling code, so that the plugin can specify the origin of calls that come from internally loaded code from other origins. This way such code can be executed with only the privileges of the origin of the code, and not the privileges of the plugin page's origin."

They've essentially described denying cross-site scripting. This is all very well, but if there's a flaw in the implementation, you still have XSS attacks. There's no sandboxing, no requirement to mark components safe for scripting or initialization, no way for the user to limit which sites can and cannot use script. It's all or nothing. IE has all the above (and the cross-site scripting restrictions).

Fundamentally, with the ability to load unmanaged components, the code can do anything at all. They don't describe any way for components to be recognised as written by a particular publisher or to limit the user's ability to download unsigned controls (possible through Group Policy for IE). The slight limitation is that of course the browser can only load components which conform to the required interface; there are many fewer of those for the Netscape plug-in API than there are COM components.

Let's look at how things work on XP SP2. In Firefox, say I want to watch Strong Bad's latest e-mail and I don't have the Flash player. Clicking that link gives me a big blank area saying 'Click here to get the plugin'. Clicking that gives me a lengthy dialog with a Get the Plug-in and a Cancel button, which doesn't mention security at all. Clicking Get the Plug-in button takes me to Macromedia's download page, so I click Download Now. I get a dialog asking me to save the installer. Once it's downloaded I open the installer, and step through the wizard, which shuts down the browser.

It appears that Firefox can load plugins on the fly, but Flash doesn't bother - it just restarts the browser. The suggestion seems to be that this will become easier - plugins could be downloaded after prompting the user. Netscape 4.x supported SmartUpdate, which does have code-signing, but this required LiveConnect, not present in Firefox. Firefox already supports XPCOM, XPInstall and XPConnect. The first allows binary extensions, the second allows extensions to be installed on demand, and the third allows those extensions to be scripted.

You can get an XPCOM extension which loads ActiveX controls. As a demo, in the download section, click one of the Click Here links. Firefox pops up a dialog:

Note "Install Now" is the default, although it is initially disabled for two seconds. I didn't hit Install Now because firstly I don't want it and secondly it doesn't work in Firefox at present.

Let's try the equivalent on IE. Remember this is SP2. First I have to persuade IE that Flash doesn't exist by deleting it from the registry. Now we go again to Homestarrunner.com. I get the information bar:

Clicking the bar and selecting "Install ActiveX Control" reloads the page and brings up the new security dialog:

Hitting Install downloads and installs the player, then loads the content straight away. Note that "Don't Install" is the default. IE also watermarks files that it downloaded: when you try to open one, you get this dialog:

I suspect the next generation of IE will support .NET-based plug-ins which use the .NET code access security model. It'll probably still support loading ActiveX controls, though.

IE 6.0 SP2 forthcoming?

Previously, Microsoft have said that the IE security changes in XP SP2 won't be released for other OSs. However, I was just skimming PressPass and saw this quote:

"In addition to this configuration change, which will protect customers against the immediate reported threats, Microsoft is working to provide a series of security updates to Internet Explorer in coming weeks that will provide additional protections for our customers.

"Later this summer, Microsoft will release Windows XP Service Pack 2, which includes the most up-to-date network, Web browsing and e-mail features designed to help protect against malicious attacks and reduce unwanted content and downloads. A comprehensive update for all supported versions of Internet Explorer will be released once it has been thoroughly tested and found to be effective across a wide variety of supported versions and configurations of Internet Explorer."

(from Microsoft Statement Regarding Configuration Change to Windows in Response to Download.Ject Security Issue)

Does this mean they've changed their minds? Will a separate IE 6.0 SP2 be released? I hope so.

Download.Ject patch

There's now a patch available for the Download.Ject trojan - or rather, one of the holes it used (since the site hosting the code was taken down last week).

The Register: Microsoft half fixes serious IE vuln

The Register get it wrong again, of course: I anticipate that this is the full and final patch for this issue. The ADODB.Stream vulnerability is simply that a feature - the ability to write to a file anywhere on the hard disk - was made available to the web. The patch sets the 'kill bit' in the registry, preventing this control from being directly loaded from IE, as detailed in KB 870669. MS have clearly decided that compatibility and functionality have to be sacrificed on the altar of security.

A naive user reading that could argue that IE's security model is the wrong way round - instead of killing known unsafe controls, only known safe controls should be permitted. It actually does work that way, for most zones: controls must be marked 'safe for scripting' to be scripted and 'safe for initialization' to load parameters from the web page. The Local Machine Zone is different; that will be, by default, locked down in XP SP2 RC2. The 'kill bit' is for this situation - a control marked safe is in fact unsafe in a particular scenario.

The other vulnerabilities exploited were already patched in MS04-013 (client-side) and MS04-011 (server-side). If admins had applied the 04-011 patch rather than being scared off by adverse media reports, this wouldn't have happened. It's an admin's responsibility to not be an attack vector, as well as protecting their own systems. The Internet users may even have been collateral damage - IIS 5.0 is used much more widely in intranets. Of course there were incompatibilities with this patch, largely due to a new kernel binary being released for Windows 2000. We needed a hotfix and got stuck in the Product Support queue behind a bunch of the problem reports.

Yes, this exploit is a side-effect of IE being able to load and script any COM component in the local machine zone - get over it. As I say above, this will be locked down in XP SP2 (and already is in RC2). IE's come a long way since the bad days of IE 3.0, which didn't do any checking. It's also a side-effect of the user running as an Administrator, which we all know is a bad thing. If you're not running as Admin, I suspect that Download.Ject couldn't actually install the backdoor components.

Note that Firefox has the XPConnect module, which allows XPCOM plug-ins to be scripted, potentially leading to the exact same problems for any installed extensions. It's just no-one's found any vulnerabilities yet.

[Edit] That's a bit strong, really. It will only apply to Firefox extensions, not every control/component installed on the user's system. Still, that's probably enough scope, since almost everyone will have Flash installed. [/Edit]

True Lies

Just finished watching True Lies. Hard to believe it's ten years old, actually. For a Governator picture, it's pretty good, but then the supporting cast is bloody good: Jamie Lee Curtis, Tia Carrera, Art Malik, and a certain Eliza Dushku (better known as Faith from Buffy).

Even at age 14 she was cute, although to be honest the role's fairly small.

Art Malik is currently wasting himself in Holby City.

Around this time, Hollywood had a real habit of casting Islamic/Arabic villains such as the 'Crimson Jihad' in this one. I wonder if the perception of the faces of terrorism (i.e. the predominant nationality/worldview of the terrorists) is in fact based on this, rather than on fact? Wikipedia's list of terrorist groups is only about 25% Islamist.

Freedom!

Well, of a sort. I went out and bought a wireless LAN ADSL router and a wireless card, so I now have Internet access at home from my own PC. I'm already running XP SP2 RC2, so firewalling is covered (and I'm free of Norton Internet Security, yay).

I'll admit to being sorely tempted by a wireless adapter for my Xbox (which is in yet another room) but I could only find an 802.11b one - maybe I'll order a g version from somewhere else.

Anyway I'm now patching Office (2000 SR-1a - ouch) and getting files off the other PC. Which is a story in itself - the network setup wizard in XP Home SP1 is, er, a bit broken (yes, dammit, I want to call the workgroup something other than MSHOME, thank you so very much) and NIS is getting in the way somewhere still, I think. I can see the other PC and mine in Network (My Network Places, for those who don't habitually delete My from every profile they create) but from their computer - nothing at all.

The subtext is that I want to download Whidbey beta 1 but don't want to actually write CDs for it - and anyway the other CD writer is a touch incompatible.