No2ID Flair

Following Ian's example, I've added the No2ID PledgeBank petition flair to this site. It looks like this:

The counter updates with the number of people who have signed up. Meta: I need to think about moving off Blogger. Republishing the whole site (e.g. when changing the template) takes an age.

Check your signatures and use precautions

I’ve had 24 copies of the I-Worm/Mytob.HI worm since 1am today. Either they’re sending to spam lists or there’s a lot of infections out there.

This one’s being sent with faked ‘From’ addresses, info@<domain>, admin@<domain>, service@<domain>, etc. Mostly the message is about account suspensions, password updates, etc. Don’t be fooled. Don’t open the attachments. 

Mac & PC - will the performance comparisons end?

From Paul Thurrott: a link to Java Rants, “Will partnering with Intel give Apple a Mac faster than a PC?”

No.

Let me expand. The theory goes that the processor is laden down with years of backwards compatibility that costs performance – starting afresh with a special customised ‘x86’ with fewer instructions, without this backward compatibility, could improve performance. Apple’s ‘special’ x86s could then beat ‘regular’ x86s.

The trouble with this argument is that it’s bullshit.

The Pentium 4 does feature compatibility right back to the 8086. If the platform support is right, you can, it is believed, boot the original MS-DOS 1.0. But that doesn’t slow it down.

It’s correct that the core of the P4 is RISC-like, but only in the sense that the aim of RISC was to have very simple instructions that could be decoded by simple logic circuits which would execute on the core in one cycle. Modern ‘RISC’ processors are the same, effectively, as the core of the P4 – there are multiple execution units that can execute operations concurrently, in the same cycle. On the P4 there are two Arithmetic-Logic Units [ALUs] which can actually perform two operations per cycle – one on the first half of the cycle, the other on the second. The key is in how the instructions are decoded.

Simple x86 instructions are decoded pretty much as a true RISC processor would – using logic to directly decode an x86 instruction into a core-compatible micro-operation [µop]. The resulting trace goes in the trace cache – the P4 attempts to only decode a stream of instructions once, performing loops directly from translated instructions. Anything more difficult than this – say, indexed indirect memory loads with autoincrement – goes to a microcode ROM which contains a ‘program’ of µops to implement that x86 instruction. If the microcode program is greater than 4 µops in size, the execution core has to execute directly from the microcode ROM rather than the trace cache. I think that pretty much all of the clever out-of-order stuff is suppressed when this occurs. Hit a large instruction from ROM and the processor slows to a crawl.

Anyway, after that brief technical interlude, let me explain why removing backwards compatibility won’t speed up the processor. Because most of it’s implemented in the microcode ROM. I don’t know if you recall when the P4 first came out. A lot of commentators, running their tests on Windows 98, suggested that the P4 was actually slower clock-for-clock than the PIII. Guess why? Partly that code was optimised for the PIII which had different characteristics from the P4 – but also that Windows 98 still contained a lot of 16–bit code, which hit the microcode ROM.

If you don’t use the expensive instructions, you don’t incur any cost, particularly, for them being there. Presumably there’s a little more complexity in the decoder logic to determine that the instructions are in microcode.

People often make the same mistake with respect to Windows XP. They think that the existence of the DOS and Win16–compatible subsystems slows it down. Nope. That code lives in NTVDM.EXE and WOWEXEC.EXE, which aren’t even loaded unless they’re being used. I also need to be clear that the console window is not DOS. cmd.exe, the Command Prompt, is a 32–bit Windows application that uses the console subsystem. The difference between a console app and a Windows app is that a console app has a console created for it if its parent was not running in a console. What is a console? It’s a window which is created and updated by the CSRSS process. There’s some code in there to handle back-compat graphics but again this almost certainly isn’t loaded unless it’s being used. A lot of this has been ditched for Windows x64, but that’s because the processor doesn’t support virtual-8086 mode, necessary for this support, in so-called Long Mode (64–bit mode).

cmd.exe supports (I believe) all of the DOS command interpreter, command.com’s feature set, and extends it greatly. This, plus the command-line environment, leads a lot of people to be confused. Particularly when some of the system utilities, like more, exist as files named more.com. If you look with dumpbin or depends, you’ll see that more.com is in fact a Win32 console executable.

Speaking of x64, I note that Apple aren’t going straight for x64. Their Universal Binary Programming Guidelines [PDF, 1.5MB] talk about the 32–bit register set, not the 64–bit extended set. I expect that this decision is because the Pentium M with EM64T – codename ‘Merom’ – isn’t due out until late 2006.

This whole argument presupposes that Intel will make Mac-specific processors. I think that’s highly unlikely. Apple haven’t told us their motivations, leading to a lot of speculation. My view is that they are trying to both save money, by using a more commodity processor, and enable quicker access to newer technologies, by using more commodity chipsets (oh, and save money). It took them years to get AGP in Mac hardware, and it looked like taking even more years to get PCI Express. I don’t therefore expect Intel to produce ‘special’ processors for Apple – it’ll be stock parts or nothing.

Another reason for using stock parts is so that stock software tools can be used. Apple can mostly wave goodbye to maintaining their own compiler. It’s no accident that their guidelines state they’re using the System V i386 Application Binary Interface [PDF, 1.0MB], with only minor modifications. Of course, that means that to begin with, Apple will be using the very compiler – GCC – which they used to ‘prove’ that PowerPC G5s outperformed P4s, in that contentious set of benchmarks. Unless they’ve improved the optimiser, or learned how to use it properly, ‘PCs’ may well still outperform Macs.

One tiny area in which Macs may have a temporary advantage is in exception handling. Today, on 32–bit Windows, each use of an exception handler pushes a handler frame onto the stack, a cost incurred whether or not an exception actually occurs. I believe GCC uses a table-based scheme, which in general incurs that cost only when an exception occurs. However, Windows on x64 – indeed on every architecture other than 32–bit x86 – also uses a table-based scheme.

Passing the word

Want to keep your right, as a UK citizen[*], to privacy? Sign up for the No2ID pledge here.

Via Ian.

[*]Not a subject, dammit. I do not recognise the Queen as my ruler. Rulers are things you draw straight lines with – she’s a bit bumpy for that.

Watermark rendering bug in ListView

Or at least, in the ‘My Pictures’ folder in Windows XP Explorer. Try it yourself – ensure the watermark is enabled. Then right click and select ‘New Text Document’. Click to remove the rename highlight. Then select it again and press Shift+Del. Hit Yes.

If this reproduces on your computer, you’ll see something like this:

I guess the ListView code simply calls ScrollWindowEx to move any remaining lines up, but doesn’t take account of the watermark.

New technology plug: since Avalon is a stored graph system, it should handle this kind of thing for you. That is, you tell it what components make up your rendering – it renders the appropriate parts as required.

More Mobile Developer Shafting: Compact Framework 2.0 won't run on CE 4.x

I downloaded the Compact Framework 2.0 Redistributable yesterday as well, planning to do a little testing with one of our CF 1.0 apps. (I need to discover how to force using 2.0 if both are installed). In doing so I discovered that CF 2.0 only supports Pocket PC 2003 and Smartphone 2003, Windows CE 5.0 custom platforms, and Windows Mobile 5.0 (for Pocket PC and Smartphone).

That means that new devices, introduced this year, like Symbol’s MC3000 which runs a custom CE 4.2 platform, are not supported. Why aren’t Symbol using CE 5.0? Who knows. Maybe they didn’t want to revalidate all their components on the new OS yet.

Yesterday I was at the London Eye. They appear to use PPT 2800s with the optional trigger handle to scan the barcoded tickets. This device runs Pocket PC 2002.

Customers often want upgrades and/or new applications to run on existing hardware. We’d like to be able to use the newer tools to implement those upgrades or new applications. But we can’t, because those tools won’t run on the old hardware.

The full framework is compatible back to Windows 98. We can’t get three years’ worth of backwards compatibility in the mobile world.

.NET Framework 2.0 Compatibility

Brad Abrams: Take the .NET Framework 2.0 Compatibility Challenge.

I was at a slightly loose end yesterday afternoon, so I took advantage of that time to check that Meteor Server, and the price checking/stock availability application we wrote for a large deployment at a UK retailer, still work correctly under .NET Framework 2.0 Beta 2.

Meteor Server itself is mostly VB6 components – a server executable, an application-host component executable (ActiveX EXE in VB6 terminology), a few application-interface components which have strict binary-compatibility requirements – with a few bits of C++ where the VB6 environment really ran out of steam (TCP/IP printing, since TCP is a pain with the Winsock control, and a ‘callback’ component which replaces the use of minimum-delay timers). Applications are written as COM components. Recently – in the last couple of years – we’ve been writing new applications in VB.NET or C#. We’ve generally found that more productive than VB6. To do this we’ve built Primary Interop Assemblies [PIAs] for the application-interface components.

Meteor Server isn’t really .NET-aware. We do ship configuration files for the server and application-host executables, to force the latest version of the PIAs to be used. This allows older applications to run on a newer version of the server, without having to distribute the PIAs with each app, or all versions that have ever existed with Meteor itself. Remember that .NET binds to a specific version of an assembly, not to the most recently installed. We did try using Publisher Policy but had problems with it – sometimes the Framework would fail to load an application object, often crashing with an Access Violation in fusion.dll. Due to a misunderstanding in version numbers we had to ship multiple policy DLLs when using that scheme – one for every major.minor version of the PIAs we’d released.

Using explicit configuration reduces the frequency of this problem, but not to zero – we still occasionally see this in deployment, with Framework 1.1 SP1. It’s something of a heisenbug – it never happens when you’re looking for it. And a particular nightmare when dealing with VB6 code, because the VB6 debugger does not run your code as it will be run in deployment. Instead it hosts it in-process in the VB6 environment. You’re not even running the compiled code, I believe it translates the code to p-code and interprets that. While you can turn on PDB generation and debug with VS.NET or another native code debugger such as WinDbg, the generated PDBs are poor quality and don’t, for example, often include your local variables or globals properly.

Since it’s not really .NET-aware, the default behaviour of the Framework when loading into an unmanaged host applies – the most recent version of the Framework is loaded. We support loading alternate applications under application control in the newest versions of Meteor (I think the version currently available on our website is very out-of-date). The aforementioned price-checking application is actually implemented as four applications: a menu application which loads the others, the price-check application, the stock-availability application, and a head-office application which has testing features. This does mean that all applications will need to support the latest version of Framework, or the admin will need to tweak the config files to force all applications to bind to the older runtime.

Anyway, I’m pleased to report that there were no obvious problems using Framework 2.0 with any of these applications.

StarCraft: Ghost

How long have we been waiting for Blizzard’s StarCraft: Ghost? I can’t recall – I think they first announced it in 2001.

This week at E3 Blizzard have published new trailers and gameplay movies. I can’t help thinking it might be too little too late. What blew us away three or four years ago now looks a bit, well, ordinary. Halo 2 seriously raised the bar.

What’s been taking so long? I don’t know, but it looks like Blizzard have ditched their original partner Nihilistic Software, bought up a console developer, and brought the game back in-house.

The question for Blizzard: can they release this game before the next generation of hardware arrives and they look completely stupid?

Xbox Backwards Compatibility

Xbox Backwards Compatibility

Now that it's been announced at E3, I can finally reveal that the Xbox 360 feature I work on is... Xbox backwards compatibility!

Yes, it's real. It's been fun to watch all the wild speculation over the past year or so as to whether the Xbox 360 would or wouldn't be backwards compatible. And all the wild ideas about how hard or easy it would be. It's clear that a lot of people have very little understanding of how these things work.

Were I forced to speculate, I’d expect some kind of JIT-compiler like the x86–on-Itanium IA32EL or the x86–on-Alpha FX!32 is involved. I can’t remember the old rule-of-thumb for emulation – was it that you should have a processor ten times as powerful as the emulated processor to reach parity? Of course that was for state-machine-based emulators, where essentially each instruction was executed in turn by emulating instruction fetch, decode, execute, retire.

Whether a particular Xbox game runs on Xbox 360 will depend on just how closely the assumed-JIT and its library emulate the original processor and, of course, GPU. Hopefully the fact that Jet Set Radio Future was a launch title and was bundled with some consoles (e.g. mine!) will be taken into consideration!

Dammit, more incompatibilities

From Windows Mobile Platform Migration FAQ for Developers:

Starting with Windows Mobile 5.0, only one instance of the installer (wceload.exe) can be running at a time on Pocket PC devices. This restriction has always been in place on Smartphone. As a result, CAB files that start other CAB files from a custom setup DLL may not work on Windows Mobile 5.0. The workaround for this is to have a small executable in the CAB that a custom setup.dll starts in its Install_Exit entry point. This executable can get a handle to the running wceload.exe process, wait for wceload.exe to exit, and then restart wceload.exe on the additional CABs.

We have a pretty clever (says he who wrote it) technique of packing multiple CABs into one wrapper CAB file, which enables us to install Compact Framework, SQL CE, our barcode scanner hardware abstraction layer (ScanHAL), our simple large-key input method, and the application itself, all in one go. Looks like I’ll need to rewrite it slightly at the install-to-RAM stage.

Well, I hope it takes a while before devices appear

Today, Microsoft announced Windows Mobile 5.0 (codenamed ‘Magneto’). They’ve made two bone-headed decisions from a development perspective:

• ActiveSync 4.0 no longer supports TCP/IP synchronisation
• According to this page, eVC 4.0 cannot be used to develop for Windows Mobile 5.0. You must use Visual Studio 2005.
• Also, it seems you can’t use VS.NET 2003 to develop Compact Framework applications for WM5. Likewise, you must use VS2005.

The first can be worked around – you don’t need to use the ActiveSync transport or startup server for Platform Manager, but it’s a heck of a lot more convenient than using the TCP/IP Transport with the Manual startup server. With that combination you need to enter a horribly long command line (example: CEMGRC.EXE /T:TCPIPC.DLL /Q  /D:192.168.1.2:5913) on the device. VS.NET 2003 and 2005 do not use Platform Manager, though – trying to get both hooked up to the same device – as I tried to do today to actually view the contents of a memory block allocated in C# code, because opening the debugger memory window kills the debugging session – is highly problematic.

The second and third are idiotic. Hint to MS: VS2005 is not yet released! Yes, there’s a beta. No, I wouldn’t trust it for production development.

And there’s a new development. Mobile device development is a feature in the Standard Edition and above. It is not a feature of the Express Editions. Microsoft claim that there will be a Compact Framework SDK. Previously eVC was free. This made a big difference.

It’s not as if you can mix-and-match that well, either, because the current VS2005 beta apparently requires ActiveSync 4.0 – not actually released yet – in order to deploy to a device. Gee, thanks </sarcasm>.

Yup. Mobile developers get the shaft again. I’ve spent some time today trying to work out a way of avoiding getting blocked when calling InternetCloseHandle when a dial-up networking link has gone down. We seem to have the choice of either rebooting when blocked (enterprise app, so not completely out-of-bounds but undesirable, especially when the hardware cold boots when it was supposed to warm boot) or leaking handles, or rewriting to use WinInet asynchronously. WinInet isn’t well documented for the synchronous case, for the async case it’s downright abysmal.

How does a New submenu Command know where to create the new item?

On the command line, that’s how.

You must specify %1 in the command line somewhere, otherwise Explorer simply doesn’t do anything (it reports “Not enough storage to complete this operation”, an erroneous error message). The %1 is replaced by the full path to the file that would have been created.

Looking at Briefcase’s .bfc registry key shows that there’s also a second option passed to FormatMessage to create the command line, and it’s a number – the command line includes %2!d! which is indicative of the use of FormatMessage. It’s not clear what this second option means, however.

So that’s a step towards building our Burn Folders. Since our shell extension will be a DLL, we’ll probably include the New command as an entry point and use rundll32 to launch that. That keeps everything together.

Musings: How to produce a Burn Folder in Windows?

One of the new features in Apple’s Mac OS X ‘Tiger’ is ‘Burn Folders’ – a folder which you can easily write to a CD by selecting the ‘Burn’ option. It contains links to the files or folders to burn, not copies of the files. In other ways it’s just like any other folder. This allows the user to stage multiple CDs at once, and keep frequently-written CD data in an easy-to-write folder. How hard would it be to create one of these folders in Windows?

The Windows (Explorer) shell is extensible. One of the things you can extend is the shell namespace. The Recycle Bin is a shell extension. So is My Documents. A Burn Folder might therefore be something like Recycle Bin. It’s even more like a Briefcase, a feature much touted back in Windows 95 – it’s still there in XP, but its functionality is largely replaced by Offline Files for the situation where you’re taking your computer away from the office.

Let’s assume that when you create a Burn Folder, we actually create a real folder with that name with some special properties, and in it, when you drop an item, we store a Shell shortcut to the item you dropped. We should also support copy and paste, pasting links. By using shortcuts we should be able to leverage the Distributed Link Tracking service to keep track of where items are if you move the original file. I’m not sure what we do if the original file was removed – presumably we have to try to resolve all the links before starting to burn, and if some are missing we need to report that.

So, we have a namespace extension, and it sounds like we’re probably going to use a folder as the junction point. We’ll have to extend the New submenu to allow such a folder to be created (I guess we’ll have to use the Command option). We’ll have to modify the Explorer toolbar to show the Burn button. To address a complaint Paul Thurrott made about ‘Tiger’s Burn Folders, we should probably also modify the status bar to show the size of the linked files (i.e. how much space is required).

That covers the UI, then we have to be able to write to the disc. Here we’re pretty much stuck with just being able to handle CDs and writing them using the Image Mastering API.

This might be a fun thing to do. Something to add to my stack.

Full-screen is a new feature??

Paul Thurrott: Tiger math 2: When a feature isn't a feature

Apparently Apple are touting full-screen video playback as a new feature in QuickTime 7 in Mac OS X ‘Tiger’. Except that you actually don’t get it in ‘Tiger’, you have to pay for an upgrade to QuickTime Pro – the upgrade to the Pro edition of 6.5 cost $29.99.

In contrast, Windows Media Player has full-screen support out-of-the-box. I seem to recall that it has done ever since Media Player 6.4 on Windows 2000. Certainly the version of Media Player 6.4 that ships with Windows XP (run mplayer2 from Program Files\Windows Media Player), version 6.4.09.1130 on my computer, has this feature.

Scary...

On top, Emperor Palpatine from the Star Wars trilogy. On the bottom, Pope Benedict XVI.

Or is it the other way around?

Tory posters defaced in Reading

Seen on Microsoft Typography news: Anger after Tory ads vandalised.

It wasn’t me, but I can understand why they were vandalised. The Tories’ whole campaign is based heavily on ignorance – xyz can’t be that hard, can it?

Yes, usually it can. I’m surprised – well, actually I’m not – that Labour and the LibDems have not actually rebutted how hard it is to do xyz.

Bring Back Chris Brumme!

Chris Brumme’s blog was one of the first I started to read, after Scoble, Chris Sells and Don Box. He’s managed a tricky task of explaining very complex subject matter (how the .NET CLR works) well.

Sadly he’s not posted in nearly a year, and the last entry was just announcing updates to existing articles.

So I’m joining Chris Anderson and Brad Abrams in saying:

Putting things in perspective

Mozilla and all their fans, and FOSS people in general, are fond of saying that Firefox has had over 25 million downloads. Sounds like a lot, right?

I was on the Skype site earlier today. We use it at work in preference to calling mobile phones. (I was on the site because users were having trouble logging in – some kind of authentication server failure.) Skype are currently reporting 84 million downloads, and one million users using SkypeOut, the Skype-to-PSTN service.

Is Firefox really all that popular, then?

MS Press, what are you thinking?

It looks like a couple of Jeffrey Richter’s classic and essential Windows programming books have gone out of print:

• Programming Applications for Microsoft Windows, 4th Ed.
• Programming Server-Side Applications for Microsoft Windows

The .NET Framework and the C# programming language have huge productivity benefits, but there are times when you still need to get close to the metal. When you do, you need the best training possible. If you need to do it, those two books and Windows Internals, 4th Ed. are your best guide – IMO – to understanding what’s happening and how to make best use of the facilities at your disposal.

How to set the owner of an object in Windows XP

Normally, in Windows NT-family security, you cannot set the owner of a secured object (such as a file or folder, or a registry key) to someone else – you can only take ownership for yourself or a group you’re a member of. If you’re unprivileged you can only take ownership if the Discretionary Access Control List (ACL) gives you permission to take ownership. There is also a privilege – SE_TAKE_OWNERSHIP_NAME – which allows you to take ownership even if you’re not permitted in the ACL. This privilege is normally and by default only assigned to the Administrators group.

This ability to take ownership is in marked contrast to Unix where administrators can set ownership, but users cannot take it.

But what if you want to set ownership to a different user? If you try to execute this without any additional privileges, under I think any security principal, you’ll get an error. If you (like me) were using ADsSecurityUtility, it’ll be encapsulated in an HRESULT, code -2147023589 (0x8007051b). Anyone familiar with HRESULTs will recognise the 0x8007 high word as representing SEVERITY_ERROR | FACILITY_WIN32, i.e. a Win32 error code encoded as an HRESULT, so we can look up 0x51b = 1307 in WinError.h which is ERROR_INVALID_OWNER, “This security ID may not be assigned as the owner of this object.”

Looking this error code up in a search engine reveals the problem, that other people have had this problem before, and the fix. If you have the SE_RESTORE_NAME privilege enabled you’re permitted to set the owner of a secured object. Which makes sense, really, as you want restored data to have the same owner as it did when it was backed up.

Privileges come in two types – those which are enabled by default if they are permitted to a user (or a group of which the user is a member), and those which are disabled by default. Of the latter type, some Win32 APIs enable the privileges they require automatically. SE_RESTORE_NAME is not one; you must call AdjustTokenPrivileges to enable the privilege if not already enabled.

By default the Administrators and Backup Operators groups only have the SE_BACKUP_NAME and SE_RESTORE_NAME privileges. You can modify this and other privileges through Group Policy – for your own machine use the Local Security Policy shortcut in Administrative Tools. It’s really not recommended to modify these settings unless you’re absolutely certain you know what you’re doing. If you must, see under Security Settings > Local Policies > User Rights Assignment.