Monday, 29 August 2005

Answering the mail - anatomy of a solution

I’m a bit slow in answering questions from the contact box. A couple of months ago, a commenter asked how the MC3000 application for New Look was implemented. Since we’re using this system as a case study it’s probably fair to post details if I don’t go into too much detail.

Actually I’m not sure where New Look’s name came up. I don’t think I’ve mentioned it here. It must have been some other forum.

As far as the MC3000 software goes, the main application is just Meteor Client. Meteor is our application server product which allows text-mode applications to be written using Windows-based desktop/server development tools – typically more rapidly than the equivalent handheld application. On this occasion, the four applications (Price Check, Stock Availability and the ancillary Menu and Head Office applications) were developed with Visual Basic .NET using a SQL Server 2000 data source. The total number of handhelds deployed is over 1200 I believe.

Meteor Client was originally written for Symbol’s Series 3000 DOS handhelds and the original/common code is written in C targetting a Symbol library called ADL and Symbol’s BiosXxx routines (wrappers around both the standard IBM PC BIOS interrupts and Symbol’s own BIOS features). It was ported to a Windows console application which is mainly used for development and testing although it has been used in deployment occasionally. It was also ported to PalmOS for the Symbol SPT 1700 and Monarch’s Pathfinder Ultra DOS device, although those ports are not currently maintained.

Meteor Client on Windows CE was originally targetted at Pocket PC-based devices such as the PPT 2700. I started the porting work about three years ago. Pocket PC devices don’t have console support and even if they did, the console would still – I think – have ended up with a certain amount of ‘furniture’ we didn’t want. Meteor Client is meant to be a locked-down application, no other applications can be used.

Without this console support, I had to provide all on-screen rendering. The only monospaced font I could rely on was Courier New, which I detest – it would be illegible at the screen sizes we need – and we needed the potential for custom font capabilities anyway. Therefore we use bitmapped fonts using bitmaps I drew, designed to use the maximum space available on the Pocket PC screen and others at two sizes, 8 rows by 20 columns (matching the PDT 3100) and 16 rows by 21 columns (matching the PDT 6800). Further fonts have been drawn to offer better displays on devices with different screen sizes – the MC3000 has a 320x320 screen which can all be used – and there are now 8 sizes ranging from a 9x9 pixel character cell up to a 15x39 pixel character.

The Windows CE port uses C++ for the Windows CE-specific code, and uses ATL for windowing support and a small amount of additional data structure support. MFC is overkill and I’d originally intended to do without it, but when it came to integrating barcode support I remembered that the code I’d written up to that point used bits of MFC since the new CE apps we were writing at that point were all written with MFC (the .NET Compact Framework wasn’t released until 2003). As part of the process to turn the port code into a separate library for porting DOS applications, I’ve eliminated this dependency.

Actually, the core of the porting library is really the keyboard buffer. Windows’ keyboard system requires that a window accept keyboard input messages. There’s a bit of a mismatch between the DOS behaviour, which made both scan code and character code available at the same time, and Windows, which generates a WM_CHAR message with the character code after the WM_KEYDOWN message with the virtual key code, but this wasn’t too hard to work around. Originally all the screen and keyboard code ran on the same thread as the application code, which meant that the screen didn’t update until you called either a keyboard wait function or a delay function, but I’ve recently – as part of the porting library effort – moved all the screen functions, and hence the keyboard functions, to their own thread. Cue some fairly complicated synchronisation code.

For barcode scanning I originally used a chunk of code that was, although slightly enhanced and made into a static library, some of the first code I wrote after graduating. Its main problem was that it only supported Symbol’s scanning APIs – it would run on a non-Symbol device but barcode scanning would be disabled. Over December to February 2002 we came up with a more abstract API that we call the Scanner Hardware Abstraction Layer, or ScanHAL for short. We now support Intermec and HHP’s APIs as well as Symbol’s, and have a package that can detect what device it’s running on and therefore install the right ScanHAL (there’s a null stub for devices with no scanner). This code is all C++ and, due to an error that was overlooked when building the first few versions, is tricky to use from C as all the entry point names are type-decorated (i.e. using the C++ linkage naming convention – we forgot to use ‘extern “C”’). We use this library, enhancing it as new features are required, in all our .NET Compact Framework applications too (if scanning is required). A colleague wrote the API and the initial version of the Symbol and Intermec HALs; I’ve enhanced both and wrote the HHP HAL, while Ian’s extended all the HALs at least once.

The New Look handheld software doesn’t just consist of Meteor Client though. There are a few other things too. Primary among them is support for firmware updates over-the-air (i.e. over the network). Symbol’s AirBEAM supports over-the-air upgrade but it doesn’t support scheduling an upgrade and for obvious reasons we didn’t want to push a firmware upgrade – which since they’re using the colour terminal is over 30MB – during work hours. Symbol do have a tool intended for making an update image on an SD card which we’ve adapted. Meteor Server pushes a notification of a new version to Meteor Client, which schedules the upgrade to occur at a random time within an administrator-specified time window. When that time occurs the updater program I wrote downloads the required files over HTTP (using the WinInet library), checks that the files downloaded correctly using the CRC32 algorithm, then runs our unpacker for Meteor Client and the other software, then Symbol’s updater. When the updater finishes successfully it cold boots the device which causes the new software to install. The install-after-cold-boot mechanism is another tool that’s evolved over my four years with the company – primarily my work.

To ensure that the update happens when the administrator wants it to, the device’s clock must be fairly accurate. Windows CE does supply a Simple Network Time Protocol client, but Symbol didn’t include that component in the MC3000 platform. We’d had trouble with its reliability on another device previously. I therefore wrote one myself, which took a couple of days – it’s less than 750 lines of C++ code (including blanks and comments) which includes converting from UTC to the local time zone since this part of Windows CE is often unreliable – it will tend not to take daylight saving into account, or do it twice. We therefore turn off the AutoDST setting in the registry – we update the clock every few hours anyway so being slightly wrong at the start and end of daylight saving doesn’t matter. It’s somewhat simplified code – the time is only computed to an accuracy of one second, which means no fixed point maths, and no roundtrip delay calculation is done. This provides plenty of accuracy for our purposes though.

The final part is the keyboard map. We do two things here: first, we map the two keys at the top of the MC3000’s keyboard which have, respectively, a green and a red surround so that they get key codes. The default keyboard map leaves them unmapped. These keys, being directly below the screen, are used as soft keys in the applications. Secondly, we found that since there’s no indicator light near the Alpha key, users could not tell when Alpha mode was active – we had a few support calls in the trial deployment asking why the handheld was simply beeping when any key was pressed, which was simply because Alpha was on. New Look use the 28–key keyboard. Style codes, colour codes and SKUs are all numeric, so there’s little need for alphabetic input, but the customer wanted to keep access to alphabetic characters for future applications. We therefore remap Alpha to a function key code and, in Meteor Client, toggle the on-screen keyboard when that key code is received.

That’s just the client side. This project, being the first to use multiple co-operating Meteor Servers to provide scale-out and failover capability, required some extensive work on the server side too.

Sunday, 14 August 2005


In my last post I said Microsoft would be releasing some official information on a topic that had caused plenty of FUD on forums. They haven’t yet done so. This is kind of annoying. The guy I was talking to hasn’t responded to my query about where the statement would appear. So the world – or at least, that part of the world that discusses these things on forums – carries on consuming and spreading FUD.

Some parts of the Windows team reacted very quickly to the suggestion that Monad could allow the spread of viruses. This team needs a boot up the backside, IMO. Scoble, are you listening?

OK, enough beating about the bush – I asked about OpenGL, and specifically this topic. Now I come to re-read the topic, it’s clear – if the app asks for accelerated pixel formats, and there is an Installable Client Driver available, Windows Vista will switch off the desktop compositor while the ICD is in use. What effect this will actually have I don’t know – presumably it will temporarily revert to using software desktop rendering, i.e. directing each window to draw in turn, clipping the parts that are obscured, as Windows has done for about 18 years (Windows 1.0 only allowed tiled, not overlapping windows). And of course you’ll only notice this if you’re using a windowed OpenGL app – full-screen apps (like games) will no doubt want the desktop compositor turned off!

The stuff about taking a 50% performance hit is simply FUD, IMO. The presumption is that translating OpenGL calls to Direct3D calls is expensive, but I don’t really see that. Numerous sites have had posters claiming that OpenGL is a more performant API than D3D, but without a program released for both APIs with as much time spent on both implementations, benchmarked on a range of hardware to eliminate driver variability, there really is no way to tell. I think the basis for this is essentially iD fanboys – since iD use OpenGL for their Quake, Return to Castle Wolfenstein and Doom 3 engines, it must obviously be better, right? I honestly don’t know, except to say that the games I’ve played have always seemed ‘fast enough’ if my system’s been within recommended spec, whatever API was used.

Wednesday, 10 August 2005

If you want to know something, ask

Conspiracy theorists: if you want to know why Microsoft are doing something, try asking them.

I recently saw some information on forums that seemed wrong, so I did a bit of research and found a presentation that seemed to apply. Microsoft presenters normally put their email addresses on the cover, and this was no exception, so I emailed the presenter asking him to clarify a few points. I got a nice reply back which covered what I was after.

Yes, I know that’s very wooly. I asked if I could blog the conversation and he said no. However, there should be some official information appearing later today. When it does, I’ll update (if I remember).

When in doubt, ask. If they can’t say, they’ll normally say so.

Thursday, 4 August 2005

Where's the damn SDK?

As I’m sure you’ve already seen, Windows Vista beta 1 was released about a week ago. This means I’m thinking about resurrecting my Changes to Win32 API in Longhorn series. Except, I can’t yet – because the SDK hasn’t been released yet!

Some kind of developer preview this is…

(Yeah, OK, the WinFX SDK has been released. But I’m still something of an unreconstructed Win32 guy.)

Loving the new monitor

I finally decided I needed more desktop space: both computer desktop, and the real desktop my monitor sits on – and this computer sits under. So I decided it was time to replace my old faithful Iiyama VisionMaster Pro 400 17” CRT with a flat panel. I looked at a number of sites and picked the ViewSonic VP191b 19” panel. So I can now use 1280x1024 comfortably.

It took a little while to adjust it to how I like it – colours seemed a little washed out – and it’s very bright. For tips on adjusting your monitor, see Charles Poynton’s article on ‘Brightness’ and ‘Contrast’ controls.

It cost a lot – nearly £400 after VAT and delivery – but I reckon it’ll last. It’s got something to live up to – the Iiyama lasted seven years with me, and is still going strong.