Death of the NDR

NDR = Non-Delivery Report

If I send you mail, but for whatever reason it doesn't get delivered, your server, or an intermediate server, may send an NDR. If this happens, I'll never know about it. My SpamBayes configuration now considers 100% of email sent by postmaster@ to be spam.

Mail from postmaster@ can now be broken down into four categories:

• Spam posing as NDRs to persuade users to read it, and to slip through anti-spam barriers;
• Viruses posing as NDRs to get you to open the attachments, and for the reasons above;
• Bounce messages from other domains where people have sent spam using your email address or, in my case, anything with my domain - my mail ISP, Demon, gives a child domain of demon.co.uk to each subscriber; I can use anything I like before the @;
• Genuine failures to deliver a message.

However, I don't actually send all that much email. I tend to receive notifications of comments to this blog, mailing list mail, and notification of replies to comments on web sites such as CodeProject. There's a little personal mail but not much.

I've recently become the Exchange administrator at work (an entirely new installation - we've only just got an Active Directory domain going and moved everyone to that). It doesn't look like you can configure it not to send a copy of the failed message back to the sender - should I configure it not to send NDRs at all?

However, if we did that, perhaps almighty fuck-ups like the one that happened last week - where we asked to begin the process of transferring a domain, and the ISP went ahead and did the whole transfer, rewriting the MX records to point to the Exchange server - wouldn't be detected as quickly.

Korby reckons Windows will use VS Team System

"For larger teams and those who require integrated work item tracking and other software configuration management features, we've got a shiny, new Team Foundation server that we think you (AND the Microsoft Windows development team) will be using for many years to come." -- Korby Parnell.

Actually, the post was mainly about Visual SourceSafe, a product I cannot in conscience recommend to anyone. Any tool whose Best Practice guide recommends that you run a scheduled repair utility, so that the database corruption doesn't grow excessive, is just poor - particularly one you're relying on for your core business assets. Now, in fairness, a lot of the corruption experienced by many people is down to the usual culprits: flaky hardware, bad implementations of network file sharing in Windows 9x. Fundamentally, though, you have multiple remote clients modifying the same files - which has traditionally also caused problems for Microsoft Access - with no decent blocking lock support.

We know that Microsoft were using an internal system before Windows 2000 that basically forced a serialised development style; when it went into the final stabilisation stage, and the team were beginning to look towards Whistler (eventually Windows XP/Server 2003), they started to use a new internal system called SourceDepot (Source: Mark Lucovsky's USENIX presentation "Windows: A Software Engineering Odyssey" [PPT, HTML version]). Rumour has it (never officially confirmed) that SourceDepot is derived from a Perforce source-code license. It's a pretty good bet that Microsoft aren't allowed to sub-license their mods to anyone else.

VS Team System's version control component (code-name Hatteras) looks surprisingly (but no doubt superficially) like SourceGear Vault, from a descriptive point-of-view. Eric Sink (CEO of SourceGear) indicates the impact he thinks this will have on Vault (basically, innocent bystander caught in the cross-fire between IBM's ClearCase and Microsoft). So MS may be continuing in their long-standing tradition of dog-fooding - or Windows may have again grown out of its source control solution.

However, the future looks decidedly dim for SourceGear's SourceOffSite - similar features appear to be destined to appear in VSS 2005.

The Ten Bugs in the Known Universe

http://www.multimedia-people.co.uk/misc/bugs.html (originally by Andy Lowe, apparently MIA).

How did I get here? Let's follow the trail backwards:

John Robbins' Bugslayer column tips
Scott Hanselman's Blog
John Lam's Blog
Don Box

Sometimes I need a breadcrumb indicator in IE.

Eric Lippert on marshal-by-reference vs marshal-by-value

Eric uses the .NET term for marshal-by-value: Serialization.

1) Build a telephone system between Seattle and Hong Kong.  I get a telephone receiver with "CLIENT PROXY" written on it. You get a telephone receiver with "SERVER STUB" written on it.  Instead of talking to you, I talk into Proxy.  Proxy talks to Stub somehow -- I really don't care how the phone system works, so long as it does -- Stub talks to you.  We get the illusion that we're actually talking to each other, when we're actually talking to hunks of plastic, but the information content is the same, so who cares?  Maybe there is some delay and expense, but the proxy does a good enough job of sending and receiving messages that we can communicate across the barrier.

2) Sequence your DNA into a string.  Run your brain through a Molecular Neuron Defrobnicator that extracts all your memories and saves them to disk.  Put the DNA string and memory data onto CD-ROMs, and FedEx the box of CD-ROMs to Seattle.  Once I get them in Seattle, I rebuild your DNA from the sequence information using nanorobots. I inject the rebuilt DNA into an egg cell.  We use the egg cell to grow a copy of you in the lab.  When the brain is developed enough, I use my Molecular Neuron Refrobnicator to insert your memories into the clone's brain.

Marshal-by-ref versus Serializable Objects

This is great beer, where did I leave my head?

Currently drinking: Young's Special London Ale @ 6.4% by vol.

Nice.

Shame I can't type, really...

New element discovered!

Probably a repost...

A major research institution has recently announced the discovery of the heaviest chemical element yet known to science. The new element has been named "Governmentium". Governmentium has 1 neutron, 12 assistant neutrons, 75 deputy neutrons, and 111 assistant deputy neutrons, giving it an atomic mass of 312.

These 312 particles are held together by forces called morons, which are surrounded by vast quantities of lepton-like particles called peons. Since governmentium has no electrons, it is inert. However, it can be detected as it impedes every reaction with which it comes into contact. A minute amount of governmentium causes one reaction to take over 4 days to complete when it would normally take less than a second.

Governmentium has a normal half-life of 2 to 6 years; it does not decay, but instead undergoes a reorganization in which a portion of the assistant neutrons and deputy neutrons exchange places. In fact, governmentium's mass will actually increase over time, since each reorganization causes some morons to become neutrons, forming isodopes. This characteristic of moron-promotion leads some scientists to speculate that governmentium is formed whenever morons reach a certain quantity in concentration. This hypothetical quantity is referred to as "Critical Morass".

Thanks to Roger Allen, who posted this on CodeProject.

Programming Taboos

I remember why I wanted to keep that last document. It ties in with things that Larry and Raymond (can't find any references right now) and of course John Robbins have said, and also a thread on the CodeProject Lounge about gotos in the MFC source code.

I believe that, if we're going to deal with leaky abstractions, we have to know and understand what's going on under the abstraction. If you don't, when the system does something crazy - and it will - you won't be able to fix it. While our virtual machines are leaky, and especially while they're unreliable, you have to understand both the VM and the real machine beneath. This is particularly true in the edge cases where you're trying to interface between the VM and some native code.

Java tells you to wrap up your native code in a nice interface using the Java Native Interface. Fine, but that requires you can write C++ code (and I think it ties you to a particular JVM implementation - not sure on this). .NET tells you to marshal using [DllImport] attributes (or Declare statements, for VBites) which are a bit neater, but you suffer the leaking abstraction when the type you're trying to marshal has many incompatible options for implementation on the native side. I've answered (or tried to answer) a number of questions about P/Invoke interop marshalling, and people are always trying to do something crazy with strings. There are many ways to marshal a string, and some are better than others (hint: on the way in, pass an LPCTSTR; if you need to pass a string out, declare two parameters, an LPTSTR and a buffer size, and use a StringBuilder on the managed side).

Anyway, if you need to crack a tough problem, it helps if you have all the tools in your toolbox - you just don't have to use the dangerous ones. Don't get hung up on the taboos - if a tool is useful and appropriate for the situation, use it.

The great clearout, part 1

I suspect this is going to be more than one entry...

This weekend, my parents have a visitor coming to stay from Düsseldorf, Reading's twin town. She needs somewhere to sleep, surprisingly, so I have to clear out my sister's old bedroom - which, since I returned from University, has been both my mum's sewing room and my guitar practice/Xbox/TV/extra storage room. A number of things I brought back from Uni have never been sorted out properly - I've been in a 'ah, it's only temporary' mindset for the best part of three years.

So I decided to tackle one of the boxes - an old Amazon box containing various files (and, for some reason, a bunch of telephone cable, some cable tacks and a phone socket - I think I planned to fit this at the last student house but never got round to it). Inside, I found a few course notes, a collection of unused Christmas cards (I don't remember sending many cards, but I have a bunch of spares anyway) and an absolute stack of papers relating to my second second year (I'll talk about that some time, if I haven't already) group project.

This project required a small hotel management system to be written in Ada with a web front-end (actually, the web front end wasn't essential, but more marks were available if you did). I essentially designed and implemented the whole system - basically I came up with what was a reasonably efficient design (based on hash tables) using Ada generics. However, I wasn't too good at communicating the design or the implementation to my team-mates - and it did use a lot of the language's features that we'd barely covered in lectures. It just overwhelmed them.

Anyway, I've recycled all the printout and design notes because, well, it's basically irrelevant to anything I'm doing now (different OS, language, environment, etc) and I've got all the source code on my hard disk anyway.

Other noteworthy documents I found (and I'm going to link to, then recycle):

• Callbacks in C++ using template functors
• Gotos Considered Harmful and Other Programmers' Taboos (PDF)

Unwarranted speculation

Looks like OpenNETCF's back up - with this registrar. Maybe they were just changing hosts and DNS updates were delayed?

Obscure programming languages

Dare talked about the lack of XSLT 2.0 and XPath 2.0 in the forthcoming .NET Framework 2.0, and linked to Mark Fussell, who linked to 99 Bottles of Beer in various languages (programming languages, that is).

I think Ian would particularly like the Common LISP one which is made up of a complicated format string - in the shape of a beer bottle!

Ian's told me before about the BrainF**k language, but I'd not seen it before. Anyway, it seems that other people have written languages for bovines (beer-drinking bovines) and simians (alcoholic apes), along similar principles.

I can't help thinking that the T-SQL version should be set-based, though...

New features

Anyone reading in an aggregator might have noticed a bunch of posts disappeared for a short while this morning, then came back later. You might also find that the last month or so of posts suddenly reappeared.

Blogger have implemented comments, and I've gone back through and enabled them (a manual process) for everything that was appearing on the front page. I don't yet know whether items posted through BlogJet will get a comment link - I hope so.

Because of this change, I've removed the Contact link over on the right. I'll still know if you've left a comment. This might improve the amount of spam I'm getting, though I don't think it'll have any effect on all the spam sent with return addresses at my domain, for which all the bounce messages end up in my inbox grrrrr.

Reading List

What are the best development books? (blogs.msdn.com/jobsblog)

I tried to answer the question, but I virtually ended up listing the contents of my Programming shelf. There's very little dead weight on there. I've got something of a shelf at work, some of which gets used, some not. Other less immediate references stay at home. Here, then, is the home list: not a lot of commentary. Links go to Amazon UK, no kickback to me if you buy one, feel free to use another vendor. In some cases Amazon lists a different title; I've typed what appears on the cover.

Extreme Programming

Extreme Programming Explained (Beck)
Extreme Programming Installed (Jeffries, Anderson, Hendrickson)
(about four years ago, Ian's then employer were considering XP. I try to use some of the principles as appropriate but not yet properly using XP)

Programming Languages and Libraries

Programming in Ada 95 (Barnes) - the language taught in my first programming courses at University. Actually not my book, it belongs to Colin.

The C++ Programming Language, 3rd ed (Stroustrup)
The Annotated C++ Reference Manual (Ellis, Stroustrup)
Ruminations on C++ (Koenig, Moo) - actually, I've never finished reading this
Effective STL (Meyers) - I barely get to use STL

C++ In Depth Box Set, consisting of:

• Accelerated C++ (Koenig, Moo)
• Exceptional C++ (Sutter)
• More Exceptional C++ (Sutter)
• Essential C++ (Lippman)
• Modern C++ Design (Alexandrescu)

(read that last one if you want a template-induced headache - I maintain that Andrei Alexandrescu was the only good thing ever to come out of RealNetworks)

The Java Programming Language (Arnold, Gosling, Holmes)
The Java Language Specification (Gosling, Joy, Steele, Bracha)
I actually don't know Java that well. I bought these books to assist with my degree final project - a tool to produce diagrams of the static class structure of a program. I started out targetting C++ but discovered that C++ is, to put it mildly, a bugger to parse. I threw myself into the C++ parser so hard that I spent too little (i.e. practically no) time on the drawing and layout side. Eventually, to get something working, I abandoned C++ for Java - but too late. I just barely scraped the project, getting the minimum 40% pass mark, dragging my overall result down to a 2:2.

Windows Programming

I have so many books on Windows and Windows-oriented programming and Windows tools, it's difficult to know how to organise this. That shouldn't be too surprising, given as I work as a Pocket PC and desktop developer for a Microsoft-oriented ISV.

MFC Books

Teach Yourself MFC in 24 Hours (Morrison) - the main book I, er, taught myself MFC from. The main difference between this and other MFC books is that it's entirely code-focused - there's very little use of the Wizards. I maintain that if you want to be able to maintain an MFC program - indeed, any program - you need to know what it's doing, and relying on Wizard-generated code without understanding it is foolish.
Programming Windows with MFC, 2nd Ed (Prosise)

Low-Level & Raw Win32

Programming Applications for Windows, 4th Ed (Richter)
Programming Server-Side Applications for Windows (Richter, Clark)
Inside Windows 2000, 3rd Ed (Solomon, Russinovich)

.NET

Applied Microsoft .NET Framework Programming (Richter)
Programming Microsoft .NET (Prosise)
Shared Source CLI Essentials (Stutz, Neward, Shilling)

COM

Inside COM (Rogerson)
Transactional COM+ (Ewald)
(Essential COM is at work, as are Inside ATL and ATL Internals)

Other Windows

Debugging Applications for Microsoft .NET and Microsoft Windows (Robbins)
International Programming for Windows (Schmitt) - this was actually an error made by my sister (who works in Exchange tech support in the UK) when I asked for Programming Windows, 5th Ed (Petzold) (which is at work). Nevertheless I kept it and read it. Despite the UK being so much closer to countries with other languages and character sets, failure to program for international markets is just as prevalent as in the US (and we have the disadvantage to be based in the GMT time zone for half the year).

Miscellaneous

TCP/IP Network Administration (Hunt) - I ran, among others, a shared computer network at Aston Brook Green between 1998 and 2000. I was responsible for - well, made myself responsible for - DNS and DHCP, which was served from a Linux kernel 2.0.38 (RedHat 5.2, IIRC) box. It was pretty reliable in comparison to Windows 98, but then I wasn't playing games on it or running GUI apps. This was before I ever used Windows NT, which changed everything.

The Mythical Man-Month (Brooks)
Code Complete (McConnell) - I note a second edition is due in about a month.

Programming with POSIX Threads (Butenhof) - there are no good references on Win32 threads, such as when and how to use them. Programming Applications gives little guidance on signalling another thread - it's all about synchronisation. So I picked this up instead.

lex & yacc (Levine, Mason, Brown) - another legacy of my final project. If you want to build a parser, don't use these tools, they're way too confusing (and not powerful enough for C++, if you want an intelligible parser). Elkhound looks much more interesting.

Refactoring (Fowler)

That's everything I have at home; I may update this entry on Monday with what I have at work.

OpenNetCF lets domain slip

I can only assume that the people at www.opennetcf.org forgot to renew their domain registration - and now a Chinese hosting company has stolen the registration.

whois -h whois.pir.org opennetcf.org
Domain ID:D96652469-LROR
Domain Name:OPENNETCF.ORG
Created On:20-Mar-2003 14:15:41 UTC
Last Updated On:07-May-2004 10:54:48 UTC
Expiration Date:20-Mar-2006 14:15:41 UTC
Sponsoring Registrar:R64-LROR
Status:INACTIVE
Registrant ID:ONLC-637769-4
Registrant Name:Buy a domain
Registrant Organization:chinachanel
Registrant Street1:xiao meng china channel

Info courtesy of www.demon.net/external. 

BALEETED!

To confirm my suspicions about the patch I referred to in the last post not having been applied, I downloaded a Linux 2.6.5 kernel tarball. Having read the appropriate source files, I'm now scouring the appropriate section of the hard disk.

Ah, that's better - I don't feel so dirty...

I don't know where to start...

CRN: Microsoft Shelves NGSCB Project As NX Moves To Center Stage (via The Inquirer).

Boy, this article and the comments are so wrong I almost don't know where to begin. There follows a slightly edited version of my comment on the article.

NGSCB and No Execute are completely different things. No Execute applies only - I emphasize ONLY - to the ability to tell the processor not to execute code from given pages. Because Windows is a protected-mode operating system, only the kernel can set or clear this bit in the Page Table Entry. NX can help prevent the exploit of buffer overrun bugs.

Windows NT has always supported the ability to set execute permission on memory independent of read and write permission - look up the VirtualProtect API and the PAGE_EXECUTE flag. It requires hardware support, which has been lacking on x86 until now. AMD's 64-bit processors implement a No-Execute bit in the page table entry when the processor is running in 64-bit mode or in Physical Address Extensions 32-bit mode. Intel's Itanium processors also include an Execute bit in the page table.

Frankly, it's taken far too long to get execute protection on the x86, so AMD should be lauded for finally implementing it.

The Linux patch that another poster referred to: the follow-ups to that message basically damn the proposal on the grounds that it breaks GCC. Things may have changed in 7 years, of course. It works by reducing the length of the Code Segment not to cover the end of the address space - this causes the x86 processor to generate an Access Violation (oops! we're on a *nix - I mean a segmentation fault) if you try to set the instruction pointer outside this range. A clue to the general effectiveness of this proposal lies in the fact that this still isn't included in the kernel source tree (actually, as posted the code had no effect apart from taking another slot in the Global Descriptor Table because the CS register was set to the newly defined USER_HUGE_CS [the old behaviour] on return from taking a trap, and there was no apparent way to set it to anything else).

Replicating this on Windows would limit the number of threads dramatically and require a large change in the way DLLs are loaded. All thread stacks would have to be in the area of memory not covered by the code segment, which would have to be at the end because a segment represents a contiguous sequence of virtual addresses. The No Execute protection allows any virtual address to be protected; rather, that all virtual addresses that don't contain code will be protected by default.

NGSCB is about securing users' data and keys in an area only accessible through secure APIs. It has nothing to do with security vulnerabilities, except that both fall under a very rough umbrella of 'security'.

[edit: confirmed my suspicion about the Linux patch not being included in 2.6, and note that the article was revised after I posted this entry.]