Sunday, 4 November 2007

VS2005 Smart Device projects don't Dispose components

I’ve posted about this on Connect and on Codeproject’s Lounge, and informed my colleagues, but not yet on here. I realise I’m repeating myself for some of my audience, but this is in a more readily searchable location.

I was trying to work out why Compact Framework applications seemed to be consuming a fair amount of memory, often causing the division between Program and Storage memory on Pocket PC 2003 to shift greatly toward Program, often causing inability to create files. I created a simple Windows Mobile 5.0 application with two forms, both with menu bars with actions on them, and had the first form continually create and destroy the second. I then loaded up .NET Compact Framework Remote Performance Monitor to monitor the application. I was seeing 400KB+ of GC heap still being used after a collection. Looking at the GC heap (in .NET CF 2.0 SP2’s RPM) showed that a large number of instances of the second form were still referenced.

Drilling down showed that the MainMenu and MenuItem objects that had been owned by the forms were still rooted, by their finalizers (shown as [root: Finalizer]). The forms were still referenced because I had event handlers connected to the menu items. That meant that all the other controls on the form were also still referenced, by the members of the form class.

To understand why they’re still rooted, even after GC, you have to know how finalization works. When a GC occurs, the collector marks every object that’s referenced by following roots, then sweeps away all those objects that are no longer referenced. When doing the sweep phase, if the object requires finalization (i.e. implements a Finalize method and GC.SuppressFinalize has not been called for that object), instead of actually being deleted and the memory reclaimed, it is placed on a finalization queue. This in itself is a source of roots. The finalizer is not run during the GC itself to reduce the time spent in GC. Instead, a separate finalizer thread processes the queue of objects to be finalized. The object will continue to be reported to GC until its finalizer is run, so may survive multiple GCs if memory demand is high and the finalizers are slow.

The reason I recommend that as far as is possible, you avoid the finalizer is that unless you’re very careful, you can end up with thread affinity problems (trying to destroy something from the wrong thread, or having to marshal back to the creating thread) and you can end up blocking the finalizer thread indefinitely. The issue of freeing memory later than would otherwise be possible isn’t as much of an issue on the desktop, but it also affects GC’s tuning of how often to run. It’s definitely an issue on the limited memory available on devices. For more on how GC works in the Compact Framework, see Steven Pratschner’s “Overview of the .NET Compact Framework Garbage Collector”. I’ve recommended before that you always call Dispose, on any object that implements it (although I’ve discovered it isn’t safe to Dispose the Graphics object passed to you in a PaintEventArgs structure).

So where does the finalizer come from? I studied the assemblies in Reflector. A side note here: the assemblies in Program Files\Microsoft Visual Studio 8\SmartDevices\SDK\CompactFramework\2.0\v2.0\WindowsCE do not actually contain the IL code, only the metadata. For the actual implementation see under SmartDevices\SDK\CompactFramework\2.0\v2.0\Debugger\BCL. Recent versions of Reflector will load assemblies from this location when you create a new assembly list in the File, Open List dialog (or when starting a newly downloaded copy, for the default list).

In Compact Framework 2.0, the System.ComponentModel.Component class implements a finalizer, so anything which derives from this class automatically gets one too. (It calls the virtual Dispose(bool) method, passing false as the parameter). The Dispose method in this class calls GC.SuppressFinalize, so if you dispose of a component properly, it won’t end up on the finalization queue. This matches what the desktop Framework has done since .NET 1.1 at least (I don’t have 1.0 installed to check).

For both desktop and device projects, a newly-created form declares a components member of type System.ComponentModel.Container, to contain the components dragged onto the designer surface. A Dispose(bool) override is generated for you (in Form.Designer.cs/.vb, for .NET 2.0 projects) which calls Dispose on the container, if components is not null. (Presumably the intent was that the container wouldn’t be created until needed, but in fact the initial code in InitializeComponent for a new form does create a Container and assigns it to components.) Container’s Dispose method disposes anything that was added to the container.

For a desktop project, when you drop something that derives from IComponent (I think) onto the form, and which doesn’t derive from Control, the designer generates code to add that component to the container, so it will be disposed of when the form is disposed. Simple. However – and here’s the bug – the device designer doesn’t. In fact it even deletes the creation of the Container object. Result, all your components end up running their finalizer, and thereby consuming memory past the first GC after they died, potentially increasing the overall memory use of the process.

To avoid the problem, you have to write the code to dispose the components yourself. The most straightforward way is to add the code that Visual Studio should have generated to your form’s constructor, after the call to InitializeComponent. That will typically look like:

this.components = new System.ComponentModel.Container();
this.components.Add( mainMenu1 );
// etc for other components

The difficulty is knowing exactly what to add and remembering to update it when you add or remove components. Reflector can help a bit – use the Derived Types view under Component to see the classes that are affected. The other one that affected us was the HardwareButton class (in Microsoft.WindowsCE.Forms). It doesn’t actually override Dispose(bool) so I think you ought to write code to clear AssociatedControl.

This bug still exists in VS2008 Beta 2. To help ensure it doesn’t continue in future versions, please vote for the bug on Connect.

In .NET Compact Framework 1.0, there isn’t a systematic finalization like this – Component doesn’t have a Finalizer or a Dispose method, although weirdly it does have a virtual Dispose(bool) method and a Disposed event. It also doesn’t implement IComponent, so you can’t add a Component-derived class to a Container! And it doesn’t implement IDisposable either.

The MainMenu class unfortunately does have a Finalizer, but it does not have a Dispose method. After a bit of digging around, you can see that you can at least clean up the native resources by setting the form’s Menu property to null. It still causes all the managed objects to remain referenced (if it has menu items that have event handlers). You’ll have to call GC.SuppressFinalize yourself once you’ve forced it to clean up.

The more I think about it, the more I think that garbage collection overpromises and under-delivers. At least the C++ resource-acquisition-is-initialization and automatic variable destruction model ensures that resources are freed in a timely fashion. That’s why the C++/CLI designers hooked that model up to IDisposable – there is no using block because local GC handle destruction does the job.