Custom XML Serialization for the .NET Compact Framework

.NET provides a whole slew of utilities for serializing objects into an XML form. But as I wrote in my previous post, .NET Compact Framework has serious problems with this serialization. The good news is that you can leverage all of the existing Attributes and tricks that you think should work (if it weren’t so buggy) and use them in your own serialization scheme.

Get Started

For example I want to know if I should skip a given member? There are a number of different things I can check. Is a Reference type null? Is there and XmlIgnore attribute? Is there a PropertyNameSpecified value set to false? All of those questions can easily be answered using reflection.


///

/// Should the current property be skipped based on rules
/// such as the existence of a propertySpecified value set to false?
///

/// The MemberInfo to check /// The object that contained this member /// true if this member should be skipped
public bool SkipMember(MemberInfo member, object o)
{
object val = null;
if (member.MemberType == MemberTypes.Field)
{
val = ((FieldInfo)member).GetValue(o);
}
else if (member.MemberType == MemberTypes.Property)
{
val = ((PropertyInfo)member).GetValue(o, null);
}

if (null == val)
return true;

string propertyToTest = member.Name + "Specified";

PropertyInfo specifiedProperty = o.GetType().GetProperty(propertyToTest);
if ((null != specifiedProperty && !(bool)specifiedProperty.GetValue(o, null)))
return true;

FieldInfo specifiedField = o.GetType().GetField(propertyToTest, FIELD_BINDING_FLAGS);
if ((null != specifiedField && !(bool)specifiedField.GetValue(o)))
return true;

return member.IsDefined(typeof(XmlIgnoreAttribute), false);
}

I can use a similar “fall-through” strategy to determine the name of the element to write using the XmlElement attribute for example. Now that I know I can answer some basic questions about an Object using the built-in mechanisms that .NET uses for serialization I can get down to serious serialization.

We’re all Object-Oriented programmers these days right? Right!? So to start I decided that the best way to handle this problem was to decompose it into a bunch of simpler problems.

ITagWriter

There are two things that we can write in XML. Either an XML Element or an XML Attribute. So, I created an interface ITagWriter with two concrete implementations to correspond to these two XML types: AttributeTagWriter and ElementTagWriter. These classes allow me to write the structure of the XML Document.


///

/// Interface to implement to write different Xml tags
/// Either Elements or Attributes.
///

internal interface ITagWriter
{
///

/// Write the opening Xml tag with the given name
///

/// The XML Document to write the tage to. /// The name of the tag void WriteStart(XmlWriter doc, string tagName);

///

/// Write the appropriate end tag
///

/// The XML Document to write the tage to. void WriteEnd(XmlWriter doc);
}

IValueWriter

With the ability to write the structure, I then need to be able to write out the values of the various objects and their properties. Just like with the ITagWriter interface, I decided to create an IValueWriter for the various kinds of values that I would need to write. The types I came up with were ObjectWriter, CollectionValueWriter, EnumValueWriter, SimpleValueWriter, and XmlElementValueWriter.


///

/// Interface to implement to write different kinds of values.
///

internal interface IValueWriter
{
///

/// Write the Entry value to the XmlDocument
///

/// The XML Document to write the tage to. /// The meta-information and value to write. void Write(XmlWriter doc, CustomSerializationEntry entry);
}

You’ll notice the CustomSerializationEntry class is the parameter for the IValueWriter.Write() method. This class contains all of the metadata and the value about the various properties of an Object. This alows us an easy way to ask questions about a given property. Is it a Collection? Is it an Enum? Is there a sort order? Basically the idea is to encapsulate all of the things that are interesting from a serialization point of view.

To help manage the interaction I also created a basic TypeLookup class. The job of this class is to determine what type of ITagWriter and IValueWriter to use for a given CustomSerializationEntry instance. This allows us to centralize that decision making in a single class. The centralized knowledge keeps the individual writer implementations much simpler. They just need to ask for the correct writer and then call the methods defined in the interface. They don’t need to care what type they are writing. All hail power of encapsulation and abstraction!

Start Serializing

I bootstrap the serialization by creating an ObjectWriter to handle the outermost object. From there, the ObjectWriter takes over, constructing CustomSerializationEntry objects for each of the serialized object’s properties. The type of the property determines the type of IValueWriter that is used to write the property value.


///

/// Serialize an object using the given
/// xmlRoot as the root element name.
///

/// /// ///
public string Serialize(object o, string xmlRoot)
{
StringBuilder sb = new StringBuilder();
using (XmlTextWriter writer = new XmlTextWriter(new StringWriter(sb)))
{
writer.Formatting = Formatting.Indented;

XmlWriter xmlDoc = XmlWriter.Create(writer);
WriteRootElement(xmlDoc, o, xmlRoot);
}

return sb.ToString();
}

private static void WriteRootElement(XmlWriter doc, object o, string rootElement)
{
doc.WriteStartDocument();

ObjectWriter writer = new ObjectWriter(new TypeLookup());
writer.Write(doc, o, rootElement);

doc.WriteEndDocument();
}

The ObjectWriter itself creates a CustomSerializationEntry for all the properties that should be written. It then loops over the properties. Notice how it uses the TypeLookup (lookup) to ask for the proper value writer for each of the properties.

// ...
public void Write(XmlWriter doc, object o, string elementName)
{
doc.WriteStartElement(elementName);
IEnumerable entries = GetMemberInfo(o);
foreach (CustomSerializationEntry currentEntry in entries)
{
lookup.GetValueWriter(currentEntry).Write(doc, currentEntry);
}
doc.WriteEndElement();
}
// ...

Conclusion

OK, so I left out a lot of details! But If I gave you all of the answers it wouldn’t be any fun now would it. I hope you can see how decomposing the problem of serialization turns it a series of relatively simple problems that you can answer. So, if I can do this in about 500 lines of code, how come Microsoft can’t implement a decent XML Serializer for the .NET Compact Framework.

.NET Compact Framework Serialization Bugs

XML Serialization in the .NET Compact Framework seems to be buggy enough that it is generally not useful if you need to serialize a class that conforms to a schema of some sort. If all you need to do is serialize and deserialize representations of a class you are probably fine. But if you need to use the serialized data to interoperate with a service (for example) it likely will not work.

Enums

I wrote previously about Problems with Enum Serialization on the .NET compact framework. To summarize: The XmlEnum attribute allows you to change the value that is serialized. One of the reasons to do this is to limit the valid values in the document. Of course, Enums have naming restrictions such as not being able to have spaces in them. So, one of the main reasons you would do this would be to put spaces in the name that was serialized.

The problem is that the XmlEnum under the .NET Compact Framework truncates the value at the first space. So, in the example below your Enums would serialize to “Some” and “Another” instead of the correct “Some Value” and “Another Value”.


public enum Foo {
[XmlEnum("Some Value")
Some,
[XmlEnum("Another Value")]
Other
}

Controlling Serialization of Value Types

.NET has reference types and value types. Reference types are stored on the heap and can be null. Value types are stored on the stack and can not be null. What is a value type and what is a reference type is not always obvious. int and double are value types but so is DateTime. string is a reference type though.

The serialization code is smart enough to not serialize null values. But what about those value types? They can not be null, so how do you determine if they should be serialized or not? There are two ways you can do it.

  1. DefaultValue
  2. PropertySpecified

DefaultValue

One way is to specify a default value. If the property has that default value it will not be serialized.

Example:

[XmlAttribute("age"), DefaultValue(-1)]
public int Age;

I have not found any problem with this on the .NET Compact Framework, but I haven’t used it extensively.
DefaultValue really does not make sense for every case. What about a value where negative, positive and zero all make sense? What about a boolean value where true and false are both meaningful and different from null? Valid values are really a business concept and many business concepts will not constrain them in such a way that you can specify a DefaultValue to control serialization, so this is not always useful.

See: more information on DefaultValue.

PropertySpecified

.NET serialization also allows you to specify a control value to check to see if a property should be serialized. The form of the control value is propertynameSpecified. It is a boolean value. If the value is false, then the property it controls will not be serialized. If the value, is true then it will be serialized.

Example:

[XmlIgnore]
public bool FooSpecified = false;

[XmlElement("foo)]
public int Foo {
get { return this.foo; }
set {
this.foo = value;
this.FooSpecified = true;
}
}

This is fine in the full framework. The problem is with the .NET Compact Framework. When the serializer comes across a propertynameSpecified value that is false, the serialization of that class stops. This means if you have 5 properties and the second property has a control value set to false, only the first value will be serialized!

Coming Soon

In a future post, I will write about how you can relatively easily write your own XML Serializer for the .NET Compact Framework in about 500 lines of code.

Update: follow-up post on Writing a Custom XML Serializer has been posted.

Automated Subversion Tagging With MSBuild

I’ve written previously about using MSBuild With Nunit as well as a bit of a Manifesto on Relentless Build Automation. I believe that automating the build and deployment process is a necessary step to ensure the reliable delivery of quality software.

Release Management

One of the things that we as software developers have to do is regularly make releases, either to QA or to a customer. When we make releases we need to be able to continue to develop the software to add new features and fix bugs. Tagging is a safety process that we use in a version control system to allow us to easily get to the code that we used to build a specific release. When you Tag, you can always rebuild that release. So, if weeks or months down the road you need to fix a critical bug, but don’t want to release new features, you can get back to the Tag, create a Branch, fix the bug and release the new build to your users.

How Do We Ensure We Can Recreate Releases

How can we ensure that we will be able to recreate a release that we make to either QA or a customer? Use Automation to Tag your builds when you create them of course.

I’ve contributed a new SvnCopy Task to the MSBuild Community Tasks project which was just accepted and committed. It is currently in the Subversion repository for the project but should be available shortly in an official build. This will allow you to easily automate the process of Tagging or Branching your builds when your release. Subversion uses the Copy metaphor for both Branching and Tagging operations which is different from some other version control systems.

Example:

<Target Name="GetRemoteRevisionNumber">
    <SvnInfo RepositoryPath="$(SvnRemoteRoot)">
        <Output TaskParameter="LastChangedRevision" 
             PropertyName="RemoteSvnRevisionNumber"  />
    </SvnInfo>
    <Message Text="Revision: $(RemoteSvnRevisionNumber)"/>
</Target>
 
<Target Name="Tag" DependsOnTargets="GetRemoteRevisionNumber">
    <Error Condition="$(SvnUserName)==''" 
        Text="You must set your Subversion Username."/>
    <Error Condition="$(SvnPassword)==''" Text="You must set your Subversion Password."/>
    <SvnCopy RepositoryPath="$(SvnRemoteRoot)/trunk"
        DestinationPath="$(SvnRemoteRoot)/tags/REV-$(RemoteSvnRevisionNumber)"
        Message="Auto-tagging Revision: $(RemoteSvnRevisionNumber)"
        Username="$(SvnUserName)" password="$(SvnPassword)"/>
    <Message Text="Tagged: REV-$(RemoteSvnRevisionNumber)"/>
</Target>

You can then integrate the process of creating a Tag every time you generate a build by tying together Tasks with dependencies. In the example below, the GenerateTestBuild calls GenerateCabFiles and Tag to automatically build the installer and Tag Subversion with the current revisions number.

<Target Name="GenerateCabFiles" DependsOnTargets="Build">
    <Exec WorkingDirectory="." 
        Command="devenv &quot;$(SolutionFileName)&quot; /build $(Configuration) /project  $(ConfigCabFileProject)"/>
    <Exec WorkingDirectory="." 
        Command="devenv &quot;$(SolutionFileName)&quot; /build $(Configuration) /project  $(UiCabFileProject)"/>
</Target>
 
<Target Name="DeployCabFiles" DependsOnTargets="GenerateCabFiles">
    <MakeDir Directories="$(DestRoot)\$(BuildFolderPrefix)$(SvnRevisionNumber)"/>
    <MakeDir Directories="$(DestRoot)\$(BuildFolderPrefix)$(SvnRevisionNumber)\FormXmls"/>
    <Copy SourceFiles="@(CabFiles)" 
        DestinationFolder="$(DestRoot)\Build$(SvnRevisionNumber)"/>
    <Copy SourceFiles="$(AutoUpdaterConfigFile)" 
        DestinationFolder="$(DestRoot)\Build$(SvnRevisionNumber)"/>
    <Copy SourceFiles="$(AutoUpdaterConfigFile)" DestinationFolder="$(DestRoot);"/>
    <Copy SourceFiles="@(FormXmlFiles)" 
        DestinationFolder="$(DestRoot)\$(BuildFolderPrefix)$(SvnRevisionNumber)\FormXmls"/>
</Target>
<Target Name="GenerateTestBuild" DependsOnTargets="DeployCabFiles;Tag"/>

Hopefully this will help you get started on some more automation.

Update:
MSBuild Community Tasks version 1.2 has been released containing this code. You can get it here.

Resources

MSBuild Community Tasks
Subversion

MSBuild with NUnit

I’ve written about Unit Testing and Build Automation in the past, but mostly dealing with Java projects and tools (because I usually write about things I’m working on at the time). Well, I’ve started a .NET project for the first time in a while so I want to solve some of the samde problems in this environment.

Why MSBuild?

In the Java world, the natural choice for automation is usually Ant. NAnt was created for the as a work alike for the .NET platform and is a very good tool. I’ve build custom extensions for database automation in the past and used it successfully. So why would I consider MSBuild? With the advent of Visual Studio 2005, Microsoft included MSBuild as the build tool for its projects. All of the project files generated by Visual Studio are MSBuild files in disguise. This makes it really easy to leverage the project files to chain together a nice automated build.

Why NUnit?

Microsoft is a true believer in Not Invented Here and as such they created their own Unit Testing framework that looks and acts exactly the same as NUnit. The MS XUnit framework only comes with the higher-end Team System versions of Visual Studio. In addition it is very hard to deploy the DLLs to build systems. The only way to get the proper DLLs installed is to install the full Visual Studio. That seems like a really big problem to me. With NUnit you can include the needed DLLs in your version control system and reference them in the project which makes it very easy to build and test code and use a Continuous Integration process.

How to Integrate MSBuild and NUnit

Here’s the good news, all of the work has been done for you. The good people at Tigris.org (the creators of Subversion) have created a series of MSBuild Tasks filling in the gaps that MSBuild was missing. One of the included Tasks is an NUnit task. Their hard work and the simple script below should get ou started with a build that can run your tests as well.

Example Build File

<?xml version="1.0" encoding="utf-8"?>
<Project DefaultTargets="Build" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
     <!-- Import the MSBuild Tasks -->
	<Import Project="$(MSBuildExtensionsPath)\MSBuildCommunityTasks\MSBuild.Community.Tasks.Targets" />
	<PropertyGroup>
		<Configuration Condition=" '$(Configuration)' == '' ">Debug</Configuration>
		<ClassLibraryOutputDirectory>bin\$(Configuration)</ClassLibraryOutputDirectory>
		<ProjectDir>ProjectName</ProjectDir >
		<ProjectTestDir>ProjectNameTest</ProjectTestDir >
		<ProjectFile>$(ProjectDir)\ProjectName.csproj</ProjectFile >
		<TestProjectFile>$(ProjectTestDir)\ProjectNameTest.csproj</TestProjectFile >
	</PropertyGroup>
 
	<!-- Build projects by calling the Project files generated by VS -->
	<Target Name="Build">
		<MSBuild Projects="$(ProjectFile)" />
		<MSBuild Projects="$(TestProjectFile)" />
	</Target>
 
	<!-- Run Unit tests -->
	<Target Name="Test" DependsOnTargets="Build">
		<CreateItem Include="$(ProjectTestDir)\$(ClassLibraryOutputDirectory)\*.Tests.dll">
			<Output TaskParameter="Include" ItemName="TestAssembly" />
		</CreateItem>
		<NUnit Assemblies="@(TestAssembly)" />
	</Target>
</Project>

The “Test” target above uses the NUnit task to run all of the NUnit Tests found in the DLL. There are a number of different ways to create ItemGroups ins MSBuild, all of which are odd and confusing. If anyone has any ideas on what the best practices in this regard, let me know.

One final plug for my favorite Visual Studio plugin, ReSharper. It includes support for NUnit. It will help you create NUnit Tests and then lets you run them and debug them from within your IDE. It’s almost like you have a real IDE all of a sudden!

References

NUnit
MSBuild Tasks
Integrating MSBuild with CruiseControl.NET

Pragmatic Unit Testing in C# with NUnit
NUnit Pocket Reference
Deploying .NET Applications: Learning MSBuild and ClickOnce