Custom XML Serialization for the .NET Compact Framework

.NET provides a whole slew of utilities for serializing objects into an XML form. But as I wrote in my previous post, .NET Compact Framework has serious problems with this serialization. The good news is that you can leverage all of the existing Attributes and tricks that you think should work (if it weren’t so buggy) and use them in your own serialization scheme.

Get Started

For example I want to know if I should skip a given member? There are a number of different things I can check. Is a Reference type null? Is there and XmlIgnore attribute? Is there a PropertyNameSpecified value set to false? All of those questions can easily be answered using reflection.


///

/// Should the current property be skipped based on rules
/// such as the existence of a propertySpecified value set to false?
///

/// The MemberInfo to check /// The object that contained this member /// true if this member should be skipped
public bool SkipMember(MemberInfo member, object o)
{
object val = null;
if (member.MemberType == MemberTypes.Field)
{
val = ((FieldInfo)member).GetValue(o);
}
else if (member.MemberType == MemberTypes.Property)
{
val = ((PropertyInfo)member).GetValue(o, null);
}

if (null == val)
return true;

string propertyToTest = member.Name + "Specified";

PropertyInfo specifiedProperty = o.GetType().GetProperty(propertyToTest);
if ((null != specifiedProperty && !(bool)specifiedProperty.GetValue(o, null)))
return true;

FieldInfo specifiedField = o.GetType().GetField(propertyToTest, FIELD_BINDING_FLAGS);
if ((null != specifiedField && !(bool)specifiedField.GetValue(o)))
return true;

return member.IsDefined(typeof(XmlIgnoreAttribute), false);
}

I can use a similar “fall-through” strategy to determine the name of the element to write using the XmlElement attribute for example. Now that I know I can answer some basic questions about an Object using the built-in mechanisms that .NET uses for serialization I can get down to serious serialization.

We’re all Object-Oriented programmers these days right? Right!? So to start I decided that the best way to handle this problem was to decompose it into a bunch of simpler problems.

ITagWriter

There are two things that we can write in XML. Either an XML Element or an XML Attribute. So, I created an interface ITagWriter with two concrete implementations to correspond to these two XML types: AttributeTagWriter and ElementTagWriter. These classes allow me to write the structure of the XML Document.


///

/// Interface to implement to write different Xml tags
/// Either Elements or Attributes.
///

internal interface ITagWriter
{
///

/// Write the opening Xml tag with the given name
///

/// The XML Document to write the tage to. /// The name of the tag void WriteStart(XmlWriter doc, string tagName);

///

/// Write the appropriate end tag
///

/// The XML Document to write the tage to. void WriteEnd(XmlWriter doc);
}

IValueWriter

With the ability to write the structure, I then need to be able to write out the values of the various objects and their properties. Just like with the ITagWriter interface, I decided to create an IValueWriter for the various kinds of values that I would need to write. The types I came up with were ObjectWriter, CollectionValueWriter, EnumValueWriter, SimpleValueWriter, and XmlElementValueWriter.


///

/// Interface to implement to write different kinds of values.
///

internal interface IValueWriter
{
///

/// Write the Entry value to the XmlDocument
///

/// The XML Document to write the tage to. /// The meta-information and value to write. void Write(XmlWriter doc, CustomSerializationEntry entry);
}

You’ll notice the CustomSerializationEntry class is the parameter for the IValueWriter.Write() method. This class contains all of the metadata and the value about the various properties of an Object. This alows us an easy way to ask questions about a given property. Is it a Collection? Is it an Enum? Is there a sort order? Basically the idea is to encapsulate all of the things that are interesting from a serialization point of view.

To help manage the interaction I also created a basic TypeLookup class. The job of this class is to determine what type of ITagWriter and IValueWriter to use for a given CustomSerializationEntry instance. This allows us to centralize that decision making in a single class. The centralized knowledge keeps the individual writer implementations much simpler. They just need to ask for the correct writer and then call the methods defined in the interface. They don’t need to care what type they are writing. All hail power of encapsulation and abstraction!

Start Serializing

I bootstrap the serialization by creating an ObjectWriter to handle the outermost object. From there, the ObjectWriter takes over, constructing CustomSerializationEntry objects for each of the serialized object’s properties. The type of the property determines the type of IValueWriter that is used to write the property value.


///

/// Serialize an object using the given
/// xmlRoot as the root element name.
///

/// /// ///
public string Serialize(object o, string xmlRoot)
{
StringBuilder sb = new StringBuilder();
using (XmlTextWriter writer = new XmlTextWriter(new StringWriter(sb)))
{
writer.Formatting = Formatting.Indented;

XmlWriter xmlDoc = XmlWriter.Create(writer);
WriteRootElement(xmlDoc, o, xmlRoot);
}

return sb.ToString();
}

private static void WriteRootElement(XmlWriter doc, object o, string rootElement)
{
doc.WriteStartDocument();

ObjectWriter writer = new ObjectWriter(new TypeLookup());
writer.Write(doc, o, rootElement);

doc.WriteEndDocument();
}

The ObjectWriter itself creates a CustomSerializationEntry for all the properties that should be written. It then loops over the properties. Notice how it uses the TypeLookup (lookup) to ask for the proper value writer for each of the properties.

// ...
public void Write(XmlWriter doc, object o, string elementName)
{
doc.WriteStartElement(elementName);
IEnumerable entries = GetMemberInfo(o);
foreach (CustomSerializationEntry currentEntry in entries)
{
lookup.GetValueWriter(currentEntry).Write(doc, currentEntry);
}
doc.WriteEndElement();
}
// ...

Conclusion

OK, so I left out a lot of details! But If I gave you all of the answers it wouldn’t be any fun now would it. I hope you can see how decomposing the problem of serialization turns it a series of relatively simple problems that you can answer. So, if I can do this in about 500 lines of code, how come Microsoft can’t implement a decent XML Serializer for the .NET Compact Framework.

.NET Compact Framework Serialization Bugs

XML Serialization in the .NET Compact Framework seems to be buggy enough that it is generally not useful if you need to serialize a class that conforms to a schema of some sort. If all you need to do is serialize and deserialize representations of a class you are probably fine. But if you need to use the serialized data to interoperate with a service (for example) it likely will not work.

Enums

I wrote previously about Problems with Enum Serialization on the .NET compact framework. To summarize: The XmlEnum attribute allows you to change the value that is serialized. One of the reasons to do this is to limit the valid values in the document. Of course, Enums have naming restrictions such as not being able to have spaces in them. So, one of the main reasons you would do this would be to put spaces in the name that was serialized.

The problem is that the XmlEnum under the .NET Compact Framework truncates the value at the first space. So, in the example below your Enums would serialize to “Some” and “Another” instead of the correct “Some Value” and “Another Value”.


public enum Foo {
[XmlEnum("Some Value")
Some,
[XmlEnum("Another Value")]
Other
}

Controlling Serialization of Value Types

.NET has reference types and value types. Reference types are stored on the heap and can be null. Value types are stored on the stack and can not be null. What is a value type and what is a reference type is not always obvious. int and double are value types but so is DateTime. string is a reference type though.

The serialization code is smart enough to not serialize null values. But what about those value types? They can not be null, so how do you determine if they should be serialized or not? There are two ways you can do it.

  1. DefaultValue
  2. PropertySpecified

DefaultValue

One way is to specify a default value. If the property has that default value it will not be serialized.

Example:

[XmlAttribute("age"), DefaultValue(-1)]
public int Age;

I have not found any problem with this on the .NET Compact Framework, but I haven’t used it extensively.
DefaultValue really does not make sense for every case. What about a value where negative, positive and zero all make sense? What about a boolean value where true and false are both meaningful and different from null? Valid values are really a business concept and many business concepts will not constrain them in such a way that you can specify a DefaultValue to control serialization, so this is not always useful.

See: more information on DefaultValue.

PropertySpecified

.NET serialization also allows you to specify a control value to check to see if a property should be serialized. The form of the control value is propertynameSpecified. It is a boolean value. If the value is false, then the property it controls will not be serialized. If the value, is true then it will be serialized.

Example:

[XmlIgnore]
public bool FooSpecified = false;

[XmlElement("foo)]
public int Foo {
get { return this.foo; }
set {
this.foo = value;
this.FooSpecified = true;
}
}

This is fine in the full framework. The problem is with the .NET Compact Framework. When the serializer comes across a propertynameSpecified value that is false, the serialization of that class stops. This means if you have 5 properties and the second property has a control value set to false, only the first value will be serialized!

Coming Soon

In a future post, I will write about how you can relatively easily write your own XML Serializer for the .NET Compact Framework in about 500 lines of code.

Update: follow-up post on Writing a Custom XML Serializer has been posted.

My Foray Into the World of Smartphones

Ok, so I might be a little behind the times when it comes to mobile phones and SmartPhones. It was a big advance for me when I got one with a color display and MIDI ringtones two years ago. Well, my contract was ending and I’ve been doing some mobile development recently, so I got curious. If I want a SmartPhone, what should I get and why?

To start with I want a phone naturally. Because of that it’s important for it to have a good address and phone book. Secondly I want email. I’m picky about email, so I run my own mail server using IMAP over SSL and I want it to work with that. Next, I would like to get maps and directions. I like to look up how to get places, even when I’m already on the go. I also want to get on the web once in a while. I’m not planning on doing most of my browsing, but when someone asks me “What 5 countries border the Caspian Sea?” (You know who you are), I want to get online and check it out. Of course there are some other random applications that I might run since this thing will be more powerful than the computers that put a man on the moon. Finally it has to work with Mac OS X.

The Options

  • Blackberry
  • PalmOS
  • Symbian
  • Windows Mobile 5

Blackberry came onto the scene as the first service to provide a push technology for mobile messaging (as far as I know). You didn’t have to constantly check an email box to see if something had arrived, but instead you would be notified when you had a new message. This of course was a hit with the dot com bubble set. Now Microsoft and its Windows Mobile platform offer the same functionality, so it’s no longer a differentiator. As a software geek all that is well and good, but more important is how can I write my own apps?

Blackberry

Blackberry’s latest offerings are built on top of a Java platform. MIDP and JavaME for some phones is very limiting. My last phone had a JVM, but it could not access any of the phone’s functions or the address book for example. That made it good for one thing basically, playing games. A mobile phone is not exactly what I’d call a platform for playing games. The Blackberry implementation is fully integrated into the phone. They offer a full SDK to do everything from manage address book entries, access the file system to respond to call events. Blackberry has the Blackberry JDE which is an OK environment. You can also integrate the JDE with Netbeans for mobile development.

PalmOS

PalmOS started life as the PDA operating system.
The primary PalmOS development languages are C and C++. They do have a Java development platform (as well as Pascal, Forth and SmallTalk). The C based tools are built on top of the Eclipse platform which is nice. Palm has started using both their own PalmOS as well as Windows Mobile. That worries me a little bit that they might be ready to abandon their own OS development and just become a hardware manufacturer?

Symbian

Symbian is the most popular SmartPhone OS in the world with something like 75% of the market. In the US though, it’s almost impossible to find one. They basically only want to support GSM, the standard of everyone in the world but the US (don’t get me started on that rant). Symbian OS uses a C development environment for writing applications. I honestly didn’t look into it much be cause there are only a couple of available devices in the US.

Windows Mobile

Windows Mobile is another popular handheld OS. Being an Operating System from Microsoft, it uses the same development tools as the Windows OS, namely Visual Studio. Windows Mobile has both C APIs and .NET APIs for you to use. Windows Mobile is a popular platform and there are a wide variety of devices for it as well.

My Decision

I like Java. I know C#. I’ve done C++, but I really don’t want to ramp up with C++ for what would be once-in-a-while development of personal apps. If I want to do Java or C# that basically put me in the Blackberry or Windows Mobile camps. PalmOS was definitely tempting, but I wanted to use the main development environment and not an also ran.

I decided to go with the Blackberry. Specifically, I chose the Blackberry Pearl. First and foremost, it’s a phone. It’s not too big, it’s the size and shape of a phone. It’s got buttons that you can use while you’re on the phone (unlike the kinds where you slide the keyboard out).

The first thing you should download are all of the Google Mobile Tools. They offer a great application for Google Maps that includes lots of goodies like searching for local businesses and a traffic overlay for highways. It’s a great mapping solution. They also have a GMail app that gives you a nice interface to GMail. Of course you can set up GMail as one of your email accounts and just use the built in editing features. But if you want to get at your mail for searching and archival, then check out the GMail application.

Opera Mini is ok as well. I’ve been trying it a bit, but mostly I’ve been using the built in Blackberry Browser. Opera Mini feels a little bit less integrated (as might be expected) because things like the menu system does not feel native to me. Either one works though. Most sites are not optimized for mobile of course (this one is a perfect example, which I hope to fix) so your mileage my vary.

The mail system works great with my setup. All of my messages are forwarded to my phone. If I read them on the phone, then they show as read in my Mail inbox. I can also delete them on both at once. This of course works with my own personal IMAP over SSL server.

Blackberry supplies PocketMac for syncing your Blackberry. It’s not the best application in the world and it’s a little buggy. But generally it gets the job done and lets you sync your phone with AddressBook, iCal, etc. Hopefully we’ll get native support for iSync at some point. There’s no native Mac OS X support for Windows Mobile either, so you have to use Missing Sync. I haven’t tried it, but maybe they could get on board?

Conclusion

Did I make the right decision? Should I have gone with PalmOS? What are your experiences being a geek who wants more than a random business person with your SmartPhone?

Coming soon, hopefully some more info on developing for Blackberry.

References

Getting Started with Blackberry Pearl on Mac OS X
PocketMac Download
Use Netbeans for Blackberry Development