Roll Your Own XML Templater in Ruby

In this post, we'll use Jim Weirich's XML Builder gem and the magic of Tilt to build our very own XML template engine.

What is a Template Engine?

A template engine is any code or software responsible for creating documents out of a given data model and template. You're probably already very familiar with one such engine. ERB allows you to write Ruby code in any plain text document that will then be evaluated and compiled into a final finished doc.

In Rails, we build HTML templates and use ERB within these templates so that we can repeatedly render the same HTML again and again, with different data.

In our Rails controllers, we're used to assigning some instance variable to be used in a view, and then calling the render method:

def index
  @cats = Cat.all
  render "index"
end

This pattern of passing some variables into a template file and compiling the resulting document via a render method is one we'll be mimicking in our own hand-rolled XML templater.

Why an XML Template Engine?

A template engine is useful whenever we need to render the same template again and again.

In the example we'll be using in this post, we're responsible for distributing music to a number of online stores, like Amazon, iTunes, Google Play, etc. These stores follow an industry standard that requires music content metadata (info about the album's tracks, artists, cover art, etc.) to be included with every content delivery. This industry standard requires that this metadata be in XML format. So, every given single or album that we are delivering to a store should be used to write an XML document that is included with each delivery.

While it is possible to use ERB to generate Ruby + XML documents, it is not as semantic or robust as using a builder like the gem we'll be working with.

First of all, in order to use ERB to template Ruby within an XML file, we'd have to actually write XML, just like we actually write HTML and inject Ruby in our views. We'd end up with an XML template that looks something like this:

<Cats>
  <% @cats.each do |cat|
    <Cat>
      <Name><%=cat.name%></Name>
       ...
    </Cat
  <% end %>
</Cats>

Personally, I find writing XML to be completely awful. If I need to generate a large and complex XML document, I do not want to write out each node by hand.

This cat also does not want to write XML by hand. source

The Builder gem allows for a much for semantic and less painstaking approach to writing XML.

xml = Builder::XmlMarkup.new
xml.dog { |d| d.name("Moebi"); d.mood("sleepy") }

will return:

<dog>
  <name>Moebi</name>
  <mood>sleepy</mood>
</dog>

Much more pleasant to write, and much more Ruby-like.

The Builder gem also gives us access to XML-specific support features that we simply don't have with just plain old ERB. For example, we can validate the XML itself (i.e. is the structure valid), and we can validate our XML documents against a given schema.

Okay, we're convinced that we don't want to use ERB to build our XML documents.

Let's also assume that we are not generating XML documents from our Rails controller actions. We want to be able to generate large and complex XML documents anywhere.

We could simply use the Builder gem to define really (really really) long methods that are responsible for building XML documents. That doesn't sound like clean code to me however. If we're looking for a way to take a given album and use it to write an XML document, we might end up with something like this:

# app/models/album.rb

def to_xml
  xml = Builder::XmlMarkup.new
  xml.MessageHeader do |header|
     header.Message self.message 
  end
  xml.Artist do |artist|
    artist.Name self.artist.name
  end
  xml.ReleaseList do |release_list|
     self.tracks.each do |track|
       ...
     end
  end
end

Even in this incomplete example, we can see the bad design. Not only will we have one large (or even several medium-sized, if we use helper methods) method that is nearly impossible to read, but we are violating the Single Responsibility Principle.

The Album model should not know how to write XML. Which is to say it should not contain the logic for generating an XML document. We should keep our XML-building logic in specific XML templates, and give these templates access to the album instance that it needs to generate the final document.

Instead, we'll build our own templating engine that allows to us to write XML documents along these lines:

album = Album.find(album_id)
xml   = Builder::XmlMarkup.new
render "name_of_template_file", album, {xml: xml}

Let's get building!

Using Tilt to Template Ruby

Our custom XML templater will use the Tilt Ruby templating engine to evaluate XML templates in the context of a given Ruby object, i.e. our album instance.

Tilt initializes with an argument of the path to the template file to be rendered:

templater = Tilt.new(path_to_template_file)

We can then call render on our template engine instance:

templater.render(album_instance, options)

The render method's first argument is the evaluation scope. In other words, it sets the scope within the template being rendered to the object that is passed in. So, inside our template file (coming soon!), self becomes the album (or single or track or whatever) instance that we pass in as the first argument here.

render also takes in a hash of options. In this case, we want to pass in our XML Builder instance so that we can continue to use it to build our XML.

Let's say we have an XML template, album.builder. Note: Tilt only supports rendering of files with specific extensions, for a full list, look here.

# album.builder
xml.instruct!(:xml, :version=>"1.0", :encoding=>"utf-8")
xml.MessageHeader do |header|
   header.Message self.message 
end
xml.Artist do |artist|
  artist.Name self.artist.name
end
xml.ReleaseList do |release_list|
  self.tracks.each do |track|
     ...
  end
end

We can render our template like this:

album = Album.find(album_id)
xml   = Builder::XmlMarkup.new(indent: 2)
Tilt.new("album.builder").render(album, xml: xml)

When we call render on our Tilt instance, it sets the scope of the template to the scope of the album instance, and passes in our XML builder instance as a local variable. Within the context of our template then, self refers to the album instance and xml refers to our XML instance.

The result of our above call to Tilt will be our string of XML:

<?xml version="1.0" encoding="utf-8"?>
<MessageHeader>
  <Message>Deliver Album</Message>
</MessageHeader>
<Artist>
  <Name>Madonna</Name>
</Artist>
...

Now that we understand how Tilt and XML Builder will work together to compile XML templates, let's build our custom templating class.

Building the XML Templating Engine

We'll define a class XmlFormatter, that will act as our engine. This class will know how to render a given XML template, in the context of a given object. For example, given an instance of our Album model, the template engine will know how to compile the album XML template.

Our XmlFormatter will initialize with an instance of the object to be formatted, for example an album. And it will initialize an instance of the XML builder, courtesy of the Builder gem.

# app/services/xml_formatter.rb
class XmlFormatter
  attr_reader :formattable

  def initialize(formattable)
    @formattable = formattable
    @xml         = Builder::XmlMarkup.new(indent: 2)
  end
end

Our formatter will respond to a method, format, which calls on a helper method that uses Tilt.

# app/services/xml_formatter.rb
def format
  render file_name, formattable, xml: @xml
end

def render(file_name, object, options)
  file = "#{template_path}/#{file_name}.builder"
  Tilt.new(file).render(object, options)
end

def template_path
  Rails.root.join("app", "xml_templates")
end

def file_name
  formattable.class.name.downcase
end

Note: We'll build our templates soon, but for now, this code assumes that we have a directory, app/views/xml_templates, and that directory contains a file, album.builder.

Now we, can call our template engine like this:

formatter = XmlFormatter.new(album)
formatter.format

Now we're ready to build our templates.

Building the XML Templates

The code in our XmlFormatter class assumes we have our XML templates in a directory, app/xml_templates/. It further assumes that we define our individual templates as <model_name>.builder. Since we're working with the example of templating album-specific XML, we'll define our template file, album.builder

# app/xml_templates/album.builder
xml.instruct!(:xml, :version=>"1.0", :encoding=>"utf-8")

xml.tag!("ern:NewReleaseMessage") do |ern|
  ern.ResourceList do |resource_list|
    tracks.each do |track|
      xml.SoundRecording do |sound_recording|
        sound_recording.SoundRecordingId do |sound_recording_id|
          sound_recording_id.ISRC isrc
        end
        sound_recording.ResourceReference "A#{number}"
        sound_recording.ReferenceTitle do |reference_title|
          reference_title.TitleText title
        end
        ...
      end
    end
    xml.Image do |image|
      image.ImageType 'FrontCoverImage'
      ...
    end
  end
end

Even from this small example snippet, we can see that this template file is going to get way out of hand. We need to write the required XML to describe each track, each artist, the album artwork, and more. That is a lot of XML to pack into one template.

What would we do if we were writing a super long HTML view file? We would write a series of partials! It would be really sweet if we could do something similar here. We know that our Tilt instances respond to render, so we could do something like this:

# app/xml_templates/album.builder
xml.instruct!(:xml, :version=>"1.0", :encoding=>"utf-8")

xml.tag!("ern:NewReleaseMessage") do |ern|
  ern.ResourceList do |resource_list|
    tracks.each do |track|
      Tilt.new("app/xml_templates/track.builder").render(track, xml: resource_list})
    end
  Tilt.new("app/xml_templates/album_image.builder").render(album_image, xml: xml)
  end
end

Here, we generate a new instance of our Tilt engine, give it the path to a new template, track.builder. Then, we call render on that instance, setting the scope of the template to be rendered to the track instance, and setting a local variable xml to the resource_list XML node. Our track.builder template would then look something like this:

# app/xml_templates/track.builder
xml.SoundRecording do |sound_recording|
  sound_recording.SoundRecordingId do |sound_recording_id|
    sound_recording_id.ISRC isrc
  end
  sound_recording.ResourceReference "A#{number}"
  sound_recording.ReferenceTitle do |reference_title|
    reference_title.TitleText title
  end
  ...
end

We could then call on still further template partials from there:

# app/xml_templates/track.builder
xml.SoundRecording do |sound_recording|
 Tilt.new("app/xml_templates/sound_recording.builder").render(track, xml: sound_recording})
end

There are two drawbacks to this approach that I can see. First of all, its not very semantic. It's a drag to have to write out the full call to instantiate Tilt and render a template, every time we want to render a template. Second of all, it's repetitive. We've already wrapped up the logic of instantiating Tilt and rendering a template with a given object in our XmlFormatter class.

Instead, let's teach our Album, Track, and any other classes we'd like to XML-format, how to respond to a semantic render method. That way, we'll be able to call on our partials like this:

# app/xml_templates/album.builder
xml.instruct!(:xml, :version=>"1.0", :encoding=>"utf-8")

xml.tag!("ern:NewReleaseMessage") do |ern|
  ern.ResourceList do |resource_list|
    tracks.each do |track|
      render "track", track, xml: resource_list
    end
  render "album_image", album_image, xml: xml
  end
end

Teaching Our Models How to Render XML

We'll define a module that we can mix in into any model that needs to be formatted as XML. We will use this module to teach any model that includes it how to generate the appropriate XML.

First, we'll define a #to_xml method that kicks off our XML generation by calling on XmlFormatter#format

# app/models/concerns
module XmlFormattable
  def to_xml
    formatter.format
  end

  def render(file_name, object, options)
    formatter.render(file_name, object, options)
  end

  def formatter
    @formatter ||= XmlFormatter.new(self)
  end
end

Now we'll define a render method that wraps a call to XmlFormatter#render, passing in the arguments of the template file to be rendered, the object whose scope we are using to render, and the options hash that sets our XML instance local variable:

# app/models/concerns
module XmlFormattable
  def to_xml
    formatter.format
  end

  def render(file_name, object, options)
    formatter.render(file_name, object, options)
  end

  def formatter
    @formatter ||= XmlFormatter.new(self)
  end
end

This module should be included in any model that needs to render XML:

class Album < ActiveRecord::Base
  include XmlFormattable
end

class Track < ActiveRecord::Base
  include XmlFormattable
end

Now we can generate fully compiled XML templates for a given album instance like this:

album.to_xml

This will call on our XmlFormatter#format method, rendering the XML template, album.builder, within the scope of the given album. This will in turn render XML template partials via:

#app/xml_templates/album.builder
...
xml.ResourceList do |resource_list|
  tracks.each do |track|
    render "track", track, xml: resource_list
  end
end

Where render is getting called on the album instance, and delegated to the formatter instance.

Our template engine is complete! We've built an abstract, reusable XML template engine with our XmlFormatter class, and implemented a flexible pattern of XML rendering with the help of our XmlFormattable module.