Importing Content into Collections

When migrating content between two systems, it’s often easier to get data out of the old system than to import that data into the new. That’s not surprising, as content management systems are designed for outputting content. Reading content back in from a myriad of different formats can be quite a task.

This is the task we found ourselves facing when design the new Runway import functionality. What file format should we teach Runway to interpret, and how should we able users to map that incoming data to where it needs to go inside Runway? Our first thought was if we supported one common data format, a developer needing to import content could first write the code to get their data into that format, and then import their file.

The more I thought about it, the less sense that made. Why write code to move your data to a file, and then have Runway read that file back and have you go through some other process of mapping the data to the right places? Why not just write code to push the data directly into Runway via an API? That way, if you can get access to your content in a PHP context you can push it directly into Runway, and if you can’t, you just need to write the code to get that content into a format that can be read back in a PHP context. Best case scenario is simpler, and worst case scenario is no worse.

So that’s what we did, and created the Collection Importer API. Here’s how it works.

Getting started

To get started, either create a script that includes the Runway runtime, or use a master page. Create an instance of the Perch API, and a new CollectionImporter. Then tell the importer which collection to import into by name. (Here, it’s the ‘Articles’ collection.)

$API = new PerchAPI(1.0, 'my_importer');
$Importer = $API->get('CollectionImporter');
$Importer->set_collection('Articles');

The importer uses a template to know how to process the incoming content. This is how Runway knows how to handle each of the fields. Everything from Markdown to dates, to images needs to be handled differently, and it’s the tags in your template that define this. For our example, the (rather dull) template might look like this:

<perch:content id="title" type="text" label="Title" required="true" />
<perch:content id="body" type="textarea" label="Body" markdown="true" />
<perch:content id="slug" type="slug" label="Slug" required="true" />
<perch:content id="date" type="date" label="Date" required="true" format="d F Y H:i" />
<perch:categories id="category" set="articles">
    <perch:category id="catTitle" />
</perch:categories>

You can see that fields like body need to be treated as Markdown, and date really is a date field, regardless of its name.

So we need to create a Template object and configure it to use our template. We then pass this to the Importer for it to use.

$Template = $API->get('Template');
// Configure the template path and tag namespace
$Template->set('content/my_article', 'content');
// Assign the template to the Importer
$Importer->set_template($Template);

You’re now ready to start importing content. If your collection is new and you want to clean out previous test imports, you can call empty_collection(). Don’t do that on a collection with valuable content, it’s a hard delete.

$Importer->empty_collection();

Adding content

At this point you want to loop through your data source (whatever that may be) and map your content to IDs from your collection template. For each collection item you want to add, call add_item() with an associative array where the keys are the IDs from your template, and the values are the content you want to assign to that field.

The add_item() method validates required fields and will throw an exception if any item of content cannot be imported. For this reason, it’s best to use a try/catch block:

try {
  $Importer->add_item([
    'title'    => 'Breaking news!',
    'body'     => 'Some **exciting news** has _just_ broken...',
    'slug'     => 'breaking-news',
    'date'     => '2012-04-28 12:23:09',
    'category' => ['articles/news', 'articles/flippin-exciting'],
  ]); 
} catch (Exception $e) {
  die('Error: '.$e->getMessage());
}

And that’s it! Provided that no exceptions are thrown, all our content is imported and ready to use.

 Keeping it simple

Rather than specifying a complex file format to put your content into, the Importer lets you just work with any content you have. And if common file formats emerge that need to be imported, that code still only needs to be written once. An enterprising add-on developer may even look at putting together an app to accept and import common formats - I’m sure there’s an opportunity there.