Extending the Migrate Plus JSON Parser in Drupal 8

The migrate_plus module provides a great JSON parser for easy migration of JSON data. With a small amount of code in a new class you can tweak the parser a bit. In my case I wanted to make a few simple changes:

  1. Speed up initial processing of the data
    1. Almost all of my data is at the top level in the JSON structure (there is no “Members” wrapper, for example).
    2. The stock JSON parser runs a selectByDepth() method if you use a item_selector: 0 , which slows things down considerably; it loops through every record, which is a bear if you have 46,000 records.
  2. Make it easy to manipulate the data (think prepare row)
    1. Using my overridden JSON parser as a base for additional parser classes it became nice to implement a simple prepareRows method to massage the data for each particular migration.
    2. I have several migrations coming out of the same data source (content type + supporting paragraphs). It’s nice to create a parser that extracts only the paragraph data so that we maintain an accurate count of the records for the “child” migration.

Here’s the current version of my base class. This will probably change over time, but I’ll try to keep this post updated. I have highlighted the rows that are different from the stock JSON parser.

 

Tagged with: , ,
Posted in Development

Leave a Reply

Your email address will not be published. Required fields are marked *

*