Drupal 8 Migrate Process Plugin: migration_lookup_target_ids

I was having some difficulties using the migration_lookup process plugin. I wrote this plugin so that I could make sense of how the migration lookup was working and the data that it was returning. The plugin ended up being something that I use after every migration_lookup as it seems to ensure error-free lookups. YMMV. Read more ›

Tagged with: , ,
Posted in Development

Drupal 8 Migrate – Tips and Tricks

This is an ongoing brain dump of quick tips for working with Drupal 8 Migrate. These items may seem true to me as I’m writing them today, but I’m constantly being reminded that things aren’t always as they seem with Migrate. It’s a wonderful system but can be tricky to understand. I recommend reading through my other Migrate-related posts, especially the post on debugging. Read more ›

Tagged with: , ,
Posted in Development

Extending the Migrate Plus JSON Parser in Drupal 8

The migrate_plus module provides a great JSON parser for easy migration of JSON data. With a small amount of code in a new class you can tweak the parser a bit. In my case I wanted to make a few simple changes:

  1. Speed up initial processing of the data
    1. Almost all of my data is at the top level in the JSON structure (there is no “Members” wrapper, for example).
    2. The stock JSON parser runs a selectByDepth() method if you use a item_selector: 0 , which slows things down considerably; it loops through every record, which is a bear if you have 46,000 records.
  2. Make it easy to manipulate the data (think prepare row)
    1. Using my overridden JSON parser as a base for additional parser classes it became nice to implement a simple prepareRows method to massage the data for each particular migration.
    2. I have several migrations coming out of the same data source (content type + supporting paragraphs). It’s nice to create a parser that extracts only the paragraph data so that we maintain an accurate count of the records for the “child” migration.

Read more ›

Tagged with: , ,
Posted in Development

Monitoring a Drupal 8 Migration Executed with Drush

Update 2 is proving to be the most useful method.

Update 1

Somehow I missed the documentation regarding the --feedback  argument for the drush mi  command. It’s similar to the methods below in that it helps you see that the migration is working, though you don’t see the totals. You can ask for feedback in seconds or items.

Update 2:

Another method that is a bit dirty but very useful is to add a single line to core/modules/migrate/src/MigrateExecutable.php.

Adding the highlighted line to \Drupal\migrate\MigrateExecutable::import will let you see exactly where the migration is (which row is being processed).

If this is something you want permanently, consider posting an issue on d.o to request more verbose logging. This is my hackey solution for my immediate needs.

The output (in your logs, and in the drush output) will be something like this (institution_node is my migration, inst_guid is the only id):

 


Original Post

This isn’t exactly an ideal solution, but it will do in a pinch. I’m working on a migration of more than 46,000 nodes (in a single migration among many). I needed a way to monitor the progress but I wanted to do it as quickly as possible (no coding). Here’s what I came up with, which I run in a new iTerm window.

All we’re doing is asking how many records are in the “member_node” migration map every 5 seconds. If we started at 6350 items we know that by the end of 20 seconds we have created 35 records.

We could also query the target content type itself:

Here we can analyze the data the same way to see how many records the migration is creating.

I recognize that this is pretty crude, but it’s effective; you are able to see that it’s working, instead of just staring at an unchanging prompt and watching your CPU jump.

Tagged with: , , ,
Posted in Development

Alter Date Format in a Drupal 8 Migration

Per https://www.drupal.org/node/2820490 you can now use the format_date plugin to alter date formats like this:

I do always recommend using skip_on_empty before any other process plugins.

Tagged with: , ,
Posted in Development

Link to a View (with Contextual Filter value) If It Will Have Results (Drupal 8)

Returns a link render array.

Tagged with:
Posted in Development

Drupal Views – Show One (Most Recent) Item Per Group

Alright, the title is a bit misleading because I’m not using Groups, but it is the best explanation of what I’m achieving with this example. My goal is to show a list of the most recent blog post in each category, where clicking the category takes you to the associated post.

I have a content type called Blog Post which has the following fields:

  • Date (field_blog_date – date field – single value)
  • Category (field_blog_tr_category – taxonomy reference field – single value)
  • Items (field_blog_cr_items – content reference field – unlimited values)

Read more ›

Tagged with: , , ,
Posted in Development

Automatic Redirect by Path in Drupal 8

This example shows how to intercept a Drupal page request, determine if the desired path matches the one we wish to redirect, find the “latest” node of a particular content type, and redirect to that node instead of the original page.

I have a page on my site called Updates. This landing page exists only so that there is a menu item and proper breadcrumb trail; users never get to this landing page. Rather, they’re redirected to the latest update when they attempt to visit the landing page. Updates are nodes of the type update. Each update node (when viewed as a page) shows a Recent Updates block in the sidebar with links to of all of the most recent updates.

The landing page is a Views-based page (so it doesn’t get accidentally deleted). There is also an All Views-based page that lists links to all updates. If the system cannot find the latest update the user will be redirected to the All page. Read more ›

Tagged with: ,
Posted in Development

Drupal 8 – Formatting a Date Field in Twig

Here are the four easiest ways I’ve found to show dates via Twig inside of a node template.

They all result in the same output.

Use the field’s formatter settings; include wrappers

This is the only example that includes wrappers. The rest output just the date.

 

Use the field’s formatter settings

This will use the format defined in Content type » Manage Displays » Your View Mode.

 

Use Twig date filter + a defined Drupal date format

I have defined, via the Drupal UI, a date format. Here’s the config for it:

uuid: 1a50c79f-96df-45de-9a21-32e0d8327806
langcode: en
status: true
dependencies: { }
id: short_mdyyyy
label: 'Short - m/d/yyyy'
locked: false
pattern: n/j/Y

 

Use Twig date filter

Tagged with: , ,
Posted in Development

Quick and Dirty Debugging of Drupal 8 Migrations

Introduction

If you’re working on a migration in Drupal 8 and you’re finding yourself a little confused by the YAML-based configuration, I strongly recommend using a proper debugger so you can step through the code. You cannot step through the YAML, but you can step through the code that the YAML triggers. This blog post is my attempt to share a few tips/tricks I’ve learned along the way. This isn’t meant to be an exhaustive step-by-step debugging tutorial, so you may have to do some Googling to fill in the blanks.

Disclaimer: Some of the information is PHPStorm-specific, but you should be able to adapt to whichever debugging tools you use. Read more ›

Tagged with: , ,
Posted in Development