Migrating Paragraphs to Layout Builder in Drupal

There have been many site-builder tools for building flexible layouts in Drupal including Field Collection, Panels, and Paragraphs. But only one of them is part of Drupal core: Layout Builder. Layout Builder provides a clean and user friendly drag-and-drop editorial experience. But don’t take my word for it. If you haven’t read the amazing blog post, The Big, Bad Layout Builder Explainer by Caroline Casals you owe it to yourself to do so. Go ahead, I’ll wait.

“I want to use Layout Builder, but my very large site has tens of thousands of Paragraphs, and you can’t migrate Paragraphs to Layout Builder”, you might be saying. Well let me tell you it is possible to do, and I have done it. In fact, I have migrated over 100,000 paragraphs into Layout Builder, and you can too! 
 

A Warning

Migrating Paragraphs to Layout Builder is a sometimes difficult, often tedious process. There are several key challenges that can eat into your development time, making the case for moving to Layout Builder that much more of a hard sell to stakeholders.

One of such challenges is the often occurring Paragraph within Paragraph architecture. This is common as Paragraphs are often used for layout as well as structured content. For example, a three-column layout can be built with Paragraphs. Each column would have a corresponding Paragraphs entity reference revision field. Migrating these types of architectures is complex. Due to this we won’t cover them today, but once you get the hang of the process it should become clear where the customizations are needed to accomplish a migration with container Paragraphs. 
 

Setup

For the purpose of this writeup we will assume you are migrating Paragraphs to Layout Builder on Nodes during a Drupal upgrade from D7 to D8. Since Layout Builder works with fieldable entities, it is possible to use Layout Builder for things like Menus as well, as you may have seen with Mike Potter’s blog post, Creating a Mega Menu using Layout Builder in Drupal 8.

In this writeup I will often refer to a repository which contains a full working example of a migration from Drupal 7 with Paragraphs to Drupal 8 with Layout Builder.

https://github.com/dsasser/d8-migrate-paragraphs-layout-builder

Before beginning, ensure that you have enabled Layout Builder for your content type using override section storage. Read about how to set up default layouts on Drupal.org. Modify the default layout by removing unwanted blocks or otherwise changing the layout per your specifications. In the example repo, I have set the body field to display in the default layout to demonstrate using default and override section storage together. More on this later.

Layout Builder Storage

To understand how the migration needs to be written, we must first understand how to store data in Layout Builder. Layout Builder doesn’t store data like a traditional Drupal field might. It uses what is called a “section”. Sections are composed of one or more regions, and in those regions are placed blocks of various types.

Layout Builder uses three types of Section storage: default, override, and temporary.

Default Section Storage

When you enable Layout Builder for a Content Type, you do so at the display mode administration UI at /admin/structure/types/manage/[content-type]/display. There you will find a ‘Use Layout Builder’ checkbox. When you check this box, you see a second checkbox appear, ‘Allow each content item to have its layout customized’. If you leave the second checkbox unchecked, it means that you will be using the same layout for each Node of this type in this display mode. This is the default Section storage: each Node has the same layout. Editors can not change the layout or add blocks to it on an individual Node basis. The default layout is stored in config for the entity view mode in third party settings.

Override Section Storage

By enabling the second checkbox mentioned above with the label ‘Allow each content item to have its layout customized’ you are telling Layout Builder that each entity can have a different layout and therefore is editable on an individual Node basis at the node/[nid]/layout tab. When enabling this storage mode, Layout Builder adds a field to the Node with a machine name of layout_builder__layout. This field stores the sections and other metadata used by Layout Builder in a serialized blob in the database.

Using override storage is a bit like using both default and override storage together. A default layout is used to create the initial layout of a new node, and override storage is what allows customizing this layout on a per-node basis. This combination is possible to retain during the migration as well, as we will see in a later section.

Temporary Section Storage

When editing a layout, prior to saving, the layout sections are stored in the core SharedTempStoreFactory service, similar to how Views stores unsaved changes. We will not be working with this type of storage during the migration process, but I pointed it out here so that you have a well rounded understanding of Layout Builder section storage.

Section Storage Construction

So we know that a layout contains one or more sections, sections contain regions, and regions contain blocks, but this is not the whole story, and in fact not exactly correct either. You see, a section is really a container for section components and other properties, and knowing how it comes together is crucial to writing the migration.

Let’s look at a node that has Layout Builder override storage enabled and see how it is constructed. We can see how the layout field data is constructed by loading the Node and calling the toArray() method on it. This gives us the exact pattern needed for the migration destination:  
 

Layout

The first element we encounter is the ‘layoutId’. Layouts are created as yml files and are stored at the root of a module or theme. You can read more about defining and building layouts on Drupal.org.

Next we see the ‘layoutSettings’. I won’t go into detail about layouts too much here, but know that the layout, template, and settings all define the behavior of the layout itself. For instance, for a multi-column layout, you might want to allow the editor to provide a width for each column, and layout settings are where you would do that. The settings are exposed in Drupal forms in the sidebar when adding or editing a section.

We will use the core-provided ‘layout_onecol’ layout: a single column layout with no configuration options, and a single region called ‘content’ for simplicity.

Components

As you can see in the figure above, a component is a SectionComponent object, and a section can contain one or many components. Components are a metadata wrapper for a block: they store data about a block including the region of the layout it should be placed in, configuration, and weight.

Region

A component ‘region’ tells the section where to place the component in the layout. Some layouts only have one region, in the case of the node we are viewing it is, ‘content’. For multi-column layouts, you might have ‘left’ and ‘right’ regions, or ‘first’, ‘second’, etc.

Configuration

The ‘configuration’ array in a section component is the beating heart of the component. This specifies exactly what kind of block belongs in the region, and block display settings that tell Layout Builder how to render it. In this case, we are seeing an inline_block:basic block. This means that in this component will be placed a content block with a machine name of ‘basic’, with a revision id of 16966. This is the glue that tells Layout Builder to render a basic block with this revision id. If there were other settings that controlled the display of the block, they would be here as well.

Weight

Like other areas of Drupal, the components in a section’s region will be sorted by weight, allowing you to add them in any order provided you also supply the correct weight in the migration, should that be available.

Let's Get To It

Wow that is a lot of information! If you managed to get through all that, you are now armed with that enough to get started, so let’s dive into the good stuff.

The Process

There is no one-size-fits-all solution to migrating Paragraphs to Layout Builder. How you go about it depends on your situation, and it is going to require writing some code to get it done. Overall, however, there are a few necessary steps regardless of your path.

Paragraphs to Blocks

The first step in the process is to get your paragraphs migrated to blocks. There is no secret sauce here, it is just a basic migration. One thing that I would recommend is having a single migration for each Paragraph type, rather than a single monolithic migration that handles all of them. This makes writing any necessary logic for a given paragraph easier to target. And, you can use the migration group shared configuration for properties and fields that are common among all the paragraphs, if that is beneficial to you.

Obviously, before you can migrate the Paragraphs to blocks, you need to build and configure those blocks on the destination. You might end up with a one-to-one relationship between the Paragraphs bundles and block types, or you might take the opportunity to consolidate or otherwise change the content model. This is ok because at the end of the day we just need blocks, so however you arrive there will work for this migration.

Linked below is an example of a blockquote Paragraph migration that creates new content blocks with a machine name of ‘blockquote‘:

migrate_plus.migration.paragraphs_blockquote.yml

And the corresponding group migration configuration:

migrate_plus.migration_group.paragraphs_to_block.yml

Reusable Blocks

If your site has many thousands of content blocks, consider setting the ‘reusable’ field to 0, or you may experience site performance issues. This is due to how block permissions are calculated on each page. By setting the reusable field  in your process mappings, the Block will become essentially invisible to Drupal for use anywhere. It will not appear in the Block layout UI, it will not appear in Layout Builder’s sidebar when adding a block, etc. And it means it will not be considered when core builds Block permissions or calculates region conditions, at least on pages where the Block isn’t used. Set the reusable field like so in your migration process configuration:

reusable:
  plugin: default_value
  default_value: 0

Node Migration

Once the blocks are created by running the paragraph to block migrations, we are ready to write the Node migrations. The below example may be for a single content type, but this pattern can be applied to any or all of the Node bundles involved in your particular situation.

Here is the Article node migration used in the example repo:

migrate_plus.migration.article.yml

And the related group migration configuration:

migrate_plus.migration_group.content.yml

The important bits for Layout Builder migration are in this portion of the process configuration:

  default_temp:  
    plugin: default_layout
    bundle: article
  paragraphs_temp:
    plugin: paragraphs_layout
    source_field: field_paragraphs
  layout_builder__layout:
    plugin: get
    source:
      - '@default_temp'
      - '@paragraphs_temp'

Here we can see that there are two temporary field mappings which use custom process plugins: default_layout and paragraphs_layout, and we are adding the output of these two temporary fields to the layout_builder__layout field for the Article.

Layout Process Plugins

This is where the magic happens. The core of our customizations necessary for migrating Paragraphs to Layout Builder are “simply” a few migration process plugins. These plugins return sections that we store in temporary fields. We then assemble each temporary field in the order we want them displayed on the page by using them as sources for our layout_builder__layout destination field.

Default Layout Process Plugin

The default layout process plugin is very simple. As we have learned, the default layout is stored in configuration in the view mode of the entity as third party settings. So the job of this process plugin is to grab that configuration and return it. This is easily accomplished using the config factory service and Layout Builder’s API for converting an array into a section object:

$config = $this->configFactory->get("core.entity_view_display.node.{$bundle}.default");
$sections_array = $config->get('third_party_settings.layout_builder.sections');
$sections = [];

if (!empty($sections_array)) {
  foreach ($sections_array as $section_data) {
    $sections[] = Section::fromArray($section_data);
  }
}
return $sections;

You can find the default_layout process plugin code here in the example repo.

Paragraphs Layout Process Plugin

At last, the piece that brings everything together: the Paragraph process plugin. This plugin is doing some heavy lifting and because of this can get rather complicated very quickly. For this reason, the provided plugin I wrote handles “non-container” paragraphs only. We aren’t doing accordions or tabs or anything like that here, but as I mentioned earlier, it is possible to do. Perhaps I’ll touch on that in a future blog post.

At the end of the transform method, the process plugin needs to return one or more sections, and in order to do that, performs the following tasks:

• Create a section.
• Iterate over the source field values.
• For each value, find the paragraph type.
• Find the correct block migration for the identified paragraph type.
• Find the migrated block.
• Find the revision id of the block.
• Create a component from the block information.
• Append the component to the section.
• Return the section.

You can find the paragraphs_layout process plugin code here in the example repo.

Create a Section

This is relatively straightforward since we have only one layout template to worry about. In the LayoutBase class, I have a method called createSection() which takes in several parameters with default values. Since we only have one layout in this migration, there is no need to pass in any values to this method, but it is written to be flexible enough to handle any type of layout required for your use case.

public function createSection(array $components = [], $layout = 'layout_onecol', array $settings = []) {
  return new Section($layout, $settings, $components);
}

Find Paragraph Type

When iterating over the source field values, we have only the paragraph id and revision id to work with. Using that information, however, we can obtain the paragraph type for a given id by querying the source migration database. In the example below, the migrateDb property is a connection to the source database.

public function getParagraphType($id) {
  $types = &drupal_static(__FUNCTION__);
  if (!isset($types[$id])) {
    $query = $this->migrateDb->select('paragraphs_item', 'p');
    $query->fields('p', ['bundle']);
    $query->condition('p.item_id', $id, '=');
    $types[$id] = $query->execute()->fetchField();
  }
  return $types[$id];
} 

This method is found in the ParagraphsLayout process plugin class.

Create New Component

To create a new component for the section, we need to locate the block that was created during the related paragraph to block migration. To do this, we leverage the migrate lookup service provided by core. In order to do this, however, we need to know the machine name of the migration that corresponds to the paragraph type in question.

Since migrations store source to destination mappings id’s in a mapping table named for the migration involved, we can use the migrate lookup service to locate the block that was created during the migration. All we need to do this is the source id and the migration id. To make this process easy, I used the Article migration source configuration to store a mapping of the paragraph type to block migration id in the constants section.  

source:
  plugin: d7_node
  node_type: article
  constants:
    map:
      block_quote: 'paragraphs_blockquote'
      rich_text: 'paragraphs_rich_text'

This mapping can be used in process plugins by pulling out the configuration like so:

$map = $row->getSource()['constants']['map'];

Now that we have the migration id for the current paragraph type, we can lookup the block the migration created when it ran. The code from below is taken from the LayoutBase class:

public function lookupBlock($migration_id, $id) {
  // Find the block from the related migration.
  $source = [$id];
  $block_ids = $this->migrateLookup->lookup($migration_id, $source);
  if (empty($block_ids)) {
    throw new LayoutMigrationMissingBlockException(sprintf('Unable to find related migrated block for source id %s in migration %s', $id, $migration_id), MigrationInterface::MESSAGE_WARNING);
  }
  return reset($block_ids)['id'];
}

If the block was located, the lookupBlock() method returns the id of the block, or throws a LayoutMigrationMissingBlockException which is a custom exception that we trap with a try/catch block in the ParagraphsLayout process plugin’s transform() method. The exception logs the missing block information to the migrate message table for the given node. Without this trapping and logging, this information would be lost and troubleshooting would be difficult.

Layout Builder needs a block revision id, not the block id, so we need to query this from the destination database. This and other information necessary for creating a new section component resides in the createComponent() method in the LayoutBase class:

public function createComponent(LayoutMigrationItem $item, $region = 'content') {
  // Find the block from the related migration.
  $block_id = $this->lookupBlock($item->getMigration(), $item->getId());
  // Get the block type. Use a db query instead of loading the entity for
  // performance.
  $query = $this->db->select('block_content_field_data', 'b')
    ->fields('b', ['type'])
    ->condition('b.id', $block_id, '=');
  $block_type = $query->execute()->fetchField();
  if (!$block_type) {
    throw new MigrateException(sprintf('An unknown error occurred trying to find the block type from migration item type %s with id %s.', $item->getType(), $item->getId()));
  }
  // Get the latest revision id for the block.
  $block_revision_id = $this->blockContentStorage->getLatestRevisionId($block_id);

  // Create a new component from the block.
  return $this->createSectionComponent($block_revision_id, $block_type, $item->getDelta(), $region);
}

Finally, we can create a new section component:

public function createSectionComponent($block_latest_revision_id, $block_type, $weight = 0, $region = 'content') {
  return SectionComponent::fromArray([
    'uuid' => $this->uuid->generate(),
    'region' => $region,
    'configuration' =>
      [
        'id' => "inline_block:{$block_type}",
        'label' => 'Layout Builder Inline Block',
        'provider' => 'layout_builder',
        'label_display' => '0',
        'view_mode' => 'full',
        'block_revision_id' => $block_latest_revision_id,
        'block_serialized' => NULL,
        'context_mapping' => [],
      ],
    'additional' => [],
    'weight' => $weight,
  ]);
}

This component will be appended to the section that is ultimately returned by the process plugin:

$section->appendComponent($this->createComponent($item)); 

This process is repeated for each paragraph on the source Article node.

Section Flattening

The default_layout process plugin might return one or more sections, or if you modify the paragraphs_layout process plugin, it too may need to return more than one section. Because of this, we might end up with a value in the destination field that will not be expected by field storage, and therefore rejected entirely. To resolve this, I used an event subscriber that subscribes to the ‘pre row save’ event, and checks for the layout_builder__layout field in the destination. If found, it flattens the multidimensional array into the single dimension expected by entity field storage. I think the docblock for the subscriber method does a good job of explaining it further:

/**
  * Migration pre-row save event subscriber.
  *
  * This method is used to flatten the layout_builder__layout field into a
  * single-dimensional array. This is needed because some of the layout plugins
  * can add multiple sections to this field and this is not a structure
  * supported by the field. Consider the following array, where
  * [Layout Section] is a Drupal\layout_builder\Section object:
  *
  * @code
  *   $layout_builder__layout => [
  *     0 => [Layout Section],
  *     1 => [
  *       0 => [Layout Section],
  *       1 => [Layout section],
  *     ],
  *     2 => [Layout Section],
  *   ];
  * @endcode
  *
  * This method will produce a flattened layout field resulting in the
  * following:
  *
  * @code
  *   $layout_builder__layout => [
  *     0 => [Layout Section],
  *     1 => [Layout Section],
  *     2 => [Layout Section],
  *     3 => [Layout section],
  *   ];
  * @endcode
  *
  * @param \Drupal\migrate\Event\MigratePreRowSaveEvent $event
  *   A migration event.
  */

All that is left is to run the Article migration either in the UI or using Drush.

Conclusion

This process, while contrived for a single content type with just a few paragraphs, can be applied to scale to whatever your architectural needs may require. The paragraphs_layout process plugin can be expanded upon to support additional use cases such as for migrating container paragraphs.

We have been on quite a journey from learning how Layout Builder storage works, how sections are constructed, and finally how to write the migration plugins for moving from Paragraphs to Layout Builder. This process, as you have seen, can range from semi-complex to extremely convoluted, and as such needs careful consideration before commitment. But if you are ready and willing to make the leap to core-provided flexible layouts, then I hope I have made that at least somewhat easier for you. Good luck with your migration, should you choose to embark on it!