A presentation at MidCamp 2020 in in Chicago, IL, USA by April Sides
weekbeforenext weekbeforenext aprilsides
⭑ Discovery and Planning ⭑ Strategies and Workflow ⭑ Magical Nerdery
Photo by Elodie Oudot on Unsplash
⭑ Drupal 7 Multisite (85+ sites) ⭑ Drupal 8 Multisite (~6 at a time) ⭑ Hosted on Acquia ⭑ Hosted on Acquia ⭑ 27 Content Types (15 migrated) ⭑ 20 Content Types (14 populated) ⭑ 14 Taxonomy Vocabularies (9 migrated) ⭑ 17 Taxonomy Vocabularies (10 populated) ⭑ Paragraphs, Field Collections and Entity Embeds ⭑ “Micro-content” Types and Entity Embeds
https://www.drupal.org/project/migration_planner
Photo by mali maeder from Pexels
⭑ WYSIWYG string queries: ✩ Object tags ✩ Image tags ✩ Embedded entities ✩ Absolute links ✩ Span tags ✩ iFrames ✩ Script tags ✩ Tables ✩ ✩ ✩ ✩ ✩ ✩ ✩ ✩ Custom classes Custom forms Email links Style tags and attributes Social network links Video iFrames Links to PDFs File Lists
⭑ ⭑ ⭑ ⭑ ⭑ ⭑ ⭑ ⭑ Content by date thresholds Parent Content Unpublished Content Content with documents Paragraphs Field Collections Specific field values Redirects ⭑ Metatag string queries: ✩ Description ✩ Abstract ✩ Token ✩ Creator ✩ Canonical ✩ Title ✩ Image ✩ Keyword ✩ Video ✩ Twitter ✩ Open Graph
$ drush @[SITENAME].[ENVIRONMENT] squeal /sites/[SITENAME].georgia.gov/files/ga_squealer_reports/ [SITENAME].georgia.gov_YYYY-MM-DD.xlsx
https://www.drupal.org/project/squealer
Photo by chuttersnap on Unsplash
❌ ⭑ ⭑ ⭑ ✅ Structural elements: ⭑ Data elements: ✩ Content types and field definitions ✩ Select nodes and field data ✩ Vocabularies ✩ ✩ Paragraph and field collection bundles Select taxonomy terms Views Webform submissions ⭑ Menus ⭑ Webforms ⭑ Files
⭑ ⭑ Core: ⭑ Contributed: ✩ Migrate migrate ✩ Migrate Files (extended) migrate_file ✩ Migrate Drupal migrate_drupal ✩ Migrate Plus migrate_plus ✩ Migrate Source CSV migrate_source_csv ✩ Migrate Tools migrate_tools ✩ Drupal Upgrade migrate_upgrade ✩ Migrate Source UI migrate_source_ui Custom: ✩ ga_migrate ✩ ga_migrate_site ✩ ga_migrate_source_ui
⭑ Local development environment: ✩ ⭑ QA environment: ✩ ⭑ Lando Tugboat.qa DevOps magic: ✩ CircleCI ✩ Quay.io
https://tugboat.qa
⭑ Focus on field mapping by complexity/content type: ✩ Basic fields ✩ Rich fields ✩ File/image fields ✩ Paragraphs/Field Collections ✩ WYSIWYG cleanup
⭑ Create/edit migration configuration files directly in config sync directory ⭑ Preserve nids for standalone nodes ⭑ Migrate unpublished content ⭑ Prioritize ability to rollback and re-import ⭑ Use ga_migrate_site for site specific overrides ⭑ Log skips and exceptions using custom logging solution ⭑ Solution order: ✩ Configuration, core and contrib ✩ Custom source/process plugins and services in ga_migrate module ✩ hook_migrate_MIGRATION_ID_prepare_row()
Photo by Dorelys Smits on Unsplash
ga_migrate_log( $message, // String: detailed message with ids. $migration_id, // String: current migration id. $audience, // String: ‘DSGa’ or ‘dev’. $needs_fix, // Boolean: Does this need to be fixed. $severity, // String: ‘warning’, ‘notice’, or ‘error’. $category, // String: Short descriptor. $row_id // Integer: The current row id. );
Photo by Farzad Mohsenvand on Unsplash
⭑ Add/remove sites from Tugboat ⭑ Run migrations on Tugboat ⭑ Client QA’s migrations on Tugboat ⭑ Development team fixes issues ⭑ Run migration on Production ⭑ Client prepares sites for launch ⭑ Sites are launched
Photo by James Wheeler from Pexels
/modules/custom/ga_migrate_site /sites/[SITENAME].georgia.gov/modules/custom/ga_migrate_site
… ga_migrate: 0 ga_migrate_site: 0 … core.extension.yml
interface GaMigrateSiteInterface { // Press_release nodes with release date greater // than threshold are trashed. const GA_MIGRATE_SITE_PRESS_RELEASE_THRESHOLD = 3; // The date unit that accompanies the threshold value. const GA_MIGRATE_SITE_PRESS_RELEASE_THRESHOLD_UNIT = ‘y’; // The site node skip list. const GA_MIGRATE_SITE_SKIP_LIST = []; } /modules/custom/ga_migrate_site/src/Plugin/GaMigrateSiteInterface.php
// Implements hook_migrate_prepare_row(). // Implements hook_migrate_MIGRATION_ID_prepare_row(). /sites/[SITENAME].georgia.gov/modules/custom/ga_migrate_site/ga_migrate_site.module
Photo by Victor Larracuente on Unsplash
node field_content two_column gta_paragraph_related_links gta_paragraph_image one_column gta_paragraph_text_area
⭑ Convert Paragraphs into to Drupal 8 markup ✩ ⭑ Includes entity embeds Stack new content in Body field
… prepared_field_content: plugin: ga_micro_content_to_text source: - field_content micro_content: field_content: paragraphs_item …
⭑ What should the Drupal 8 markup be for this paragraph when placed in the WYSIWYG Body field? ✩ Text markup ✩ Entity embed code ✩ File download link ✩ Alignment adjustments
node field_content two_column 6 3 gta_paragraph_related_links gta_paragraph_image one_column 2 5 gta_paragraph_text_area 4 1
… body/0/value: plugin: concat source: - ‘@prepared_field_content’ - body/0/value …
Photo by Anita Austvika on Unsplash
site_page node link_collection node field_related_links field_rich_links Link Link Node reference Node reference Node reference Node reference Link Link Link Link ga_d7_site_page_field_related_links
site_page node link_collection node nid topic_page node body entity embed code ga_d7_node_site_page
… prepared_field_related_links: plugin: migration_lookup migration: ga_d7_site_page_field_related_links source: nid no_stub: true plugin: ga_entity_embed_code entity_type: node … ga_d7_node_site_page
… body/0/value: plugin: concat source: - body/0/value - ‘@prepared_field_related_links’ … ga_d7_node_site_page
⭑ ga_d7_node_site_page ✩ �� ga_d7_site_page_field_related_links ��
site_page node link_collection node field_related_links field_rich_links Link Link blog_post reference blog_post reference site_page reference NULL Link Link Link Link ga_d7_site_page_field_related_links
… site_page_stub_entity_references: plugin: sub_process source: field_related_links process: entity_references: … plugin: migration_lookup migration: ga_d7_node_site_page source: field_related_content/0/nid stub_id: ga_d7_node_site_page …
site_page node link_collection node field_related_links field_rich_links Link Link blog_post reference blog_post reference site_page reference topic_page reference Link Link Link Link ga_d7_site_page_field_related_links
… site_page_stub_entity_references: plugin: sub_process source: field_related_links process: entity_references: … plugin: migration_lookup migration: ga_d7_node_site_page source: field_related_content/0/nid stub_id: ga_d7_node_site_page …
⭑ ga_d7_node_site_page ✩ ⭑ �� �� �� ga_d7_site_page_field_related_links ga_d7_site_page_field_related_links ✩ ga_d7_node_blog_post ✩ “ga_d7_node_site_page” �� ��
⭑ The non-specified dependency migration id is alphabetically before the current migration: ✩ ga_d7_node_site_page ✩ ga_d7_site_page_field_related_links
⭑ ⭑ ⭑ The non-specified dependency migration id is alphabetically after the current migration: ✩ ga_d7_index_list_field_related_links ✩ ga_d7_node_index_list The fix hack: ✩ ga_d7_1_node_index_list ✩ ga_d7_index_list_field_related_links Migration Dependency When Stubbing Content: https://www.drupal.org/project/drupal/issues/3024634
👏
⭑ Georgia GovHub: a case study by Darren Peterson at Drupal GovCon 2019 ✩ ⭑ This is also coming to DrupalCon Minneapolis 2020! A forest of designs without subthemes: Implementing Georgia.gov’s front-end in Drupal 8 by Marc Drummond at Drupal GovCon 2019
⭑ Real Life Data Migrations on the Lullabot Podcast ⭑ An Overview for Migrating Drupal Sites to 8 by Juampy NR ⭑ Running and Testing Drupal 8 Migrations in CircleCI by Juampy NR ⭑ 31 Days of Drupal Migrations with Mauricio Dinarte on the Lullabot Podcast ⭑ Managing Authentication During API Migrations by April Sides ⭑ and more to come!
As Digital Services Georgia upgraded their Drupal 7 multisite platform to Drupal 8, they capitalized on the opportunity to make improvements to their content model. Data migrations were customized to move and shape data to fit into new content types and fields.
Let’s take a look at some of the strategies, tools, and techniques used to migrate site data from the GeorgiaGov Platform to Georgia GovHub.
Topics that will be covered:
for free. You
can too.