Configuration management in Drupal 8 vs Drupal 7

DrupalCamp Cebu 2017
Thibaud (Tibo) Guilpain

In this session, we will look at configuration management, what it is, how it was done in Drupal 7 and how it is now done in Drupal 8.

(French) For those wondering where the accent comes from...

Pixel Onion is an Singapore based digital agency. We help medium and large organisations with digital strategy, user-centric design and web platform development. We try to empower our clients and make their life as easy as possible.

Started with Drupal 7 in 2012 and now moving to Drupal 8

What we are going to talk about

What is configuration management?
How was/is it done in Drupal 7?
What about Drupal 8?
Contribution modules for configuration management

I would like to have a feel of the people attenting this session, can you raise your hand if:

you are a site builder?
you are a developer?
dev with d8 experience?
d7 experience?
d8 experience?

What is configuration management?

In general, for a website and in Drupal

Configuration management (or CM)

"a systems engineering process for establishing and maintaining consistency of a product's performance, functional, and physical attributes with its requirements, design, and operational information throughout its life."

~Wikipedia

Thanks wiki, but hum what does that mean for us?

CM for a website5

CM for a website

What is a website?

CM for a website

What is a website?

Code or logic

CM for a website

What is a website?

Code or logic
Configuration of the code

CM for a website

What is a website?

Code or logic
Configuration of the code
Content (articles, pictures, videos and ...)

And in Drupal?6

And in Drupal?Code  
6

And in Drupal?Code   -> Drupal core + contributed & custom modules + libraries + theme
6

And in Drupal?Code   -> Drupal core + contributed & custom modules + libraries + theme
Configuration
6

And in Drupal?Code   -> Drupal core + contributed & custom modules + libraries + theme
Configuration -> Database
6

And in Drupal?Code   -> Drupal core + contributed & custom modules + libraries + theme
Configuration -> Database
Content
6

And in Drupal?Code   -> Drupal core + contributed & custom modules + libraries + theme
Configuration -> Database
Content -> Database
6

Ok, so ?

Where is the problem?

Configuration and content are in the database.

Config and content in the db you said?

Yes, and extracting the configuration from it is not easy. single dev could work on his own and deliver the website to the client, but as soon as it is in production and you need to make changes or updates, you would be stuck. You cannot put the client's website on maintenance, work on your update, and push the new database with the changes into production and then reope the website.

Where is the problem?

Configuration and content are in the database.

Could we separate the tables in the database and just export the relevant configuration ones?

No, not easily.

Config and content in the db you said?

How was/is it done in Drupal 7?

The hardcore way

All in custom modules' code:

all configuration is created programmatically using install and update hooks
- blocks
- views
- enable modules
- set variables
- ...

Yes I had some hard core colleagues that really didn't like using contrib modules unless they did exactly what they wanted and nothing more.

It is really good to understant how everything works, but really hard to get there. Also requires a lot of knowledge and most of the time reverse engineering.

The Features module

Can be used to export most of the configuration and has integration with most other modules.

-> Self disciplne is required

Or it can become a real mess. No real standard here though, each developer has his/her own way of sorting it all out.

Other modules?11

Other modules?Features master: allows you to export active modules, themes and permissions into a special master feature.
11

Other modules?

Features master: allows you to export active modules, themes and permissions into a special master feature.
Devinci: Allows your code to be context aware and switch between environments and automatically change some configuration in the process.

Conclusion on CM in Drupal 7Still quite complex
12

Conclusion on CM in Drupal 7

Still quite complex
Lots of experience needed to use all these tools efficiently

Conclusion on CM in Drupal 7

Still quite complex
Lots of experience needed to use all these tools efficiently
Always a pain point in Drupal development

What about Drupal 8?

CMI, the jewel of D8.

The Configuration Management Initiative (CMI)14

Since configuration management was so complicated in D7, and not part of the core, the community felt it needed an overall for Drupal 8, so here comes the CMI.

The Configuration Management Initiative (CMI)

Basic principles:

Since configuration management was so complicated in D7, and not part of the core, the community felt it needed an overall for Drupal 8, so here comes the CMI.

The Configuration Management Initiative (CMI)

Basic principles:

Active configuration:

Since configuration management was so complicated in D7, and not part of the core, the community felt it needed an overall for Drupal 8, so here comes the CMI.

The Configuration Management Initiative (CMI)

Basic principles:

Active configuration:
- Comes from installed modules and changes in UI

Since configuration management was so complicated in D7, and not part of the core, the community felt it needed an overall for Drupal 8, so here comes the CMI.

The Configuration Management Initiative (CMI)

Basic principles:

Active configuration:
- Comes from installed modules and changes in UI
- Stored in the database

Since configuration management was so complicated in D7, and not part of the core, the community felt it needed an overall for Drupal 8, so here comes the CMI.

The Configuration Management Initiative (CMI)

Basic principles:

Active configuration:
- Comes from installed modules and changes in UI
- Stored in the database
- Can be read

Since configuration management was so complicated in D7, and not part of the core, the community felt it needed an overall for Drupal 8, so here comes the CMI.

The Configuration Management Initiative (CMI)

Basic principles:

Active configuration:
- Comes from installed modules and changes in UI
- Stored in the database
- Can be read (with overrides)

Since configuration management was so complicated in D7, and not part of the core, the community felt it needed an overall for Drupal 8, so here comes the CMI.

Think display of content

The Configuration Management Initiative (CMI)

Basic principles:

Active configuration:
- Comes from installed modules and changes in UI
- Stored in the database
- Can be read (with overrides)
- Can be read and edited (without overrides)

Since configuration management was so complicated in D7, and not part of the core, the community felt it needed an overall for Drupal 8, so here comes the CMI.

Think display of content

Think configuration forms

The Configuration Management Initiative (CMI)

Basic principles:

Active configuration.

The Configuration Management Initiative (CMI)

Basic principles:

Active configuration.
Exported configuration:

The Configuration Management Initiative (CMI)

Basic principles:

Active configuration.
Exported configuration:
- In YAML files

The Configuration Management Initiative (CMI)

Basic principles:

Active configuration.
Exported configuration:
- In YAML files
- Exported via the UI: Manage > Configuration > Development > Configuration Synchronization (admin/config/development/configuration)

The Configuration Management Initiative (CMI)

Basic principles:

Active configuration.
Exported configuration:
- In YAML files
- Exported via the UI: Manage > Configuration > Development > Configuration Synchronization (admin/config/development/configuration)
- or via drush: drush cex & drush cim

drush config-export drush config-import

The Configuration Management Initiative (CMI)

Interactions between both set of configuration:

The Configuration Management Initiative (CMI)

Interactions between both set of configuration:

Export:
- deletes all the configuration files
- then re-creates all the files

Very simple and straightforward logic

The Configuration Management Initiative (CMI)

Interactions between both set of configuration:

Export:
- deletes all the configuration files
- then re-creates all the files
Import:
- makes the difference between imported and active configuration
- modifies only what is relevant

Very simple and straightforward logic

Needed for conf delete.

TODO :CHECK drush cim --partial !!!!!!!!!!!!!!!!!

Deployment between environments17

Both are copies of each other, like a production and a devlopment environment.

Deployment between environmentsExport the active configuration to the YAML files
17

Both are copies of each other, like a production and a devlopment environment.

Deployment between environments

Export the active configuration to the YAML files
Add the files to Git and push to code repository.

Both are copies of each other, like a production and a devlopment environment.

Deployment between environments

Export the active configuration to the YAML files
Add the files to Git and push to code repository.
Pull the files into destination environment

Both are copies of each other, like a production and a devlopment environment.

Deployment between environments

Export the active configuration to the YAML files
Add the files to Git and push to code repository.
Pull the files into destination environment
Import the YAML files as the active configuration.

Both are copies of each other, like a production and a devlopment environment.

Warning: database has to be the same

Team work, and how not to fail at it

Workflow to follow for a safe process:

Team work, and how not to fail at it

Workflow to follow for a safe process:

Do your site building

Or you would have nothing to export...

Team work, and how not to fail at it

Workflow to follow for a safe process:

Do your site building
Export config: drush cex

Or you would have nothing to export...

Export your configuration into files so that they can be merged with your colleagues' work.

Team work, and how not to fail at it

Workflow to follow for a safe process:

Do your site building
Export config: drush cex
Commit: git add & git commit

Or you would have nothing to export...

Export your configuration into files so that they can be merged with your colleagues' work.

to have a safe haven to go back to. So these first three steps are to prepare you for the actual merging of your work with your teammates'

Team work, and how not to fail at it

Workflow to follow for a safe process:

Do your site building
Export config: drush cex
Commit: git add & git commit
Merge: git pull

Or you would have nothing to export...

Export your configuration into files so that they can be merged with your colleagues' work.

to have a safe haven to go back to. So these first three steps are to prepare you for the actual merging of your work with your teammates'

Merge and solve eventual merge conflicts. Be careful of git here, as if your colleague deleted a config file and you needed it for something new you added, you might not notice. Example: your colleagues deletes a field. it deletes the basic conf of the field and the field instance, since only 1 content type was using it. At the same time, you added that field to another content type. So the associated conf is just a new file for the field's instance. So you merge, it deletes the field default config but keep your field instance file. -> the conf is broken. Best if you see it now, but other wise detected by one of the next steps.

Team work, and how not to fail at it

Workflow to follow for a safe process:

Do your site building
Export config: drush cex
Commit: git add & git commit
Merge: git pull
Update dependencies: composer install

Or you would have nothing to export...

Export your configuration into files so that they can be merged with your colleagues' work.

to have a safe haven to go back to. So these first three steps are to prepare you for the actual merging of your work with your teammates'

Make sure your modules and all dependencies are up to date.

Team work, and how not to fail at it

Workflow to follow for a safe process:

Do your site building
Export config: drush cex
Commit: git add & git commit
Merge: git pull
Update dependencies: composer install
Run updates: drush updb

Or you would have nothing to export...

Export your configuration into files so that they can be merged with your colleagues' work.

to have a safe haven to go back to. So these first three steps are to prepare you for the actual merging of your work with your teammates'

Make sure your modules and all dependencies are up to date.

Update the db to take your teammate's hook update and module updates into account

Team work, and how not to fail at it

Workflow to follow for a safe process:

Do your site building
Export config: drush cex
Commit: git add & git commit
Merge: git pull
Update dependencies: composer install
Run updates: drush updb
Import config: drush cim

Or you would have nothing to export...

Export your configuration into files so that they can be merged with your colleagues' work.

to have a safe haven to go back to. So these first three steps are to prepare you for the actual merging of your work with your teammates'

Make sure your modules and all dependencies are up to date.

Update the db to take your teammate's hook update and module updates into account

Reimport the config, to make sure it is not broken. D8 will tell you if the config is broken. Will alos tell you if a file is missing as per my example.

Team work, and how not to fail at it

Workflow to follow for a safe process:

Do your site building
Export config: drush cex
Commit: git add & git commit
Merge: git pull
Update dependencies: composer install
Run updates: drush updb
Import config: drush cim
Check all works fine

Or you would have nothing to export...

Export your configuration into files so that they can be merged with your colleagues' work.

to have a safe haven to go back to. So these first three steps are to prepare you for the actual merging of your work with your teammates'

Make sure your modules and all dependencies are up to date.

Update the db to take your teammate's hook update and module updates into account

Reimport the config, to make sure it is not broken. D8 will tell you if the config is broken. Will alos tell you if a file is missing as per my example.

Test it all, see if all works as expected (automated tests would be a great help here....)

Team work, and how not to fail at it

Workflow to follow for a safe process:

Do your site building
Export config: drush cex
Commit: git add & git commit
Merge: git pull
Update dependencies: composer install
Run updates: drush updb
Import config: drush cim
Check all works fine
Push: git push

Or you would have nothing to export...

Export your configuration into files so that they can be merged with your colleagues' work.

to have a safe haven to go back to. So these first three steps are to prepare you for the actual merging of your work with your teammates'

Make sure your modules and all dependencies are up to date.

Update the db to take your teammate's hook update and module updates into account

Reimport the config, to make sure it is not broken. D8 will tell you if the config is broken. Will alos tell you if a file is missing as per my example.

Test it all, see if all works as expected (automated tests would be a great help here....)

Now you are finally good to push your code to the repo.

Mistakes and consequences

If you fail to follow the previous steps and:

Mistakes and consequences

If you fail to follow the previous steps and:

Import before export then all your work is lost, no backup

you just overrode your work with your colleagues' after all

Mistakes and consequences

If you fail to follow the previous steps and:

Import before export then all your work is lost, no backup
Merge before export then Export deletes your colleagues work, can be solved with git

you just overrode your work with your colleagues' after all

Mistakes and consequences

If you fail to follow the previous steps and:

Import before export then all your work is lost, no backup
Merge before export then Export deletes your colleagues work, can be solved with git
No updb or updb after config import then db might be broken

you just overrode your work with your colleagues' after all

Mistakes and consequences

If you fail to follow the previous steps and:

Import before export then all your work is lost, no backup
Merge before export then Export deletes your colleagues work, can be solved with git
No updb or updb after config import then db might be broken
No composer install then code may be outdated

you just overrode your work with your colleagues' after all

Mistakes and consequences

If you fail to follow the previous steps and:

Import before export then all your work is lost, no backup
Merge before export then Export deletes your colleagues work, can be solved with git
No updb or updb after config import then db might be broken
No composer install then code may be outdated
Merge before commit then conflicts need to be solved by hand, no safe state to return to

you just overrode your work with your colleagues' after all

Mistakes and consequences

If you fail to follow the previous steps and:

Import before export then all your work is lost, no backup
Merge before export then Export deletes your colleagues work, can be solved with git
No updb or updb after config import then db might be broken
No composer install then code may be outdated
Merge before commit then conflicts need to be solved by hand, no safe state to return to
Forgotten import then next export will be missing merged config. Hard to solve in git

you just overrode your work with your colleagues' after all

Bonus workflow: module updates20

Bonus workflow: module updatesUpdate code: composer update
20

Bonus workflow: module updatesUpdate code: composer update
Update db: drush updb
20

Bonus workflow: module updatesUpdate code: composer update
Update db: drush updb
Export updated config: drush cex
20

Bonus workflow: module updatesUpdate code: composer update
Update db: drush updb
Export updated config: drush cex
Commit: git add & git commit
20

Bonus workflow: module updatesUpdate code: composer update
Update db: drush updb
Export updated config: drush cex
Commit: git add & git commit
Then go through the safe team workflow from 2 slides back.
20

Overriding configuration

Use case: change existing configuration values in different environments $conf in Drupal 7 becomes $config in Drupal 8.

Overriding configuration

Use case: change existing configuration values in different environments $conf in Drupal 7 becomes $config in Drupal 8.

// Get system site maintenance message text. This value may be overriden by
// default from global $config (as well as translations, see below).
$message = \Drupal::config('system.maintenance')->get('message');

Overriding configuration

Use case: change existing configuration values in different environments $conf in Drupal 7 becomes $config in Drupal 8.

// Get system site maintenance message text. This value may be overriden by
// default from global $config (as well as translations, see below).
$message = \Drupal::config('system.maintenance')->get('message');

To override the value in settings.php, use:

$config['system.maintenance']['message'] = 'Sorry, our site is down now.';

When using $config override outside of settings.php, use a preceding global $config; Note that values overridden via $config within settings.php will not be viewable from the Drupal administration interface

But can only alter existing conf
Can’t add new conf
Cannot unset conf
Cannot manage existing modules (installed or enabled)
Cannot override some conf such as Bartik color and such

Problems not solved by CMI22

Problems not solved by CMISharing configuration with another site (instead of another environment)
22

CMI has a built in check that makes sure the uuid of the site is the same, if it is not, won't let you

Problems not solved by CMI

Sharing configuration with another site (instead of another environment)
Having different configuration per environment: devel ?

CMI has a built in check that makes sure the uuid of the site is the same, if it is not, won't let you

What if I want to have devel only on my dev env? What if I need some dev environment permissions or views? Cannot solve this with what I just showed you.

Contrib modules for configuration management

Features, config split & more...

Config read only

config_readonly

This module is to prevent users from altering the configuration at all. No edit permitted.

Config read only

config_readonly

This module is to prevent users from altering the configuration at all. No edit permitted.

To enable readonly mode, enable this module and add this to your settings.php:

$settings['config_readonly'] = TRUE;

if (isset($_ENV['AH_SITE_ENVIRONMENT']) && $_ENV['AH_SITE_ENVIRONMENT'] === 'prod') {
    $settings['config_readonly'] = TRUE;
}

To allow all changes via the command line and enable readonly mode for the UI only:

if (PHP_SAPI !== 'cli') { $settings['config_readonly'] = TRUE; } You could similarly toggle read-only mode based on the presence or absence of a file on the webserver (e.g. in a location outside the docroot).

if (!file_exists('/home/myuser/disable-readonly.txt')) { $settings['config_readonly'] = TRUE; }

Configuration split

config_split

Define sets of configuration that will get exported to separate directories when exporting, and get merged together when importing. It is possible to define in settings.php which of these sets should be active and considered for the export and import.

Configuration split

config_split

Define sets of configuration that will get exported to separate directories when exporting, and get merged together when importing. It is possible to define in settings.php which of these sets should be active and considered for the export and import.
Works based on config_filter

So, config filter adds a filter in CMI just before the files are written.

Then can put plugins there. Basically hooks that are run on import/export Such as config split...

Configuration split

config_split

Defines black list:

Black listed configuration are banned from the default config export folder
Config redirected to the split

and grey list:

Grey list configuration is not banned from default config
Any changes from the default config already exported is redirected to the split
On import the changes from the split override the default exported config

Blacklist:

Configuration items that are explicitly managed by the split in question. These configuration items will be removed from the default directory. You can now store that configuration item in one or more configuration split, but not in the default directory.

Greylist:

Configuration items on the grey list are not automatically removed from the default configuration directory. If a configuration item exists in the currently active split, it takes precedence over the copy stored in the default directory. If a copy of the greylisted configuration item does not exist in the currently active split, the configuration item from the default directory is used.

For example, if you want to override site performance settings in your local split in order to disable css / js aggregation, the greylist would be a good option. This would allow you to store your performance settings for dev / test / prod in the default directory, while also storing an altered version for local development in your local split. You can also achieve this using the blacklist, but you would need to store a copy of the performance settings in each split. If performance settings are the same for dev / test / prod and only vary for the local split, this is duplicative

Configuration split

config_split

When to use it?

-> When you need to define configuration for different environments

Configuration split

config_split

When to use it?

-> When you need to define configuration for different environments

How?

-> Define configuration splits per environment, use environment variable to choose which split to enable on each environment.

Configuration installer

config_installer

Installation profile taking over the the Drupal installer in order to set up the website based on exported configuration.

Configuration installer

config_installer

Installation profile taking over the the Drupal installer in order to set up the website based on exported configuration.

-> Let's you create new environments without relying on db export.

Configuration installer

config_installer

Example workflow:

First developer:
- Initialise git repo
- Install site locally
- Exports config to sync
- Commit and push to shared git repo
Other dev (or envs):
- Clone code
- Have config_installer profile available
- Install site starting from exported config

Features

features

Allow you to add a set of configuration to a website. But not used as a replacement of CMI.

However, it is difficult not to have it overlap with CMI. Does any of you use feature with Drupal 8?

As the module creator said, Features can now focus on doing what it was meant for. Sharing a common set of configuration to a new website so as to skip some recurring development. Photo gallery example.

Enable the feature on the new website
feature uninstall removes the config

Questions?

How to reach me

Thibaud@pixelonion.com

Made with remark.js

↑, ←, Pg Up, k	Go to previous slide
↓, →, Pg Dn, Space, j	Go to next slide
Home	Go to first slide
End	Go to last slide
b / m / f	Toggle blackout / mirrored / fullscreen mode
c	Clone slideshow
p	Toggle presenter mode
t	Restart the presentation timer
?, h	Toggle this help