Announcing Features for Drupal 8

Features_D8

The first Alpha version of the Features module for Drupal 8 is now available! “But wait!” you say. “I thought we didn’t need Features anymore now that we have the Configuration Management Initiative (CMI) in Drupal 8?”

This article will explain why some sites will still need Features, how Features works with CMI, and how Drupal 8 Features differs from the Drupal 7 version.

Features and CMI

Built in Drupal 8 core, CMI cleanly and consistently separates “content” (articles, blogs, etc) from “configuration” (views, content types, fields, etc). Rather than forcing each module to create its own import and export format, CMI uses YAML (yml) files to store configuration data. CMI also provides mechanisms for deploying configuration between sites, including development, test, and production environments.

Drupal 7 didn’t have CMI, and because the Features module could export and import configuration data as code modules, Features was often used for configuration management and deployment. But Features wasn’t designed to be a full configuration management system. Features was initially written to “bundle reusable functionality”.  The classic example was creating a Photo Gallery which was a bundle of a content type, some fields, a view, and an image style.

Features_D8_Edit

To create a Photo Gallery module in Drupal 8, you need to use the “single export” option in CMI to export your content type, then export your fields (and field storage), then export your view, then export your image style, and then hope that you haven’t missed any other important settings. This very manual process is subject to errors. Luckily, the Features module in Drupal 8 allows you to pick and choose what configuration data you want to add to your custom module, simplifying the process.

Features in Drupal 8 uses CMI as a consistent storage mechanism, in addition to importing and exporting configuration. Both the user interface and drush command line support are similar to the Drupal 7 version many developers are accustomed to.

New Features in Features

Rather that rewriting Features from scratch for D8, we started with the excellent “config_packager” module from nedjo. His module was created to export “packages” of configuration in D8, which is really the intended use-case of Features. During the past few months I have worked closely with nedjo to merge his config_packager concepts into the new Features module, which gives users some very cool new functionality. It’s been a great example of how open source collaboration often produces better results than either individual effort.

Features_D8_PluginsAssignment Plugins

One awesome idea from config_packager was plugins that can automatically assign existing site configuration into packages. When you first install Features and go to the listing page within the Configuration Management section of your site, you will see that it auto-detects your content types (article, page) and creates Features for exporting these content types, automatically including any fields, views, or other configuration assigned to that content type.

You can control which assignment plugins are enabled, and what order they run. Each plugin can have it’s own configuration. For example, the “Base” assignment plugin lets you choose which component should be the “base” for organizing configuration. By default, the base is “content type.” If you change the base to “views” then your site configuration will be organized into Features based upon each view on your site.

These assignment plugins solve the problem of organizing and modularizing your configuration. They also better modularize functionality within Features, such as auto-detecting dependencies, adding profile configuration, or excluding configuration that has already been exported to a module.

Bundles

Assignment plugins are a great concept, but we wanted to support multiple plugin settings to enable easy switching between “show config organized by content type” and “show config organized by views.” In addition, we wanted to add “namespaces” to Features to better isolate configuration exported to different profiles. For example, “namespaces” make it easier to support sites running multiple distributions, such as Open Atrium running on Panopoly, where some Features are prefixed by oa_* and other Features are prefixed by panopoly_*.

In D8 Features we implemented “bundles” to specify the namespace (prefix) of Features modules, similar to the “package” tabs along the left side of the Features listing in D7. These bundles can be assigned to a specific profile, such as Panopoly or Open Atrium and will filter the Features listing to only show modules within the bundle’s namespace. Assignment plugin settings are stored within the bundle, allowing different bundles to organize configuration in different ways. You can also easily export configuration to a different bundle, making it finally easy to copy your Features from one namespace to another.

Features is for Developers

Drupal 7 didn’t have CMI — core support for configuration management was added via Features. If a custom module contained configuration exported by Features, you needed to have the Features module enabled (in most cases), even on Production, to use that module. In Drupal 8 we wanted to remove the dependency on the Features module. When you create a module using Features D8, the module will work on any other D8 site even without Features being installed!

In Drupal 8, configuration is owned by the site and not by modules. This is a very important design decision behind CMI and how your site configuration is managed and deployed. Configuration provided by a module is imported into the site only when the module is enabled. Once the configuration is imported, any changes made to the config files in the module are ignored by the site. The only way to re-import the configuration is to uninstall the module and then reinstall it.

Drupal 8 also does not allow you to enable a module that contains configuration that already exists on the site. This can cause problems with updating configuration provided by a module. For example, if the Photo Gallery view within our Gallery module needs to be changed, you normally need to uninstall the module and then reinstall it. However, some configuration is not removed when you uninstall a module. Thus, you can have a situation where you uninstall the module and then cannot reinstall it because the configuration already exists. You must manually delete the view provided by the module before you can re-enable the module.

These hurdles and restrictions can make developing modules that contain configuration difficult. As a developer, Features provides functionality to help with these issues. When the Features module is active, it allows you to enable a module that contains configuration already on your site. Once a module is enabled, Features allows you to Import changes from that module into your site without needing to first uninstall the module. This prevents the situation of having a module that cannot be re-enabled, or a module that cannot be uninstalled because of dependencies and makes it much easier to update the configuration stored in the module during development.

You only need to install the Features module in your development environment. Once you have updated your feature module and imported the configuration changes into your dev environment, you then use normal Drupal 8 CMI to deploy configuration to staging and production. If you still want to keep the Features module enabled on staging or production for doing config imports, you can disable the Features UI submodule and only use the drush interface.

Overriding Configuration

If you change configuration in Drupal 8 that was imported from a module, Features will detect and display this change. If you import the new module configuration, it will overwrite the active site configuration. If you export the changes you will update the config data stored in your module. CMI controls the granularity of this configuration. For example, an entire View is a single configuration entity. Thus, there is no way to only import a simple change, such as the title of the view. Importing the view will overwrite the entire view, replacing any other changes that might have been made.

So far there isn’t any way to manage partial configuration changes in Drupal 8. CMI was specifically designed not to handle partial configuration. Currently a module that needs to update partial configuration needs to implement an update hook. However, the code to update a partial configuration object is very specific to the configuration being changed. There is no overall consistent way to “merge” configuration between the module and active site. The config_synchronizer module is the beginning of some good ideas on monitoring site config changes to determine if it is safe to import new config from a module and might be the start of a better Features Override module in the future.

drupal8My Thoughts on Drupal 8

Rewriting Features for D8 was a very interesting project for me. I started with little knowledge of the guts of Drupal 8. I’m actually amazed by the amount of work that has gone into Drupal 8 to make it look and feel like Drupal (Views, Content Types, etc) and yet have a completely different architecture based on Symfony.

Working on Features for Drupal 8 was actually a joy.

Once I got the hang of some of the new concepts such as services and routing and plugins, it was actually fun to create Features. Sure, there are still some rough edges and some core config formats changed even during the Drupal Beta that caused some things in Features to break.

Ultimately I challenge developers to give Drupal 8 a chance and start “poking the tires” to give it a spin. In only four weeks I was able to learn D8 and produce a useable module for something as complex as Features (while also learning config_packager and doing a lot of refactoring). It’s not nearly as onerous to develop in D8 as I had been led to believe by some bloggers. I’m very excited for this new chapter of the Drupal story and also happy that Features will be playing a much smaller part in that story than ever before.

Conclusion

If you want to learn more about Features and see a demo, come to my session at DrupalCon LA next month: Features for Drupal 8. If you want to help, go to the Features project page on drupal.org and download the latest 3.x version and start playing with it.  Post your bugs and suggestions to the issue queue. We could really use some help writing automated tests in order to release a Beta version. I am looking forward to our bright future in Drupal 8 where we no longer need to curse about Features and can focus back on building reusable functionality.

Announcing Phase2’s 2015 DrupalCon Lineup

It’s about that time of year again. DrupalCon 2015 is fast approaching, and all of us at Phase2 are busily planning events, sessions, and of course travel to Los Angeles! We are excited to announce that the Phase2 team will be represented on several session tracks this year. Here’s a rundown of what we’ll be presenting in LA.

DconLA lgoo

Business & Strategy

As CEO Jeff Walpole discussed in his blog post earlier this week, we’re really excited about the power and potential of D8 for enterprise organizations. At DrupalCon, CTO Frank Febbraro and Head of Strategic Accounts Michael Le Du will be joined by our client Evan Liebman of Memorial Sloan Kettering to discuss why the cancer center decided to adopt D8 while it was still in its beta state. They’ll examine MSK’s experience implementing one of the first major D8 platforms in the U.S. and explain why all enterprises should seriously be considering a move to D8. Don’t miss: Drupal 8, Don’t Be Late (Enterprise Orgs, We’re Looking at You).

Site Building

Software Architect Mike Potter will demo the latest version of the Features module in his session Features for Drupal 8. He’ll delve into the architecture of Features in D8, how Features integrates with D8 CMI and other modules, and when and how to override Features. Learn how, in D8, Features returns to its original purpose of bundling functionality rather than just managing configuration.

14213355409_f3515a9367_o

The popular Drupal is Not Your Website session returns with Software Architect Tobby Hagler, who explores how to build a Drupal website that will interoperate with other web components, live behind CDNs, and make heavy use of caching layers, yet still maintain a positive custom user experience. In essence, the Drupal CMS you are building is not actually the “thing” that Web users are directly looking at; Drupal is not your website. Check out this primer for a sneak peak.

DevOps

Want to use Drupal Grunt Tasks on your next project? Stop by Director of Engineering Joe Turgeon’s session, Using Grunt to Manage Drupal Build and Testing Tools. Grunt is a well-supported and flexible JavaScript-based task runner. Building on his 2014 BADCamp talk, Joe will demonstrate the free and open-source Grunt Drupal Tasks project, which provides a set of common tasks related to building and testing Drupal sites.

User Experience Design

At Phase2, we do things a little differently when it comes to design. Join Senior Designer Joey Groh, Developer Evan Lovely, and Developer Micah Godbolt of the Sass Bites podcast to discover The New Design Workflow. The panel, representing some of Phase2’s most experienced designers and front-end devs, will provide an inside look at our best practices, tips and tricks, as well as an overview of some of our most successful co-design projects. Plus, hear us weigh in how Drupal 8 will interface with your favorite front-end tools like PatternLab.

OLYMPUS DIGITAL CAMERA

Coding & Development

As part of the Coding & Development track, Senior Developer Joshua Turton will present How, When & Why to Patch a Module. Ever find a module that does 98% of what you need it to do, and there’s no way to make it do that last 2%? Ever need to fix a bug in a module you’ve run into, but you’re the only one who’s ever had this problem? You need a patch – and to attend this session.

Plus, another must-see session: Jose Eduardo García Torres, CTO at Anexus IT (a Phase2 partner), will give a talk called Speeding Up Drupal 8 Development Using Drupal Console.

Can’t wait to see you all in Los Angeles!

State of the Public Platform: Our Insights on Digital Government

The past year has seen some unprecedented and exciting advancements for Phase2’s work in the digital government space. Not only did we embark on several major projects with local, state, and federal partners, we partnered with IBM to complete work on the first federal agency-wide, cloud-based platform built on OpenPublic, our content management system tailored for government.

smc-logo  uspto logo  DOI logo

It’s been a busy twelve months, and we have certainly learned a lot – some of which we’ve shared in our digital strategy white paper series. As our Director of Government Practice, I’d like to share some of the lessons these projects have taught us – and what you can expect in 2015 and beyond for digital government.

The Value of Consolidation & Collaboration

Working in partnership with Phase2 and IBM, the Department of Interior became the first federal agency to implement the OpenPublic platform hosted on IBM’s SoftLayer cloud.This department-wide, enterprise-scale platform is a major milestone not just for Interior, but all federal agencies. I fully anticipate that the coming year will see many other agencies follow in the DOI’s footsteps. Why? It’s all about the value of consolidation and collaboration.

Before implementing the OpenPublic platform on SoftLayer cloud, the DOI had to contend with the cost and complexity of hundreds of individual sites. Managed separately, these sites induced high fixed operational costs, including infrastructure costs (servers, storage, and load balances), as well as services costs (disaster recovery, patch management, performance tuning, etc.). Not only did the fixed price structure limit the department’s flexibility in responding to fluctuations in end-user traffic, it made third party integrations and mobile applications expensive and difficult to support. This is an issue most federal agencies grapple with.

3DOI

Our solution was to establish an agency-wide Drupal platform to migrate and consolidate all of Interior’s hundreds of sites. The platform is hosted in the IBM SoftLayer cloud and built using OpenPublic, our Drupal distribution pre-configured to meet government web requirements. The collaboration between Phase2 and IBM was a crucial element of the platform’s success. OpenPublic provided key functionality, while the SoftLayer cloud provided an enterprise-scalable PaaS infrastructure. Each site migrated to the platform retains its domain, separate database table structure, and site design, and each can be administered and updated separately by the agency team currently managing it. As a result, site administrators kept control of each site, but they are now hosted in a reasonably priced, elastic, enterprise-level cloud environment.

The Value of Apps

Within our Drupal distribution for government, OpenPublic, we simplified content management in the enterprise public sector space by “appifying” all the distribution’s functionality and content types. We encapsulated what was once a wilderness of modules and configuration in a clean collection of Apps, making all OpenPublic’s out-of-the-box functionality simple to configure for non-technical site administrators. This new App strategy made it easier and cheaper for governments to implement an OpenPublic CMS solution.

SM2

The introduction of Apps into OpenPublic was the culmination of years of developing government systems, most recently San Mateo County’s multi-site platform. This project really drove home these organizations’ need to turn features on and off without affecting other parts of the platform. As Bethany Thames of San Mateo County elaborated, “Each department’s identity and requirements are tied to their lines of business and the community they serve. The County departments wanted to maintain their unique identities within the overall County brand. OpenPublic allowed the County to maintain a strong central brand while meeting user demand for autonomy and flexibility.”

Apps like the Services App, Media Room App, Security App, and Workflow App allow those departments to pick and choose which distinct segments of functionality make the most sense for them. The “appification” of all functionality in this manner is truly knocking down traditional barriers to Drupal site maintenance and scalability.

The Value of the Static CMS

The single biggest point of failure of a Drupal site is Drupal. If any part of Drupal goes down, the site goes down. Wouldn’t it be great to build a site with Drupal but have a static public-facing site, instead of a constantly generating dynamic site? We took this approach with the U.S. Patent & Trademark Office, implementing what we referred to as the “Sleep at Night CMS.”

Static websites are ideal for all government agencies: no performance or security issues, no worries about redundant failover, no getting woken up in the middle of the night because something has gone wrong. Instead, by implementing Drupal’s static module, we allowed USPTO to take advantage of Drupal’s powerful content management capabilities while delivering the site to the public as static HTML. In effect, this approach mitigates the majority of performance, security, and redundant failover risks. When changes and additions are made within Drupal, any affected generated pages are updated along with it. Drupal is never exposed to end users, eliminating reliability issues. If Drupal has a problem, it just stops updating the generated site until it is fixed.

USPTO1

How did we do it? Get the details from Senior Developer Randall Knutson.

The Value of Open

Phase2 uses open technology in everything we do, but this choice is particularly salient when it comes to government work. Government agencies strive to reduce unnecessary costs for their taxpayers, and avoid the recurring licensing fees of proprietary software is a major benefit to open source solutions. Bypassing proprietary vendor lock-ins allows government to leverage the sustainable innovation of an open community working collaboratively to create, improve, and extend functionality, in addition to utilizing the community’s best practices for development. And because open technology is in the public domain, any agency can download, test drive, and learn about potential content management systems before choosing a provider.

San Mateo County recognized the value of openness in government technology and opened their code to the GitHub community. We were ecstatic that one of our clients embraced the open practices which are not only inherent in our work but laid the foundation for the development of OpenPublic. By making the innovative technology that went into building San Mateo County’s platform available for wider use, San Mateo County contributed to the government’s objective to foster openness, lower costs, and enhance service delivery. The “Open San Mateo” project demonstrates the power of open source to improve not just one government agency, but hundreds simultaneously by making the code available to other governments.

sm5

Similarly, web CMS and open data have become increasingly integrated over the past year. Government open data portals, like this example in New York City, build on open source tools to integrate external data into traditional website pages – in the best cases, using proven content strategy and UX design techniques to connect data sets and CMS content. This allows visitors on government websites to access raw data sets and interactive visualizations in a context sensitive way.

As an example, we are partnered with Socrata and have built a Drupal module that provides an integration point for Socrata within Drupal via Socrata’s SODA2 interface. This is significant because it will allow ordinary people, not just web developers, to use open data to solve everyday problems. For more, here are my thoughts on a less bifurcated user experience.

What’s Next…

Over the coming months, we expect to see a continuation of the trends mentioned above. In particular, keep an eye out for:

  • More consolidated CMS platforms for large federal agencies

  • Open source continuing to disrupt the SharePoint behemoth for internal collaboration and communication

  • Further integration of web CMS and open data

  • Increased code sharing across agencies

As for Phase2, we are hard at work on several government projects, including an overhaul of North Carolina’s web platform. You guessed it – they are moving to a Drupal multi-site platform to unify all state agencies and provide efficiency across all state departments. Also in the works is the migration of DOI.gov website onto the agency’s new OpenPublic platform.

What trends have you seen in the digital government space recently? Do you agree that many agencies are moving towards larger, more consolidated platforms built on open technology? Let us know in the comments below! For more updates on Phase2’s work in government, join our email mailing list! Finally, if you are interested in how to create an informed digital strategy for government, be sure to check out our white paper series for information on technology selection, platform models, and how to leverage a Drupal CMS to achieve your mission.

Accelerating with Drupal 8

Today is an exciting day for the Drupal community! Collectively, we’re all moving a few steps closer to a full release of Drupal 8 with the help of a program called Drupal 8 Accelerate. This is a pilot program from the Drupal Association designed to put $250,000 of community funds towards eliminating the last 50 critical issues between us and release.

The Drupal Association has been an incredible leader in the effort to release Drupal 8, pledging to set aside $62,500 to match every dollar donated to the provide Drupal 8 Acceleration Grants.

What’s the latest with Drupal 8 Accelerate?

But we knew we could do even more to turbocharge this project. Today we are announcing that D8 Accelerate is now getting a huge boost from seven anchor sponsors, who have pledged to “match the match,” amplifying every donation made and accelerating the community’s investment in Drupal 8.

Phase2, Acquia, Appnovation, Lullabot, Palantir, PreviousNext, and Wunderkraut have collectively pledged another $62,500 to match the Drupal Association’s matches of community donations. This is an all-out, everyone-in community effort to move D8 from beta to release. Our goal is to bring the total to $250,000 available for grants by September. We are now more than half way there.

drupal-relay_1

Why should we all want Drupal 8 to succeed?

The answer is simple: D8 will empower us to use Drupal the way many of us have wanted to for a long time. D8 improves the API layer, multi-lingual capabilities, theming and the editor experience. It also makes is much more powerful for developers (which matters a lot to us at Phase2).

Historically, it has been a challenge to integrate new libraries or different front-end elements without a lot of leg work. Imagine, for example, how the availability of Twig theming will enhance your projects. Or how flexible implementations can be with dependencies on meaningful external software integrated through Symfony routing. We will even be able to more seamlessly incorporate mobile apps into the digital strategies we develop, correcting one of the main weak points of previous Drupal releases.

Put simply, Drupal 8 is a win for our collective clients, and therefore it is a win for all of us.

Phase2 & Drupal 8

At Phase2, we want Drupal 8 to succeed because our clients have increasingly big needs and major challenges, and we believe that Drupal 8 is moving in the direction to address those. For that reason, we’ve made investing in Drupal 8 a priority, not only by way of the Drupal 8 Accelerate program, but also in the form of contributed code and shared knowledge gleaned from major enterprise Drupal 8 implementations.

Taking on early Drupal 8 implementations enables us to commit our people to the D8 cause, while directly supporting our client’s mission. It also provides us with a group of advanced scouts to report back from the front lines and develop training for the rest of our team.

Principle among these scouts was Software Architect Jonathan Hedstrom, whose contributions to D8 include Drush support, core patch reviewing, testing and re-rolling, writing tests, modules upgrades (Redis), and more. In addition to Jonathan, Senior Developer Brad Wade made important front-end contributions, while Software Architect Mike Potter has been a significant part of Features development.

We’ll be sharing a lot of what we learned from our D8 work so far at DrupalCon Los Angeles, so stay tuned for our session announcements next!

 An all-out, everyone-in effort

It took the whole Drupal community – including individuals, companies, the Drupal Association – to get D8 to the place it is now. We are honored to have contributed alongside everyone involved. It has certainly been a heavy lift for many community members, so to each of these people and organizations, we say thank you. The success of Drupal 8 is the most important priority of our community.

Screen Shot 2014-09-04 at 11.14.45 AM

However, Drupal 8 still needs a strong push to get over the finish line. So we must ask one more time for the support of our fellow Drupalers. We all have a major stake in the success of the project, and everyone can play an instrumental role getting it out the door. Even the smallest donation makes a difference when every dollar you donate is now matched, compounding your impact. You can read more about how the funds actually support the grant program to achieve the work on the Drupal Association D8 Accelerate page.

If you would like to donate, please visit the D8 Accelerate Fundraising site and please consider using my profile as a way to easily make your contribution so we can start enjoying those launch parties!

Announcing New Release and Version Scheme For Open Atrium

For the last several months I’ve had the privilege of leading the new dedicated products team here at Phase2. Having a team solely focused on products does not mean we are shifting away from Open Source, but it does mean we are going to be changing our practices a bit to better support the community and our clients. We now have a more predictable development schedule for our products like Open Atrium, and we want to pass the benefits of that to the community and our clients. To that end, we will be working on regular releases with a more consistent version scheme.

Releases will have the following types:

  • Maintenance Releases –  these will happen regularly (approximately once a month) and will include bug fixes, security patches and minor feature improvements/tweaks. We’ll signal these with version numbers that end in 1-9 (e.g. the “1” in Open Atrium 2.31).

  • Feature Releases – these will happen once a quarter and will add major new functions. They will generally require a little more care in upgrade because they may include big updates. These releases will end in zero, like our recent Open Atrium 2.30 release.

  • Major Releases – we have big ideas and plans, and some of them will require that we break compatibility and/or force a migration. We will be working on these big ideas, and we’re aiming to have a new major release each year. Our next will be Open Atrium 3.0 in early 2016.

As we build our solutions, we want to be able to move fast and make lots of improvements, but we need to balance that with a strong testing/review cycle. Rather than keep our activities behind the curtain, we will still be working in public git repositories so that anyone can see where we are going. And, for folks that do not want to follow the day-to-day of development but want to be more involved, we’ll be following a release candidate strategy. Before new feature releases, as soon as we feel we’re feature complete, we’ll publish a release candidate (e.g. 2.30-rc1). We’ll then put the RC through its paces and if things go well and we get no reports of issues from the community, that will become the final release.

So, future releases will have versions that look like this:

Diagram of Open Atrium release cycle

By both developing in the open and putting our releases out for review before they are final, we hope to strike the right balance between being visible and collaborative in the community, and offering our clients access to well reviewed and tested releases. As we work to bring more solutions to market, rest assured that we’ll keep “open” as a strong part of our DNA.

Personally, I’m really excited to be working with a great group of folks, both on my team and in the community. Lets push it, and do something great together! If you would like to stay informed about Open Atrium developments, be sure to sign up for the Phase2 newsletter!

How to Override Features

The Features module helps address the shortcoming in Drupal 7 of how to manage and deploy site configuration data.  There are times when you need to change a Feature.  The Features Override module can help with this, but sometimes doesn’t solve the problem completely.  In this article I will discuss all the different ways to override your Features and the common problems associated with that.  While this has been an issue for years, it’s become more common recently with the advent of Distributions such as Open Atrium and Panopoly which heavily use Features and which expect the users to upgrade to new versions.  Maintaining your site customizations when upgrading a distribution using Features Override has become a more common need.

The Problem

How do I override configuration from an existing Feature  within my site-specific project?

Solutions

Cloning the base module (not recommended)

If you need to make extensive changes you might need to clone the base module and then customize it.  Disable the original module, uninstall it, then enable your site-specific module.

Pros Cons
  1. You have full control over all aspects of the original module.
  2. Keeps the overrides nicely separate in their own modules that can be enabled/disabled/deployed.
  3. The original Features will never be shown “overridden” (because the original is uninstalled!)
  1. Any future bug fixes, improvements, updates to the original module will not be available.
  2. Since future security patches to the original module won’t be available, this might add a security risk to the project or force maintainers to monitor the original module for updates and apply them to the cloned version.

Using settings.php

If you just need to change a variable (from variable_get) you can do this via the $conf[] array in your site settings.php file.  This method is typically used for environment-specific settings, such as databases, search servers, caching settings, etc.  You can also set the $conf[] array within a custom module if you need to deploy the change across all your environment.  Avoid using variable_set() to change the variable since that will update the database directly as mentioned below.

Pros Cons
  1. The original feature containing the variable (via Strongarm) won’t be marked as “overridden”
  1. Only works for Variables (Strongarm Features)
  2. settings.php not typically part of the code repo, so need to ensure it is version controlled some other way.
  3. Sometimes developers forget to check settings.php for variable overrides, making ongoing support tricky.
  4. Best for Environment variables rather than for any generic Drupal variable.

Update the Database directly

You can create an update hook, or other Drupal hook (hook_enable, hook_install) and update the database directly via SQL or other API functions.  For variables you can use variable_set() to save the new value to the database.  This is only recommended as the last resort for configuration not supported by Features Override.

If you must update the database directly, be sure to only do it once and not in a hook such as hook_init that runs with each page load.  Updating the database on each page load can kill the performance of your site.  With variable_set() you can do this by first calling variable_get() and only saving the value if it needs to be changed.

Pros Cons
  1. Sometimes it’s the only way.
  2. Can keep the overrides separate in their own modules
  3. Only applies specific changes and allows future updates to the base Feature to be used.
  1. Reverting the base feature will change the database back to it’s original value, forcing you to re-run whatever SQL is needed to override it. Can be alleviated by Locking the original base Feature to prevent any reverts.
  2. If the original feature is locked, future bug fixes or improvements might not be available.
  3. The original feature will always be marked as “overridden”.
  4. Some configuration is not stored in the DB, such as Views (or other ctools-based components like Panels) that exist purely in code.

Implement an “alter hook” in code

Most configuration has some sort of “alter” hook that allows it to be modified after it is loaded from the database.  For example, Views calls hook_views_default_views_alter(&$data) to allow you to change any part of a view, whether that view comes from the DB or is in code.  You can create a custom module that implements the desired alter hooks and override the data directly.

Pros Cons
  1. Base Feature will not be marked as “overridden”.
  2. Keeps the overrides nicely separate in their own modules that can be enabled/disabled/deployed.
  3. Only applies specific changes and allows future updates to the base Feature to be used.
  1. Not all configuration has the needed hooks.
  2. Each component has a different hook name and different data structure to modify.

Use Features Overrides module

Similar to the above “Implement alter hook” the Features Override module is designed to create the alter hook implementations for you and export those as new Features modules that you can enable, disable, deploy.

Pros Cons
  1. Base Feature will not be marked as “overridden”.
  2. Keeps the overrides nicely separate in their own modules.
  3. Only applies specific changes and allows future updates to the base Feature to be used.
  4. Features Overrides does the work of figuring out which alter hooks can be used and how to override the data.
  5. Provides a UI for inspecting overrides.
  6. Allows an entire component to be altered (e.g. entire view), or just a single change (e.g. just the title of a view)
  7. Works properly with Features. An Override is just another Feature module on the site.
  1. Not all configuration has the needed hooks.
  2. Can be tricky to “override an override”.

Typical Problems

Feature is stuck at being “overridden”

The most common difficulty is a base Feature marked as “overridden” that does not go away when the feature is reverted.  ”Overridden” simply means that the value of the configuration stored in the DB is different from what is stored in the exported Feature code file.  Features literally re-generates the feature export internally from the DB and compares the code with what is stored on disk.  Doing a “drush fd featurename” simply shows the code diff between the code stored on disk (shown in red) with the code generated from the current database (shown in green).

To determine if a Feature is overridden, Features actually takes the generated code, sanitizes it, sorts it, executes it, and then does an md5 checksum on it.  If the two checksum values are different, the Feature is marked as “overridden”.  However, the “drush fd featurename” command shows the actual code difference regardless of if it’s marked as “overridden”.

This behavior means that “drush fd” might output code differences that actually don’t cause the feature to be overridden.  For example, if you change the order of dependencies in your module.info file and then run “drush fd” on the feature, you’ll see it display those changes, even though a “drush fl” won’t show the feature as overridden.

This makes it difficult sometimes to debug why a feature is overridden, since not all of the output of “drush fd” is relevant.  You need to look for actual value changes, rather than just re-ordering changes.

Adding a new module

Sometimes, just enabling a new contrib module will cause your Features (especially field features) to be “overridden”.  If the new module adds some new settings, formatters, widgets, etc then these settings will be active in the DB but not reflected in the code.  So the Feature will be shown overridden.  Reverting the feature won’t have any affect because there is no way to remove the new settings from the DB without uninstalling the new module.

These kinds of overrides are annoying, but not actually that important.  They don’t prevent the feature from being reverted when a new version of the base module is released.  They are not caused by any site-specific change that would be lost by a revert.  Typically it’s best to just document these cases and leave them as overridden.  Just be careful to not become complacent and make a site customization in the future to the same feature and then ignore that it’s overridden and lose your changes when you revert.  You should periodically do a “drush fd” on overridden features just to be sure what is shown is still just from the new module you installed.

A disabled module

Similar to the above, but the opposite case.  If you have disabled a module that was used when the original Feature was exported, then the code might have settings for that module that are no longer in your database.  Once again you won’t be able to remove this by reverting.  Again, you can just document these cases.  But make sure you really need to have the module disabled.  Having a module enabled that you are not using is not a big performance impact.  The time saved in long-term maintenance of the site it typically more important than worrying about an extra module.

A bad override

If you have used some of the not-recommended approaches for overriding a base Feature, such as writing directly to the DB, then you will end up with an overridden feature that cannot be reverted.  Or if the Feature is reverted, you might lose the change that was written to the DB and need to reapply it.  Even if the Feature is locked to prevent reverting, it will still be listed as overridden.

Another type of “bad override” is using the wrong kind of alter-hook to try to modify configuration.  If the alter hook is called in a way that makes it look like a DB change to Features, then Features is going to see a difference between the DB and code and mark it as overridden.

Overridden Views and Panels

Some modules, such as Views and Panels have their own “override” functionality.  These “pure” Features can exist purely in code with nothing in the DB.  When you make a UI change to a View, it gets copied from the code into the DB and your change is made to the DB.  The View is then marked as “overrides code” in the Views listing.  A View that came from code can be “reverted” within the Views UI.  This essentially just deletes the version of the view in the DB so that the code takes affect again.

Sometimes you can get a View stored in the DB that still matches the code.  For example, you change a View in the UI (causing it to be copied to the DB), then you edit the View and undo the change.  Now it matches the code again, but the view is still in the DB.  In this case, the View Feature will not be shown as “overridden”, but in the Views listing it will still say “overrides code”.

When changing a pure Feature like a View via code (like in an alter hook), your changes will take affect immediately, or possible after a cache clear.  No revert will be necessary because there is nothing in the DB to update.  However, in the above case where you have changed a view and then changed it back so it’s stored in the DB, changing the code will not affect the View until you revert the Feature to remove the DB copy of the old view.

In general, if you are having issues with Views, Panels, or other ctools-like pure Features, make sure they are not overridden in their respective UIs.  For example, click the Revert option for the View within the Views listing to ensure it’s actually using the View stored in code.

Wrong version of Features

If the code in the Feature was exported using a different version of Features than what is on your site, there might be changes that cause overrides.  Ensure your version of Features matches the version used to generate the export.  This usually only applies to major version numbers and is not normally a problem.  Normally updates to the Features module are made in backwards-compatible ways to prevent this.  But certainly be sure your version of Features on your site is not older than the version that generated the base code.  Always try the latest version of Features to see if it helps.

Nobody should be using the 1.x branch of Features anymore.  There is no reason for this.  All Drupal 7 sites should be on the 2.x branch.

Unsupported Component

As mentioned, not all configuration components support Features Override, or only support it partially.  For example, Panelizer works with Features, but doesn’t work well with Features Override because it still depends on the specific order of items in it’s config array and when the alter-hook generated by Features Override tries to change something, that array order might not be preserved.  Sometimes in this case you can create your own alter-hook implementation that overrides the code properly.

This might also be an opportunity to help improve Features Override or suggest patches to the module in charge of the configuration to better support Features Override.

Living with Overrides

What if you have tried all of this advice and your Feature is still marked as overridden?  And what if this override represents a specific site change that you need to maintain and ensure never gets reverted?  The solution to this is to “lock” your feature component.  Go to the Structure/Features page and select the Feature that is marked as overridden.  Click the small “lock” icon next to the overridden component in the right column.  The component will still be listed as “overridden” but when you revert your Features the locked component will be skipped, ensuring that your customization remains in place.

When locking your Features, try to only lock the specific component rather than locking the entire Feature.  The downside to locking a component is that any changes to that component from a new version of your distribution won’t take affect.  This is similar to the consequences of the “Update the Database directly” option listed above.  However, sometimes this is the only alternative.

Conclusion

Just keep in mind that both Features and Features Override are just implementing hooks (normal hooks for Features, alter-hooks for Features Override) that are exposed by Drupal or contrib module.  If those hooks don’t exist or don’t work properly there isn’t much Features or Features Override can do about it.  Drupal 7 doesn’t have any sort of configuration management in core, so every contrib module does it differently.  ctools-based modules have more of a standard framework, as does entity API.  Features itself tries to add code to handle core components such as Fields, Content Types, etc that don’t have the needed hooks.  But there is often a limit to how much Features can do and just patching stuff into the DB is not usually a good solution.

Don’t “hate” Features or Features Override.  Life was much worse without them.  They are the best solution to solving configuration management problems in Drupal 7.  They have been developed over years of experience.  Try to help improve them before trying to implement your own configuration or override architecture.

Drupal 8

Don’t count on Drupal 8 magically solving all of these issues.  The Configuration Management Initiative (CMI) within Drupal 8 was focused on handling site configuration, but not on “bundling functionality”.  It provides an API, storage mechanism, and framework for deploying configuration.  CMI helps a lot, but there are still issues around around component granularity and overriding configuration.  In Drupal 8, configuration is owned by the site rather than owned by a module.   This will make bundling functionality (ala Features original intent) more difficult.  We will be working on Features for D8 soon, but this is still mostly unknown territory.  For now, take a look at some existing modules such as config_packager and config_devel.

I’ve submitted a session to DrupalCon Los Angeles on Features in Drupal 8.  Add a comment and support!

The Pros And Cons of Headless Drupal

Drupal is an excellent content management system. Nodes and fields allow site administrators the ability to create complex datasets and models without having to write a singe mysql query. Unfortunately Drupal’s theming system isn’t as flexible. Complete control over the dom is nearly impossible without a lot of work. Headless Drupal bridges the gap and gives us the best of both worlds.

What is headless Drupal?

Headless Drupal is an approach that decouples Drupal’s backend from the frontend theme. Drupal is used as a content store and admin interface. Using the services module in Drupal 7 or Core in Drupal 8, a rest web service can be created. Visitors to the site don’t view a traditional Drupal theme instead they are presented with pages created with Ember.js, Angular.js, or even a custom framework. Using the web service the chosen framework can be used to transfer data from Drupal to the front end and vice versa.

Pros

So what makes Headless Drupal so great? For one thing it allows frontend developers full control over the page markup. Page speed also increases since display logic is on the client side instead of the server. Sites can also become much more interactive with page transitions and animations. But most importantly the Drupal admin can also be used to power not only web apps but also mobile applications including Android and iOS.

Cons

Unfortunately Headless Drupal is not without its drawbacks. For one layout control for editors becomes much more difficult. Something that could be done easily via context or panels now requires a lot of custom fronted logic. Also if proper caching isn’t utilized and the requests aren’t batched properly lots of roundtrips can occur, causing things to slow down drastically.

Want to learn more about Headless Drupal?  Check out the Headless Drupal Initiative on Drupal.org.  And on a related topic, check out “Drupal 8 And The Changing CMS Landscape.

Ultimate Flexibility: Open Atrium’s New Related Content Feature

atrium-logo (1)The constant struggle between content editors and web developers:

  • Content editors want to embed rich media, callouts, and references to related content anywhere within their WYSIWYG.  They love the Body field and want more buttons added to the WYSIWYG.
  • Web developers want to add additional fields for media, attachments, related content to support rich content relationships and access control.  Web developers hate the Body field and wish they didn’t need a WYSIWYG.

In the latest 2.30 version of Open Atrium, we attempt to help both content editors and web developers with a new approach to building rich story content using the Paragraphs module.  Rather than having a single WYSIWYG body, content editors can build their story using multiple paragraphs of different types and layouts. The order of these paragraphs can be rearranged via drag and drop to create long-form content.

Site builders can add their own custom paragraph types to their sites, but Open Atrium comes with four powerful paragraph types “out of the box”:

(1) Text Paragraphs

OA2_Test_Paragraph

The simplest paragraph type is the “Text” paragraph.  It works just like the normal Body field with it’s own WYSIWYG editor.  But an additional Layout field is added to control how the text is rendered.  There are options for multiple columns of wrapping text within the paragraph (something nearly impossible to do with the normal Body field), as well as options for left or right floating “callouts” of text.

(2) Media Gallery

OA_Media_Paragraph

The “Media Gallery” paragraph handles all of the images and videos you want to add to your story.  It can replace the normal Attachments field previously used to add media to a document.  Each Media paragraph can contain one or more images, videos, or files.  The Layout field controls how that media is displayed, providing options for left or right floating regions, or a grid-like gallery of media.  Videos can be embedded as preview images or full video players.

When floating media to the left or right, the text from other paragraphs will flow around it, just as if the media had been embedded into the WYSIWYG.  To move the images to a different part of the story, just drag the media paragraph to a new position in the story.

In Open Atrium, images directly embedded into the Body WYSIWYG field becomes Public, bypassing the normal OA access control rules.  However, anything added to a Media paragraph works more like the Attachment field and properly inherits the access permission of the story document being created.  Thus, the Media paragraph provides a way to embed Media within your story while retraining proper privacy permissions.

(3) Snippets

OA_Snippet_Demo

The “Snippet” paragraph type allows you to embed text from any other content on your site.  You can specify whether the Summary, Body, or full Node is embedded and also control the Layout the same as with Text paragraphs.  You can also either display the Title of the referenced content or hide the title, or override the title with your own text.

One of the best features of Snippets is the ability to lock which revision you want to display.  For example, imagine you want to embed a standard operating procedure (SOP) within your story document.  You create a Snippet paragraph that points to the SOP.  However, if the related SOP node is updated in the future, you don’t want your old document to change.  For compliance purposes it still needs to contain the original SOP text.  By “locking” the snippet to the old revision, the old document will continue to display the original SOP even if the SOP is updated later.  If you “unlock” the snippet, then it will display the latest version of the related SOP.

Open Atrium access controls are also respected when displaying snippets.  If you reference content that the user doesn’t have permission to view, that snippet will be removed from the displayed text.  Users still only see the content they are allowed.  This provides a very powerful way to create rich documents that contain different snippets of reusable content for different user roles and permissions.  Similar to adding additional fields with Field Permissions, but much more flexible and easy to use.

(4) Related Content

OA_Related_Demo

The “Related Content” paragraph type is similar to Snippets, but displays the Summary or Full rendered node of the related content.  Like the Media paragraph, the Related Content can contain one or more references to other content on the site.  The Layout provides options for displaying the content as a table of files, or a list of node summaries (teasers), or as full node content.  When full node content is used, any paragraphs used in the related content will also being displayed (paragraph “inception”!).  In addition, any special fields from the full related node can be shown.  For example, a Related Event will show the map of the event location.  A Related Discussion will show all of the discussion replies and even provide the Reply Form, allowing you to reply to a related discussion directly from the story document itself!

Related Content is also a bi-directional link.  When you view the related content node, a sidebar widget called “Referenced From” will show all of the stories that reference the node being viewed.

A Real World Example

To pull all of this together with a real-world example, imagine that you are scheduling a Meeting.  You want to create an Agenda for that meeting and allow your team to discuss and edit the agenda before the meeting.  In Open Atrium you can now do this all from a single document:

  1. Create the Event for the Meeting, adding your team to the Notifications
  2. Add a Related Content paragraph for the meeting Agenda document
  3. Add a Related Content paragraph for the agenda Discussion

Open Atrium is smart about where this related content is created.  If you already have a section for documents, the Agenda will be created within that section.  If you already have a section for discussions, the related discussion will be placed there.  You can change these locations if you wish, but the default behavior reflects the most common information architecture.

When your team members receive the email notification about the meeting and click the link, they will be taken to your Event and will see the agenda document and discussion as if they were a normal part of the event body.  They can view the agenda content directly and can post replies directly into the discussion reply field.  They don’t need to go to separate places on the site to see the document or discussion.  If you *do* view the document or discussion nodes directly, such as from a search results page, you’ll see a link back to the meeting event in the References From list in the sidebar.

Conclusion

Not only do the Paragraph features help content editors build rich stories quickly and easily, they allow web developers to create related documents, linked content, better search results, better data structures.  It’s still not a magical unicorn wysiwig of content editor’s dreams, but it’s a significant step for Open Atrium and Drupal. It opens a whole new world of collaboration where all related content can be viewed together.

Looking for more information about Open Atrium? Sign up to receive Open Atrium newsletters and updates! Plus, get an in-depth look at Open Atrium’s new features in our winter release webinar – the recording is now available!

Join Phase2 for Drupal Global Sprint Weekend in NYC & PDX

This upcoming weekend, the global Drupal community is kicking off the New Year with sets of local sprints on January 17 and 18. The event is called Drupal Global Sprint Weekend, and as of this blog post there are almost 50 sprints scheduled all around the world. Phase2 is excited to be a part of this international event, hosting sprints at both our New York City and Portland offices and sponsoring a sprint at AnexusIT in Costa Rica.

Screen Shot 2015-01-12 at 10.07.09 AM

For the past 2.5 years, we’ve worked with Robbie Holmes to host Drupal community events, called Drupal Play Days, at the Phase2 office in New York City. Over time, the agenda has grown more diverse to include topics like SASS and Compass, Vagrant and Puppet, and of course Drupal 8. Our own Mai Irie has also used a lot of recent Play Days to hold planning meetings for NYCCamp events held at the United Nations.

This month, we are excited to combine our Drupal Play Day with the Drupal Global Sprint Weekend at the Phase2 offices. This means that you’re welcome to drop in, no matter your level in Drupal, from 9 AM to 5 PM. I repeat, you don’t have to be a Drupal developer to come out – all skill sets and skill levels are encouraged to participate. The plan is to split into pairs to work on Drupal core issues tagged with #SprintWeekend. Phase2 will provide Wi-Fi, heat, and food.

Please remember to sign up on the Drupal Groups event page, indicating if you will be sprinting in New York CityPortland, or Costa Rica, so that we can plan appropriately for food and seating needs. In the meantime, be sure to spread the word using #SprintWeekend on Twitter.

We can’t wait to see you this weekend! Don’t forget to bring your laptop ;)

Play with Display: Altering Drupal Field Collections

How do you insert a text field into a rendered entity, all within a field collection? This question came to mind on a recent Drupal 7 build. What at first sounded complicated turned into a concise and elegant solution.

The basis of the requirement broke down into the need for events to list those participating in the event. The build already had existing Event and Profile content types, so the new functionality would only require implementing the list of Profiles participating in an event. Each participant would include their role at the event, along with links to contact them.

blog

To implement this, an Event Participant field collection was added as a field to the Event content type. Each instance of the field collection included a single Profile content type and a single text field for their role at the Event. The Profile would then be displayed as a rendered entity within the field collection so that the Profile’s image and contact links could be displayed. The point at issue then arose. How would the Participant’s role at the event be placed within the rendered Profile teaser, along with the other Profile fields? To the user it needed to look like a single unified entity, with the Profile’s image, role at the Event, Profile name, and contact links all included.

Screen Shot 2014-12-11 at 4.27.43 PM.png

After some initial brainstorming a handful of different approaches came to mind. The initial impression was to use template_preprocess_field() since it allows the alteration of values in the theme layer. The participant role field would be removed from it’s initial location and inserted into the participant profile object. This did not work though. Despite the repositioning of elements within the render array, the changes occur too late in the rendering execution. The render array had already been rendered and thus the display remained the same.

After quickly realizing a theme function wasn’t the answer, hook_node_view() came to mind. The hook allows for the alteration of a node before it is rendered and so all that would be needed would be to shift the location of the Participant Role field within the $node array. The issue with this implementation and the reason it wasn’t chosen, is that it would require reaching out to the global state to determine what changes are needed. In other words hook_node_view() doesn’t realize what context it is called under and would thus require a hack of sorts to determine when it was being called under an Event node. This could be worked around by checking the URL path, but this is not best practice as it is brittle and prone to breaking with future site updates.

As they say the third time’s the charm. The final solution was to use hook_field_collection_item_view(). This hook allows for the alteration of a field collection item before rendering. Unlike many of the other options, this hook is called for each field collection item, allowing one to both check the field collection name and adjust it’s internal field ordering, before any rendering occurs. Using this hook, each Event Participant field collection item is altered, removing the Role field and inserting it into the Profile rendered entity. Simple as that. Now each Event Participant is correctly displayed as a single unified entity, with an image, name, and a set of contact links.

Through several quick, small, iterations an elegant, straight-forward solution was found to what at first sounded like a complicated feature request. Now users viewing an Event can see a list of Participants, with each Participant displayed as a single item. Check out more on field collections on the Phase2 blog!