Making Drupal upgrades easy forever

One of the key reasons that Drupal has been successful is because we always made big, forward-looking changes. As a result, Drupal is one of very few CMSes that has stayed relevant for 15+ years. The downside is that with every major release of Drupal, we've gone through a lot of pain adjusting to these changes. The learning curve and difficult upgrade path from one major version of Drupal to the next (e.g. from Drupal 7 to Drupal 8) has also held back Drupal's momentum. In an ideal world, we'd be able to innovate fast yet provide a smooth learning curve and upgrade path from Drupal 8 to Drupal 9. We believe we've found a way to do both!

Upgrading from Drupal 8.2 to Drupal 8.3

Before we can talk about the upgrade path to Drupal 9, it's important to understand how we do releases in Drupal 8. With the release of Drupal 8, we moved Drupal core to use a continuous innovation model. Rather than having to wait for years to get new features, users now get sizable advances in functionality every six months. Furthermore, we committed to providing a smooth upgrade for modules, themes, and distributions from one six-month release to the next.

This new approach is starting to work really well. With the 8.1 and 8.2 updates behind us and 8.3 close to release, we have added some stable improvements like BigPipe and a new status report page, as well as experimental improvements for outside-in, workflows, layouts, and more. We also plan to add important media improvements in 8.4.

Most importantly, upgrading from 8.2 to 8.3 for these new features is not much more complicated than simply updating for a bugfix or security release.

Upgrading from Drupal 8 to Drupal 9

After a lot of discussion among the Drupal core committers and developers, and studying projects like Symfony, we believe that the advantages of Drupal's minor upgrade model (e.g. from Drupal 8.2 to Drupal 8.3) can be translated to major upgrades (e.g. from Drupal 8 to Drupal 9). We see a way to keep innovating while providing a smooth upgrade path and learning curve from Drupal 8 to Drupal 9.

Here is how we will accomplish this: we will continue to introduce new features and backwards-compatible changes in Drupal 8 releases. In the process, we sometimes have to deprecate old systems. Instead of removing old systems, we will keep them in place and encourage module maintainers to update to the new systems. This means that modules and custom code will continue to work. The more we innovate, the more deprecated code there will be in Drupal 8. Over time, maintaining backwards compatibility will become increasingly complex. Eventually, we will reach a point where we simply have too much deprecated code in Drupal 8. At that point, we will choose to remove the deprecated systems and release that as Drupal 9.

This means that Drupal 9.0 should be almost identical to the last Drupal 8 release, minus the deprecated code. It means that when modules take advantage of the latest Drupal 8 APIs and avoid using deprecated code, they should work on Drupal 9. Updating from Drupal 8's latest version to Drupal 9.0.0 should be as easy as updating between minor versions of Drupal 8. It also means that Drupal 9 gives us a clean slate to start innovating more rapidly again.

Why would you upgrade to Drupal 9 then? For the great new features in 9.1. No more features will be added to Drupal 8 after Drupal 9.0. Instead, they will go into Drupal 9.1, 9.2, and so on.

To get the most out of this new approach, we need to make two more improvements. We need to change core so that the exact same module can work with Drupal 8 and 9 if the module developer uses the latest APIs. We also need to provide full data migration from Drupal 6, 7 and 8 to any future release. So long as we make these changes before Drupal 9 and contributed or custom modules take advantage of the latest Drupal 8 APIs, up-to-date sites and modules may just begin using 9.0.0 the day it is is released.

What does this mean for Drupal 7 users?

If you are one of the more than a million sites successfully running on Drupal 7, you might only have one more big upgrade ahead of you.

If you are planning to migrate directly from Drupal 7 to Drupal 9, you should reconsider that approach. In this new model, it might be more beneficial to upgrade to Drupal 8. Once you’ve migrated your site to Drupal 8, subsequent upgrades will be much simpler.

We have more work to do to complete the Drupal 7 to Drupal 8 data migration, but the first Drupal 8 minor release that fully supports it could be 8.4.0, scheduled to be released in October 2017.

What does this mean for Drupal developers?

If you are a module or theme developer, you can continually update to the latest APIs each minor release. Avoid using deprecated code and your module will be compatible with Drupal 9 the day Drupal 9 is released. We have plans to make it easy for developers to identify and update deprecated code.

What does this mean for Drupal core contributors?

If you are a Drupal core contributor and want to introduce new improvements in Drupal core, Drupal 8 is the place to do it! With backwards compatibility layers, even pretty big changes are possible in Drupal 8.

When will Drupal 9 will be released?

We don't know yet, but it shouldn't matter as much either. Innovative Drupal 8 releases will go out on schedule every six months and upgrading to Drupal 9 should become easy. I don't believe we will release Drupal 9 any time soon; we have plenty of features in the works for Drupal 8. Once we know more, we'll follow up with more details.

Thank you

Special thanks to Alex Bronstein, Alex Pott, Gábor Hojtsy, Nathaniel Catchpole and Jess (xjm) for their contributions to this post.

Distributions remain a growing opportunity for Drupal

Yesterday, after publishing a blog post about Nasdaq's Drupal 8 distribution for investor relations websites, I realized I don't talk enough about "Drupal distributions" on my blog. The ability for anyone to take Drupal and build their own distribution is not only a powerful model, but something that is relatively unique to Drupal. To the best of my knowledge, Drupal is still the only content management system that actively encourages its community to build and share distributions.

A Drupal distribution packages a set of contributed and custom modules together with Drupal core to optimize Drupal for a specific use case or industry. For example, Open Social is a free Drupal distribution for creating private social networks. Open Social was developed by GoalGorilla, a digital agency from the Netherlands. The United Nations is currently migrating many of their own social platforms to Open Social.

Another example is Lightning, a distribution developed and maintained by Acquia. While Open Social targets a specific use case, Lightning provides a framework or starting point for any Drupal 8 project that requires more advanced layout, media, workflow and preview capabilities.

For more than 10 years, I've believed that Drupal distributions are one of Drupal's biggest opportunities. As I wrote back in 2006: Distributions allow us to create ready-made downloadable packages with their own focus and vision. This will enable Drupal to reach out to both new and different markets..

To capture this opportunity we needed to (1) make distributions less costly to build and maintain and (2) make distributions more commercially interesting.

Making distributions easier to build

Over the last 12 years we have evolved the underlying technology of Drupal distributions, making them even easier to build and maintain. We began working on distribution capabilities in 2004, when the CivicSpace Drupal 4.6 distribution was created to support Howard Dean's presidential campaign. Since then, every major Drupal release has advanced Drupal's distribution building capabilities.

The release of Drupal 5 marked a big milestone for distributions as we introduced a web-based installer and support for "installation profiles", which was the foundational technology used to create Drupal distributions. We continued to make improvements to installation profiles during the Drupal 6 release. It was these improvements that resulted in an explosion of great Drupal distributions such as OpenAtrium (an intranet distribution), OpenPublish (a distribution for online publishers), Ubercart (a commerce distribution) and Pressflow (a distribution with performance and scalability improvements).

Around the release of Drupal 7, we added distribution support to Drupal.org. This made it possible to build, host and collaborate on distributions directly on Drupal.org. Drupal 7 inspired another wave of great distributions: Commerce Kickstart (a commerce distribution), Panopoly (a generic site building distribution), Opigno LMS (a distribution for learning management services), and more! Today, Drupal.org lists over 1,000 distributions.

Most recently we've made another giant leap forward with Drupal 8. There are at least 3 important changes in Drupal 8 that make building and maintaining distributions much easier:

  1. Drupal 8 has vastly improved dependency management for modules, themes and libraries thanks to support for Composer.
  2. Drupal 8 ships with a new configuration management system that makes it much easier to share configurations.
  3. We moved a dozen of the most commonly used modules into Drupal 8 core (e.g. Views, WYSIWYG, etc), which means that maintaining a distribution requires less compatibility and testing work. It also enables an easier upgrade path.

Open Restaurant is a great example of a Drupal 8 distribution that has taken advantage of these new improvements. The Open Restaurant distribution has everything you need to build a restaurant website and uses Composer when installing the distribution.

More improvements are already in the works for future versions of Drupal. One particularly exciting development is the concept of "inheriting" distributions, which allows Drupal distributions to build upon each other. For example, Acquia Lightning could "inherit" the standard core profile – adding layout, media and workflow capabilities to Drupal core, and Open Social could inherit Lightning - adding social capabilities on top of Lightning. In this model, Open Social delegates the work of maintaining Layout, Media, and Workflow to the maintainers of Lightning. It's not too hard to see how this could radically simplify the maintenance of distributions.

The less effort it takes to build and maintain a distribution, the more distributions will emerge. The more distributions that emerge, the better Drupal can compete with a wide range of turnkey solutions in addition to new markets. Over the course of twelve years we have improved the underlying technology for building distributions, and we will continue to do so for years to come.

Making distributions commercially interesting

In 2010, after having built a couple of distributions at Acquia, I used to joke that distributions are the "most expensive lead generation tool for professional services work". This is because monetizing a distribution is hard. Fortunately, we have made progress on making distributions more commercially viable.

At Acquia, our Drupal Gardens product taught us a lot about how to monetize a single Drupal distribution through a SaaS model. We discontinued Drupal Gardens but turned what we learned from operating Drupal Gardens into Acquia Cloud Site Factory. Instead of hosting a single Drupal distribution (i.e. Drupal Gardens), we can now host any number of Drupal distributions on Acquia Cloud Site Factory.

This is why Nasdaq's offering is so interesting; it offers a powerful example of how organizations can leverage the distribution "as-a-service" model. Nasdaq has built a custom Drupal 8 distribution and offers it as-a-service to their customers. When Nasdaq makes money from their Drupal distribution they can continue to invest in both their distribution and Drupal for many years to come.

In other words, distributions have evolved from an expensive lead generation tool to something you can offer as a service at a large scale. Since 2006 we have known that hosted service models are more compelling but unfortunately at the time the technology wasn't there. Today, we have the tools that make it easier to deploy and manage large constellations of websites. This also includes providing a 24x7 help desk, SLA-based support, hosting, upgrades, theming services and go-to-market strategies. All of these improvements are making distributions more commercially viable.

How Nasdaq offers a Drupal distribution as-a-service

Nasdaq CIO and vice president Brad Peterson at the Acquia Engage conference showing the Drupal logo on Nasdaq's MarketSite billboard at Times Square NYC

Last October, I shared the news that Nasdaq Corporate Solutions has selected Acquia and Drupal 8 for its next generation Investor Relations and Newsroom Website Platforms. 3,000 of the largest companies in the world, such as Apple, Amazon, Costco, ExxonMobil and Tesla are currently eligible to use Drupal 8 for their investor relations websites.

How does Nasdaq's investor relations website platform work?

First, Nasdaq developed a "Drupal 8 distribution" that is optimized for creating investor relations sites. They started with Drupal 8 and extended it with both contributed and custom modules, documentation, and a default Drupal configuration. The result is a version of Drupal that provides Nasdaq's clients with an investor relations website out-of-the-box.

Next, Nasdaq decided to offer this distribution "as-a-service" to all of their publicly listed clients through Acquia Cloud Site Factory. By offering it "as-a-service", Nasdaq's customers don't have to worry about installing, hosting, upgrading or maintaining their investor relations site. Nasdaq's new IR website platform also ensures top performance, scalability and meets the needs of strict security and compliance standards. Having all of these features available out-of-the-box enables Nasdaq's clients to focus on providing their stakeholders with critical news and information.

Offering Drupal as a web service is not a new idea. In fact, I have been talking about hosted service models for distributions since 2007. It's a powerful model, and Nasdaq's Drupal 8 distribution as-a-service is creating a win-win-win-win. It's good for Nasdaq's clients, good for Nasdaq, good for Drupal, and in this case, good for Acquia.

It's good for Nasdaq's customers because it provides them with a platform that incorporates the best of both worlds; it gives them the maintainability, reliability, security and scalability that comes with a cloud offering, while still providing the innovation and freedom that comes from using Open Source.

It is great for Nasdaq because it establishes a business model that leverages Open Source. It's good for Drupal because it encourages Nasdaq to invest back into Drupal and their Drupal distribution. And it's obviously good for Acquia as well, because we get to sell our Acquia Site Factory Platform.

If you don't believe me, take Nasdaq's word for it. In the video below, which features Stacie Swanstrom, executive vice president and head of Nasdaq Corporate Solutions, you can see how Nasdaq pitches the value of this offering to their customers. Swanstrom explains that with Drupal 8, Nasdaq's IR Website Platform brings "clients the advantages of open source technology, including the ability to accelerate product enhancements compared to proprietary platforms".

Sharing innovation with your competitors

Some of the largest brands in the world are emerging as leading sponsors and contributors of Drupal. Pfizer, for example, has been using Drupal to improve its internal content workflow processes. Not only is Pfizer a major user of Drupal, they are also making their Drupal improvements available for everyone's benefit, including their competitors. This kind of innovation and collaboration model is relatively unheard of and is less likely to happen with proprietary software.

Another great example is Boston.gov. Last year the City of Boston migrated Boston.gov to Drupal. Shortly after the launch of Boston.gov, they released Boston.gov's source code to the public domain. By open-sourcing their project, the city of Boston is challenging the prevailing model. Anyone can see the code that makes Boston.gov work, point out problems, suggest improvements, or use the code for their own city, town or organization.

The City of Boston isn't the only government agency that is changing their way of innovating. In 2012, the White House released the code behind "We the People", the Drupal-based application that allows the American people to submit petitions directly to the President of the United States. By releasing the code that supports "We the People", any government in the world can take advantage of the project and implement it in their own community.

Next, the international media group Hubert Burda Media employs a team of six Drupal developers that build and maintain Thunder, a Drupal 8 distribution that can be used by any of the 164 brands that Burda supports. Last year, Burda open-sourced Thunder, allowing competitors to benefit from Burda's development, know-how and best practices. As part of their work on Thunder, Burda is an active contributor to Drupal 8's media initiative. Burda is also inviting its competitors to contribute to Thunder.

Some may wonder what is beneficial about sharing innovation with competitors. Today, technology is becoming more and more complex and the rate of change is accelerating. It is becoming increasingly difficult for any one organization to build an entire solution and do it well. By contributing back and by working together, these organizations can keep a competitive edge over those that don't use open source and collaborate. What looks strange to some, is actually perfectly logical to others. Those that contribute to open source are engaging in a virtuous cycle that benefits their own projects. It is a tide that raises all boats; a model that allows progress to accelerate due to wider exposure and public input. It's a story that is starting to play out in every industry -- from pharmaceutical companies, to media and publishing, to government.

Challenge the prevailing model

As I wrote in my 2016 Acquia retrospective, I believe that the use of open source software has finally crossed the chasm -- most organizations don't think twice about using open source software. The next step is to encourage more organizations to not just use open source, but to contribute to it. Open source offers a completely different way of working, and fosters an innovation model that is not possible with proprietary solutions. Pfizer, Boston.gov, the White House and Burda are remarkable examples of how organizations benefit from not only using but contributing to open source.

In order to help people understand the power of this model we have to change the lens through which organizations see the world. It's hard to disrupt the status quo, but fortunately we now have powerful examples that highlight how great organizations are using open source to change their operating model.

If you want to help challenge the prevailing model at your own organization, here are the basic steps that your organization can implement today:

  1. Embrace open source in your organization and make it successful.
  2. Assess whether any of your customizations are truly custom or if they can be used by others.
  3. Contribute back your customizations to the open source project, advance it in the open and encourage others to contribute.

Acquia a leader in the 2017 Forrester Wave for Web Content Management Systems

Today Acquia was named a leader in the The Forrester Wave: Web Content Management Systems, Q1 2017. This report is especially exciting because for the first time ever Acquia was recognized as the leader for strategy and vision, ahead of every other vendor including Adobe and Sitecore.

I take great pride in Acquia's vision and strategy. As I wrote in my 2016 retrospective, I believe that the powerful combination of open source and cloud has finally crossed the chasm, and will continue to drive change in our industry.

Today, cloud is the starting point for any modern digital experience project, but this wasn't always the case. Acquia was one of the first organizations to explore Amazon Web Services in 2008 and to to launch a commercial cloud product in 2009. At the time that was a bold move, but we have always seen the cloud as the best way to provide our customers with the necessary tools to build, deploy and manage Drupal.

Since its founding, Acquia has helped set the standard for cloud-based delivery of web content management. This year's Forrester report shows that many of our competitors are only now introducing a cloud delivery platform. Acquia's eight year commitment to the cloud is one of the reasons why we were deemed the leading vendor for strategy.

Second, organizations can no longer afford to ignore open source software. Acquia is the only open source leader in both the Forrester Wave and the Gartner Magic Quadrant for Web Content Management. There are a lot of inherent benefits to open source software, including speed of innovation, cost, quality, security and more. But the thing that really sets open source apart is freedom. This extends to many different users: for developers freedom means there are no limitations on integrations; for designers that means creative freedom, and for organizations that means freedom from vendor lock-in.

In addition to our commitment to open source and cloud, Forrester also highlighted Acquia's focus on APIs. This includes our many contributions to Drupal 8's API-first initiative, but it also includes our APIs for Acquia Cloud, Acquia Cloud Site Factory and Acquia Lift. APIs are important because they enable organizations to build more ambitious digital experiences. With Open APIs, organizations can use our software in the way they choose and integrate with different technology solutions.

This is why Acquia's solution -- using open source, cloud, and open APIs -- is so powerful: it prepares organizations for the current and evolving demands of digital experience delivery by allowing them to innovate faster than traditional solutions.

Thank you to everyone who contributed to this result! It's another great milestone for Acquia and Drupal.

Updates from Dries straight to your mailbox