As the Drupal community and lots of newcomers all descend on San Francisco this week, I am making just one (maybe not so bold) prediction: This conference is the tipping point. Later, we will look back and pinpoint this as when Drupal really went big.
Viewing Drupal as both a product, and as a platform with a maturity curve is a concept that has stayed with me since Dries used a technology product adoption curve approach to illustrate the maturity of Drupal during his traditional State of Drupal keynote in Paris (courtesy of ultimike from Drupal Easy).
As I consider the way in which Drupal use is growing wildly in 2010, I think that the project itself is reaching a point of maturity we can no longer measure simply in terms of progress on releases, code contributions or downloads alone. There are lots of measures of so called "community progress" which have given way to many more exciting signs of maturity with respect to the larger market and ecosystem.
Of particular note this year is the adoption of Drupal as the platform of choice for large established organizations in the publishing, media, entertainment and government spaces. To me, the embracing of Drupal by large organizations defines the point at which a technology "has arrived" on the scene and is likely to be taken seriously. It is also a good sign that it will stick around as an important technology with an installed base for a while to come.
Why is this all happening now? It is likely a combination of many things: maturity of the product itself, size of the installed base, growth in community popularity and participation, and lack of affordable alternatives for comparable web 2.0 functionality. I actually prefer to view it more as the maturity of Drupal as a platform through which the Drupal technology community is pushing Drupal forward - potentially straight into enterprise software territory. Here is my "top ten style" list of the technology trends I see fueling this growth.
10. Flexible Architecture
Brute force is a bad way to build sites. One of the great things about Drupal is how you can use it to create sites in increasingly smarter ways like multi-site configurations where sites sharing the same core with shared components like modules can be setup in permutations like: virtual sites, multi-sites, and micro-sites. Another great thing is the installation profile that allows pre-fab site construction to occur. There are a lot of real world use cases for why these are important to big organizations developing sites. The key is repeatability. No one wants to re-invent each site in a cluster of similar properties when they share lots of the same components, themes and even content. The Drupal community is leading the way on strategies to accomplish re-use through these schemes more than any other CMS I know.
9. Deployment Improvements
I am fascinated by the idea that there are still limitless opportunities for more efficient ways to integrate, test, deploy and maintain code. In our firm, deployment is a serious and sacred subject because nothing makes the pros stand apart from the amateurs more than how you manage and control your code and environments. It is also incredibly important to us that we minimize the amount of time these activities take relative to the more creative (and ultimately more value adding and rewarding) activities of solution design and code development. As we build more and more Drupal sites and deploy them to larger and increasingly complex environments, we need tools to make this process better. The best practices are far from universalized and institutionalized at this point, but many leading tools and processes are pushing this forward including Aegir from DevelopmentSeedand continuous integration techniques from firms like Chapter Three and Four Kitchens.
8. The Cloud
Sure this trend is not unique or native to the Drupal community, but clearly using the flexibility of cloud-based environments to host sites and deploy assets has reached critical mass. For some, this includes deploying file assets like audio and video to S3 to free server resources, for others it means using Amazon EC2 for scalable deployment environments. It seems like new products and strategies for hosting sites in the cloud are being deployed every week. Chapter Three's "Pantheon" framework for creating and managing Amazon Management Interfaces (AMIs) is a good example home grown in the Drupal community. Acquia provides complete cloud based hosting as its only infrastructure offering. Whether you plan to host in the cloud or augment physical servers with cloud-based services, this trend provides huge opportunity for software developers to make hardware "programmable" and more responsive to our ability to script and control resources without physical dependencies.
7. Semantic Web
Having a keynote session in Paris dedicated to semantic web is a great reminder that our community is on the leading edge of how to program for web3.0. Drupal 7 code development was frozen on-time with some notable exceptions and RDF, despite its trickiness and difficulty, made the cut. Including support for RDF and RDFa representations of data probably didnt even make the roadmap for most CMS products in 2009 because it is such a forward thinking approach towards the semantic web. RDF is what makes our data portable, machine readable and light years ahead of other design-centric CMSs. Our community is full of developers, academics, and companies that understand why this is important and are setting a course to make Drupal the semhead's CMS of choice. When we released our suite of openCalais modules in 2008 for using this amazing web web service for semantic metatagging of content we learned how respected Drupal was in the semantic web community - a group of very smart folks that don't dote on technologies lightly. I have no doubt that inspiring technologists working on the Drupal RDF team like Arto Bendiken, Miglius Alaburda, Ben Lavender, Jeff Miccolis, Frank Febbraro, Stéphane Corlosquet and of course, Dries will keep us bleeding edge on the semantic web for some time to come.
The features concept has the potential to make significant impacts on the Drupal architecture by establishing a re-use model that developers use to create end user functionality. The concept of features was all over the Paris conference and has since been moving quickly through the community from theory to reality. While OpenAtrium stands as the best example of how to use features, the possibilities for how to make this impact many aspects of Drupal are just beginning to be discussed. Essentially the concept of features is that "ready to go" functionality can be created, re-used and shared if the components of that functionality (includes content types, views, modules, theme) could be bundled and exported as a self contained entity. It is exciting and refreshing to see that innovations can occur in the way Drupal sites are constructed from within the community and above the core level - it means there is a lot of new ground to cover even still.
5. User Experience
Drupal has embraced one its worst perceived weaknesses and hit usability and user experience hard this past year in preparation for D7. Designers Mark Boulton and Leisa Reichelt took on the user experience to change the way the world sees Drupal forever. The crux of this issue is that Drupal must invest in improvements to UI as well as features in core. The challenge is striking the balance that keeps Drupal on the leading edge for developers who want flexibility, extensibility and performance over UX. But another way to look at the focus on UX is to question whether Drupal has the best and most mature ways of allowing theming and design on the front end too. In our OpenPublish distribution, we determined the answer to that question was decisively no and we set about making it easier to theme knowing that more advanced publications would be more interested in it as a result. We recognized the same with our administrative interfaces and switched to a slightly modified version of Admin2 created by DevelopmentSeed that is much simpler, more intuitive and (when combined with a theme like Rubik) frankly better looking than any admin we had used before.
Bringing these user experiences questions to the fore front this past year has helped to bring designers and developers together in a way that the Drupal community sorely needed - even when there was disagreement. It takes a balance of form and function to create a truly robust product. All efforts to improve Drupal user experience are exciting because it is one less thing that the rest of the world can say Drupal "doesnt do well." I love that this community seeks these things out and attacks them with vigor.
Have you read "Cracking Drupal" by Greg Knaddison ? It was all the rage of Drupal books for developers. What it says to me is that this community is taking security seriously enough to have books written on the subject instead of a series of checklists (ahem!) I am certainly no security expert, but I do spend a lot of time discussing the pros and cons of Drupal vs. other CMS solutions with some pretty tough customers and have for many years. Less and less do I hear concerns about Drupal and more and more I hear it about competing platforms. I think this is yet another area where the community has recognized a potential weakness and improved its focus and collective energies. The Drupal security team doesnt sit around updating a wiki with lists of compromised modules, they act. What does for the rest of the community is establish an urgency around ensuring that vulnerabilities dont take control of the platform and its reputation.
3. Consumer Drupal
In 2009, Acquia introduced Beta Drupal Gardens and set sites on the consumer market for SaaS sites dominated by players like WordPress.com and other products for the microsite marketplace. Lullabot has a similar concept that addresses different aspects of the Do It Yourself SaaS site called Buzzr. Clearly the need for something to capture what is more or less the general consumer marketplace is going to reach a pinnacle in 2010. But many developers and early community defenders were also questioning the strategy behind this concept. Clearly not everyone is onboard with the idea of looking at Drupal as a tool for the common man. Honestly I did not either at first, but I think the key to understanding what is happening here is to think of this as one more part of the puzzle of making Drupal go big, real big. We have to try to think of the Drupal community as a diverse ecosystem of consumers and providers of all types reaching different market segments. No doubt mid market popularity of Drupal among non-profits, private companies and publishers is already skyrocketing and large enterprises are growing more comfortable with open source in general and Drupal specifically. What we are lacking is overwhelming mass market momentum to achieve staying power and build brand ubiquity. I would certainly never advocate that mass market appeal is what is going to make Drupal more interesting, more fun or more rewarding. However, it would be good to gain some staying power by having a version that appeals to a wider audience.
Some things do change, but everyone is still asking the same question: Does Drupal perform at an enterprise level? This is one of those discussions that just won't die, but the evidence is all around us that Drupal can handle enterprise use cases. For sure sending an open source platform up against commercially funded and supported tools that cost hundreds of thousands of dollars for organizations that get millions of page views and expect 24/7/365 uptime can be a bit daunting. But the proof is mounting that with the help of infrastructure tools like CDNs, proxy caching, redundant virtualized servers, and some good old fashion well planned architecture, it can be done. Perhaps the community needs to raise a showcase site to present how heavily trafficked sites and publications are running Drupal every day? Or maybe just better education about what truly effects performance and how Drupal can address each point? Whatever it takes, this may be the biggest weight holding Drupal down right now and one that is destined to be lifted this year if more high traffic examples are unveiled.
1. Drupal Product Development
We went big on developing Drupal distributions ("distros") over the last two years because we felt we had more to offer back to the community than modules - we wanted to build stuff that showed how flexible and powerful Drupal could be. Modules help Drupal developers, but distros like OpenPublish can be and are used by non-community business users. When you build dozens of Drupal sites, you see patterns, commonalities, and ways to package products that serve specific needs. My colleague Irakli our Director of Product Development, wrote a prolific post on our strategy that I think still summarizes our views on this.
Many of the most proficient Drupal development shops including DevelopmentSeed, Lullabot, Acquia, Phase2, and Four Kitchens build distros for their own reasons, but I suspect a lot of the inspiration behind this trend is because distros give us a creative outlet to show how to make things from Drupal that are cool and work. When we released Tattler (app), a topic monitoring tool for the Web, we wanted it to demonstrate a unique use case for Drupal which opens up more ideas and spurs innovation. It was a proof of what could be done not a complete product we wanted to sell. Of course there is a danger to all of this re-packaging of Drupal. Those of us doing it, need to set some better ground rules and remain committed to not branching from core community principles and core architecture. A lot of consensus has emerged on this front over even just the last six months.
Distros also help to grow the ecosystem. Hosting companies are emerging with plans to sell packages of these distros directly to consumers as fully managed installations. Themers like AdaptiveThemes plan to sell custom and off the shelf themes for specific distros like OpenPublish. I have no idea how many people have been attracted to using Drupal by distros like OpenPublish and Open Atrium but from anecdotal evidence I have, its a lot. We hear things like "I didnt know where to start with Drupal or what to do with it, but now I do..." According to DevelopmentSeed close to 100,000 people have downloaded OpenAtrium so far. How many of those were capable of developing a high quality intranet based upon Drupal before that? Pretty powerful stuff.
So my conclusion is that Drupal for the Enterprise is here - at least technically. Now it is time for the community to drive it home. Big customers are shopping Drupal and many more than you might expect are using it. But it is not just about open source vs. proprietary anymore, nor is the issue simply whether Drupal can perform, scale and be secure enough. The tools to do wondrous technical things exist. The truth that the Drupal community now needs to realize is that the art of the big engagement is as much about process, project management, best practices and professionalism. I dont think Drupal is quite at the point where legions of Accenture and IBM consultants are going to flood "zero to hero" sessions at Drupalcon, but with success comes attention and with attention comes scrutiny so get ready Drupal your biggest trials and glory are still ahead.