This blog post accompanies the training session Phase2 did at Drupal Camp Costa Rica. Phase2 prides itself in its ability to deliver big scalable Drupal sites to our clients. As time goes on, we have noticed some common patterns in projects that involve building big Drupal sites. Big Drupal sites accomodate big teams of people, from developers to content editors. This post outlines a bird's eye view of some common concepts we've encountered.
is a way to put configuration into code. This allows us to maintain it in a versioning system, and to always have a “baseline” of how your site should be structured.
In features, we usually keep:
- Content types
- field base definitions
- field instances
Somtimes, depending on the site, we keep site variables in Features using strongarm module. Most times these variables are related to content types.
In features, we usually avoid:
- menus, & menu links
- Site variables when they are not related to content types.
For collaboration among a large team, we prefer git. Git can be utilized via githhub, bitbucket, another third party service or an internal git server. An "upstream" repository is created that only code reviewers can commit to. Individual developers fork the main repository and set the upstream repo as a git remote. Git remotes allow individual developers to pull changes into their forks from the upstream repository.
- git remote -v - list all git remotes
- git remote add <remote name> <git repo> - adds a git remote to your repository.
Drush and Continuous IntegrationDrush
is a command line tool for managing a Drupal site. It allows easy and quick scripting for many development processes. Since it uses the command line and not the Drupal GUI it is much faster.
Drush commands for maintenence
- drush cron
- drush cc all
- drush dl <modulename> (to download a contrib module)
- drush upc <modulename> (to update a contrib module)
- drush fra (to revert all features)
- drush updb (to run all outstanding update hooks)
Drush commands for iteration
- drush sql-drop (to drop all tables in the database)
- drush sqlc (to enter the mysql command line interface)
- cat db.sql | `drush @alias sqlc` (to import an exported db into the site db)
- drush si (to install a site, rather than using /install.php)
Drush in contrib
- drush fra (and fu, etc.)
- drush solr-delete-index (and solr-index, etc.)
- drush vbo stufffff
- drush bam
Drush with users
- drush uli (to get a password reset link for a user)
- drush upwd jsmith --password=”changeme” (to reset a user password)
- drush urol “administrator” jsmith (to add a role to a user)
- drush urrol “administrator” jsmith (to remove a role from a user)
- drush uinf jsmith (info on a user account)
is your virtual butler. Jenkins can monitor the execution of repetitious jobs, such as deploying a project or running cron. Mainly, we use Jenkins for:
- building projects continuously
- monitoring executions of cron jobs
- syncing production sites down to stage/dev instances for accurate representations and content in those environments
- archiving databases for development use
- Jenkins does not include all options out of the box
- Parameterized Trigger Plugin - This plugin lets you trigger new builds when your build has completed, with various ways of specifying parameters for the new build.
- GitHub Plugin — This plugin integrates Jenkins with Github projects.
Continuous Integration steps we follow with Jenkins
- Commit code to a repo
- Push that code up
- the repo update triggers jobs on Jenkins that are watching the repo
- Code is deployed automatically or manually through job execution
- Drush commands are run to update db and revert features
- Jenkins reports back commit messages, success or failure
Consistent Development Environments
Asking developers to constantly configure their own web servers will result in inconsistencies of experience across the team. To avoid this we use Vagrant to create lightweight portable development environments. Vagrant provides a virtual machine that can mimic your project's production environment. Vagrant completely removes the concept of developers saying, "Why doesn't it work on the servers? It works on my machine."
The solution to scaling Drupal is very similar to scaling any open source php-based software. Depending on our clients' needs, a variety of tuning and software implementation is used. A few examples of scaling softwares are:
Varnish in it's simplest terms is a web application accelerator. Although there are many web accelerator products out on the web, they deal in HTTP, FTP, SMTP and other protocols. Varnish only handles HTTP. It is a reverse proxy cache that is usually only constrained by the physical limits of a network. It is very flexible in that it allows a sysadmin to write policies on how different incoming requests are handled.
There is a Drupal module that helps your site integrate with Varnish.
Memcached is a distributed memory object caching system. Rather than pages, memcached caches object. Memcached allows you to take memory from parts of your system where you have more than you need and make it accessible to areas where you have less need.
There is a Drupal module that helps your site integrate with Memcached.
Another interesting way to scale the performance of your Drupal site is to us a Master / Slave mysql combination for it's persistence layer. You will also see a boost in performance if these servers use SSD's instead of spinning disks.
Hopefully these tips will help you maintain and scale your big Drupal site that requires seamless collaboration between a large team of developers and meet requirements for it to serve content to many users. Phase2 was very proud to have participated in Drupal Camp Costa Rica!