Running Behat tests on BrowserStack

Behat, in combination with Mink, allows for running tests against a variety of browser emulators. One of these is Selenium. BrowserStack is a service that sits on top of Selenium, and allows for testing a wide variety of real browsers on real operating systems. Behat (by way of the Mink Extension) has natively supported BrowserStack for some time, but the documentation is limited and outdated. To get started, you'll need a BrowserStack username and access key (available at https://www.browserstack.com/accounts/automate once logged in). You'll need the Selenium 2 driver, as the BrowserStack-specific items simply wrap that driver. Your composer.json file might look something like this:

{
  "require": {
    "behat/mink": "~1.5",
    "behat/mink-goutte-driver": "~1.0",
    "behat/mink-selenium2-driver": "~1.1",
    "behat/behat": "~3.0,>=3.0.5",
    "behat/mink-extension": "~2.0"
  }
}

If you're using Drupal, and the Behat Drupal Extension, it can be as simple as:

{
  "require": {
    "drupal/drupal-extension": "~3.1"
  }
}

...since the Behat Drupal Extension already requires the various drivers and extensions above. Since the BrowserStack driver is a wrapper to the Selenium 2 driver, much of the configuration will be the same. Here is a sample behat.yml file with some BrowserStack-specific configurations:

default:
  suites:
    default:
      contexts:
        - FeatureContext
        - Drupal\DrupalExtension\Context\DrupalContext
        - Drupal\DrupalExtension\Context\MinkContext
  extensions:
    Behat\MinkExtension:
      show_cmd: "open %s"
      base_url: http://local.dev
      browser_stack:
        username: 'USERNAME'
        access_key: 'ACCESS KEY'
        capabilities:
          browser: 'firefox'
          browserVersion: 42
          # This is required for local testing.
          browserstack-tunnel: true
      goutte: ~
      javascript_session: 'browser_stack'
    Drupal\DrupalExtension:
      blackbox: ~
      drush:
        alias: 'local'
      drupal:
        drupal_root: './build/html'
      api_driver: 'drupal'

Note the browserstack-tunnel option. This is needed if the site you are testing is running on your local server. BrowserStack has documentation on running browsers against local servers. Once you've downloaded the binary to enable this, that just needs to be running in a terminal while Behat runs (./BrowserStackLocal  {ACCESS KEY}).Once you have your tests running, they can be watched in real time (or played back later) via the Browserstack dashboard. For instance, given this simple feature:

Feature: Demonstrate Browserstack capabilities
  @api @javascript
  Scenario: Administrators can manually run cron
      Given I am logged in as a user with the "administrator" role
      And I am on the homepage
      And I click "Manage"
      And I click "Reports"
      And I click "Status report"
     Then I should see the link "run cron manually"

In addition to all of the normal capabilities of the Selenium 2 driver, BrowserStack adds operating system, device, and others. To test a wide variety of browser and version combinations, using a continuous integration server to automate this by way of environment variables will reduce the complexity of the behat.yml file. If this is not possible, using Behat suites to configure different browser combinations would also be an option. If using Travis CI, this is an example from a .travis.yml file to setup multiple different browser versions:

env:
  - BEHAT_PARAMS='{"extensions": {"Behat\\MinkExtension": {"browser_stack": {"capabilities": {"browser": "Chrome", "browserVersion": 45}}}}}'
  - BEHAT_PARAMS='{"extensions": {"Behat\\MinkExtension": {"browser_stack": {"capabilities": {"browser": "Firefox", "browserVersion": 42}}}}}'

This will run all tests in both Firefox and Chrome. Further combinations involving different operating systems could be added in a similar fashion.For more information on Behat and Drupal, see the documentation.