Using JMeter to test performance of Drupal with authenticated users

When building a site in Drupal (or any content management system, really), there are often concerns about how the site will perform. It’s fairly easy to take measures to reduce the impact of anonymous traffic on server resources (cacheing, using CDNs, etc). However, much of what Drupal offers is the ability for users to interact with the site far beyond that of a traditional static site.

Tobby Hagler, Director of Engineering
#Drupal | Posted

When building a site in Drupal (or any content management system, really), there are often concerns about how the site will perform. It’s fairly easy to take measures to reduce the impact of anonymous traffic on server resources (cacheing, using CDNs, etc). However, much of what Drupal offers is the ability for users to interact with the site far beyond that of a traditional static site. So now you have a complete site built, and you’re sure it can handle your anonymous traffic, and you want to test how well your site will perform when users log in and start doing things (such as leaving comments).

JMeter is a tool from Apache that allows you to load test a site. Following the steps in their Recording Tests step-by-step guide (PDF), you can quickly set up a basic test plan to run against your site. You can do things like set the number of simultaneous threads and how many times each thread will run to get an idea of site performance under light, medium, and heavy traffic conditions.

I won’t go into great detail here about how to set up the test plans, since there’s plenty of authoritative material on the Web already. I will assume that, following the instructions in JMeter’s documentation, you can set up a basic test that will run through various pages of your site simulating the actions of a typical anonymous user.

Now that you have an idea how your site will run under normal circumstances, you want to know how it will perform when authenticated users start creating accounts, logging in, leaving comments, and blogging.

Let’s assume that you are creating a JMeter test plan using the proxy service. In your browser, visit your front page, log in (it’s OK to log in as you, we can adjust this later), browse around and visit a number of different pages, and log out. Now you’ll have a test plan that mimics an authenticated user’s basic behavior.

Testing a Drupal Form

On your thread, add a HTTP Cookie Manager. Set the Cookie Policy to “compatibility” and check “Clear cookies each iteration.” This is a necessary step in order to maintain the session cookie Drupal uses for each user. Without this, each step of your test will instantiate a new session.

Because of how the Drupal Forms API works, there are some additional steps that we need to set up when running a JMeter test plan to submit any form. In your existing test plan, find the login form at /user. Add a Post Processor to it called Regular Expression Extractor.

Give it a name, like form_build_id, and set the Regular Expression to id=”(form-.{32})”. Set the Template to $1$ — this refers to the first parenthetical set; in this case, there is only one set, form-.{32}. You’ll need to set the Match number to the number of the form you want. For instance, assume the user login page has two forms: a search box (because the theme has Search on every page) and the user login form. In this case, the Match number will be 2.0. If the only form on the page is the user login form, set it to 1.0. Note that the number is 2.0 and not 2.

Next, find the actual login form submission, which will be something like /user?destination=node. This is a POST request, and we will need to edit some values. Edit the value for form_build_id and set it to ${form_build_id}. This will pass the extracted form_build_id from the previous page and insert the value here and allow the Drupal Forms API to process the form.

Authenticated Users

For the purpose of this test plan, you will need to have a number of test users that will closely resemble your regular users (as in, no special roles, etc.). The best way to do this is with the Devel module, using the Generate Users feature. For the purposes of this test, it is necessary to know the password of each users; in most cases the quickest approach is to set the password for each user to be their username (these are test users that you will remove after this test, after all). In my scenario, all random users started with a UID over 100, so I executed this MySQL command: mysql> <span class="caps">UPDATE</span> users <span class="caps">SET</span> pass=<acronym title="name">MD5</acronym> <span class="caps">WHERE</span> uid>100;

Select Config Elements and add a User Defined Variables element. You’ll want to add some of your random user names as variables (user_1=nopacrawowe, user_2=spofrophew, and so on).

Next, you’ll add another Config Element called Random Variable. You’ll give it the variable name “usernum” and set the minimum and maximum values to 1 and 10 (this will correspond to the number in user_1 in the previous step, so set your range accordingly). You’ll also want to make sure Per Thread is True.

Finally, you’ll need to edit the user login form submission (the same one we edited before). For both the name and pass parameters, set the value to ${__V(user_${usernum})}. This will choose a random number from our range in the Random Variable element, and insert a corresponding value from our User Defined Variables element.

Now you can run your test. You should see random users logging in and out. To confirm this, check Drupal’s logs. One way to do this in a Linux environment is to execute: # tail -f /var/log/messages | grep "Session "

Commenting

Now that you have users logging in and out, it’s time to make these test users do something that authenticated users would do. One of the simplest things they can do is leave a comment on a blog post.

Let’s assume that when you created your test plan via JMeter’s proxy, you visited a blog post, submitted a comment, and were redirected back to the blog post. Since the form_build_id and form_tokens have already been used, your test will not submit anything until we edit the POST request as we did in the user login scenario.

Find your comment submission form page. It may be on the node page itself (such as /node/1234), or on a separate page (/comment/reply/1234). You will need to add two Regular Expression Extractor elements.

The first Regex element will be identical to the user login regex. Be sure to set the Match number to the position this form element appears in the source HTML.

The second Regex element will be for the form_token. This wasn’t needed when logging in, but will in order to submit a comment. Set the Reference Name to form_token and the Regular Expression to id=“edit-form-token” value=”(.*?)”.

Finally, set the send parameters in the POST request as we did before. You will need to edit the form_build_id value to be ${form_build_id} as we did before, and the form_token value to ${form_token}.

As an optional step, you can also use ${form_token} in the body or title of the comment. This way, you can confirm that JMeter is properly inserting comments.

More Users (Advanced, Optional)

At this point, you have a pretty usable test. However, you may want to test much larger data sets to see how your site will be performing in six or twelve months. In this case, I recommend making several copies of your database, which will have 1,000 users, 50,000 users, and maybe even 1 million users.

When handling the user login portion of this test plan, entering users in by hand is impossible. However, there’s a fairly easy way to edit your existing plan. The JMeter test plan is just an XML file, and you can edit it with your favorite text editor. Using the following snippet of PHP, you can have Drupal generate the necessary XML for you. Simply paste the results of the following code snippet in your test plan XML where you currently have your user variable elements.

1
2
3
4
5
6
7
8
9
10
11
$results = db_query("select uid,name from {users} where uid>100 order by rand() limit 2500");
$count = 1;
while ($data = db_fetch_array($results)) {
  print '
  <elementProp name="user_' . $count . '" elementType="Argument">
  <stringProp name="Argument.name">user_' . $count . '</stringProp>
  <stringProp name="Argument.value">' . $data['name'] . '</stringProp>
  <stringProp name="Argument.metadata">=</stringProp>
  </elementProp>';
  $count ++;
}

Tobby Hagler

Director of Engineering