Running Jenkins Behind nginx

Here at Phase2, we use the Jenkins continuous integration server for many of our projects. Jenkins allows us to accomplish a number of tasks including:

• Running Drush make and Compass to build a full Drupal site on each commit
• Running cron
• Running Behat tests and reporting their results to end users, either with the HTML Publisher plugin or by running them through Sauce Labs and pulling back video with the Sauce OnDemand plugin
• Checking code style using PHP CodeSniffer and reporting violations over time using the Violations plugin

There are many hosts that provide hosted Jenkins environments, including CloudBees and OpenShift. In certain cases, however, we find that we need to run Jenkins in a client’s environment. This could be because it needs to integrate with a client’s LDAP, Crowd, or RSA server, or simply because IT policies require it to be hosted inside the firewall.

Hosting Jenkins

Hosting Jenkins is pretty simple. The Jenkins project provides RHEL/Centos and Debian/Ubuntu packages (including an LTS repository, which you should probably install unless you need the latest and greatest,) and a full-featured Puppet module.

No matter which variant you choose, Jenkins ships with its own servlet container called Winstone that can serve up HTTP traffic itself. The default configuration has Jenkins listening on all interfaces (0.0.0.0) on port 8080. The port and prefix that it listens on are configurable in its configuration file. On RHEL/CentOS this will be at /etc/sysconfig/jenkins, while on Debian/Ubuntu it will be at /etc/default/jenkins.

In many cases, Jenkins benefits from having a reverse proxy in front of it. Apache or nginx can handle sending responses or static files to slow clients and not tie up Jenkins threads. Apache or nginx can also cache static files for a short time, or serve them directly off the filesystem to lower load on Jenkins. Nginx in particular is also very efficient at terminating SSL and is a better choice than Winstone in this regard. In addition, many authentication solutions like CAS or RSA’s WebAgent are tightly tied into Apache or nginx and Jenkins may need to use these same authentication solutions. Jenkins makes this easy with the Reverse Proxy Auth Plugin.

For this example, we will use nginx as the chosen reverse proxy. Unless you have a compelling reason to use Apache, nginx will likely result in using fewer system resources under the same amount of traffic.

Configuring Jenkins and a Proxy

When you’ve decided that you’d like to run Jenkins behind a proxy, it’s easy to do so. This post assumes that you want to run Jenkins at https://example.com/jenkins. To get Jenkins ready to run behind a reverse proxy, we’ll first want to have it stop binding to all interfaces and just have it bind to the local loopback address (127.0.0.1). This behavior is controlled by the httpListenAddress option in JENKINS_ARGS. Making Jenkins available at /jenkins is similarly controlled by theprefix argument in JENKINS_ARGS.

Here is an example of the Jenkins configuration set up suitably for use behind a reverse proxy:

JENKINS_PORT="8080"

JENKINS_ARGS="--prefix=/jenkins --httpListenAddress=127.0.0.1"

With that in place, you also need to configure an nginx server {} block to serve Jenkins. On Debian/Ubuntu, this should be placed in /etc/nginx/sites-enabled/example.com and on RHEL/CentOS this should be placed in /etc/nginx/conf.d/example.com.conf. Below is a simple example configuration.

(If you use this you will need to provide your own SSL certificate and key.)

server {

  listen                *:443;

  # Listen over SSL.

  ssl                  on;

  ssl_certificate      /opt/ssl/example.com.pem;

  ssl_certificate_key  /opt/ssl/example.com.key;

  # Some SSL configuration options removed for brevity.

  ssl_session_timeout  5m;

  ssl_session_cache shared:SSL:50m;

  server_name           example.com;

  access_log            /var/log/nginx/example.com.access.log;

  location ^~ /jenkins {

    proxy_pass          http://localhost:8080;

    proxy_read_timeout  90;

    # Optionally, require HTTP basic auth.

    # auth_basic "Please authenticate to use Jenkins";

    # auth_basic_user_file /opt/nginx/htpasswd;

  }

}

Using this file you should be able to load up https://example.com/jenkins and log into Jenkins. If you make an httpasswd file, you can uncomment the auth_basic and auth_basic_user_file directives and nginx will prompt the user for authentication before letting them access Jenkins. However, if you visit “Manage Jenkins” or use some functions of Jenkins, you may find that you get redirected to http://localhost:8888, which isn’t correct.

Fixing redirects behind a reverse proxy

Getting redirected to localhost:8080 is a manifestation of the “It appears that your reverse proxy set up is broken” issue described on the Jenkins wiki. The issue in this case is that Jenkins expects the reverse proxy in front of to rewrite the Location header, so if Jenkins redirects you by sending Location: http://localhost:8080/jenkins/, Jenkins needs the proxy to transparently rewrite that to Location: https://example.com/jenkins/ before the header is sent to your browser. I banged my head on this a bit when I started using nginx in front of Jenkins since the official wiki does not have this solution listed. Luckily, nginx can easily do this by making use of the proxy_redirect directive.

The proxy_redirect directive takes a pattern as its first argument and then the replacement. A sample working proxy_redirect for this setup could look like this.

1

    proxy_redirect      http://localhost:8080 https://example.com;

For even more resilience, we could use nginx’s built-in $scheme variable to ensure that this block will work without modification whether it is in a server {} block that is serving HTTP or HTTPS traffic, like so.

    proxy_redirect      http://localhost:8080 $scheme://example.com;

That leaves our whole location {} block looking like the following.

  location ^~ /jenkins {

    proxy_pass          http://localhost:8080;

    proxy_read_timeout  90;

    # Fix the “It appears that your reverse proxy set up is broken" error.

    proxy_redirect      http://localhost:8080 $scheme://example.com;

    # Optionally, require HTTP basic auth.

    # auth_basic "Please authenticate to use Jenkins";

    # auth_basic_user_file /opt/nginx/htpasswd;

  }

With this in place, you can enjoy Jenkins behind an efficient reverse proxy.