Selenium Jenkins - How to Do it Yourself and Sauce Labs Advantage

by Azhar Uddin Technical Architect

So you’re looking to integrate Selenium WebDriver (or RC) with Jenkins, in order to run full end-to-end / UI automation testing as part of your build process? Well, we have good news and bad news:

The good news is that it’s relatively easy to run Selenium tests as part of the Jenkins build, assuming you’re already using a testing framework like JUnit or TestNG. We’ll explain a few ways to achieve it further on this page.

The bad news is that there are quite a few challenges to consider:

Builds with Selenium tests take much longer to run.

You’ll find it hard to cover all the relevant browsers and platforms.

You might have a lot of maintenance work installing browsers/operating systems.

Scaling up your testing will be complex and you’ll need to deploy Selenium Grid and/or rewrite your tests to support multi-threading. Get best Selenium with Jenkins Training.

We at Sauce Labs run over a million Selenium tests every day on our cloud testing platform. We have an easy-to-use Selenium Jenkins Plugin that allows you to run your Selenium tests, with effortless scalability and cool features. (Keep in mind Sauce is a commercial service, but we have a free trial and is free for open source projects.)

If you’d like to do Selenium with Jenkins yourself, using open source software, check out our summary table of Jenkins Selenium solutions, or visit the sections below to see how it’s done and the pros and cons of each solution:

Option 1: Selenium on one machine- headless testing - running Selenium directly on your Jenkins workstation with a headless browser, probably the easiest option.

Option 2: Selenium on one machine - real browsers - running Selenium on your Jenkins workstation with real browsers, limited to the capacity of your workstation.

Option 3: Selenium Grid local- real browsers on multiple machines - running Selenium on several machines on premises, using Selenium Grid, and activating them via Jenkins.

Option 4: Sauce Labs Jenkins Plugin how to setup the plugin and run Selenium Jenkins tests on over 250 browser/OS combinations in the Sauce cloud, with no need to install browsers and set up special infrastructure in house.

Heads Up: Challenges You Might Face When Running Selenium Tests with Jenkins

It’s not fun reading about problems. But it’s even less fun running into them further along your project! So we hope to save you a bit of pain with our take on Selenium Jenkins integration challenges.

(This discussion is based on our experience with over 500 million Selenium tests run on the Sauce Labs cloud testing platform - many of them through CI platforms.)

SELENIUM TESTS TAKE A LONG TIME TO RUN

It’s true that in a typical project, there can be thousands of unit tests but only dozens or hundreds of Selenium tests. Nevertheless, those Selenium tests will take much much longer to run than all your unit tests put together. Selenium tests run on a real browser, they need to perform actual browser operations and often wait for an HTTP server to respond. Plus starting and stopping the browser takes time - you get the picture.

The implication is that if you include Selenium tests as part of your build, the build will take much longer to run - so that if currently you’re running a build on every commit, or several times a day, you may have to resort to running the build overnight, and you might need to upgrade your Jenkins workstation or even add more machines to your Jenkins cluster. There are two ways to achieve this:

1. Setting up a local Selenium Grid.

2. Running your tests on Sauce Labs with the Sauce Jenkins Plugin.