Downloading Jenkins Logs
a way to analyze Jenkins logs offline

Recently, I encountered a problem on one of the integration test run by Jenkins. This particular test was failing “sometimes”. The problem was that sometimes, the Selenium integration was timing out because a page was too slow, but it was hard to find which part of the test was the one failing. I needed some statistical information on how the test was running, but the one we have on Jenkins didn’t expose that information. As an alternative to change Jenkins configuration, I could analyze Jenkins test logs. Jenkins provides an RSS feed with all the run information, including a URL to a gzip-ed file containing the logs I need. In this article, I will describe the code I create to download this logs files to do further analysis locally.

First, I need the simplest framework I could find just to make a GET request to get the RSS feed, and then to get the gzip-ed log file. In the past, I used scalaj-http for simple REST-full service consumption on scripts. It is a simple-blocking-wrapping of the good-old-java HttpUrlConnection. That will do for this problem. The following is the build.sbt I created for the project:

The last line on the build.sbt is using assembly plugin. This plugin needs to be configured by adding the file /project/assembly.sbt as described here.

To facilitate the HTTP communication, a http.Util object was created containing functions for Basic authentication and GET operations:

Using this functions, the main object JenkinsReader is created:

The code expects to be executed passing Jenkin’s username/password and the output directory as parameters (lines 17 - 19). This code use Lightbend(Typesafe) Config to configure the following data:

The code is very straightforward, here is some description of it:

This code could be better as some errors could be handled (Ex. the code could not write to a particular path), but I hope it will be useful to somebody as it was for me.

Happy coding!

*****
Written by Darien Martinez on 03 April 2016