Dec
09
2018

How to setup a Python-Jenkins API connection with Python 3

Jenkins Dashboard

I’m working a project to create Python program that scrapes jobs from a Jenkins server, with the end goal of storing them in a Mongo NoSQL database as an archive.  At work we have more than a dozen build servers that will ultimately will be consolidated to a few. 

My plan(since we are moving to a cloud service)  is to use a Jenkins Docker container hosted from a cloud service to dynamically pull the jobs(from my MongoDB) and run builds on demand!

I decided to use the python-jenkins module, but immediately ran into problems.  Nothing serious, just tracking down a few libraries to build the Python modules acquired via pip3.  Afterward I tried to connect to a Jenkins server only to be denied with a 401 ‘invalid password’ error.  I spent all day trying to figure this one out, but there is little online to help.  All the examples I saw were Python2 with HTTPS configurations.  

Goals of this Post:

  • provide a clear example of connecting to Jenkins HTTP REST API via Python-Jenkins

Extra details:

  • Python 3.4, 3.5, 3.6 tested from Fedora 27 Client 
  • Python 3.5 tested from Fedora 24 Client
  • CentOS 6.10 – Jenkins Server ver 2.154



The Python-Jenkins module is one of the first search results you will encounter looking online.  The directions on it’s website seem geared towards Python 2 because the directions show the old way of handling string substitution.  I made some minor adjustments to be Python 3 compatible.

This tutorial is only about creating a Jenkins REST API connection object with the Python-Jenkins module using Python 3.  This tutorial does not cover Jenkins server setup or Python 3.4 and 3.5 environment.  Nor do I delve into the methods or use cases.

First install the Python-Jenkins.  I use pip3, but you might want to use the package manger of your favorite distro.  Also, note that your distro’s package manger might name it differently, than the Pypi package name. 

For example in might be called “python34-jenkins” in yum or apt.  Make sure you are installing the correct lib as the link at the top of the page.

[root@mattcom1 ~]# pip3 install jenkins
Collecting jenkins
Downloading https://files.pythonhosted.org/packages/2c/dd/b65fd97de6c3a4aa74d1a92760291b7cd30d84e9729254c6e4cc376cc360/jenkins-1.0.2.tar.gz
Installing collected packages: jenkins
Running setup.py install for jenkins … done
Successfully installed jenkins-1.0.2
[root@mattcom1 ~]#

Drop out of root and test it,

[root@mattcom1 ~]# exit
logout
[matt@mattcom1 Desktop]$ python3
Python 3.6.5 (default, Apr 4 2018, 15:01:18)
[GCC 7.3.1 20180303 (Red Hat 7.3.1-5)] on linux
Type “help”, “copyright”, “credits” or “license” for more information.
>>>import jenkins
>>>

The import works without error!  Good!  Lets create a function that will create a  connection and return the version of Jenkins.

Then I executed my script,

[matt@mattcom1 jenkins_backup]$ ./jenkinsconn.py
connecting to Jenkins…
Traceback (most recent call last):
File “/usr/local/lib/python3.6/site-packages/jenkins/init.py”, line 829, in get_version
response = self._response_handler(self._request(request))
File “/usr/local/lib/python3.6/site-packages/jenkins/init.py”, line 533, in _response_handler
response.raise_for_status()
File “/usr/lib/python3.6/site-packages/requests/models.py”, line 935, in raise_for_status
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: http://10.1.11.231:8080/
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File “./jenkinsconn.py”, line 69, in
non_working_jenkins_conn()
File “./jenkinsconn.py”, line 58, in non_working_jenkins_conn
version = jconn.get_version()
File “/usr/local/lib/python3.6/site-packages/jenkins/init.py”, line 835, in get_version
% self.server)
jenkins.BadHTTPException: Error communicating with server[http://10.1.11.231:8080/]
[matt@mattcom1 jenkins_backup]$

The HTTP 401 or 403 error appears.  So I tried from a Python 2 CLI, with the same result.  I tried again, on a lightly used production Jenkins server and I still got a 401.  So, I created a new Jenkins user and tried a fresh token, still the same dreaded 401!  Finally in act of random desperation I formatted the request as,

I tried my script again,

[matt@mattcom1 jenkins_backup]$ ./jenkinsconn.py
connecting to Jenkins…
Hello from Jenkins 2.154
[matt@mattcom1 jenkins_backup]$

Voila!  It worked!  An entire day figuring this out!  I hope I saved you some time!  Thank you for reading!  Below is the working script, properly formatted for Python 3/python-jenkins.





No Comments »

RSS feed for comments on this post. TrackBack URL

Leave a comment