Automate Git Repository Setup with Python & Fabric
I love Git. I find Git to be an intuitive way to track many versions between many systems. Now all my default config files and scripts utilize Git. However, logging into a server and creating the Git repo, then cloning it is a number of steps that detracts from getting work done.
This tutorial outlines a basic example of using Fabric(Python automation module) to automate the process of creating a ‘mycode.git’ repository and cloning it. This tutorial was written for Cent OS 6.5, but could be adapted for Ubuntu and Fedora. Basic knowledge of Linux and Python 3 is required.
If you need help setting up Python3, pip, and Fabric on CentOS 6 please refer to my earlier tutorial: https://www.savelono.com/linux/how-to-install-python-3-and-pip-3-for-centos-6.html
Goals of this post:
- Write Python/Fabric function to automate the Git setup process
- Write Python/Fabric function to automate cloning Git repository on local host
Introduction to Fabric
Fabric is a Python library and SSH abstraction layer. It’s purpose is to easily automate local and remote shell commands as tasks with Python. Here is a description from www.fabfile.org:
Fabric is a Python (2.5-2.7) library and command-line tool for streamlining the use of SSH for application deployment or systems administration tasks.
It provides a basic suite of operations for executing local or remote shell commands (normally or via sudo) and uploading/downloading files, as well as auxiliary functionality such as prompting the running user for input, or aborting execution
This type of automation is good for me because SSH access is the only common thread for accessing my customers Linux servers. Fabric excels in situations with many servers located on different networks. If all my servers were on one network, I’d probably go with Puppet or Chef or some other “state” based system.
Lets start with the Python modules and default Fabric options. The env.hosts should be set to your Git servers IP address and be configured in a similar manner to my previous tutorial. Also, notice we created a blank class for handling errors called, “FabricException”. We will use this later to capture Fabric: aborts, errors, & warnings.
The “env.*” variables are used by fabric to conduct automation tasks. If left blank you will be prompted by Fabric at execution.
fabfile.py:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 |
#!/bin/python3 from fabric.api import * from fabric.contrib import files # Local git managment automation tools # copyright matt birkland, Savelono.com 2017 # License: GPLv2 # # # Set the username env.user = "git" # Set the password [NOT RECOMMENDED] env.password = "" env.hosts = "10.1.10.10" class FabricException(Exception): pass env.abort_exception = FabricException |
Below is our main Fabric function. Your fabfile.py can have as many Fabric functions as you want but I tend to divide them up by purpose. We will alternate between the “local()” and “run()(for remote tasks)” functions of which make up the backbone of Fabric automation. The function starts by Prompting for a project name, followed by install directory. It does some basic error checking and then displays your entry.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 |
# # newgit pseudo code # ------------ # # - Prompt for name of newcode # - Prompt for install directory # - Test that directory starts with / # - Test for tailing / & remove it # - Display Git Repo build info # - Test to see if project already exsist # - Create git folder and intialize # - Clone to local folder of choice # #------------------------------------------ def newgit(): while True: git_name = input("Please enter name. Example, 'myproject' :") if not git_name: print("You must enter a name. Please try again or quit(ctrl-c).") print() continue else: break print() print() print("Current ~/git/ tree:") print("----------------------------------------------------") local("ls -al ~/git") print("----------------------------------------------------") print() while True: git_dir = input("Please enter folder(default ~/git). Example, '/home/matt/software' :") if not git_dir: git_dir = "~/git" break # check for slash as first character in valid unix directory elif git_dir[:1] != "/" and git_dir[:1] != "~": print("ERROR: {}/{}".format(git_dir,git_name)) print("Please enter valid directory path. Example, '/home/matt/git/somedir' or ~/git/somedir.") continue # check for trailing slash and remove it elif git_dir[-1:] == "/": git_dir = git_dir[:-1] break else: break print("====================================================") print(" Git Server info") print("---------------------------") print("Git Server: {}".format(env.hosts)) print("Git User: {}".format(env.user)) print("Git Project: {}:/git/{}.git".format(env.hosts,git_name)) print() print(" Git Local info") print("---------------------------") print("Git Poject name: {}".format(git_name)) print("Git Directory: {}/{}".format(git_dir,git_name)) print("=====================================================") print() print() |
Now that we have settled on a name we will create the install Git repository in /git. Remember, in my last tutorial(on installing Git) we created a soft link between /home/git/code and /git(to shorten the string when we connect remotely).
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 |
while True: this_ok = input("Is this correct? ('yes' or 'no') :") if this_ok == "yes": try: with settings(warn_only=False): check = run("mkdir ~/code/{}.git".format(git_name)) print() print() break except FabricException: print("Git repo already in use! Please run again and select a different name for the git project") print() exit(1) elif this_ok == "no": print("Exiting...") exit() else: print() print("=====================================================") print(" Please Enter 'yes' or 'no'. Lowercase only please. ") print("=====================================================") print() continue # Initialize Git directory try: with settings(warn_only=False): intialize_git = run("git init --bare ~/code/{}.git".format(git_name)) print() print("Remote Git repo configured!") print() print() except FabricException: print("Unknown error") print() exit(1) |
Finally we will create the local directory(if needed) and clone the repo there!
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 |
# Intialize Git directory try: with settings(warn_only=False): intialize_git = run("git init --bare ~/code/{}.git".format(git_name)) print() print("Remote Git repo configured!") print() print() except FabricException: print("Unknown error") print() exit(1) # Create Project directory try: with settings(warn_only=False): clone_git = local("mkdir {}".format(git_dir)) print() print() except FabricException: print("Directory exist already!") print() # Clone Git repo try: with settings(warn_only=False) and lcd("{}".format(git_dir)): clone_git = local("git clone git@10.1.10.10:/git/{}.git".format(git_name)) print() print("Git repo cloned to {}/{}!".format(git_dir,git_name)) print() exit(1) except FabricException: print("Unknown error") print() exit(1) |
Here is a link to the entire fabfile.py unabridged.
Let’s test it!
[10.1.10.10] Executing task ‘newgit’
Please enter name. Example, ‘myproject’ :mygitproject
Current ~/git/ tree:
—————————————————-
[localhost] local: ls -al ~/git
total 48
drwxrwxr-x 9 matt matt 4096 Jun 20 02:58 .
drwx–x—+ 59 matt matt 12288 Jun 20 14:43 ..
drwxrwxr-x 6 matt matt 4096 Jun 20 03:04 fabric
drwxrwxr-x 3 matt matt 4096 Apr 9 22:48 nflrank
drwxrwxr-x 4 matt matt 4096 Dec 30 01:08 puppet
drwxrwxr-x 5 matt matt 4096 Apr 10 14:10 python
drwxrwxr-x 3 matt matt 4096 Jun 20 03:03 savelono-projects
drwxrwxr-x 4 matt matt 4096 Nov 24 2016 swnpc
drwxrwxr-x 4 matt matt 4096 Jun 2 11:42 voiceip
—————————————————-
Please enter folder(default ~/git). Example, ‘/home/matt/software’ :
====================================================
Git Server info
—————————
Git Server: 10.1.10.10
Git User: git
Git Project: 10.1.10.10:/git/mygitproject.git
Git Local info
—————————
Git Poject name: mygitproject
Git Directory: ~/git/mygitproject
=====================================================
Is this correct? (‘yes’ or ‘no’) :
Type “yes” if the information is as intended.
[10.1.10.10] run: git init –bare ~/code/mygitproject.git
[10.1.10.10] out: Initialized empty Git repository in /home/git/code/mygitproject.git/
[10.1.10.10] out:
Remote Git repo configured!
[localhost] local: mkdir ~/git
mkdir: cannot create directory ‘/home/matt/git’: File exists
Fatal error: local() encountered an error (return code 1) while executing ‘mkdir ~/git’
Aborting.
Directory exist already!
[localhost] local: git clone git@10.1.10.10:/git/mygitproject.git
Cloning into ‘mygitproject’…
git@10.1.10.10’s password:
warning: You appear to have cloned an empty repository.
Checking connectivity… done.
Git repo cloned to ~/git/mygitproject!
Disconnecting from 10.1.10.10… done.
[matt@mattcom1 newgit]$
The directory ~/git exist already, so our exception captured it and the program continued.
our server:
/home/git/code/mygitproject.git
[root@git mygitproject.git]# ls
branches config description HEAD hooks info objects refs
[root@git mygitproject.git]#
Our Git client(from which we executed the fabfile.py):
/home/matt/git/mygitproject
I’ll create a file ‘savelono.py’, add it as a file Git tracks and push it to the upstream repo.
[matt@mattcom1 mygitproject]$ chmod a+x savelono.py
[matt@mattcom1 mygitproject]$ git add .
[matt@mattcom1 mygitproject]$ git commit
[master (root-commit) ea8c083] Add savelono.py
1 file changed, 0 insertions(+), 0 deletions(-)
create mode 100755 savelono.py
[matt@mattcom1 mygitproject]$
[matt@mattcom1 mygitproject]$ git push
git@10.1.10.10’s password:
Counting objects: 3, done.
Writing objects: 100% (3/3), 218 bytes | 0 bytes/s, done.
Total 3 (delta 0), reused 0 (delta 0)
To git@10.1.10.10:/git/mygitproject.git
* [new branch] master -> master
[matt@mattcom1 mygitproject]$
That’s it! Hopefully this comes in use to others that want to automate Git with Python/Fabric. Thank you for reading.
No Comments »
RSS feed for comments on this post. TrackBack URL