Locally hosted DevOps Part I: Setting up VMs and a Git Server

I’ve been playing around with DevOps related stuff this weekend and I thought I’d share some progress.

The end goal is that I want to set up a toy DevOps pipeline from code through to git through to Kubernetes cluster deployment using Rancher, all hosted on virtual machines on my local machine. The reasons for this are… somewhat unclear but I feel like it will give me a better understanding of all this containerization and micro-services stuff that I hear so much about.

TL; DR

This is a long post. The contents are:

• Setting up a virtual machine
  • Choosing which virtualization to use (Use QEMU/KVM, there’s no real need to think here)
  • Downloading a Linux ISO for the VM OS.
  • Setting up the VM with storage and so forth
  • Assigning a static IP to the VM
• Creating a git repository on the virtual machine
  • SSH access to the VM
  • Creating a bare git repository
  • Cloning it on another machine

This is mainly a prelude to the later posts where we’ll get to the real devops stuff which is

• Build automation
• Kubernetes deployments

You can safely skip this post if you prefer to use a service like GitHub or BitBucket. I just wanted to host the entire pipeline on local which is why I went a bit mental here.

Creating virtual machines

This process may vary for you depending on your local setup.

Setting up virtualization

I’m running Manjaro Linux as my host OS and I set up QEMU/KVM virtualization long ago, together with the graphical virtual machine tools that I need. There are a few blog posts you can follow to get this process completed, I won’t lie it was a pain in the neck and I did it long ago. Also it tends to vary depending on your specific OS so you kinda need to google around. Here is a recent one for Manjaro if you want a starting point.

I’d certainly recommend QEMU/KVM but you can also try VMWare and VirtualBox as alternatives.

Setting up an Ubuntu virtual machine

The main thing you want to do is download Ubuntu 18.04 LTS rather than 20.04 LTS – I did try with 20 first but the problem was installing Docker didn’t go so smoothly, and I kinda want Docker as part of the base setup for the VMs.

You may wonder why Ubuntu rather than Manjaro – it’s just me mucking around, but also Ubuntu is like… the bog standard most boring Linux distro that there is, and boring is good when you’re doing server stuff. Also it’s a supported distro for Rancher which I’m planning on exploring. ivermectina para perros contra garrapatas

I chose to download Ubuntu server ISOs for the VMs. Ubuntu server is just Ubuntu minus all the user space applications and graphics stuff, so it runs faster and takes up less space. ivexterm gotas

OK so you’ve downloaded the ISO, now what. Setting up a VM with the graphical tools on Manjaro is super easy.

• First you open the Virtual Machine Manager.
• Click the + button to add a new VM.
• Select “Local Install Media”. ivermectin dosage calculator
• Browse to find your ISO.
• Depending how much memory you have available, try to give the thing a few CPU cores and at least 4GB of memory. I went for 4 CPU cores (I have 8 available) and 8 GB memory.
• It defaults to creating 20 GB storage. I usually go for qcow2 storage for my VMs cos they can grow as the VM grows and are not all allocated immediately, so my host OS doesn’t suffer.
• The rest you can leave as defaults.

Once you create the VM it will boot up and you’ll need to complete a normal Ubuntu server install configuration, together with creating a root user account etc. Make sure that at this stage when it asks you about SSH you say “Yes Please”.

Granting a static IP to the VM

Because I’m kinda setting up a network of virtual machines simulating a network of several different servers, I want them to have static IPs so that when I configure e.g. a git repository on my Git server and clone it on another machine, the links won’t get broken the next time I start up because of the machines having a different IP address.

I’m following this blog post to perform this procedure, but for the record here’s how I did it on my machine.

First, you can see IP addresses for your virtual machines like this:

[davido@david-pc ~]$ sudo virsh list
[sudo] password for davido: 
 Id   Name            State
-------------------------------
 1    Manjaro         running
 8    ubuntu18.04     running
 9    ubuntu18.04-2   running

[davido@david-pc ~]$ sudo virsh domifaddr ubuntu18.04-2
 Name       MAC address          Protocol     Address
-------------------------------------------------------------------------------
 vnet2      52:54:00:60:c1:44    ipv4         192.168.122.155/24

The two important points there are the IP address and the MAC address for the named VM.

To set the named machine to have a particular IP address is as simple as this:

sudo virsh net-update default add-last ip-dhcp-host \
     "<host mac='52:54:00:60:c1:44' name='git-server' ip='192.168.122.155' />" \
     --live --config

You can check that the call succeeded with:

sudo virsh net-dumpxml default

Look for the section that looks like this:

  <ip address='192.168.122.1' netmask='255.255.255.0'>
    <dhcp>
      <range start='192.168.122.2' end='192.168.122.254'/>
      <host mac='52:54:00:60:c1:44' name='git-server' ip='192.168.122.155'/>
    </dhcp>
  </ip>

Putting SSH access in place

For this section of the blog post and the next I followed this blog post here, so you can do that too if you prefer to go direct to the source.

In order to access the git server via SSH, our first step is to create a user on the Ubuntu VM we’ve been working with.

sudo useradd git -m
sudo passwd git

This will add a user called git to our VM and set their password. The -m option is so that a home directory gets created for them, as this is not guaranteed on Ubuntu server.

Now we create SSH keys on the machine we want to access the server from, in my case that’s the host OS.

ssh-keygen -t rsa

I did not set a password on the ssh key that was generated, I left most of the settings as default.

Next we copy that key to the Ubuntu server using the user and password we just defined. So running this on the Host OS (Manjaro) I did this:

cat ~/.ssh/id_rsa.pub | ssh git@192.168.122.155 \
    "mkdir -p ~/.ssh && cat >>  ~/.ssh/authorized_keys"

Now we can test if have SSH access like this:

ssh git@192.168.122.155

Spoiler alert: it worked. Stay logged in on SSH for the next bit.

Creating a bare git repository on the server

So to create a git repo on the server it’s as easy as entering these commands at the SSH command prompt:

mkdir -p ~/projects/base-microservice.git
cd ~/projects/base-microservice.git
git init --bare

The reason this is a “bare” repository is that it does not have a working directory. No one is gonna be doing development on the server, we’re just gonna clone the repo to our local and push to the server.

Let’s clone the repo to local and do some dev work on it. exit will get us out of the git server SSH.

git clone ssh://git@192.168.122.155:/home/git/projects/base-microservice.git
cd base-microservice

We can do a test push:

echo "Hello World"  > hello.md
git add .
git commit -m "Test commit"
git push

Conclusion

So this post was basically about setup. After this the next steps are setting up build automation and deployments, which… we’ll cover in the next few blog posts 🙂