Debugging with git bisect

This is a post  for appreciating “git bisect” and how it can be one of the most powerful tool to find out root cause of a broken build or a broken branch.

Here is simple example of  how “git bisect” can be used to find  a bad commit.

Lets assume that we have a git repository which has hundreds of commits and currently the HEAD of the master branch is broken i.e. has a bug. Our objective is to find out which commit introduced the bug in to the code base.

Before starting the git bisect process we need to know a couple of things. First we need know a good commit i.e. a old commit at which code worked as expected. This is not very difficult to find out as it is most likely the last release of the code.  Also we need to know the steps to test the code and reproduce the issue. It will help us to find out if certain commits are good or bad during the git bisect process.

Git bisect uses binary search algorithm between the good and bad commit to  find out the commit  that introduced the bug.

Here are the commands to start the git bisect work flow. Lets call the current HEAD commit as “original HEAD”

$ git bisect start

$ git bisect bad

$ git bisect good  <commit ID>

Bisecting: 130 revisions left to test after this (roughly 4 steps)

Once the above commands are executed, git bisect will change the HEAD to  the middle of the commits between the “original HEAD” and good commit. Read about binary search if you want to know how it decides to which commit the HEAD needs to be moved.

At this point we are expected to test the code and find out if we are able to reproduce the issue. After the testing we need to again  tell git bisect if it is bad commit (see below) i.e. we are able to reproduce the issue else it is a good commit.

$git bisect bad
Bisecting: 65 revisions left to test after this (roughly 3 steps)

Or

$ git bisect good
Bisecting: 65 revisions left to test after this (roughly 3 steps)

We need to continue the process few times and git bisect will give you the commit which introduced the issue/bug.

In my experience I always get to the commit (which introduced the issue) in 4 to 5 steps of git bisect.  Which I think is an awesome thing.

So go ahead and try git bisect if you have not tried it yet and do not forget to use it when you broken builds.

 

 

 

Creating Live media image

If you are a GNU/Linux user, their is a high chance that you have used live media images i.e. live CD, live DVD or ISO. I was curious about  how live images are created and recently got a chance to do some hands on.

There are multiple tools available for creating live media. However I am going to use livecd-tools to create live ISO. livecd tools need kickstart files to create the images and we will use CentOS upstream kickstart files [1].

As a first step install livecd-tools [2]. I using CentOS Vagrant box to create live ISOs.

$ sudo yum install livecd-tools git -y
$ git clone https://github.com/CentOS/sig-core-livemedia
$ cd sig-core-livemedia/kickstarts

#You can use any one of the .cfg files in /sig-core-livemedia/kickstarts
$ livecd-creator --config <Kickstart file>

If you are new to kickstart files and want to know more about it refer the documentation: https://github.com/rhinstaller/pykickstart/blob/master/docs/kickstart-docs.rst

[1] https://github.com/CentOS/sig-core-livemedia

[2] https://github.com/rhinstaller/livecd-tools

Running System V init script in CentOS7

RHEL 7 and CentOS 7 moved to systemd from System V init facility.  So if we want to run a script at boot,  it is highly recommended that you should write a systemd unit file.  However to ease the transition to systemd, CentOS7 /RHEL7 offers a backward compatibility mode for init scripts.

In a System V init system you need to add an entry of the script to /etc/rc.local (or equivalent) and the script will be executed during boot at the end of multi user run level.

In CentOS7 /etc/rc.local is a symbolic link to /etc/rc.d/rc.local.

root@localhost ~]# ls -l /etc/rc.local 
lrwxrwxrwx. 1 root root 13 Apr  4  2016 /etc/rc.local -> rc.d/rc.local

To enable the init compatibility mode, you need to make  /etc/rc.d/rc.local executable and then add the script to /etc/rc.local.

Here are the steps you need to perform to to start a script e.g. hello.sh during boot.

$ chmod +x /etc/rc.d/rc.local

#make the script executable
$ chmod +x hello.sh

#Make a entry in /etc/rc.local
$ cat /etc/rc.local
!/bin/bash
# THIS FILE IS ADDED FOR COMPATIBILITY PURPOSES
#
# It is highly advisable to create own systemd services or udev rules
# to run scripts during boot instead of using this file.
#
# In contrast to previous versions due to parallel execution during boot
# this script will NOT be run after all other services.
#
# Please note that you must run 'chmod +x /etc/rc.d/rc.local' to ensure
# that this script will be executed during boot.

touch /var/lock/subsys/local

/root/hello.sh

Reference : https://www.centos.org/forums/viewtopic.php?t=48140

Get rid of password prompt for Vagrant commands on Libvirt

If you use Vagrant with KVM Libvirt , there is a high chance that you are annoyed by the  password prompts for every Vagrant commands e.g. vagrant up/ssh/destroy unless you are running the commands as root.

The issue is that the a typical Linux user does not have access permission to libvirt socket so to access it we need to provide extra permission.  Interestingly libvirt uses Policy Kit ( man polkit) to decide access permissions and we can add an explicit rule to polkit to give an user privilege to access libvirt.

To fix the issue you need to create a file in /etc/polkit-1/localauthority/50-local.d as mentioned below.

There are other methods to fix this issue too. You can also create a user group, give the group privilege and add the user to the group.

# cd  /etc/polkit-1/localauthority/50-local.d
# cat vagrant.pkla 
[Allow lmohanty libvirt management permissions]
Identity=unix-user:<USER_NAME>
Action=org.libvirt.unix.manage
ResultAny=yes
ResultInactive=yes
ResultActive=yes

References
[1] https://niranjanmr.wordpress.com/2013/03/20/auth-libvirt-using-polkit-in-fedora-18/
[2] https://ttboj.wordpress.com/2013/12/09/vagrant-on-fedora-with-libvirt/

A Docker Workshop

In Fudcon Pune 2015 we had conducted a Docker introductory workshop. It was well attended and  we got positive feedback about it.  While preparing for the Linux container track in Fudcon we had decided that we will put all the documentation in github. The idea was to keep the content open for collaboration so that others can contribute and reuse the content. Thanks to Neependra for the idea.

Here is the github link which has the workshop content.

The github project also contains some useful material e.g. Hands on Kubernetes which you might useful.

This workshop will take around 3 hours to complete. This is really useful If you are new to docker and wants to learn by doing some hands on.

Multi-Container Application Packaging With Nulecule

I got an opportunity to talk about “Project Atomic and Multi-container application packaging”  in recent docker meetup Bangalore. I have posted my slides to slideshare [1].

However I thought of giving more context and further pointers to the presentation through this blog.

As mentioned in my slide, Nulecule is a specification to define a multi-container applications.  So that we can get rid of the custom shell scripts, long docker run commands, moving the required configuration files and instructions about how to deploy the application to the end user.

Checkout the YouTube video about why we need  Nulecule: From the Nulecule Nest to an Atomic App

However just having a specification will not solve the problem. We need a code to do the required work to run multi-container application using Nulecule specification and that is Atomic App project .

Atomic App performs all actions to run the application by reading the Nulecule spec. Atomic App is used as a docker image.

To run the Atomic App installer for your application , atomic command line is used.

Here is the workflow if you want to build Atomic App installer for your application using Nulecule specification.   Please keep in mind that when I say application, I am actually talking about multi-container application.

Step-1

Write Nulecule specification files. Which also  includes manifesto files for underlying required orchestration platform (e.g. Kubernetes, OpenShift, Docker Compose, Apache Mosos Marathon etc)

Here is a blog from Ratnadeep about how he created a nulecule-ized application -> http://www.rtnpro.com/nuleculizing-an-docker-image/

Step-2

Create the layered docker image (use Atomic App docker image as base image) of the application. Which will include Nulecule specification and Atomic App code.

You can push the new image to your local docker registry or public docker hub for your use.

Step-3

Running the Atomic App image for the application. Here is a example of running a helloapache Atomic App -> https://hub.docker.com/r/projectatomic/helloapache/

Note that there are three way to run an Atomic App i.e.

  • Option 1: Non-interactive defaults
  • Option 2: Unattended
  • Option 3: Install and Run

Here is a YouTube video which shows a demo : WordPress Nulecule Demo

For further read check  Vasik’s presentation or Nulecule github project.

Get Involved:

Nulecule Poject: https://github.com/projectatomic/nulecule
Atomic App: https://github.com/projectatomic/atomicapp

Check the README files of the above projects for relevent communication channels for participating in the project.

[1] http://www.slideshare.net/then4way/project-atomicnulecule
[2] http://www.rtnpro.com/nuleculizing-an-docker-image/
[3] http://www.slideshare.net/VavPavl/nulecule

Cherry pick a PR (pull request) from github

Sometime you might want to test pull requests (from github) in local machine  by cherry picking it. This usually happens before it get merged in the upstream repo and released by the project.

I searched the internet but  did not get good reference about how to do it. After little bit of trial and error I came up with below steps.

Cherry picking a pull request:

For example you want to cherry pick https://github.com/fgrehm/vagrant-cachier/pull/164

Cherry picking a commit: