Debugging with git bisect

This is a post  for appreciating “git bisect” and how it can be one of the most powerful tool to find out root cause of a broken build or a broken branch.

Here is simple example of  how “git bisect” can be used to find  a bad commit.

Lets assume that we have a git repository which has hundreds of commits and currently the HEAD of the master branch is broken i.e. has a bug. Our objective is to find out which commit introduced the bug in to the code base.

Before starting the git bisect process we need to know a couple of things. First we need know a good commit i.e. a old commit at which code worked as expected. This is not very difficult to find out as it is most likely the last release of the code.  Also we need to know the steps to test the code and reproduce the issue. It will help us to find out if certain commits are good or bad during the git bisect process.

Git bisect uses binary search algorithm between the good and bad commit to  find out the commit  that introduced the bug.

Here are the commands to start the git bisect work flow. Lets call the current HEAD commit as “original HEAD”

$ git bisect start

$ git bisect bad

$ git bisect good  <commit ID>

Bisecting: 130 revisions left to test after this (roughly 4 steps)

Once the above commands are executed, git bisect will change the HEAD to  the middle of the commits between the “original HEAD” and good commit. Read about binary search if you want to know how it decides to which commit the HEAD needs to be moved.

At this point we are expected to test the code and find out if we are able to reproduce the issue. After the testing we need to again  tell git bisect if it is bad commit (see below) i.e. we are able to reproduce the issue else it is a good commit.

$git bisect bad
Bisecting: 65 revisions left to test after this (roughly 3 steps)


$ git bisect good
Bisecting: 65 revisions left to test after this (roughly 3 steps)

We need to continue the process few times and git bisect will give you the commit which introduced the issue/bug.

In my experience I always get to the commit (which introduced the issue) in 4 to 5 steps of git bisect.  Which I think is an awesome thing.

So go ahead and try git bisect if you have not tried it yet and do not forget to use it when you broken builds.




Creating Live media image

If you are a GNU/Linux user, their is a high chance that you have used live media images i.e. live CD, live DVD or ISO. I was curious about  how live images are created and recently got a chance to do some hands on.

There are multiple tools available for creating live media. However I am going to use livecd-tools to create live ISO. livecd tools need kickstart files to create the images and we will use CentOS upstream kickstart files [1].

As a first step install livecd-tools [2]. I using CentOS Vagrant box to create live ISOs.

$ sudo yum install livecd-tools git -y
$ git clone
$ cd sig-core-livemedia/kickstarts

#You can use any one of the .cfg files in /sig-core-livemedia/kickstarts
$ livecd-creator --config <Kickstart file>

If you are new to kickstart files and want to know more about it refer the documentation:



Running System V init script in CentOS7

RHEL 7 and CentOS 7 moved to systemd from System V init facility.  So if we want to run a script at boot,  it is highly recommended that you should write a systemd unit file.  However to ease the transition to systemd, CentOS7 /RHEL7 offers a backward compatibility mode for init scripts.

In a System V init system you need to add an entry of the script to /etc/rc.local (or equivalent) and the script will be executed during boot at the end of multi user run level.

In CentOS7 /etc/rc.local is a symbolic link to /etc/rc.d/rc.local.

root@localhost ~]# ls -l /etc/rc.local 
lrwxrwxrwx. 1 root root 13 Apr  4  2016 /etc/rc.local -> rc.d/rc.local

To enable the init compatibility mode, you need to make  /etc/rc.d/rc.local executable and then add the script to /etc/rc.local.

Here are the steps you need to perform to to start a script e.g. during boot.

$ chmod +x /etc/rc.d/rc.local

#make the script executable
$ chmod +x

#Make a entry in /etc/rc.local
$ cat /etc/rc.local
# It is highly advisable to create own systemd services or udev rules
# to run scripts during boot instead of using this file.
# In contrast to previous versions due to parallel execution during boot
# this script will NOT be run after all other services.
# Please note that you must run 'chmod +x /etc/rc.d/rc.local' to ensure
# that this script will be executed during boot.

touch /var/lock/subsys/local


Reference :

Using Docker registry in Atomic Developer Bundle

With Atomic Developer Bundle you can easily setup OpenShift on your workstation and OpenShift creates a local docker registry which can be used independently.

To make these commands work, you need to have a working Vagrant setup with Virtualbox or Libvirt/KVM provider.

For this blog I am using ADB 2.1 . I also  believe these instructions will work for CDK 2.0.

$ vagrant plugin install vagrant-service-manager
$ git clone
$ cd adb-atomic-developer-bundle/components/centos/centos-openshift-setup/
$ vagrant up

After the above commands you should get an OpenShift single node setup based OpenShift Origin. If you want to use a specific version of Origin  use the relevant tag from Docker hub in and then do “vagrant up”.

There are various ways of using the OpenShift  from ADB/CDK i.e. web console or through oc binary/command line. I am going to login to the Vagrantbox and start using the docker registry.

$ vagrant ssh

Get the login credential

$ oc whoami -t

Login to the docker registry

$ docker login -u admin -p tF8vQU7xBaM4KA4iKgRmjWFlQex1oKJQr8nwAvblczE -e hub.openshift.c
WARNING: login credentials saved in /home/vagrant/.docker/config.json
Login Succeeded

Pull an image from docker hub and push it to the local registry

$ docker pull fedora
Using default tag: latest
Trying to pull repository ... latest: Pulling from library/fedora
7891603e1bb1: Pull complete 
6932b0d5be7d: Pull complete 
Digest: sha256:cfd8f071bf8da7a466748f522406f7ae5908d002af1b1a1c0dcf893e183e5b32
Status: Downloaded newer image for

Push the image to the local docker registry

$ docker push
The push refers to a repository [] (len: 1)
6932b0d5be7d: Pushed 
7891603e1bb1: Pushed

Now you can pull the image from the local docker registry

$ docker pull
Trying to pull repository ... latest: Pulling from sample-pro
Digest: sha256:eb1987b9de75cd307f22ac8ed73cd20dcc39220904b8274eb2b92b55f383da30
Status: Downloaded newer image for


Get rid of password prompt for Vagrant commands on Libvirt

If you use Vagrant with KVM Libvirt , there is a high chance that you are annoyed by the  password prompts for every Vagrant commands e.g. vagrant up/ssh/destroy unless you are running the commands as root.

The issue is that the a typical Linux user does not have access permission to libvirt socket so to access it we need to provide extra permission.  Interestingly libvirt uses Policy Kit ( man polkit) to decide access permissions and we can add an explicit rule to polkit to give an user privilege to access libvirt.

To fix the issue you need to create a file in /etc/polkit-1/localauthority/50-local.d as mentioned below.

There are other methods to fix this issue too. You can also create a user group, give the group privilege and add the user to the group.

# cd  /etc/polkit-1/localauthority/50-local.d
# cat vagrant.pkla 
[Allow lmohanty libvirt management permissions]


A Docker Workshop

In Fudcon Pune 2015 we had conducted a Docker introductory workshop. It was well attended and  we got positive feedback about it.  While preparing for the Linux container track in Fudcon we had decided that we will put all the documentation in github. The idea was to keep the content open for collaboration so that others can contribute and reuse the content. Thanks to Neependra for the idea.

Here is the github link which has the workshop content.

The github project also contains some useful material e.g. Hands on Kubernetes which you might useful.

This workshop will take around 3 hours to complete. This is really useful If you are new to docker and wants to learn by doing some hands on.

Multi-Container Application Packaging With Nulecule

I got an opportunity to talk about “Project Atomic and Multi-container application packaging”  in recent docker meetup Bangalore. I have posted my slides to slideshare [1].

However I thought of giving more context and further pointers to the presentation through this blog.

As mentioned in my slide, Nulecule is a specification to define a multi-container applications.  So that we can get rid of the custom shell scripts, long docker run commands, moving the required configuration files and instructions about how to deploy the application to the end user.

Checkout the YouTube video about why we need  Nulecule: From the Nulecule Nest to an Atomic App

However just having a specification will not solve the problem. We need a code to do the required work to run multi-container application using Nulecule specification and that is Atomic App project .

Atomic App performs all actions to run the application by reading the Nulecule spec. Atomic App is used as a docker image.

To run the Atomic App installer for your application , atomic command line is used.

Here is the workflow if you want to build Atomic App installer for your application using Nulecule specification.   Please keep in mind that when I say application, I am actually talking about multi-container application.


Write Nulecule specification files. Which also  includes manifesto files for underlying required orchestration platform (e.g. Kubernetes, OpenShift, Docker Compose, Apache Mosos Marathon etc)

Here is a blog from Ratnadeep about how he created a nulecule-ized application ->


Create the layered docker image (use Atomic App docker image as base image) of the application. Which will include Nulecule specification and Atomic App code.

You can push the new image to your local docker registry or public docker hub for your use.


Running the Atomic App image for the application. Here is a example of running a helloapache Atomic App ->

Note that there are three way to run an Atomic App i.e.

  • Option 1: Non-interactive defaults
  • Option 2: Unattended
  • Option 3: Install and Run

Here is a YouTube video which shows a demo : WordPress Nulecule Demo

For further read check  Vasik’s presentation or Nulecule github project.

Get Involved:

Nulecule Poject:
Atomic App:

Check the README files of the above projects for relevent communication channels for participating in the project.