Continues Integration in DevOps

We have seen in the past article on the high level of the continues integration is the process of automated build, deployment and publish the code on the repository tool in order to continuously validate the code.

In the open source project development world, The continuous integration is very well implemented through GitHub repository tool with team collaboration tools like slack, open source communities, chat, etc..

Here in this scenario, the developers initiate the project in their own machine IDEs (most of IDE has the integration with GitHub repositories tool) and once the code has been developed and by clicking the build button on the IDE tool, it will get the code be deployed on the branch folder in the repositories. There are audiences who watch that particular project, clone that branch folder into their own environment and execute the result in the form of core functionality test by x person, security scan by y person & code review by z person who actually initiates this open source project. It is either developer do unit testing by his own hand or the watcher do the test and update the test report to the developer with feedback from team communities. Also, each and every project followed with versioning, as some may be happy with the version xxxx.0.1 that they use in their production zone or some audience might be expecting additional features to be released on the next version xxxx.2.0.

The outcome of collaborative working culture is to encourage the developer to share their code on the common repository tool to integrate their code with other’s code and also share the unit test case scenario for the peers to run on the same code and get the feedback of test result/report. As the result, it delivers the quality software code to the end users.

With that same approach be followed in the in-house development environment, the continuous integration ensures to go the next steps of continuous delivery.

Implementing the continuous integration:

The continuous integration relies on three foundational elements such as

  1. Version control system
  2. Continuous integration system
  3. Automated build process

Version Control System:

Earlier days the version control system was handled by the team under VCU (version control unit) who were responsible for maintaining the codes in folders structured like Branch & Trunk based on the environment such as Test, UAT & Production. Such manual code maintenance was followed with the paper-based process like the developer has to get the approval from respective team lead or manager in the code-release form which has to be submitted to VCU team, who collect the code from the developer and keep it respective environment folder. Here, if a developer does even small changes on the fly in any of one environment then the respective changes must be submitted manually to the VCU team.

Due to this whole manual process, there was no concept of code sharing and packaging. Also, If person “A” do changes on login.jsp while person “B” already doing changes then it is complete VCU team responsibility to ensure code integration happened during the deployment.

Centralized version control system:

To have all these processes automated and properly maintained the code, the organization start using centralized version control tools such as TFS (Team foundation server), SVN (Subversion). In this centralized version control systems, all the codes are maintained. The centralized tools leverage the developer to check-out the code for code changes and also check-in into the branch for the code update. In this centralized version control process, the pull & push request of code will keep sync into the centralized server, so if another developer does check-out to his local machines, they will get a notification that this particular file is been changed by person “A”. In this type of tools, it has the user interface maturity, single repository to maintain the code as common and enable the SCM (software configuration management) team to have proper access control policy, so well used for audit purpose.

Specifically this kind of centralized version control system well suited for large base code, which in another term widely used in the organization that follows the waterfall methodology.

Distributed version control system:

The distributed version control system comes into practice when organizational changes their methodology from waterfall to agile. As the agile process in place, the large code base divided into the smaller code base, so each developer of the respective team like (front-end, backend & database team) can clone the common code on their development environment and they can start working on the code by themselves.

The Github is currently distributed open source repository tool, which leverages the developer to fork the code base from another developer’s repo who shared their code in the public and also it has well access control policy as the repo owner can completely control the master node of who can do commit. Since it is distributed format branching of code is very easy to follow.

Currently GIT is famous repository tool in the open source world, as almost all IDEs has the proper integration with this GIT and also the CI tools such VSTS, Jenkins has the tokenization method to access the GIT, The process of clone, push the code commit to master node, see the history, user stories on repo changes & git hooks features associated with these kind of distributed tools.   

Git automation:

The GitHub distributed version control has the special feature of git hook which will perform client-side verification on the file before we commit the code in the common repository. In another term, we can add set of shell scripts at client side to perform pre-commit verification for sensitive information on code & comments, auto-fix the code alignments, rearrange the configuration file, remove unnecessary spaces in the code. Also with git hook server-side verification functionality, introduce the code reviewer just before the code gets committed & auto-merging based on the certain commit policies.

Continuous integration system:

As the DevOps process make the organization succeeded in term of quality product delivery to the customer in the fastest manner. CI (continuous integration) is the important approach which benefits the developer to perform hassle-free development without worrying development environment design, platform certainty, release notes preparation, package management, unit test integration & test report preparation. The CI system fully takes care of steps from code build to code deployment and also it notifies the whole group about the result of code that checked-in recently.

Automated Build process:

The main focus of continuous integration system is to encourage the developers to concentrate on developing the code with new ideas. While this CI system establish the pipeline with features of,

  1. Get the source code from repositories.
  2. Build solution to compile the code
  3. Pack the binaries for deployment.
  4. Deploy the project code to CI server.
  5. Test execution as per the test case design
  6. Publish the test/build report to members of the team.

Continuous integration tools behavior:

There are many CI tools currently available in the market, out of which VSTS (visual studio team service), JetBrains TeamCity & Jenkins CI are well popular and widely used tools in the organization.

Most of CI system designed with agent & queue concept. So when the schedule or trigger happens upon source code commits in repository tool. The CI system will initiate the build definitions with help of queue management and dispatch the build configuration to the agent server (CI) for the project specific build to be executed. Also, every CI system has the plugins for Maven, Gradle, Ant, MSBuild, Junit, Pylint, etc for automated build, unit test, functional test and regression test.

Build Definition/Project build:: The Build definition/project (each tool uses different terms) can be easily created either through webUI or files in the form of Json or YAML files. The build definition generally consists of Agent, environment variable, stages and steps to follow on each stages.

Agent: The agent can be any of local system, virtual machines , containers or on cloud.

Environment variable : It has the values that globally used on each stages in the CI pipeline.

Stages: The stages highlight the build, test, deploy and deliver the result or artifacts.

Steps : Each steps under the stages are classified and derived according to the stages goal. For eg: The steps for stage “Build” are MSbuild, Maven-build or even simple unix shell/ windows powershell script to compile the code.

Moreover, the CI tools has multiple option to verify your build, Build status notification, code analysis, code comparison, stories about the check-ins and digital code approval in place.

Benefits of CI:

  • Improving code quality based on instance test report
  • Auto Triggering / constant scheduling for automated testing upon every code change
  • Reducing build times for rapid feedback and early detection of problems (risk reduction)
  • Better managing technical debt and conducting code analysis
  • Reducing long, difficult, and bug-inducing merges
  • Increasing confidence in codebase health long before production

Monitoring the CI

If the continuous integration to be added in the pipeline of continuous delivery, It has to be validated whether the continuous integration process setup correctly and whether it really gives the benefits the organization.

WIth the below steps in mind to measure our continuous integration to go into full mile of continuous delivery.

  • Do the developers commit the code multiple times in a day into the repository tool.
  • At Least one commit happening on the branch node in the repository tool to automatically publish the code on the CI environment (development env) where automated code built and unit test happening.
  • Whether CI/development env have updated with proper auto build trigger and auto unit test scripts.
  • Do the developers given priority to the unit test result.
  • With the CI environment test result, whether developers improve the codes and fix.

Leave a Reply