Tuesday, 29 December 2015

How to migrate Parent pom from maven to gradle

Convert Maven parent/super pom to Gradle init file

When you think of moving to gradle for some reason from maven, You need to take care of lot of stuff around it, You can find lot of tools and help to convert maven pom.xml to gradle build.gradle But one of the most important task as an engineer or architect would be to convert your super pom (or parent pom) in to gradle init scripts.

Gradle init script provides same functionality as Maven super/parent pom. The basic difference is that you can call init script

  • Run time
  • As many as of them
This gives us flexibility to change the init script on run time but doubt of not tracking the changes.

Monday, 21 December 2015

Search Engine Optimization (SEO) for blog

Search Engine Optimization 

Adding right content to you website is very important aspects of making your site more useful and relevant. At same time its important that people read the blog and content. How are you going to do that, You need to make sure your content is right and searchable.

I am going to list out few tips to make your blog/site more searchable and (SEO) Seach Engine optimization.

Things to take care of while writing content and improve search result for blog

Wednesday, 4 November 2015

The Rise of the Jenkins CI Butler - by Jonathan McAllister

Author of Mastering Jenkins [Packt Publishing] [http://www.masteringjenkins.com]

Kohsuke Kowaguchi designed and developed the Hudson automation system from the ground up while he was working at Oracle in 2006.  Primitive inceptions of the Jenkins platform (then known as Hudson) simply aimed to support continuous integration practices and offer a viable replacement for CruiseControl, which was the popular CI solution at the time. The Hudson platform was initially very Java centric with tight coupling to Maven and Java development paradigms. Little did Kohsuke know that his internal Hudson tool would catapult in notoriety, change names (into Jenkins), spin off to become CloudBees and eventually become the most popular modern Continuous Integration and automation orchestration platform in use in modern history.

The current Jenkins platform provides an easily consumable unified user interface coupled with a highly scalable distributed automation solution that can build, test, and automate across diverse technologies. Its vast plugin ecosystem allows it to be extended to integrate tightly with other technologies and dynamically expand based on user needs and developer creativity. But this was not always the case. During it's early years the Hudson user interface was not the cleanest and the tool was heavily Java focused. Aside from the Java features the original iterations of Hudson were sub par in comparison to CruiseControl. The major positive feature it did have was a unique extension paradigm.

Thursday, 29 October 2015

Maven Errors - Few common Maven error and troubleshooting part 1

In this post i am going to talk about maven build troubleshooting, Here is the list few maven error and its troubleshooting.

Tag Already exist

If you see following type of error in your release build after the perform step

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-release-plugin:2.3.2:prepare (default-cli) on project : Unable to tag SCM
[ERROR] Provider message:
[ERROR] The git-tag command failed.
[ERROR] Command output:

[ERROR] fatal: tag '<some tag>' already exists

Thursday, 10 September 2015

In house tool vs enterprise tool / open source tool

Tool development and adoption

The current pace of software development cannot manage its speed without the automation.
Automation is the most important element in order to achieve agility in delivery of the product. Every part of the software development life cycle needs tools to perform their action automatically. These phases could be coding, code review, build & deployment, environment management, resource assignment, notification and many others.
There could be 2 options, which a team can opt for:
  • Use a third party tool
    • One can select the tools, which are available in market, either open-sourced or free. You can even buy enterprise solution if needed.
  • Build an in-house tool
    • Tool can be built or developed in-house for the consumption as well.
Lets take look different aspects of choosing options

Tuesday, 18 August 2015

Code Review - Write your code right

Writing code is definitely subjected to individual thinking, logics and perspective. How he writes code is completely depend on his technical and logical skills. 2 different people can write same code in entirely different tactic (May be both logics of the code are right in their own way). Even 2 persons can write bad code as well as good code as per standard.
In real world it is most likely possible that one writes the code and same will be read multiple times by many other developers to fix the bugs, improve the features, understand the code. That is why each language has its own style of coding and guideline to make it more meaningful and understandable.
Most of time the Developers struggle to understand the code written by someone else in team. I have even heard people saying that its difficult to read someone else code rather writing same from scratch.

I would like to highlight few compiled techniques in once place to make writing code better.

Wednesday, 29 July 2015

SVN mirror for Live back-up - High Availability

Other then taking backup at regular interval, we may want to provide keep live data just to make sure data is highly available.

The best option is to backup Live data as soon as it gets changed.

Here is the trick for the SVN Mirroring.

How to create Repository
  • Only user with correct access as admin can create new repository
  • Identify the repository name and cluster. You must follow the approval process before creating any new repository.
  • Create repository svnadmin create /svnroot/<Repo_name>
  • change the owner of repository to apache(if needed) chown -R apache:apache /svnroot/<Repo_name>
  • Now you need to create Mirror repository on another SVN server, following are the steps to create "Mirror repository".

How to create Mirror Repository
  • Take the hotcopy of the repository from 1st SVN server.
svnadmin hotcopy /svnroot/<Repo_name> /tmp/<Repo_name>
  • Remove the hook script if any
  • create a tar file of the repo and scp it to 2nd SVN server.
  • Log in to 2nd server and untar the file.
  • now again do hotcopy to svnroot location on 2nd server
svnadmin hotcopy <Repo_name> /svnroot/<Repo_name>

Git repository backup

Git is distributes system, the beauty of the git make is more robust and easy to recover from any last known good working copy.

However, You can not always rely on such copy and you will definitely need proper backup solution to deal with such situation.

There are many ways to take backup of git repositories.
I am listing few here.

  1. You can directly take a tar of the git repo directory as it has the whole bare contents of the repo on server. There is a slight possibility that somebody may be working on repo while taking backup.
  2. The following command will give you the bare clone of repo (just like it is in server), then you can take a tar of the location where you have cloned without any issue.
    git clone --bare {your backup local repo} {new location where you want to clone}

Saturday, 4 July 2015

Maven to gradle Migration

I am not a big fan of gradle but still this time i am taking this topic.

Maven pom file is xml based build file and Gradle is groovy based system. There are many logic involve to make one to one comparison of generic features. I would like take some generic logic and try to detail out on converting those from maven pom file to gradle build file.

  • Setup Modules
  • define repositories
  • upload of artifacts to maven repository
  • Profiling
    • jacoco test report
  • use of release plugin

Tuesday, 30 June 2015

Too many open files

If system throws "too many open files" error or something like following during build/compilation
[INFO] java.io.FileNotFoundException: (Too many open files)

This means limit to open files for the user or system has been exceeded.

You can check the limit by running following command

ulimit -a

Solution ->
open file common-session
sudo vi /etc/pam.d/common-session

Add following entry in the file
session required        pam_limits.so

open file limits.conf
sudo vi /etc/security/limits.conf

make sure both soft and hard limits are set to larger number (as per your need)
*                soft    nofile          500240
*                hard    nofile          500240

Logout and login and run command again
ullimit -a

You will see the new limit number.

Monday, 29 June 2015

Thrift compilation error

I was trying to compile and install thrift from https://github.com/apache/thrift.
While compiling the thrift for 6 and 8 version i faced few errors and had to spend lot of time fixing it   (Including searching various solutions on Net).

Let me  detail out few error and solution for it.

  • Missing thrifty.h 

src/thriftl.ll:50:21: fatal error: thrifty.h: No such file or directory
compilation terminated.
make[3]: *** [libparse_a-thriftl.o] Error 1
make[3]: Leaving directory `/home/abc/thrift/compiler/cpp'
make[2]: *** [all] Error 2
make[2]: Leaving directory `/home/abc/thrift/compiler/cpp'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/home/abc/thrift'
make: *** [all] Error 2

Solution ->
You will have thrifty.hh file instead of thrifty.h.
Location of file would be 

 cp ./compiler/cpp/thrifty.hh ./compiler/cpp/thrifty.h
and start your compilation again, It should work.
  • Non static declaration error
struct.c:28:1: error: static declaration of âstrlcpyâ follows non-static declaration
/usr/include/ruby-1.9.1/ruby/missing.h:198:20: note: previous declaration of âstrlcpyâ was here
make[4]: *** [struct.o] Error 1
make[4]: Leaving directory `/home/abc/thrift/lib/rb/ext'
setup.rb:655:in `command': system("make") failed (RuntimeError)
        from setup.rb:664:in `make'
        from setup.rb:1258:in `setup_dir_ext'
        from setup.rb:1532:in `block in traverse'
        from setup.rb:1549:in `dive_into'
        from setup.rb:1530:in `traverse'
        from setup.rb:1524:in `block in exec_task_traverse'
        from setup.rb:1519:in `each'
        from setup.rb:1519:in `exec_task_traverse'
        from setup.rb:1246:in `exec_setup'
        from setup.rb:996:in `exec_setup'
        from setup.rb:826:in `invoke'
        from setup.rb:773:in `invoke'
        from setup.rb:1578:in `<main>'

Solution ->
 You will have file name struct.c in  ./lib/rb/ext/struct.c
Comment the line 26(approx) and text static, comment the line with /*static*/
and start your compilation again, It should work.

Thursday, 18 June 2015

How to download Artifacts from maven repo

To make deployment automated, there may be need to get the artifacts from maven repo.
Release artifacts can be downloaded since we know the version But how to download the latest snapshot artifacts.

here are few http call to get  rid of such issues.

How to download latest Snapshot artifact from nexus maven repo

If you know the version you can directly get the artifact like this

wget  http://<Your repo name>/service/local/artifact/maven/content?r=snapshots&g=com.&a=app&v=2.0.25-SNAPSHOT&p=war

Setup Build - SCM System from beginning

This topic is specially for users who are new to the Build, SCM and Release Management and need to understand how can they setup the infrastructure and what are their options.

To Build the system to automate the build and deployment process there are few main areas which needs to be covered. Following are our prerequisite.

- Source code 
 - Builds System
-  Deployments
- QA Automation

To describe above here are main ingredients

Sunday, 14 June 2015

Maven Tips to optimize build timings

Few Maven Tips and tricks to make Maven builds fast and faster and efficiently.
This can help to reduce the build timings drastically and optimize your build timings

  • Run compilation parallel

mvn -T 2 clean compile
#Above will trigger 2 threads

mvn -T 1C clean install
#Above will trigger 1 thread per CPU, if you have 2 CPU core then 2 threads will be spawned.
#You can even give number in fraction (1.5, 2.5 etc).

Sunday, 7 June 2015

Managing open source component

Building any software need lots of efforts including resource, time, money etc. It is really a great pleasure when it goes live or get released. In parallel, fear may come with the doubt of having bugs (even though testing is done). You can fix bugs related to software feature or functionality but the once which can hit badly are
  • Security vulnerability 
  • Licensing risk of open source component.
  • Outdated open source component.

Above checks needs to be done during the development phase and ruled out any discrepancy related to security or legal.
WhiteSource provides one fine platform to solve such issues without putting much effort by developers, So that they can concentrate on core development instead of spending time on finding these issues which can be easily handled by WhiteSource.

WhiteSource is an open source management solution which does

  • Open source licensing and compliance management
  • Open Source security vulnerabilities alerts and management
  • Executive dashboards, policy enforcement, and reporting

WhiteSource checks your software and generates open source inventory report, including detail from open source. The WhiteSource team has good understanding of open source components and they keep the inventory updated. 

WhiteSource covers almost all commonly used languages and provide detail report, This tool can track all your open source components used knowing or unknowingly within your software.

It sends alerts for any potential issue observed in open source component used by software. It does check for outdated/expired component, It provides detail on security issues found in any opensource component.

WhiteSource does not store(keep track of) any software component which is not open source, this means it's safe and your code will not be touched.

CI tool Integration (Jenkins)

They best part of the tool is that it has plugin available for the widely used CI tool - Jenkins.
WhiteSource Jenkins plugin is the best and easy way to integrate with WhiteSource to run the checks during build and integration phase in an automatic way.
Its very easy to use, not even just for Maven project also for free style projects.
Easy to configure and use within project. Only token needs to be added and plugin automatically takes care of everything.
Multiple options are available at job level to define the project and modules to be included.
Logs are pretty descriptive when plugin start processing in Jenkins job.

WhiteSource basic principle talks about
Managing the component vs governing the component.
Leave the managing part up to WhiteSource and only concentrate governance at component level in your software.

Thursday, 9 April 2015

Jenkins Error - "Error while accessing rate limit API" for github pull request builder

If Jenkins throws following error for github pull request builder or for any https call Received fatal alert: handshake_failure (Like following)

org.jenkinsci.plugins.ghprb.GhprbGitHub connect
Can't connect to https://<github-enterprise-server>.com/api/v3 using oauth
 org.jenkinsci.plugins.ghprb.GhprbRepository initGhRepository
SEVERE: Error while accessing rate limit API
javax.net.ssl.SSLHandshakeException: Received fatal alert: handshake_failure

Wednesday, 25 March 2015

Gradle or Maven - A Dilemma

Gradle and Maven (or Both)

The world of build tools evolve from simple compilation (gcc) to make files, then evolution of ant(+ivy) and Maven, now the latest build tool is Gradle.
Its always a puzzle which build tool to choose maven, gradle or both (maven and gradle). Sometime its first time project setup or could be migration from existing tool but it always a debate to choose one of the existing one.
There have been lot of talks and discussion on Gradle, maven (and/or Gradle vs maven). There are Buzz around that Android/google adopted Gradle as its standard build tools.

Monday, 16 March 2015

Maven Repository tools Comparison

There are several maven repository tools available but biggest players are Artifactory and Nexus (sonatype). 
I am taking these 2 repository provider tools for comparison.
Artifactory and Nexus


Main purpose of this document is to compare the 2 maven repository providers (Tools).
  • Artifactory
  • Nexus (Sonatype)
I am trying to review both repo managers. This is on basis of few points like features, cost, support etc.

Saturday, 28 February 2015

Git Repository Corruption - Part1

Git Repo Corruption recovery

The biggest fear of any administrator is the Data corruption. As a Git administrator we do take regular backup, We keep our system HA, We even do disk replication. In short we try every thing to keep data safe and secure.
But still there are chances for the git repository corruption and you have to deal with it in real world.

Here are few repo corruption scenarios and solutions.

Note-> Please make sure you backup everything before touching the repo. There are instruction to fix some of the error. It is always suggested to backup your data before trying to perform any action.