Tuesday, 30 June 2015

Too many open files

If system throws "too many open files" error or something like following during build/compilation
[INFO] java.io.FileNotFoundException: (Too many open files)

This means limit to open files for the user or system has been exceeded.

You can check the limit by running following command

ulimit -a

Solution ->
open file common-session
sudo vi /etc/pam.d/common-session

Add following entry in the file
session required        pam_limits.so

open file limits.conf
sudo vi /etc/security/limits.conf

make sure both soft and hard limits are set to larger number (as per your need)
*                soft    nofile          500240
*                hard    nofile          500240

Logout and login and run command again
ullimit -a

You will see the new limit number.

Monday, 29 June 2015

Thrift compilation error

I was trying to compile and install thrift from https://github.com/apache/thrift.
While compiling the thrift for 6 and 8 version i faced few errors and had to spend lot of time fixing it   (Including searching various solutions on Net).

Let me  detail out few error and solution for it.

  • Missing thrifty.h 

src/thriftl.ll:50:21: fatal error: thrifty.h: No such file or directory
compilation terminated.
make[3]: *** [libparse_a-thriftl.o] Error 1
make[3]: Leaving directory `/home/abc/thrift/compiler/cpp'
make[2]: *** [all] Error 2
make[2]: Leaving directory `/home/abc/thrift/compiler/cpp'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/home/abc/thrift'
make: *** [all] Error 2

Solution ->
You will have thrifty.hh file instead of thrifty.h.
Location of file would be 

 cp ./compiler/cpp/thrifty.hh ./compiler/cpp/thrifty.h
and start your compilation again, It should work.
  • Non static declaration error
struct.c:28:1: error: static declaration of âstrlcpyâ follows non-static declaration
/usr/include/ruby-1.9.1/ruby/missing.h:198:20: note: previous declaration of âstrlcpyâ was here
make[4]: *** [struct.o] Error 1
make[4]: Leaving directory `/home/abc/thrift/lib/rb/ext'
setup.rb:655:in `command': system("make") failed (RuntimeError)
        from setup.rb:664:in `make'
        from setup.rb:1258:in `setup_dir_ext'
        from setup.rb:1532:in `block in traverse'
        from setup.rb:1549:in `dive_into'
        from setup.rb:1530:in `traverse'
        from setup.rb:1524:in `block in exec_task_traverse'
        from setup.rb:1519:in `each'
        from setup.rb:1519:in `exec_task_traverse'
        from setup.rb:1246:in `exec_setup'
        from setup.rb:996:in `exec_setup'
        from setup.rb:826:in `invoke'
        from setup.rb:773:in `invoke'
        from setup.rb:1578:in `<main>'

Solution ->
 You will have file name struct.c in  ./lib/rb/ext/struct.c
Comment the line 26(approx) and text static, comment the line with /*static*/
and start your compilation again, It should work.

Thursday, 18 June 2015

How to download Artifacts from maven repo

To make deployment automated, there may be need to get the artifacts from maven repo.
Release artifacts can be downloaded since we know the version But how to download the latest snapshot artifacts.

here are few http call to get  rid of such issues.

How to download latest Snapshot artifact from nexus maven repo

If you know the version you can directly get the artifact like this

wget  http://<Your repo name>/service/local/artifact/maven/content?r=snapshots&g=com.&a=app&v=2.0.25-SNAPSHOT&p=war

Setup Build - SCM System from beginning

This topic is specially for users who are new to the Build, SCM and Release Management and need to understand how can they setup the infrastructure and what are their options.

To Build the system to automate the build and deployment process there are few main areas which needs to be covered. Following are our prerequisite.

- Source code 
 - Builds System
-  Deployments
- QA Automation

To describe above here are main ingredients

Sunday, 14 June 2015

Maven Tips to optimize build timings

Few Maven Tips and tricks to make Maven builds fast and faster and efficiently.
This can help to reduce the build timings drastically and optimize your build timings

  • Run compilation parallel

mvn -T 2 clean compile
#Above will trigger 2 threads

mvn -T 1C clean install
#Above will trigger 1 thread per CPU, if you have 2 CPU core then 2 threads will be spawned.
#You can even give number in fraction (1.5, 2.5 etc).

Sunday, 7 June 2015

Managing open source component

Building any software need lots of efforts including resource, time, money etc. It is really a great pleasure when it goes live or get released. In parallel, fear may come with the doubt of having bugs (even though testing is done). You can fix bugs related to software feature or functionality but the once which can hit badly are
  • Security vulnerability 
  • Licensing risk of open source component.
  • Outdated open source component.

Above checks needs to be done during the development phase and ruled out any discrepancy related to security or legal.
WhiteSource provides one fine platform to solve such issues without putting much effort by developers, So that they can concentrate on core development instead of spending time on finding these issues which can be easily handled by WhiteSource.

WhiteSource is an open source management solution which does

  • Open source licensing and compliance management
  • Open Source security vulnerabilities alerts and management
  • Executive dashboards, policy enforcement, and reporting

WhiteSource checks your software and generates open source inventory report, including detail from open source. The WhiteSource team has good understanding of open source components and they keep the inventory updated. 

WhiteSource covers almost all commonly used languages and provide detail report, This tool can track all your open source components used knowing or unknowingly within your software.

It sends alerts for any potential issue observed in open source component used by software. It does check for outdated/expired component, It provides detail on security issues found in any opensource component.

WhiteSource does not store(keep track of) any software component which is not open source, this means it's safe and your code will not be touched.

CI tool Integration (Jenkins)

They best part of the tool is that it has plugin available for the widely used CI tool - Jenkins.
WhiteSource Jenkins plugin is the best and easy way to integrate with WhiteSource to run the checks during build and integration phase in an automatic way.
Its very easy to use, not even just for Maven project also for free style projects.
Easy to configure and use within project. Only token needs to be added and plugin automatically takes care of everything.
Multiple options are available at job level to define the project and modules to be included.
Logs are pretty descriptive when plugin start processing in Jenkins job.

WhiteSource basic principle talks about
Managing the component vs governing the component.
Leave the managing part up to WhiteSource and only concentrate governance at component level in your software.