Tuesday, 31 January 2012

Hibernate criteria query | restriction projections Order by examples

Hibernate criteria API:-

criteria are used to select the specific type of data from the database.
Let us take the usecase, where website has search capability provided, you have to search the website with different conditions, To handle this using hibernate, we can compose different parameters to Criteria object and make custom sql queries to database to retireve the data.

Hibernate criteria API is alternative to Hibernate query language or HQL and generate complex queries using different crierias. Here we can not write much sql or hql queries instead we can add the different critiera parameters, generates combatable sql queries

Criteria can be created using Session Object.

Create Criteria object using Hibernate API:-

Hibernate provides org.hibernate.Criteria to create criterias on a persistent object. This object is created from the avilable session objects.

HomeEquity is java persistence object which is mapped to home_equity in hibernate configuration.

Criteria equityCriteria = sess.createCriteria(HomeEquity.class);
 List homeEquitys = equityCriteria.list();

The above command retrieves all the rows in the table_homeequity table and store this all the rows in the list of HomeEquity Objects.

The following is the equvalent native SQL query that executes at database level is

select * from home_equity
The above results all rows in the table, instead if we wan to restrict only 20 records, then we have to configure number to restrict the data using Criteria.setMaxResults(Integer no)

Criteria equityCriteria = sess.createCriteria(HomeEquity.class);
List homeEquitys = equityCriteria.list();
the corresponding SQl query executed in different databases are

select * from home_equity limit 20  
For Oracle
select * from home_equity rownum <=20 

Sort the objects using Criteria Order by query:-

To order the list of objects with based on loan amount ordered by asecending order. we have to used criteria.addOrder method.
we can also add conditioninal order by parameter to criteria

Criteria equityCriteria = sess.createCriteria(HomeEquity.class);
List homeEquitys = equityCriteria.list();
The equivalent query executed at database is

select * from home_equity order by loan_amount asc

Criteria Distinct Query for retrieve unique objects:-

We have many number of approaches to retrieve the distinct result

First approach does the unique object filter in two steps, one step is retrieve all the objects into the memory, and next step is to filter for required unique nes
second approache using criteria projects
the qual native sql executed at database is 
select distinct loan_type from home_equity

Criteria Restrictions Query for comparison of objects:-

hibernate Criteria API provides Restrictions class which has built in methods(ne,eq,isNotNull,gt) to do comparison operators.

equityCriteria.add( Restrictions.like("name", "A%") );

the equal native sql query executed at Database is
select * from home_equity where name like 'A%'
equityCriteria.add( Restrictions.lit("loan_amount", "100000") );
the equal native sql query exected at database is
select * from home_equity where loan_amount <10000

equityCriteria.add(Restrictions.between("loan_amount", "10000", "50000"));

natvive sql for the above criteria is
select * from home_equity where loan_amount between 10000 and 50000
Hope you understand the basic criteria queries. This is part 1 for hibernate criteria queries

Read More

maven assembly plugin example with tutorial in maven2

Maven assembly plugin in maven2:-

Maven is project build management tool , it automate the build phase of any java

projects.whenever the project is build, it crease single modules with the format of jar or war.
so it builds and zip your project including configuration files and other xml files into different customized output module types lik zip,jar,wat and other tar.gz formats.

As you know maven-war-plugin generates only war files, but in the projects, you need to archive in different formats like zipped formats, in that case we will use this plugin.

usually any project builds either war or jar file. so if the project(artifact name is CloudHadoop), the result of your install command is CloudHadoop-1.0.0-SNAPSHOT.war. More over, it generates single war file.

But using this plugin, we can multiple zip files like jar,zip with one exection

if you want to generate a zip file (CloudHadoop.zip)of your project source code with it's dependencies as well as configuration files,There are three task involved to use  the maven-assembly-plugin in our project.

1.Configure maven-assembly-plugin plugin
2.Create custom assembly file
3.Run assembly maven goal command

Configure maven-assembly-plugin plugin:-

Maven is plugin based architecture so you need to configure plugin if we want to support any extra functionality.

Add the following plugin code in your pom.xml


maven-assembly-plugin is plugin is defined with plugin element with groupId and version
descriptorRef represents the type of folder structure in zipped bundled file.

the possible elements for descriptorRef are jar-with-dependencies(jar format like MANIFEST),project(zipped in project directory structure),src (only the src folder with sub folders)

descriptor specifies the cusotmized assembly xml which contains the code

execution phase represents the goal where this assembly code is executed

2.create custom assembly descriptor file:-

we can define our own custom assembly file to create our module. and the file name is zipCode.xml. you can define multiple assembly files by configuring this files in plugin declaration.

format specifies the result for format of you zipped file.
FileSets defined the files in the source directory of your project. and target directory is zipDirectory after your package goal is run.

Building your project with assembler:-

assembele plugin can be run in two ways.

one way with independent way of execution.

  mvn assembly:assembly 
.once the above goal is executed, it crease zipped file which contains files in the form of same project directory strucutre.

and other ways is attach the plugin execution with any predefined goals.

maven package

How to create multiple jar files with maven:-

listing down highlevel steps to generate multiple jar files with in single execution

1.maven-assembler-plugin configuration in pom.xml
2.define multiple exections element in maven-assembler-plugin plugin configuration and configure multiple fule custom assembler xml files
3.For each jar, define own custom assembler file with format type and source and target directories information and also include include and exclude elements
4. define the goal execution as single or attached with any predefined goals
5.Finally build your project using maven command in the command line.

Hope you understand basic idea on maven2 assembler plugin with basic example.
Read More

Monday, 30 January 2012

Best String examples with tutorials in java

String class or object in java

Strigns are sequence of characters in any language. For java, sequence of characters are stored in the form of java.lang.String object.

Strings are immutable objects, meaning once created, the contents of the string, can not be modified in memory that means string objects are readonly.

String class is final class meaning we can't able to overried the behaviour of the methods defined in strings
why String objects are read only:-
String s="hadoop";

In the above example, string 's' is created with value as hadooop, in the next line, appending the string with cloud that means, string "cloud" object is created,
and the new string is created with value 'cloudhadoop' which is referenced by s, original value "Hadoop" is unreferenced

ways to String object creation:-

there are many wasy to create a string object in java using overloaded constructors

String cloudString=new String(); string object created without value

String cloudString=new String("cloudhadoop"); string object created with cloudhadoop value

String cloudChars="{'c','l','o','u','d'}"
String cloudString=new String(cloudChars);

String cloudString="cloudhadoop"
String cloudString=new String(cloudChars);

Length of the string.

to find out the number of characters in string, String.length method is used.

Concate strings or adding the strings in java:-

we can use either concat or plus operator to add the strings. Both will be used to add the strings at the end.

String sObj="cloud";
String addedString="hadoop+sObj; or

String sObj="cloud";
String addedString="s.concat("hadoop");

Comparison of two Strings:-

String sObj="cloud";
String addedString="cloud";
In the above how many objects are create? answer is only one object but the references are created two which are pointing to the cloud object.

In java we can compare references the two strings using "==" operator
if we want to compare two string objects data ,we can use equals method.
In the above example, == returns true and equal method returns true.

we can also use equalsIgnoreCase method to ignore the string case sensitivity

How to Split string into substrings:-

strings are separeted into list of sub strings with delimeter specified using String.split(String) method

String s="Best String examples";
String delimeter=" ";
String array[]=s.split(delimeter);

output:- Best String examples

format String with example in java:-

format method is provided in String object which is similar to c language println format.

To format the one of the string, we use codes to represent the different value types

String.format("Hi %s, today is %d day of month","Kiran",29);

and the output is Hi Kiran, today is 29 day of month.

%s represents strings type

%d represents integer type

%f represents float type

capital codes represents capital types of types

Hope you understand the basics of Strings in java
Read More

Saturday, 28 January 2012

How to setup and install tomcat 7 server in windows?

Tomcat is application server which supports for running java,j2ee based applications.

Tomcat provides different services like any other applicaiton servers such as jboss, websphere

Is tomcat Application Server or Webserver?

Yes. Tomcat is a application server which provides the following services

1.Object Management like automatic object creation,modifying, and garbage collection
2.JDBC Connection pool mechanism
3.Maintaining multiple nodes in cluster setup
4.99.99 starnd up time for running applicaitons
5.Configuring the different j2ee services like mbeans, enterprise objects like message beans with in admistrative tools
6.Integration with different security frameworks like CUP with LDAP, and database realms

The advantages of the above services are that developer need not to write this and can concentrate on his business logic.

Web server is server which serves plain request annd gives the response. apache webserver need not be talk to database

But in produciton scenario,apache httpd server acts front controller which delegates the request to applicaiton server, the applicaton server talks to database and do the busines logic and send the response to webserver, finally webserver send the response to client.

Setup apache Tomcat 7.0 in windows:-

following are the sequence of steps to setup tomcat in windows.

JDK settigns for tomcat

For tomcat, jdk is required, make sure install jdk 1.6 version and configure the following environment variable
JAVA_HOME=D:\jdk path

download tomcat 7.0.25

Before downloading tomcat 7.0 to your machine, we need to find out 32 bit or 64 bit machine you are going to install, check here to find out 32 bit or 64 bit for your machine.

First try to download the download Tomcat 7.0 server from windows.apache-tomcat-7.0.25-windows-x86.zip file is downloaded to your machine, extract or unzip using winrar or zip tools to your drive(d:\)

let us say i have extracted to G drive like G:\apache-tomcat-7.0.25

tomcat server folder structure

Coming to the folder structure as follows

\bin folder :-bin folder contains scripts for setting classpaths,starting and running scripts. scripts with .bat extensions are for windows server and .sh for linux box.
setclasspath.bat is used to configure the any classpath related settings like JAVA_HOME etc.
startup.bat is to start the tomcat as non servce mode meaning if we don't install tomcat as windows service, this will increase performance for tomcat
tomcat7.exe is to start the tomcat as window servce meaning if we install tomcat as windows service
shutdown.bat is to stop or shutdown the tomcat server.
catalina.bat is to set the tomcat environment related settings like jvm options, heap size settings and garbage collector settings

\conf folder:- configuration related files in tomcat like port number and log level changes etc..
server.xml contains settings for tomcat container like listening port number and resouce configuration like jdbc datasource, security settings
tomcat-users.xml contains authentication and authorization details for tomcat administration console tool

\lib folder:-
Lib folder contains all the jars required to run for tomcat. This bootstrap class loader for tomcat meaning, jars in this folder are loaded into the container when tomcat application is started. This jars are avilable to all the applications deployed to tomcat container. we can overirde the jars with in application.
always avoid duplicate with different version of jars in lib folder with your application folder. otherwise you will be in trouble with a lot of NoClassDefFounder issues.

\logs folder :- contains all the tomcat server related logs like standard out logs, access logs, and catalina logs.if we have any issues in tomcat, This folder is first chock to look for the detailed information about the error.

\temp and work folder:- contains the temporary tomcat work folder for your tomcat. It will acts cache for your application.

\webapps folder:- This is main folder where all your applications are deployed to. if you wan to deploy your application, copy your webapplication war file to this location.

That's about the folder structure.

tomcat environment settings

make the following environment settings for your tomcat.


Starting tomcat server

start the tomcat server using startup.bat. this opens the console starts the tomcat serer

Stopping tomcat server

stopping or shutdown the tomcat server using shutdown.bat. this opens the console stops the tomcat serer

finally you have completed the tomcat setup in windows and got some basic idea on tomcat server.

Please feel free to comment if you have any questions.

Read More

Wednesday, 25 January 2012

Timers in java: Create timer in java with example?

Timer in java:-

Timer is java classe is in java.utils package, Timer is used to specify the task execution for every intervals or after holdup of some time in an application.

TimerTask in java:-

TimerTask is java class in java.utils package. TimerTask implements runnable interface which has run method to overried.
run method of TimerTask can have code for which you want to apply the timer functionality.This timer task can be scheduler by many times for specific time interval duration.

what is timers in java?

We have seen many scheduler programs in linux which are based on cron jobs. Java has also mechanism to provide the limited scheduling mechanism with Timer and TimerTask Task. if you want full pledge scheduler, you can use quartz scheduler.

Before this classes are introduced, we have to write our own custom code for scheduler task using our own calculation logic using threads.

Timer can be used in real time applications where you have large object which want to synch with database for every one hour or read the logs from the apache logs and insert into db for every 4 hours.

Timer class implements Runnable method which is for creating thread. so Timer will create threads internally which cause to execute the given task at periodic intervals of time have the control for execution of multiple task achieves the task execution at repeated intervals or we can post pone the execution for specific intervals of time.

How to create a Non Dameon timer?

Non Dameon timer is default timer implementaion using default constructor of Timer.
Timer t=new Timer();
Timer timer=Timer(false)
suppose you have created timer that means threads are created internally, once the run method of the thread execution is completed, timer should not executed
and other case is, you have to execute the timer for 10 seconds only, after that you have to cancel the timer, meaning is developer have the control to stop the execution of the timer(using cancel method of timer).
This timer causes performance effect as JVM unable to stop if non dameon timer is executing

How to create a Dameon timer?

Dameon timer are created in the background thread where it's timer is created and started.Dameon timer can be created using the following code
Timer timer=Timer(true);

Dameon timer is timer which will not be stopped even your applicaiton life ends.

As timer is completed dependten on thteads, so we will see the main difference.

difference between Dameon Thread and Non Dameon Thread?

Dameon Thread are background thread which are create and executed in background. scope is JVM running time only
Non Dameon Thread scope is application running life time. Once the application is stopped, dameon thread stopped it's execution.
This threads ends when exit method is called or run method execution is over or exception is occurred.

Timer example in java:-

how to execute timer for repeated interval of time:-

Let us see the example for usage of timer class with printing hello world message for every one second for 10 times.

import java.util.Timer;
import java.util.TimerTask;

public class HelloMessage extends TimerTask {
 private int numberoftimes = 0;
 public void run() {
  if (numberoftimes <= 10) {
   System.out.println("Hello World....");
  } else {
class TimerExample {
 public static void main(String[] args) {
  Timer helloWorldTimer = new Timer();
  helloWorldTimer.scheduleAtFixedRate(new HelloMessage(), 0, 1000);
Timer objec is create called the timer scheduleAtFixedRate() which has TimerTask job which has actual code written to execute for timer. For this we have written Cusotm java class HelloMessage which extends TimerTask and overrieds run method exectution. we have used cancel method to end the life of the timer after repeated 10 times execution.

Hope you got enough idea on basic timers in java for your execution.
Please free to comment if you have any questions or if you liked my post.
Read More

Monday, 23 January 2012

Learn Basics of Sample web.xml in java

What is Web.xml in java?

Web.xml is web deployment descriptor used for any web application written in java. Every web application
that is developed in java should have this xml file in Root-Folder/WEB-INF.Web.xml is defined as part java specification. so every vendors like tomcat, jboss or other app  servers should have to implement in their own webbased frameworks.web.xml contains all the jsp,servlet configuration, filter, listener configurations.

Mostly these days developer uses eclipse for web.xml modifications. This validates xml file against schema and gives warning if any

web.xml configuration:-

1.xml version:-

<?xml version="1.0" encoding="UTF-8"?>

The above line is starting line for any xml file which contains version and encoding type of xml.

2.Root element :-

<web-app xmlns="namespace" xmlns:xsi="schema instance"
          xsi:schemaLocation="schema location"

This is high level root element which has  xml namespace(xmlns) to identify the unique name for this xml.
As you know each xml should be validate against either DTD or schema. validation means for each tag, what are the sub elements(display-name) and how many sub elements i.e one or two
Version is servlet specification version for the web.xml.

3. Application metadata

<display-name>name of the application</display-name>
<description> short description </description>

4. Context parameter configuration

display-name represents the name of the web application.
description configured to define the short description of your module

<param-name>conext initialization parameter</param-name>
<param-value>conext initialization value</param-value>
context-param  is used to configure the key and value pair of values in web application.
This values are available through the webapplicaiton. We can configure many context-param values for web applicaiton
5.Listener configuration:-

<listener-class>Listener java class</listener-class>
This listener entry is used to configure the listeners in web application

5. Servlet configuration :-

   <servlet-class>java servlet class with package</servlet-class>
   <param-name>init param</param-name>
     <param-value>init param value</param-value>
servlet entry configures the servlet defined for any webapplicaiton with name and the servlet class.
load-on-startup is used to pre loading of a servlet. Preloading means servlet will be loaded into web container when applicaiton is started  and calls the callback methods of init methods.
we can specify the integer number which is priority order given to load when multiple servlets are presented in servlet container.

init-param is like context-param  to define the key and values. These values are avilable to specific servlet only unlike context-param in whole applicaitln
servlet-mapping defie the mappings for the defined servlet name in above step to specific url pattern.
servlet name will be called for the request url that ends with html extension.

6. jsp configuration in web.xml :-

   <param-name>init param</param-name>
     <param-value>init parameter value</param-value>

jsp can also be declared like servlet except as seen the above code
and the mapping for the jsp page is as follows


servlet-mapping defie the mappings for the defined jsp name in above step to specific url pattern.
jsp name will be called for the request url that ends with html extension.

6. Filters name defination and mapping:-


Filters are called before servlets. Mosty these are used for security purpose like login security implementaiton for checking authrozations and authentication
Filter-mapping is used to define the filter class to specific url patter.
These are the basic of configurations in web.xml. This configurations are same for tomcat or jboss.

Sample web.xml file:-
  sample web.xml for reference
      <param-name>context parameter as key</param-name>
      <param-value>Context parameter value</param-value>

Points to remember:-
1.The order all the elements should be as per Schema
2.For every start tag, there must be end tag.

Read More

Sunday, 22 January 2012

Learn tar command with examples in linux or unix

Tar command in unix:-

Tar command is abbreviated as tape archive.tar is one of the most popular zipped/achive command in unix like zip in windows to compress the files and store all the files into single file tar.gz.

tar command examples in linux
tar and gzip are more popular zipped archive commands in linux or unix like zip,winrar etc in windows.I have also blogged about Top 10 gzip examples in linux,unix  for gzip usage.

Most of the open source frameworks like maven, tomcat,jboss etc..are shipped in tar.gz format to install in unix environments.blogging all these for reference.

Tar example usage :-

1.Creating archive tar.gz for a folder or tar the files:-

Tar –cvzf   newfilename.tar.gz existing-folder-name

This will compress create the tar version/zipped as new-file-name.tar.gz from the existing-folder.The structure of files ,directories,subdirectories in existing-folder are same way created for new-file-name.tar.gz

c- creating new gz file
z - compresse the filees
v- print the list of the files shown in stdout
f- with this option, archive name are specified with in command

2.Extract/unzip tar.gz file into a folder or untar
Tar –xvzf filename.tar.gz

X option means unzip the contents of this archive
Remaing options are same as like in #1

This will extract and copy all the files to the current directory with the folder as filename where this command is running.

3.Show all the files of tar file

tar  tvf  filename.tar.gz

This will show all the files in filename.tar.gz to stdout console.
Option t – list all the contents of the archive

Advantages with tar is it will take a backup of your files
Disadvantage is single file can not be extracted from the zipped or archived file

Extract or unzip archive with gunzip,tar command :-

With this both commands, we can used to unzip the tar.gz or .gz files.’

To unzip tar.gz file, there are two steps involved. First unzip .tar.gz into tar file in the following command and then use tar command

gunzip filename.tar.gz
tar –xzf filename.tar 

filename.tar is created with the first command, this is passed to tar command extract the files in the tar.

Hope you got basic start for tar,gunzip with examples.

Please feel free to comment and share your thoughts.

Read More

Saturday, 21 January 2012

Top 10 Maven Commands tutorial

Maven is a build automation management tool in java projects. Most of the people working on java related technologies know about the maven usage. I am listing down the list of Maven commands with examples reference which is very helpful for developer. This list can be used by all the Java developers for their daily usage in projects.

Please have a look of my previous article maven installation. For the maven installation, the jdk is required.

Listing down all the maven command tutorial at a high level.

1.       Creating a sample maven standalone project:-
2.       Creating a sample web standalone project:-
3.       Clean project
4.       Compile project
5.       Building web apps:-
6.       Deploy project:-
7.       Run unit and integration tests  
8.       Ignore test execution
9.       Generate  java documentation for the project
10.   Maven debug
11.   Generate site

1. Creating a maven standalone project:-

mvn archetype:generate -DgroupId=org.cloudhadoop -DartifactId=myproject

This is the starting step for any Java project to create. This command runs at the command prompt and creates a project with project name i.e. artifact id.  This project is located in group “org.cloudhadoop” using groupid,.
After running this command for the first time, this tries to download all the required artifacts downloaded from remote repositories ( from maven .. etc) and copy to your local repository and after that, it creates a project. This project contains the src/main/java and src/test/java folders which contain the HelloWorld java program in the main folder, and test class for hell0world program in the test folder.

2. Creating a web standalone project:-

mvn archetype:generate   -DarchetypeGroupId=org.apache.maven.archetypes  
-DarchetypeArtifactId=maven-archetype-webapp   -DgroupId=com.cloudhadoop  
Most of the times, we usually create a web-based project in eclipse. With this command also, we can create web projects like spring, struts which contain, WEB-INF, lib, classes folders and web.xml. This command runs at the command prompt and creates mywebapp with project name i.e. artifact id.  This project is located in group “org.cloudhadoop” using groupid,.
After running this command for the first time, this also tries to download all the required artifacts downloaded from remote repositories ( from maven .. etc) and copy to your local repository and after that, it creates a project.
The directory structure is different from the #1 command and creates as per maven directory structure for web applications.

3. Cleaning project

mvn  clean
As you know target folder contains all your compiled classes, as well as the jar, war files when you run the project with maven install.
Clean is maven predefined goal, this command deletes all the contents in the target folder.

4. compile maven project

mvn  compile
compile is maven predefined goal, this command compile all your java classes which include src files as well as text files in your project.

5. Build web apps:-

mvn package
maven package goal is used to build maven applications.
As you know any Java project can be packed as jar or war.
The package is a maven predefined goal. By giving this command, first, compile all the java files(using compile goal)  and run all your test classes and copy all these files to target folder and create a jar, war file. The final output for this command is jar/war of your project located in the target folder.

6. Deploy/install project:-

mvn install or 
mvn deploy
When we run this command under your project,  it will do all the tasks  in “mvn package” and create the required jar/war file in target folder.
Maven install goal is used to deploy the project(jar/war) to the local repository. and local repository location is /.m2/repositories/groupid/.
Maven deploy goal is used to deploy the project to the remote repository like the nexus. so that other developers can use this artifact in their module. remote repository location is specified in pom.xml.

7. Run unit and integration tests:-

mvn test
Test goal is used to run only the test classes in your project. The test files are located under src/test/java and copy the result to your target/test-classes, target/reports if any.
8. Ignore test execution
mvn package –DskipTests or
mvn package -Dmaven.test.skip
In 4#, with maven package command run, tests are also executed. If you want to skip the unit tests, The we can use this command. You can use this option with install goal also.

9.Generate  java documentation for the project

maven javadoc:javadoc
This will generate java documentation for your project. And the generated java doc report can be found in the target folder
This will includes API documentation for your Java classes in your project.

12.   Debug Maven

mvn -X  
This command is used to start the maven goals in debug mode and gives logging information. This command gives more information like what artifact is failing for what reason.
This command can be used when you are not getting any clue for your maven project execution failure.

11.Generate site

mvn site  
The predefined goal in maven is used to generate site documentation in formatted style.

Please let me know if you like my post and share your comments.

Read More

Thursday, 19 January 2012

Top 10 gzip examples in linux or unix

gzip example usage in linux:-

Gzip is one of the frequent command used in linux . gzip command is use do to compression of a file for reducing size of file. This will saves the bandwidth if the file is transferring between different systems.Moreover the reduced size depends on the the content of the file, if the content is text, it will reduce 60% and for image, it should be 80%.

gzip linux command usage

if we want to copy the mulitple files,the files should becompressed, so that bandwidth of file is reduced.Gzip abbrivated as GUN zip

Gzip command examples:-

1.Compression the file with deleting original file.
Gzip linuxFileName

This will replace a linuxFileName.gz which has size of 80% of linuxFileName in the current directory. The filename size is reduced by this command. Once gz file is created, linuxFileName should be deleted

2.Compression the file with keeping original file.
Gzip –c  linuxFileName
This command will behave same expect deleting the original file. So original file should be kept as it is.
2. Uncompress/decompress the gz file
Gunzip fileName.gz

This will unzip the filename.gz and get the original file before using gzip command

3.Compression multiple files in a directory
Gzip -r directoryname
using -r option, recursively traverse all the files, meaning all the files in current directory including all the files subdirectory and create a directoryname.gz which contains all the files in the current directory and subdirectory
After compression, total size of the files is approximately 20% less gz file.

3. Uncompress/decompress the gz file into multiple files
Gunzip -r fileName.gz
This will unzip the filename.gz into the the multiple original files before using gzip -r command

4.Compression files fastly:-
Gzip -1 filename.txt
Gzip –fast filename.txt
The both above options compress filename.txt very fast and create filename.txt.gz folder

5.Compression files fastly:-
Gzip -9 filename.txt
Gzip –best filename.txt

The both above options compress filename.txt files slowly and create filename.txt.gz folder

Advanced gzip examples:-

6.zip each file in the current directory and creating separate gz

for filename in *.txt; do gzip -c "$filename" > "$filename.gz"; 

let us say we have file1.txt,file2.txt,file3.txt in the current directory /tmp/directory. To do this, we have to iterate each file and do the gzip command redirect(>) the output as gz file
The above command create file1.txt.gz,file2.txt.gz,file3.txt.gz
-c option keep all the original files (file1.txt,file2.txt,file3.txt) and give the file to stdout console.
if we don't specified any option, it will remove all the files, and create a gz file

Hope you got basic start for gzip with examples.

Please feel free to comment and if you have any questions, leave a comment, i would get back to you.

Read More

Tuesday, 17 January 2012

Learn Basic of HDFS in Hadoop

HDFS is distribute file system used by apache hadoop.Hadoop used HDFS for storing large data say peta bytes of data. This data stores the data and distrubtes the data in different machine in clustered architecture, because of the distribute data over multiple machines, it is highly available in process the data. HDFS runs on low cost hardware.

How this data stored in HDFS:-

Hadoop runs on different machines which are named as clusters. and process the data huge amount of data. Each cluster has set of nodes called data node and each node has set of blocks. and the set of data is duplicated in different nodes in clustered environment. This duplication is we called as replication in the database environment. so set of data is available in different node or machines, so that you can't loose the data when one is down. The number of nodes where this data is replicated is configured in hadoop system, To control all these nodes, we have name node concept. which sync with set of data nodes in cluster environment to know the health of the nodes and also stores metadata about the nodes as well as blocks

if data is grown rapidly, we can add nodes without fail over of the whole system and losing the data. This we can call it as scalable system in network terminology. This system handles the case of losing data while adding machines to the existing machines or after the machine add to the cluster.
As you know cluster has different nodes, if one node fails, hadoop handles the scenaro without losing the data and serves the required work as expected.

HDFS stores the data in the files where this files uses the underlying system.

HDFS is suitable for storing the large amount of data like peta and tera bytes of data which process the data using Map reduce for OLAP transaction.

Let us take the scenario where you have 12 pages of pdf file you want to store in HDFS system.
Assume that each page has one block, it might be different in real system.

Name Node holds three items( filename,number of blocks,block ids)
1.File name which is page number of pdf file stored under hdfs file system example (/pdf/page1)
2 Number of blocks represents count of blocks where this file is store in hdfs
3.block ids represents reference to the blocks in the name node

Data nodes holds the page data in different blocks.
Page 1 is replicated in three blocks which are of id's 1,5,6
These blocks are of on different machines
Here is the summary of pages store in different nodes on clusters.


Because of the replication of data, data will never lost even when the data node is down.The communication of namenode and data node can be throught TCP protocol.

HDFS is completely written on java framework.Data store in hdfs can be controlled using commands as well as java hdfs api's provided by apache. Commands executed on top of underlying operating system which inturns calls java apis to interact with file system.

Hope you got little bit idea on ocean of HDFS.

Please leave a comment if you like this post.
Read More

Sunday, 15 January 2012

Issues with OutOfMemoryError: PermGen space in tomcat and solution

OutOfMemoryError: PermGen space in tomcat
Recently when I have deployed a web application in tomcat 6.0.32 in windows Development environment, I got following PermGen Space errors.
so I would like to write the post on this with root cause and exceptions

Caused by: java.lang.OutOfMemoryError: PermGen space
 at java.lang.ClassLoader.defineClass1(Native Method)
 at java.lang.ClassLoader.defineClassCond(ClassLoader.java:631)
 at java.lang.ClassLoader.defineClass(ClassLoader.java:615)
 at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:141)

What actually causing OutOfMemoryError?

This issue either occur in the startup of your application or in runtime whenever the application is running.
One of the causes is
As you know, java object is stored in heap memory, when your application war is deployed and started in tomcat container, the application loads all the objects and called the startup to call back objects and loaded into the memory. so there is no enough memory to accommodate all the object in memory. it throws heap permgen errors. and the result is application failed with an error and doesn't start properly. By default, tomcat assigns 256m as heap memory for the process, so you have to increase the size based on your application size as well as your RAM capacity
and the other cause is
JVM take care of cleaning the objects automatically, but some objects hold in memory in a long time whenever your application start/stops or whenever your application process the data, it holds a different version of the same class in the memory

Fixing OutOfMemoryError: PermGen space in tomcat

We can fix this in many ways
1. set the below value in startup.bat in windows or startup.sh in Linux flavors.

set JAVA_OPTS=" -Xms1563m -Xmx1563m -XX:NewSize=1024m -XX:MaxNewSize=1024m -XX:PermSize=1024m -XX:MaxPermSize=1024m -XX:+DisableExplicitGC -XX:+CMSClassUnloadingEnabled -XX:+CMSPermGenSweepingEnabled"

2. and other way is to set the below value in startup.bat in windows or startup.sh in Linux flavors.
set CATALINA_OPTS=" -Xms1563m -Xmx1563m -XX:NewSize=1024m -XX:MaxNewSize=1024m -XX:PermSize=1024m -XX:MaxPermSize=1024m -XX:+DisableExplicitGC -XX:+CMSClassUnloadingEnabled -XX:+CMSPermGenSweepingEnabled"

3. Most of the issues related to permgen issues are with SUN JVM implementations so you can change the JVM to JRocket JVM for avoiding this types of errors.

Most of the times permgen issues are solved with the above approaches.

Still, the above approach is not solved, yes, the above settings does not helped me to solve the exception, In that case,

you need to see the logs.in the logs, I am seeing the below message.

SEVERE: A web application registered the JBDC driver [oracle.jdbc.driver.OracleDriver] but failed to unregister it when the web application was stopped. To prevent a memory leak, the JDBC Driver has been forcibly unregistered.

Yes, this is the culprit for this issue. my app is not unregistering properly after stopped the server. so the references are not cleaned and available in memory and this would be a memory leak at classloader, again after started, classloader does not load correctly and references are there in memory. this will be causing the issue.

and the solution for this is to copy the corresponding oracle jar to tomcat\lib folder to load from the tomcat classloader. hope this will solves issues

Please let me know if you are still facing the issue.
Read More

Wednesday, 11 January 2012

Find out 32 bit or 64 bit java jdk version in Java

I have encountered one of the issue with java installation which is not combatable with my 32 bit machine.

32 bit is 4 bytes of size, where as 64 bit is 8 bytes of size. That means 64 bit takes more memory than 32 machines
Most of the times, if your code is complied with 32 bit jdk version, then you need to execute this class files in 32 bit machine/64 bit machine only. This will complies and executes fine, but in some cases if there is machine specific , you are in trouble with issues,so careful while installing jdk versions.

32 bit JVM is not compatible with other 64 bit hosts/frameworks as i have encountered issues tomcat 64 bit on java 32 bit

I am listing down the ways to find out the 32 bit vs 64 bit

As per Sun Specification "There is no public API to find out the 32 bit or 64 bit".

There are number of ways to find out the 32bit version or not.

One way is as "sun.arch.data.model" is one of the system property in JVM which has 32 bit or 64 bit or unknown, so we have to write a simple program to read the property value

So i have written sample code to find out bit version
public class BitVersionDemo {
 public static void main(String args[]) {
  System.out.println("version ="
    + System.getProperty("sun.arch.data.model"));

and the output is
version =32
that means your installed jdk is 32 bit

on Linux :-

so if your system is installed java 64 bit, by running java -d32 command it gives the following message

bash-3.2$ java -d32 -version
Running a 32-bit JVM is not supported on this platform.

By giving -d64 command, the following informationn is displayed

-bash-3.2$ java -d64 -version
java version "1.6.0_27"
Java(TM) SE Runtime Environment (build 1.6.0_27-b07)
Java HotSpot(TM) 64-Bit Server VM (build 20.2-b06, mixed mode)

On windows, "java -d32 -version" is giving error so you can write above samle to find out the version

Hope this helps for your debugging.
Read More

10 Interview Questions answers for Log4j in java

What is Log4j?

Log4j is logging framework provided by apache foundation for java based applications.
In the applicaitons, if you want to log the some information, like any event triggered, or any Database updated is happened, We have the need to log the specific information or error for the useful of the application.
To debug any issues in applications, we have to log the error/exceptions in the logs. For this we will use log4j mechanism .
Log4js log the information and show this information in different targets. The different targets are called appenders (console).

How do you define logging for your applicaiton

if you want to add logging for any applicaiton, you have to download the log4j framework from the apache site.
Once log4j jars are downloaed, make sure that these jars are in classpath of your application. let say you have webapplicaiton need to be added log4j. In this case, log4j jars are copied to WEB-INFO/lib folder of your webapplication
create new file either logging.properties or log4j.xml which will be copied to WEB-INF/classes folder
logging.properties contain the all the configuraiton related to looging mechanism and logger level and package that you want to define logger level.

What are the different logging levels?

There are several logging levels that you can configure in you applicaiton
Those are FATAL,ERROR,WARN,TRACE,DEBUG,INFO OR ALL in apache logging. Default logging level is INFO.

what are defifferent types of logs?

usually in any application there two types of logs
1.Application server logs :- These are the logs configured at the applicaiton server level. for example in tomcat,
we have log files called localhost.log. tomcat.log,catalina.log, stdout.log, sterr.log. all these logs are showing with default settings defined in logging.properites located in your tomcat installation folder/conf folder.
if you want custom settings, we have to change the different parameters in logging.properties in conf folder of tomcat directory.

2.Application logs:- We can define logging at each applicaiton level, For this we have to create log4j.xml or logging.properties in WEB-INF/classes folder

what are dfifferent appendeds that can configure in log4j?

There are different appendeds that we can configure in log4j are CONSOLE,FILES,DATABASE,JMS, EVENT LOGGING

CONSOLE in log4j:- if we use this as appenders in your applicaiton, log4j logs the information in the console or command prompt window that is started with statup script.
Files in log4j:- Files Appender is used to log the information into our custom name files. when we are configuring this appender, we can spcefy the file name
Read More