Varadharajan's Journal

HBase and Eclispe IDE integration

fossman — Fri, 25 Mar 2011 09:00:44 +0000

In this post, we will be creating a sample HBase project in Eclipse IDE. It is required to have a local working instance of HBase (mine is 0.20.6 version) and Eclipse Helios version. I assume that you are aware of starting and stopping HBase.

Step 1 : Start HBase. Start Eclispe.Step 2 : Create a new Java Project (File -> New -> Java Project).
Step 3 : Fill up the form. (Here project name = Sample) and click next.

Step 4 : Select Libraries tag from it and click “Add External JARs”.

Step 5 : Browse to the $HBASE_HOME directory. ($HBASE_HOME is the directory where you have installed HBase).

Step 6 : Select hbase-0.20.6.jar and click Ok. Similarly add the (commons-logging-1.0.4.jar , log4j-1.2.15.jar, zookeeper-3.2.2.jar, hadoop-0.20.2-core.jar) present in $HBASE_HOME/lib folder and click Finish.Step 7: Create a class with main method and replace the contents with the code displayed below. The below code creates a sample table named Blogposts with 2 column family Author, Title

// HBaseTest.java
import java.io.IOException;

import org.apache.hadoop.hbase.HBaseConfiguration;
import org.apache.hadoop.hbase.HColumnDescriptor;
import org.apache.hadoop.hbase.HTableDescriptor;
import org.apache.hadoop.hbase.client.HBaseAdmin;

public class HBaseTest {
public static void main(String[] args) {
// Try to create a Table with 2 column family (Title, Author)
HTableDescriptor descriptor = new HTableDescriptor(“blogposts”);
descriptor.addFamily(new HColumnDescriptor(“Title”));
descriptor.addFamily(new HColumnDescriptor(“Author”));

try {
// Create a HBaseAdmin
HBaseConfiguration config = new HBaseConfiguration();
HBaseAdmin admin = new HBaseAdmin(config);

// Create table
admin.createTable(descriptor);
System.out.println(“Table created…”);
} catch (IOException e) {
System.out.println(“IOError: cannot create Table.”);
e.printStackTrace();
}
}
}

Step 8 : Right click project in Package Explorer. Select Run as -> Java Application

Step 9: Check the output screen for any Exceptions, if not thats it.

Step 10: Confirm creation of the table using HBase Shell.

Thats it for now.. will create more posts on HBase soon

Implementing “Sieve of Eratosthenes” in Python

fossman — Sat, 07 Aug 2010 09:09:11 +0000

For a couple of days, I’ve gathered a lot of information about prime numbers and algorithms which generates them. out of all the available algorithms, “Sieve of Eratosthenes” is one of the most normally used ones and i want to implement this algorithm in python. After an hour of battle with the Python, i successfully tamed it. The scriptgenerates all the prime numbers within the range of 2 and the number scanned from the user. The script will be available at http://rapidshare.com/files/411537514/eratosthenes.py.html

(image from wikipedia)

Using openssl for generating a SHA1 Hash

fossman — Fri, 18 Jun 2010 06:02:30 +0000

Today, i stumbled a website, where it had some good resource of generating a SHA1 hash of a file using the executable command ‘openssl’. SHA1 is a Hash used for checking the integrity of the file. The command for generating the hash is

openssl dgst -sha1

For example, consider a file image.jpg. The command that will generate a SHA1 digest is

openssl dgst -sha1 image.jpg

Back after exams :)

fossman — Fri, 14 May 2010 12:19:55 +0000

After a long pause, due to my internal and terminal exams, I’m planning to get more serious on learning some new things on operating systems. I had a short crash course on C language (Perl, Python, Ruby made me forget C )

IBM Workshop on POWER7 proved to be useful

fossman — Sun, 28 Mar 2010 11:25:43 +0000

Today, i along with my friends attended the IBM Workshop on IBM POWER7 architecture was really good. The workshop was held at MIT (Just a few streets from my home ) and we really got a good exposure on various problems faced in huge data centres and methods to minimise them. Two problems were discussed, one being power consumption (and Plans to reduce it) and the second one was something related to memory (i couldn’t get it properly). The speaker did a great job and i have drafted a mail to congratulating her for her efforts. THANKS SAMHITA ’10

Preparing for my seminar on XMPP

fossman — Fri, 26 Mar 2010 06:51:33 +0000

Today, i got a nice paper about the eXtensible Messaging and Presence Protcol at www.tml.tkk.fi/Publications/C/21/nie_ready.pdf. The main motive of me reading this, is my seminar about XMPP in computer networks class on 31 of this month. however, I am also thinking of developing an IM Client like pidgin.

Thinking of developing a new DHCP server..

fossman — Tue, 23 Mar 2010 17:49:54 +0000

Today, i thought of developing(implementing) a DHCP server and started to read out the Dynamic Host Configurations Protocol’s Request for comments(RFC) at www.faqs.org/rfcs/rfc2131.html and i think i will start the work by this weekend. Before, starting the coding process, i need a crash course on Sockets programming in C, without which the implementation will not be an efficient one.

Web Interface for Map Reduce Job Submission in hadoop

fossman — Sun, 21 Mar 2010 08:24:45 +0000

Yesterday, It took me about a couple of hours to figure out the working of hadoop (single node cluster) and today, i designed a simple web interface to submit raw data to be crunched by hadoop and then to display the map – reduced results. Till now, the web interface can be used to submit jobs for the famous hadoop API example “word count” and i am planning to extend this to user given hadoop API programs. The site uses CGI (Python) for processing request.

My Laptop is nearly dead..

fossman — Sun, 14 Mar 2010 10:20:36 +0000

Today, i messed up with my laptop. It all started by learning to create a Debian bootable USB stick as given in http://blogs.koolwal.net/2009/02/25/installing-linux-on-usb-part-7-install-debian-linux-from-usb-drive/ and at the step of extracting the boot.img.gz file, i gave /dev/sda which is my laptop hard drive instead of USB drive. After this accident, i could not reboot my system. I cannot go beyond the boot screen showing the compaq logo and i cannot access BIOS also. I had a talk with HP service center and they have asked me to visit them tomorrow. When i googled for the symptoms, i got many solutions like flashing BIOS etc.. but i don’t want to take risk.

IP and MAC address based ACL’s in Squid

fossman — Sat, 06 Mar 2010 05:23:35 +0000

I am little bit confused on how to match IP address of a computer with its MAC address while writing ACL’s in squid. My objective is to allow IP address for which the NIC’s MAC address is already defined in ACL, so that other, who are responsible for IP collision may not use proxy. For example, i will allow a computer with IP 10.2.1.77 when its mac address is only 00:0f:fe:1c:9d:63 so on. I think the following ACL will do the job.

acl comp1_ip src 10.2.1.77

acl comp1_mac arp 00:0f:fe:1c:9d:63

http_access allow comp1_ip comp1_mac

I think the above ACL’s will work. Will post the results here after testing it.