HBase and Eclispe IDE integration

In this post, we will be creating a sample HBase project in Eclipse IDE. It is required to have a local working instance of HBase (mine is 0.20.6 version) and Eclipse Helios version. I assume that you are aware of starting and stopping HBase.

Step 1 : Start HBase. Start Eclispe.Step 2 : Create a new Java Project (File -> New -> Java Project).
Step 3 : Fill up the form. (Here project name = Sample) and click next.

Step 4 : Select Libraries tag from it and click “Add External JARs”.

Step 5 : Browse to theĀ  $HBASE_HOME directory. ($HBASE_HOME is the directory where you have installed HBase).

Step 6 : Select hbase-0.20.6.jar and click Ok. Similarly add the (commons-logging-1.0.4.jar , log4j-1.2.15.jar, zookeeper-3.2.2.jar, hadoop-0.20.2-core.jar) present in $HBASE_HOME/lib folder and click Finish.Step 7: Create a class with main method and replace the contents with the code displayed below. The below code creates a sample table named Blogposts with 2 column family Author, Title

// HBaseTest.java
import java.io.IOException;

import org.apache.hadoop.hbase.HBaseConfiguration;
import org.apache.hadoop.hbase.HColumnDescriptor;
import org.apache.hadoop.hbase.HTableDescriptor;
import org.apache.hadoop.hbase.client.HBaseAdmin;

public class HBaseTest {
public static void main(String[] args) {
// Try to create a Table with 2 column family (Title, Author)
HTableDescriptor descriptor = new HTableDescriptor(“blogposts”);
descriptor.addFamily(new HColumnDescriptor(“Title”));
descriptor.addFamily(new HColumnDescriptor(“Author”));

try {
// Create a HBaseAdmin
HBaseConfiguration config = new HBaseConfiguration();
HBaseAdmin admin = new HBaseAdmin(config);

// Create table
admin.createTable(descriptor);
System.out.println(“Table created…”);
} catch (IOException e) {
System.out.println(“IOError: cannot create Table.”);
e.printStackTrace();
}
}
}

Step 8 : Right click project in Package Explorer. Select Run as -> Java Application

Step 9: Check the output screen for any Exceptions, if not thats it.

Step 10: Confirm creation of the table using HBase Shell.

Thats it for now.. will create more posts on HBase soon šŸ™‚

Posted in Cloud, Cluster, Hadoop, Technology | 2 Comments

Implementing “Sieve of Eratosthenes” in Python

For a couple of days, I’ve gathered a lot of information about prime numbers and algorithms which generates them. out of all the available algorithms, “Sieve of Eratosthenes” is one of the most normally used ones and i want to implement this algorithm in python. After an hour of battle with the Python, i successfully tamed it.Ā  The scriptgenerates all the prime numbers within the range of 2 and the number scanned from the user. The script will be available at http://rapidshare.com/files/411537514/eratosthenes.py.html

(image from wikipedia)

Posted in Algorithms, Python | 3 Comments

Using openssl for generating a SHA1 Hash

Today, i stumbled a website, where it had some good resource of generating a SHA1 hash of a file using the executable command ‘openssl’. SHA1 is a Hash used for checking the integrity of the file. The command for generating the hash is

openssl dgst -sha1 <filename>

For example, consider a file image.jpg. The command that will generate a SHA1 digest is

openssl dgst -sha1 image.jpg

Posted in Cryptography, security, Technology | Leave a comment

Back after exams :)

After a long pause, due to my internal and terminal exams, I’m planning to get more serious on learning some new things on operating systems. I had a short crash course on C language (Perl, Python, Ruby made me forget C :()

Posted in Cryptography, security | Tagged | Leave a comment

IBM Workshop on POWER7 proved to be useful

Today, i along with my friends attended the IBM Workshop on IBM POWER7 architecture was really good. The workshop was held at MIT (Just a few streets from my home :-)) and we really got a good exposure on various problems faced in huge data centres and methods to minimise them. Two problems were discussed, one being power consumption (and Plans to reduce it) and the second one was something related to memory (i couldn’t get it properly). The speaker did a great job and i have drafted a mail to congratulatingĀ  her for her efforts. THANKS SAMHITA ’10

Posted in Hardware, Technology | Leave a comment

Preparing for my seminar on XMPP

Today, i got a nice paper about the eXtensible Messaging and Presence Protcol at www.tml.tkk.fi/Publications/C/21/nie_ready.pdf. The main motive of me reading this, is my seminar about XMPP in computer networks class on 31 of this month. however, I am also thinking of developing an IM Client like pidgin.

Posted in Internet, WWW | Leave a comment

Thinking of developing a new DHCP server..

Today, i thought of developing(implementing) a DHCP server and started to read out the Dynamic Host Configurations Protocol’s Request for comments(RFC) at http://www.faqs.org/rfcs/rfc2131.html and i think i will start the work by this weekend. Before, starting the coding process, i need a crash course on Sockets programming in C, without which the implementation will not be an efficient one.

Posted in Internet, Technology, WWW | 2 Comments

Web Interface for Map Reduce Job Submission in hadoop

Yesterday, It took me about a couple of hours to figure out the working of hadoop (single node cluster) and today, i designed a simple web interface to submit raw data to be crunched by hadoop and then to display the map – reduced results. Till now, the web interface can be used to submit jobs for the famous hadoop API example “word count” and i am planning to extend this to user given hadoop API programs. The siteĀ  uses CGI (Python) for processing request.

Posted in Cloud, Cluster, Hadoop, WWW | 1 Comment

My Laptop is nearly dead..

Today, i messed up with my laptop. It all started by learning to create a Debian bootable USB stick as given in http://blogs.koolwal.net/2009/02/25/installing-linux-on-usb-part-7-install-debian-linux-from-usb-drive/ and at the step of extracting the boot.img.gz file, i gave /dev/sda which is my laptop hard drive instead of USB drive. After this accident, i could not reboot my system. I cannot go beyond the boot screen showing the compaq logo and i cannot access BIOS also. I had a talk with HP service center and they have asked me to visit them tomorrow. When i googled for the symptoms, i got many solutions like flashing BIOS etc.. but i don’t want to take risk.

Posted in Technology, Troubleshoot | Leave a comment

IP and MAC address based ACL’s in Squid

I am little bit confused on how to match IP address of a computer with its MAC address while writing ACL’s in squid. My objective is to allow IP address for which the NIC’s MAC address is already defined in ACL, so that other, who are responsible for IP collision may not use proxy. For example, i will allow a computer with IP 10.2.1.77 when its mac address is only 00:0f:fe:1c:9d:63 so on. I think the following ACL will do the job.

acl comp1_ip src 10.2.1.77

acl comp1_mac arp 00:0f:fe:1c:9d:63

http_access allow comp1_ip comp1_mac

I think the above ACL’s will work. Will post the results here after testing it.

Posted in Uncategorized | 4 Comments