Web Interface for Map Reduce Job Submission in hadoop

Yesterday, It took me about a couple of hours to figure out the working of hadoop (single node cluster) and today, i designed a simple web interface to submit raw data to be crunched by hadoop and then to display the map – reduced results. Till now, the web interface can be used to submit jobs for the famous hadoop API example “word count” and i am planning to extend this to user given hadoop API programs. The siteĀ  uses CGI (Python) for processing request.

Posted in Cloud, Cluster, Hadoop, WWW | 1 Comment

My Laptop is nearly dead..

Today, i messed up with my laptop. It all started by learning to create a Debian bootable USB stick as given in http://blogs.koolwal.net/2009/02/25/installing-linux-on-usb-part-7-install-debian-linux-from-usb-drive/ and at the step of extracting the boot.img.gz file, i gave /dev/sda which is my laptop hard drive instead of USB drive. After this accident, i could not reboot my system. I cannot go beyond the boot screen showing the compaq logo and i cannot access BIOS also. I had a talk with HP service center and they have asked me to visit them tomorrow. When i googled for the symptoms, i got many solutions like flashing BIOS etc.. but i don’t want to take risk.

Posted in Technology, Troubleshoot | Leave a comment

IP and MAC address based ACL’s in Squid

I am little bit confused on how to match IP address of a computer with its MAC address while writing ACL’s in squid. My objective is to allow IP address for which the NIC’s MAC address is already defined in ACL, so that other, who are responsible for IP collision may not use proxy. For example, i will allow a computer with IP 10.2.1.77 when its mac address is only 00:0f:fe:1c:9d:63 so on. I think the following ACL will do the job.

acl comp1_ip src 10.2.1.77

acl comp1_mac arp 00:0f:fe:1c:9d:63

http_access allow comp1_ip comp1_mac

I think the above ACL’s will work. Will post the results here after testing it.

Posted in Uncategorized | 4 Comments

My first task as a Network Administrator…

Today, i got my first task as a network administrator. The job was to install a local proxy server in our college central computing center. We have decided to have a local proxy server, so that we can reduce the load in our main proxy server and we can have even more secure rules like matching IP address with Mac Address and so on. I think this job, even though will not take a couple of hours, will be special for me.

Posted in security, WWW | Leave a comment

successfully configured a single node cluster using Hadoop

Today, it took me around 30 minutes to configure a single node cluster in debian using hadoop without even knowing anything about cloud or distributed computing. The process was very simple and straight forward. The tutorial written by Michael at http://www.michael-noll.com/wiki/Running_Hadoop_On_Ubuntu_Linux_(Single-Node_Cluster) was very helpful. My next job is to learn java and write programs for Hadoop using the Hadoop API. Lets see what hadoop is capable of in a while…

Posted in Cloud, Cluster, Hadoop | Leave a comment

Perl or Python for exploit writing ?

Perl VS Python is one of the biggest cold wars out there in the FOSS world. I agree with most people that Perl is ugly, nasty for writing lengthy codes but when it come to exploit writing or something related to creating hacking tools , it is Perl which rules Python. I dont know the reason for this but can anybody just explain which among the two can be used for writing exploits easily?

Posted in Perl | 1 Comment

Waste of Time…

From Yesterday, My DNS servers (61.1.96.69 , 61.1.96.71) given by BSNL (ISP) was not functioning and i had no means to reach the internet. But due to luck, one of my PC has cached the ip-address of www.bing.com and with that i was able to search for some to the free public DNS servers and finally made the DNS system working again.

Posted in DNS, Internet | Tagged , | Leave a comment