surprise in Hadoop log

When I started working with Hadoop I was confused by next message in logs:

DEBUG conf.Configuration: java.io.IOException: config(config) 
    at org.apache.hadoop.conf.Configuration.<init>(Configuration.java:225) 
    at org.apache.hadoop.mapred.JobConf.<init>(JobConf.java:183) 

It was Hadoop 1.0.3 and I didn't understand "what am I doing wrong?". It was just the newest Hadoop and I didn't find more information in Google, So, I was need to check source code... surprise! Look at it (line 4!):

public Configuration(boolean loadDefaults) {
    this.loadDefaults = loadDefaults;
    if (LOG.isDebugEnabled()) {
      LOG.debug(StringUtils.stringifyException(new IOException("config()")));
    }
    synchronized(Configuration.class) {
      REGISTRY.put(this, null);
    }
    this.storeResource = false;
  }

I can't believe they always log exception... strange way to get stack trace? maybe...

In memorize: REGEXP

The best online java regexp tester http://www.regexplanet.com/advanced/java/index.html

Raspberry Pi cluster for hadoop?

I just thought "Raspberry Pi cluster for hadoop?" Is it possible? does it makes any sense?

Let's think... Hadoop uses hard-drive very-very intensive. Memory.. it's good too have enough memory, but it doesn't critical; I believe 512 MB will be enough. CPU... depends on your code, but usual it's not critical point for map-reduce in general

So, with Raspberry Pi you get (just for $35!):

• RAM 512 MB
• CPU ARM11 700 MHz
• SD with Linux 4-16 GB (you will need to buy it separately)

Some time ago there was the nice article about Paspbery Pi supercomputer: 64 Raspberry Pi computers were connected into the one cluster (via Ethernet); each has 16 GB SD card and it means 1 TB storage for whle cluster (!), and costs about $4000
One concern: access speed to SD card. It isn't good enough and you will need to buy external SSD hard-drive. I assume each Raspberry Pi has to have own SSD (32-64 GB must be enough). So, this solution will be a more expensive that $4000, but cheapest than whole PC or cloud instances.

Let's try to calculate: 64 Raspberry Pi * $35 = 2240, SSD 64 GB * 64 = 4TB costs $4500, whole solution will cost $6500-$7000 for 64 physical node:)

So, does is make sense to build hadoop-oriented cluster? I believe so, what do you think?
At least, it will be a great experiment!

PS. Maybe someone wants to donate money for this experiment? kickstarter sounds resonable here

Project management: communications in distributed team

During my working experience I had working on several projects in distributed team or I had just communication with customer. The first type of collaboration is typical for out-staffing and some kind of product companies (do you remember "Rework"?), the second type is typical for outsourcing companies and some kind of product companies.

As you know the one of the most important factors in distributed team is communication. Communication is really important for team in general, and the main challenge of communications in distributed teams is a question: how to organize high-quality communication inside distributed team? The friendship in a team is important, how can we reach it in a distributed team?

Fix microphone problem in Ubuntu

I'm using Ubuntu/Kubuntu on a Dell laptop, and it doesn't seem to recognize my headset's microphone. What can I do?

It was the question that was worried me for a last week. Actually, my microphone stopped working suddenly on the previous weekends. And my microphone didn't work in Skype nor Gtalk, it was awful!
I was looking for solution for a several hours and found it just a seconds ago! Hopefully  it was described here http://preprocess.me/skype-microphone-does-not-work-on-ubuntu-heres-a-fix for Ubuntu (Ideal works under Kubuntu too)

Just main steps (to me in the future):

1. Run alsamixer
2. Go to Capture devices
3. Select capture device (I had two) and highlight it; then pres space to enable - you capture device must be highlighted in Red "Capture" label

Helping Java build friendship with Common Lisp

When I was 3rd year student I had functional programming and AI classes where we studied Common Lisp programming language. At the end of AI course I had coursework: write program to calculate differential expression, something like f(x) = x*x then f'(x)=2*x. It was pretty simple to do in Lisp...

As business layer was implemented in Lisp I decided to create user interface in Java. I had to use Swing, because JavaFX didn't exist at this time. So, I guess I started hate Swing from this time...
But here I'd like to show how you can execute Common Lisp from Java (from my personal experience with code sample and some other options). So, how to use Lisp from Java?

Push existing source code to GitHub

There are a lot of advice about that, but the most of them don't work. I found one that really works:

1. Create the remote repository, and get the URL such asgit://github.com/youruser/somename.git
   If your local GIT repo is already set up, skips steps 2 and 3

   ---

2. Locally, at the root directory of your source, git init
3. Locally, add and commit what you want in your initial repo (for everything, git add . git commit -m 'initial commit comment')

   ---

4. to attach your remote repo with the name 'origin' (like cloning would do)
   git remote add origin [URL From Step 1]
5. to push up your master branch (change master to something else for a different branch):
   git push origin master

Origin: http://stackoverflow.com/questions/4658606/import-existing-source-code-to-github