пʼятниця, 7 червня 2013 р.

Hadoop MySQL Applier

What is Hadoop MySQL Applier? And how is it differ from Sqoop?

In fact, Applier provides realtime data migration from MySQL into HDFS. The idea is pretty simple:
applier reads MySQL binary logs and replicate insert events on HDFS
only insert operations are handled and replicated, there aren't updates, deletes or DDL

субота, 1 червня 2013 р.

JMX batch updates

Some days ago, I found that it would be a good idea to monitor and save (or even set) some JMX metrics in batch style. In other words, what if I have a server farm and wish to get some JMX attribute(s) in a moment from all servers? Or even set up new value. I'd like to have it as command line tool because of I'd like to run it from console (even performed by cron). That's why I started my new repo JMXSample on GitHub. The idea is pretty easy and all implementation is located in the one file which can be compiled by javac. There are another files that are example of JMX managed application, you can ignore them.
To run this utility, you have to create configuration file (see example), which contains JMX commands line by line (currently only two commands are supported: get MBean value and set it, also only primitive Java type, String and Date are supported, check out source code for details). So, the typical line of configuration is following:
ObjectName attributeName [new_value_if_set] host port

Let's look how we can use it. I assumed you reuse sample configuration file and you run ElectroCar sample application (I wrote about it earlier) with the next parameters:

-Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=1617
-Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false

After that feel free to run batch JMXCommand util (after compilation, of course), and expected result is following:
java jmxsample.JMXConsole /absilote_path/JMX/example/conf.txt

Expected output will be next (I tagged moment, when maximal speed was updated):

[localhost:1617] Attribute MaxSpeed has value 150
[localhost:1617] Attribute CurrentSpeed has value 55
[localhost:1617] Attribute MaxSpeed has value 250
[localhost:1617] Attribute CurrentSpeed has value 90
[localhost:1617] Attribute CurrentSpeed has value 235