However, it doesn't run on Windows at all. When you write your first PigUnit script, you will get the following exception:
Exception in thread "main" java.io.IOException: Cannot run program "chmod": CreateProcess error=2, The system cannot find the file specified :In fact, it means Cygwin is not correctly installed. To fix it, you have to download and install Cygwin, after that edit PATH variable and enter the path name to cygwin directory.
Try to run again. The next possible error will be:
ERROR mapReduceLayer.Launcher: Backend error message during job submission java.io.IOException: Failed to set permissions of path: \tmp\hadoop-MyUsername\mapred\staging\MyUsername1049214732.staging to 0700
It means, your temporary directory is not set correctly (or doesn't set at all). Be honest, I tried to set up this temporary directory with the following code:
pigServer.getPigContext().getProperties().setProperty("pig.temp.dir", "D:/TMP");
pigServer.getPigContext().getProperties().setProperty("hadoop.tmp.dir", "D:/TMP");
Unfortunately,it doesn't work.... The solution is to set up system property. There are a lot of way to do it, and one of them is to tune java run configuration when you run your test, just add:
-Djava.io.tmpdir=D:\TMP\Ok, that's much better, but it's not the finish yet, there is error
java.io.IOException: Failed to set permissions of path: file:/tmp/hadoop-iwonabb/mapred/staging/iwonabb-1931875024/.staging to 0700
at org.apache.hadoop.fs.RawLocalFileSystem.checkReturnValue(RawLocalFileSystem.java:526)
that's because of error in the code.
There are several solutions to fix this bug (it is present in Hadoop for a years... :(). One of them, is to use this patch or fix code and recompile. But for me it was the best way (special, it will be fix only for specefic version, also it is difficult to maintain on several clusters on dev machines and so on).
So, I've decided to change code at runtime with Javassist
So, the solution is a very simple and self-describing:
Nice fix!! Worked like a charm from inside a JUnit test. All I had to do was add FixHadoopOnWindows.runFix() in the @SetUp()
ВідповістиВидалитиplease help me, how to test this code
ВідповістиВидалитиwhen running maven test : Tests in error:
ВідповістиВидалитиtestStudentsPigScript(com.ram.hjk.debug_pig.AppTest): The system cannot find the path specified
Thank you.
pigTest.assertOutput("D", new String[] { "(2,No)", "(3,Ha!)",);
ВідповістиВидалитиunable to find the path "D" .