четвер, 5 вересня 2013 р.

Useful unix commands


1. Let's assume you have very long CSV file and you wish to find all non-empty columns #25, so:
tail -n +1 data.csv | cut -d ',' -f 25 | grep -v "^[[:space:]]*$"

2. In the previous file you need to find number of all rows with value 'CA' in the column 25:
cat -n data.csv | cut -d ',' -f 25,1 | grep -e "CA.*"

In output, you will get column value as well as row number
3. In the huge file you wish to get row #250007 in separate file:
sed -n '250007p' data.csv | cat > line_250007.csv

4. To get number of tabulation in line
tr -cd \t < line_250007.csv | wc -c

5. Ensure file copy is complete before starting process or ensure that file is not used by other process right now:
if [ $(lsof $file | wc -l) -gt 1 ] ; then
    echo "file $file still loading, skipping it"
else
    echo "file $file completed upload, process it"
fi

6. Color script output for better visualisation
NORMAL=$(tput sgr0)
GREEN=$(tput setaf 2; tput bold)
RED=$(tput setaf 1)

function red() {
    echo -e "$RED$*$NORMAL"
}

function green() {
    echo -e "$GREEN$*$NORMAL"
}

red "This is error message"

7. Find all files in directory by regexp:
for FILE in $(ls $DIR | grep ^report_.*csv$)
do
...
done

8. Check previous command result
mkdir -p $WRK_DIR
if [ $? -ne 0 ]; then
  #do something
fi

9. Extract 12 hours from NOW and cast to specific format:
date -d "+12 hours" +\%Y\%m\%d\%H

Немає коментарів:

Дописати коментар