linux commands

port listern

    lsof -i:9000

make directory and its sub-directories

    mkdir -p mydata/hdfs/namenode

find files containing the string

    grep -Ril "text-to-find-here" /

• i stands for ignore case (optional in your case).
• R stands for recursive.
• l stands for "show the file name, not the result itself".
• / stands for starting at the root of your machine. 

Extract matched text 

    pcregrep -Moh '<PubSubError id=(\n|.)*?</PubSubError>' pubsubPusherWeb-1_1.log.2016-11-* > /home/x194594/error02.xml

• M apply the regular expression to multiple lines
• o show only the part of the line that matched
• h suppress the prefixing filename on output, by default it output the file name if targetFile is multiple files.

The regular expression '<PubSubError id=(\n|.)*?</PubSubError>'

• start with <PubSubError id=
• end with </PubSubError>
• between is (\n|.)*, which means either new line or any character, * means multiple times.
• ? means non-greedy, so it stops at the first

Grep regular expression non-greedy

grep -oP '200:{_messageid=.*?,' pubsubPubSvc-1_0.log

• P to use the perl syntax

Grep sort uniq and count

grep -oP '200:{_messageid=.*?,' pubsubPubSvc-1_0.log | sort | uniq | grep -c '_messageid'

• | sort, sorting the result
• | uniq, only return one if more than one
• c, means count

The command can get same result

grep -oP '200:{_messageid=.*?,' pubsubPubSvc-1_0.log | sort | uniq | wc -l

• wc -l  count line

If want to do something like SQL group by

    grep -oP '\-\[.*?]-T-1 INFO ]- Successfully saved error:' weblogicLog/pubsubPusherWeb-1_1.log | sort | uniq -c

uniq -c means count group by the uniq section.

Grep to display line number

    grep -n ...

Grep to display last n  lines

    grep pattern file | tail -1

Grep display first n lines

    grep -m 5 pattern file

Grep in specific files

    grep -r --include="*Spec.js" "your search text" searchDir/
or
    find searchDir -name "*Spec.js" | xargs grep "your search text"
xargs here is using the result of "find" as parameters of grep.

Grep display matched result colorfully

    grep --color "your search text" target

zgrep to grep from gz file 

    use zgrep if grep on gz file, the rest is same as grep.
    same, use zcat on gz file like cat on normal file

Link directory or file

    ln -s targetDirectoryOrFile link
to unlink it
    unlink

View CPU memory usage

    top
The column RES means physical memory used. The other columns are easy to understand.

View disk IO

    sar
or only care average
    sar | grep 'Average'

View disk usage for the files in specific folder

     du -sh * | sort -h

View disk space available 

    df -h .
or
    df -h

Split big files 

split --bytes=500M

Open file count

/usr/sbin/lsof |grep infra |wc -l

check last reboot time 

uptime
or
last reboot