cat
cat copies each file (‘ -’ means standard input), or standard input if none are given, to standard output.
Synopsis:
cat [ option ] [ file]…
head
head prints the first part (10 lines by default) of each file; it reads from standard input if no files are given or when given a file of - .
Synopsis:
head [ option ]… [ file]…
# head /var/log/auth.log
tail
tail prints the last part (10 lines by default) of each file; it reads from standard input if no files are given or when given a file of ‘ -’.
Synopsis:
tail [ option ]… [ file]…
# tail /var/log/auth.log
sort
sort sorts, merges, or compares all the lines from the given files, or standard input if none are given or for a file of ‘ - ’. By default, sort writes the results to standard output.
Synopsis:
sort [ option ]… [ file]…
options:
‘ -n ’
‘ --numeric-sort ’
‘ --sort=numeric ’
Sort numerically. The number begins each line and consists of optional blanks, an optional ‘ -’ sign, and zero or more digits possibly separated by thousands separators, optionally followed by a decimal-point character and zero or more digits. An empty number is treated as ‘ 0 ’. The LC_NUMERIC locale specifies the decimal-point character and thousands separator. By default a blank is a space or a tab, but the LC_CTYPE locale can change this.
uniq
uniq writes the unique lines in the given
input , or standard input if nothing is given or for an input name of ‘ -’.
Synopsis:
uniq [ option ]… [ input [output ]]
options:
‘ -c ’
‘ --count ’
Print the number of times each line occurred along with the line.
cut
cut writes to standard output selected parts of each line of each input file, or standard input if no files are given or for a file name of ‘ - ’.
Synopsis:
cut option… [ file]…
Options:
‘ -d input_delim_byte ’
‘ --delimiter= input_delim_byte ’
With -f, use the first byte of
input_delim_byte as the input fields separator (default is TAB).
‘ -f field-list ’
‘ --fields= field-list ’
Select for printing only the fields listed in field-list . Fields are separated by a TAB character by default. Also print any line that contains no delimiter character, unless the --only-delimited (-s) option is specified.
Note awk supports more sophisticated field processing, and by default will use (and discard) runs of blank characters to separate fields, and ignore leading and trailing blanks.
awk '{print $2}' # print the second field
awk '{print $NF-1}' # print the penultimate field
awk '{print $2,$1}' # reorder the first two fields
In the unlikely event that awk is unavailable, one can use the join command, to process blank characters as
awk does above.
join -a1 -o 1.2 - /dev/null # print the second field
join -a1 -o 1.2,1.1 - /dev/null # reorder the first two fields
Example: a quick way to see which IP addresses are most active is to sort by them:
# cat access.log |cut -d ' ' -f 1 |sort
UPDATE: even easier: the uniq command has a -c argument that does most of this work automatically. It counts the occurrences of each unique line. Then a quick sort -n and a tail shows the big ones. Also, I tend to use "cut" as above, but one of the Dreamhost guys reminded me that awk may be a little more straightforward:
# cat /path/to/access.log |awk '{print $1}' |sort |uniq -c |sort -n |tail
References:
- https://www.gnu.org/software/coreutils/manual/html_node/index.html
- https://encodable.com/tech/blog/2008/12/17/Count_IP_Addresses_in_Access_Log_File_BASH_OneLiner
