Saturday, April 2, 2016

How to determine your Debian need to restart after packages update

Does your Debian server need to restart after packages update?

You can use checkrestart from the debian-goodies. debian-goodies availabe since Debian Wheezy.
  • Install debian-goodies
# apt-get install debian-goodies
  • Check does our Debian need to restart
# checkrestart
Found 44 processes using old versions of upgraded files
(28 distinct programs)
(21 distinct packages)
...
    • Restart your Debian

    References:
    • http://serverfault.com/questions/667076/debian-how-can-i-know-if-reboot-is-required-after-update

    Friday, April 1, 2016

    Find host or IP on Fail2ban log in Centos using bash and awk

    Find out your fail2ban log and make permanent block. You need to install ipset to make your iptables rules


    • Create script bannedge2centos.sh
    #!/bin/bash
    ## create by dedetok March 2016
    ## GNU GPL v3
    echo "These IP get WARNING at least twice"
    awk '(/fail2ban.filter/ && /WARNING/) { print $NF}' /var/log/fail2ban.log | sort | uniq -c | sort -n |
    {
      while read -r line1 line2
      do
        if [ "$line1" -ge 2 ]; then
          echo "$line1 $line2"
        fi
      done
    }

    echo "These IP get Filter sshd at least twice"
    awk '(/fail2ban.filter/ && /sshd/) { print $NF}' /var/log/fail2ban.log | sort | uniq -c | sort -n | {
      while read -r line1 line2
      do
        if [ "$line1" -ge 2 ]; then
          echo "$line1 $line2"
        fi
      done
    }

    echo "These IP get Filter ftpd at least twice"
    awk '(/fail2ban.filter/ && /vsftpd/) { print $NF}' /var/log/fail2ban.log | sort | uniq -c | sort -n | {
      while read -r line1 line2
      do
        if [ "$line1" -ge 2 ]; then
          echo "$line1 $line2"
        fi
      done
    }

    echo "These hosts get Filter Fail2ban at least twice"
    awk '(/fail2ban.filter/ && /WARNING Unable/) { print $(NF-7)}' /var/log/fail2ban.log | sort | uniq -c |
     sort -n | {
      while read -r line1 line2
      do
        if [ "$line1" -ge 2 ]; then
          echo "$line1 $line2"
        fi
      done
    }
    • Create ipset rules mynetrules
    # ipset create mynetrules hash:net
    • Run bash bannedge2centos.sh  
    # ./bannedge2centos.sh
    These IP get WARNING at least twice
    2 ['199.15.112.8']
    2 ['216.117.2.180']
    2 ['82.148.206.193']
    4 ['208.100.26.231']
    4 ['31.211.102.129']
    5 ['71.6.167.142']
    10 ['88.80.15.69']
    28 known
    These IP get Filter sshd at least twice
    2 118.136.248.90
    18 84.16.74.40
    These IP get Filter ftpd at least twice
    5 88.80.15.69
    These hosts get Filter Fail2ban at least twice
    2 128-140-19-52.maxnet.ir:
    2 1-39-45-166.live.vodafone.in:
    2 178.218.202.224.ip.turontelecom.uz:
    2 181-174-60-157.telebucaramanga.net.co:
    2 222.64.uzpak.uz:
    2 238.124.206.49-ras.beamtele.net:
    2 75-242-69-115.vasaicable.co.in:
    2 78.187.230.16.static.ttnet.com.tr:
    2 85-113-26-150.static.ktnet.kg:
    2 91-219-55-106.static-pool.centr.zp.ua:
    2 brbnd47-30.mng.net:
    2 static-mum-120.63.188.93.mtnl.net.in:
    4 client.fttb.2day.kz:
    • Popuate ipset rules mynetrules
    # ipset add mynetrules 84.16.74.40
    # ipset add mynetrules 71.6.167.142
    # ipset add mynetrules 208.100.26.231
    # ipset add mynetrules 31.211.102.129
    • Add ipset rules into iptables rules
    # iptables -I INPUT -m set --match-set mynetrules src -j DROP
    • Save your new rules
    # service ipset save
    # service iptables save
    Default ipset and iptables: /etc/sysconfig/ipset, /etc/sysconfig/iptables,  and /etc/sysconfig/iptables.save 
    • To reload ipset and iptables
    # service ipset reload
    # service iptables restart
    Another interesting scrips
    • # awk '(/pam_unix/ && /sshd:auth/ && /authentication failure/) { print $NF }' /var/log/secure | sort | uniq -c | sort
    • # awk '(/pam_unix/ && /webmin/ && /authentication failure/) { print $(NF-1) }' /var/log/secure | sort | uniq -c | sort
    • # awk '(/pam_unix/ && /vsftpd/ && /authentication failure/) { print $(NF) }' /var/log/secure | sort | uniq -c | sort 
    • awk '((/not receive/ || /Failed/ || /Connection closed/ || /closed/ || /failure/ ) && /preauth/ && /sshd/) { print (NF-2),$(NF-1),$NF}' /var/log/auth.log | sort  | uniq -c
    Note:
    • You can modify these to auto block those IP into your ipset rules. But beware about your IP may incidentally get auto blocked. Check those IP or host location before add them into ipset ruls
    • When ipset size increasing, it wont impose to CPU load.


    Read your debian server log and ....

    Requirement:

    • awk
    • ipset & iptables

    Read your mail log for authentication failure:
    create this bash script ckmail.sh or download 
    #!/bin/bash
    ## create by dedetok April 2016
    ## GNU GPL v3
    echo "reading mail log file"
    awk '(/authentication failure/) { print $7," ",$(NF-1)," ",$(NF)}' /var/log/mail
    .log | sort | uniq -c | {
      while read -r line1 line2 line3 line4
      do
        if [ "$line1" -ge 5 ]; then
           echo -e "$line1\t$line2 $line3 $line4"
        fi
      done
    }
    create ipset rules for blacklisting smpt
    # ipset create mynetrulessmtpd hash:net
    Run script and pupulate mynetrulessmtpd:
    # ./ckmail.sh
    reading mail log file
    20      unknown[157.122.148.154]: authentication failure
    840     unknown[195.22.127.187]: authentication failure
    16      unknown[91.193.74.31]: authentication failure
    # ipset add mynetrulessmtpd 157.122.148.154
    # ipset add mynetrulessmtpd 195.22.127.187
    # ipset add mynetrulessmtpd 91.193.74.31
    Add ipset rules into iptables rules
    # iptables -I INPUT -p tcp --match multiport --dports smtp,smtps -m set --match-set mynetrulessmtpd src -j DROP

    Tuesday, March 29, 2016

    ipset: iptables extensions

    ipset is a framework that introduces since kernel 2.4.


    Installation
    To install it in debian jessie
    # apt-get install ipset

    Creating rules
    To create a new set of ipset using TYPENAME hash:net (The hash:net set type uses a hash to store different sized IP network addresses. Network address with zero prefix size cannot be stored in this type of sets.)
    # ipset create mynetrules hash:net
    or if you want to create set of ipset using TYPENAME hash:ip (The hash:ip set type uses a hash to store IP host addresses (default) or network addresses. Zero valued IP address cannot be stored in a hash:ip type of set.)
    # ipset create myiprules hash:ip
    or you want to create set of ipset using TYPENAME hash:ip,port (The hash:ip,port set type uses a hash to store IP address and port number pairs. The port number is interpreted together with a protocol (default TCP) and zero protocol number cannot be used.)

    # ipset create myipportrules hash:ip,port
    "If you want to store same size subnets from a given network (say /24 blocks from a /8 network), use the bitmap:ip set type. If you want to store random same size networks (say random /24 blocks), use the hash:ip set type. If you have got random size of netblocks, use hash:net."

    View available ipset rules
    To view available ipset rules 
    # ipset list

    Adding IP into ipset rules
    To add single IP into ipset rules:
    # ipset add myiprules 183.3.202.105
    To add block IP into ipset rules:
    # ipset add myiprules 193.201.227.0/24
    If host names or service names with dash in the name are used instead of IP addresses or service numbers, then the host name or service name must be enclosed in square brackets. Example:
    # ipset add myiprules [u18576666.onlinehome-server.com]

    Removing IP from ipset rules
    To remove ip or hostname from ipset rules

    # ipset del myiprules [u18576666.onlinehome-server.com]
    or
    # ipset del myiprules 183.3.202.105
    or
    # ipset del myiprules 193.201.227.0/24

    Adding ipset rules into iptables
    To add ipset rules into iptables and drop it
    # iptables -I INPUT -m set --match-set mynetrules src -j DROP
    or if you want just to block connection to ssh 
    # iptables -I INPUT  -p tcp --dport 22 -m set --match-set mynetrules src -j DROP
    Note: 
    -I to insert rules into first line
    -A to insert rules into last line
    To delete rules, replace -I or -A to -D
    How to restore or refresh ipset rules if rule name exist 
    To show list of iptables 
    # ipset -exist restore < /root/ipset.up.rules
    Additional command
    To show list of iptables 
    # iptables -L
    To show list of iptables with line number
    # iptables -L --line-numbers
    To show list INPUT of iptables 
    # iptables -L INPUT
    To show list INPUT of iptables with line number
    # iptables -L INPUT --line-numbers
    To delete line 2 from INPUT
    # iptables -D INPUT 2
     
    Help
    # ipset --help
    ipset v6.23

    Usage: ipset [options] COMMAND

    Commands:
    create SETNAME TYPENAME [type-specific-options]
            Create a new set
    add SETNAME ENTRY
            Add entry to the named set
    del SETNAME ENTRY
            Delete entry from the named set
    test SETNAME ENTRY
            Test entry in the named set
    destroy [SETNAME]
            Destroy a named set or all sets
    list [SETNAME]
            List the entries of a named set or all sets
    save [SETNAME]
            Save the named set or all sets to stdout
    restore
            Restore a saved state
    flush [SETNAME]
            Flush a named set or all sets
    rename FROM-SETNAME TO-SETNAME
            Rename two sets
    swap FROM-SETNAME TO-SETNAME
            Swap the contect of two existing sets
    help [TYPENAME]
            Print help, and settype specific help
    version
            Print version information
    quit
            Quit interactive mode

    Options:
    -o plain|save|xml
           Specify output mode for listing sets.
           Default value for "list" command is mode "plain"
           and for "save" command is mode "save".
    -s
            Print elements sorted (if supported by the set type).
    -q
            Suppress any notice or warning message.
    -r
            Try to resolve IP addresses in the output (slow!)
    -!
            Ignore errors when creating or adding sets or
            elements that do exist or when deleting elements
            that don't exist.
    -n
            When listing, just list setnames from the kernel.

    -t
            When listing, list setnames and set headers
            from kernel only.
    -f
            Read from the given file instead of standard
            input (restore) or write to given file instead
            of standard output (list/save).

    Supported set types:
        list:set            3       skbinfo support
        list:set            2       comment support
        list:set            1       counters support
        list:set            0       Initial revision
        hash:mac            0       Initial revision
        hash:net,iface      6       skbinfo support
        hash:net,iface      5       forceadd support
        hash:net,iface      4       comment support
        hash:net,iface      3       counters support
        hash:net,iface      2       /0 network support
        hash:net,iface      1       nomatch flag support
        hash:net,iface      0       Initial revision
        hash:net,port       7       skbinfo support
        hash:net,port       6       forceadd support
        hash:net,port       5       comment support
        hash:net,port       4       counters support
        hash:net,port       3       nomatch flag support
        hash:net,port       2       Add/del range support
        hash:net,port       1       SCTP and UDPLITE support
        hash:net,port,net   2       skbinfo support
        hash:net,port,net   1       forceadd support
        hash:net,port,net   0       initial revision
        hash:net,net        2       skbinfo support
        hash:net,net        1       forceadd support
        hash:net,net        0       initial revision
        hash:net            6       skbinfo support
        hash:net            5       forceadd support
        hash:net            4       comment support
        hash:net            3       counters support
        hash:net            2       nomatch flag support
        hash:net            1       Add/del range support
        hash:net            0       Initial revision
        hash:ip,port,net    7       skbinfo support
        hash:ip,port,net    6       forceadd support
        hash:ip,port,net    5       comment support
        hash:ip,port,net    4       counters support
        hash:ip,port,net    3       nomatch flag support
        hash:ip,port,net    2       Add/del range support
        hash:ip,port,net    1       SCTP and UDPLITE support
        hash:ip,port,ip     5       skbinfo support
        hash:ip,port,ip     4       forceadd support
        hash:ip,port,ip     3       comment support
        hash:ip,port,ip     2       counters support
        hash:ip,port,ip     1       SCTP and UDPLITE support
        hash:ip,mark        2       sbkinfo support
        hash:ip,mark        1       forceadd support
        hash:ip,mark        0       initial revision
        hash:ip,port        5       skbinfo support
        hash:ip,port        4       forceadd support
        hash:ip,port        3       comment support
        hash:ip,port        2       counters support
        hash:ip,port        1       SCTP and UDPLITE support
        hash:ip             4       skbinfo support
        hash:ip             3       forceadd support
        hash:ip             2       comment support
        hash:ip             1       counters support
        hash:ip             0       Initial revision
        bitmap:port         3       skbinfo support
        bitmap:port         2       comment support
        bitmap:port         1       counters support
        bitmap:port         0       Initial revision
        bitmap:ip,mac       3       skbinfo support
        bitmap:ip,mac       2       comment support
        bitmap:ip,mac       1       counters support
        bitmap:ip,mac       0       Initial revision
        bitmap:ip           3       skbinfo support
        bitmap:ip           2       comment support
        bitmap:ip           1       counters support
        bitmap:ip           0       Initial revision
     
    References:

    Tuesday, March 22, 2016

    GNU grep

    grep prints lines that contain a match for a pattern. The general synopsis of the grep command line is grep options pattern input_file_names
    There can be zero or more options. pattern will only be seen as such (and not as an input_file_name ) if it wasn’t already specified within options (by using the ‘ -e pattern ’ or ‘ -f file’ options). There can be zero or more input_file_names.

    Matching Control

    -e pattern
    --regexp= pattern
    Use pattern as the pattern. This can be used to specify multiple search patterns, or to protect a pattern beginning with a ‘ - ’. (-e is specified by POSIX.)
    -f file
    --file= file
    Obtain patterns from file, one per line. The empty file contains zero patterns, and therefore matches nothing. (-f is specified by POSIX.)
    -i
    -y
    --ignore-case
    Ignore case distinctions, so that characters that differ only in case match each other. Although this is straightforward when letters differ in case only via lowercase-uppercase pairs, the behavior is unspecified in other situations. For example, uppercase “S” has an unusual lowercase counterpart “Å¿” (Unicode character U+017F, LATIN SMALL LETTER LONG S) in many locales, and it is unspecified whether this unusual character matches “S” or “s” even though uppercasing it yields “S”. Another example: the lowercase German letter “ß” (U+00DF, LATIN SMALL LETTER SHARP S) is normally capitalized as the two-character string “SS” but it does not match “SS”, and it might not match the uppercase letter “ẞ” (U+1E9E, LATIN CAPITAL LETTER SHARP S) even though lowercasing the latter yields the former.
    -y is an obsolete synonym that is provided for compatibility. (-i is specified by POSIX.)
    -v
    --invert-match
    Invert the sense of matching, to select non-matching lines. (-v is specified by POSIX.)
    -w
    --word-regexp
    Select only those lines containing matches that form whole words. The test is that the matching substring must either be at the beginning of the line, or preceded by a non-word constituent character. Similarly, it must be either at the end of the line or followed by a non-word constituent character. Word-constituent characters are letters, digits, and the underscore. This option has no effect if -x is also specified.
    -x
    --line-regexp
    Select only those matches that exactly match the whole line. For a regular expression pattern, this is like parenthesizing the pattern and then surrounding it with ‘ ^ ’ and ‘ $’. (-x is specified by POSIX.) 

    Examples: 

    to find authentication logs for “root” on an debian system:
    # grep "root" /var/log/auth.log
    For example, we can see that when someone failed to login as an admin, they also failed the reverse mapping which means they might not have a valid domain name.
    # grep -B 3 -A 2 'Invalid user' /var/log/auth.log
    To find authentication logs at current system date
    # grep  "$(date +%b) $(date +%e)" /var/log/auth.log | grep 'fail\|preauth'
    To find authentication logs at current system hour 
    # grep  "$(date +%b) $(date +%e) $(date +%H:)" /var/log/auth.log
    To find mail logs at current system date
    # grep  "$(date +%b) $(date +%e)" /var/log/mail.info
    To find mail logs at one hour before current system date
    # grep  "$(date --date="1 hours ago" +%b) $(date --date="1 hours ago" +%e)" /var/log/mail.info

    A list of date command field descriptors from http://www.cyberciti.biz/faq/unix-linux-bash-get-time/ (as a copy)

           %%     a literal %
     
           %a     locale's abbreviated weekday name (e.g., Sun)
     
           %A     locale's full weekday name (e.g., Sunday)
     
           %b     locale's abbreviated month name (e.g., Jan)
     
           %B     locale's full month name (e.g., January)
     
           %c     locale's date and time (e.g., Thu Mar  3 23:05:25 2005)
     
           %C     century; like %Y, except omit last two digits (e.g., 20)
     
           %d     day of month (e.g., 01)
     
           %D     date; same as %m/%d/%y
     
           %e     day of month, space padded; same as %_d
     
           %F     full date; same as %Y-%m-%d
     
           %g     last two digits of year of ISO week number (see %G)
     
           %G     year of ISO week number (see %V); normally useful only with %V
     
           %h     same as %b
     
           %H     hour (00..23)
     
           %I     hour (01..12)
     
           %j     day of year (001..366)
     
           %k     hour, space padded ( 0..23); same as %_H
     
           %l     hour, space padded ( 1..12); same as %_I
     
           %m     month (01..12)
     
           %M     minute (00..59)
     
           %n     a newline
     
           %N     nanoseconds (000000000..999999999)
           %p     locale's equivalent of either AM or PM; blank if not known
     
           %P     like %p, but lower case
     
           %r     locale's 12-hour clock time (e.g., 11:11:04 PM)
     
           %R     24-hour hour and minute; same as %H:%M
     
           %s     seconds since 1970-01-01 00:00:00 UTC
     
           %S     second (00..60)
     
           %t     a tab
     
           %T     time; same as %H:%M:%S
     
           %u     day of week (1..7); 1 is Monday
     
           %U     week number of year, with Sunday as first day of week (00..53)
     
           %V     ISO week number, with Monday as first day of week (01..53)
     
           %w     day of week (0..6); 0 is Sunday
     
           %W     week number of year, with Monday as first day of week (00..53)
     
           %x     locale's date representation (e.g., 12/31/99)
     
           %X     locale's time representation (e.g., 23:13:48)
     
           %y     last two digits of year (00..99)
     
           %Y     year
     
           %z     +hhmm numeric time zone (e.g., -0400)
     
           %:z    +hh:mm numeric time zone (e.g., -04:00)
     
           %::z   +hh:mm:ss numeric time zone (e.g., -04:00:00)
     
           %:::z  numeric time zone with :  to  necessary  precision  (e.g.,  -04,
                  +05:30)
     
           %Z     alphabetic time zone abbreviation (e.g., EDT)
           By  default,  date  pads  numeric  fields  with  zeroes.  The following
           optional flags may follow '%':
     
           -      (hyphen) do not pad the field
     
           _      (underscore) pad with spaces
     
           0      (zero) pad with zeros
     
           ^      use upper case if possible
     
           #      use opposite case if possible

    References: