| Finding Passwords in Logs and Shell HistoryKrazyWorks

Networking

Unix and Linux network configuration. Multiple network interfaces. Bridged NICs. High-availability network configurations.

Applications

Reviews of latest Unix and Linux software. Helpful tips for application support admins. Automating application support.

Data

Disk partitioning, filesystems, directories, and files. Volume management, logical volumes, HA filesystems. Backups and disaster recovery.

Monitoring

Distributed server monitoring. Server performance and capacity planning. Monitoring applications, network status and user activity.

Commands & Shells

Cool Unix shell commands and options. Command-line tools and application. Things every Unix sysadmin needs to know.

Home » Featured, Security

Finding Passwords in Logs and Shell History

Submitted by Igor on June 27, 2019 – 11:15 am

Sooner or later it will happen: you type something after which you expect a password prompt then, without looking, you type the password. However, you fat-fingered the first command, and your password ended up in clear text in your shell history, maybe the system log, and who knows what remote syslog server.

Logs live forever. I’ve seen servers with logs that were older than the server (probably copied from the old server for some reason). With the help from remote syslog and various real-time indexers like Splunk or Graylog, logs are almost indestructible. Access to log data is generally poorly controlled with read permission granted to everyone more often than not.

All these reasons, combined with the fact that people make mistakes, make log files a valuable source of information for the hackers. It may seem that extracting strings that might be passwords from log data would be very challenging and time-consuming. Not so. Standard requirements for password strength make this task much more manageable.

Here’s a simple example to illustrate this point. Is this string a password: password? What about this one: P@ssw0rd1? See my point? Good, let’s continue.

Below is the script (also on GitHub) you can run on just about any modern Linux server to check users’ .bash_history and /var/log/messages for possible passwords. Naturally, you need to be root to do this. You can modify the script to only look at log files you can access with your credentials.

The script

#!/bin/bash
# Up to how many time do you think a user might enter his password in plain text in shell?
# This is to exclude some password-like strings in various log files that appear on a regular basis.
stupid_limit=5

# This is the regex that matches what may very well be a password
include_string="(?=[a-zA-Z0-9\!#@$?]{8,}$)(?=.*?[a-z])(?=.*?[A-Z])(?=.*?[0-9]).*"

# This regex excludes certain strings that look like passwords but you already know they aren't.
# Feel free to modify this to better match your environment.
exclude_string="11gSoftwareR4|Xms1536m|(Jan(uary)?|Feb(ruary)?|Mar(ch)?|Apr(il)?|May|Jun(e)?|\
Jul(y)?|Aug(ust)?|Sep(tember)?|Oct(ober)?|Nov(ember)?|Dec(ember)?)([0-9]{1,})?20[0-9]{2}"

# Here we search a few directories commonly containing logs and shell history files. Search depth is limited,
# so not to go down some NFS-mounted rabbit hole. Here we're looking specifically for users' .bash_history
# and the primary system log. You can add application and system daemon  logs as you need.
find / /home /var/log -mindepth 1 -maxdepth 3 -mount -type f -name "\.bash_history" -o -name "messages" | while read i
do
   # If something is matched
   if [ $(egrep -vi "${exclude_string}" "${i}" | grep -cP "${include_string}") -gt 0 ]
   then
     # Take a closer look
     egrep -vi "${exclude_string}" "${i}" | grep -oP "${include_string}" | sort -u | while read p
     do
       # Does it appear too often to be a user mistake?
       if [ $(grep -c "${p}" "${i}") -lt ${stupid_limit} ]
       then
         # If it looks just right, print the suspected password along with some other details
         echo "$(stat -c %U" "%y" "%n "${i}") ${p}"
         # This is an optional section commented out by default to reduce clutter. This command will
         # grep for an example of the password and a few lines before and after to put things in context
         #echo "-------------------------------------------------"
         #grep -m1 -B4 -A4 "${p}" "${i}"
         #echo "-------------------------------------------------"
         #echo ""
       fi
     done
   fi
done

If you’re a sysadmin and have access to configuration management tools like Salt or Ansible, you can run this script on multiple systems in parallel. Here’s an example of running the script via Salt CLI:

salt "prod-tomcat*" cmd.script "salt://scripts/bash_history_password_find.sh"

And below is sample output. The first three lines look like the user accidentally copy-pasted into command line an encryption key or somesuch. But the other four lines clearly contain two very stupid passwords for which jjames will receive a beating.

Sample output

prod-tomcat-node02.domain.local      tomcat    2019-03-20  05:14:16.000000000  -0400  /home/tomcat/.bash_history    nXhvR1sCAwEBBaMhMB8wHQYCDD0OBBYE
prod-tomcat-node02.domain.local      tomcat    2019-03-20  05:14:16.000000000  -0400  /home/tomcat/.bash_history    y1ynn3Ln76k0isKBwjzsEFmmFHunDjHmHVhINwq
prod-tomcat-node02.domain.local      tomcat    2019-03-20  05:14:16.000000000  -0400  /home/tomcat/.bash_history    yXkQ1kk9MkIg40HfB191vlZARsGgnWCjFfv8niYfVFkyPV
prod-tomcat-node01.domain.local      jjames    2019-01-28  11:59:58.000000000  -0500  /home/jjames/.bash_history    Hot@123
prod-tomcat-node01.domain.local      jjames    2019-01-28  11:59:58.000000000  -0500  /home/jjames/.bash_history    Hot@123
prod-tomcat-node03.domain.local      tomcat    2019-04-08  13:21:55.000000000  -0400  /home/tomcat/.bash_history    Mypass@123
prod-tomcat-node03.domain.local      tomcat    2019-04-08  13:21:55.000000000  -0400  /home/tomcat/.bash_history    Mypass@123