Networking

Unix and Linux network configuration. Multiple network interfaces. Bridged NICs. High-availability network configurations.

Applications

Reviews of latest Unix and Linux software. Helpful tips for application support admins. Automating application support.

Data

Disk partitioning, filesystems, directories, and files. Volume management, logical volumes, HA filesystems. Backups and disaster recovery.

Monitoring

Distributed server monitoring. Server performance and capacity planning. Monitoring applications, network status and user activity.

Commands & Shells

Cool Unix shell commands and options. Command-line tools and application. Things every Unix sysadmin needs to know.

Home » Commands & Shells, Featured

Bash scripting: lists and random things

Submitted by on April 16, 2013 – 12:39 pm One Comment

Imagine you have an HPC cluster with a hundred compute nodes named node001-node100. The two commands below will help you generate a list of node names – either all name on one line or one name per line. Such a list can be useful as input in a “for” or “while” loops.

Node names on one line:

eval "echo node{`seq -w -s ',' 1 100`}"

Example:
[root@wil-dev-l04]# eval "echo node{`seq -w -s ',' 1 100`}"
node001 node002 node003 ... node100

Node name one per line:
for i in `seq -w 1 100` ; do echo node$i ; done

Example:
[root@wil-dev-l04]# for i in `seq -w 1 100` ; do echo node$i ; done
node001
node002
node003
...
node100

While we are on the subject of generating tediouly long lists of sequential things, here’s a quick example of generating every usable IP address for 192.168.0.1/24 subnet
for a in `seq 192 192`
do
  for b in `seq 168 168`
  do
    for c in `seq 0 255`
    do
      for d in `seq 1 254`
      do
        echo "${a}.${b}.${c}.${d}"
      done
    done
  done
done

Example:
[root@wil-dev-l04]# for a in `seq 192 192`; do for b in `seq 168 168`; do for c in `seq 0 255`; do for d in `seq 1 254`; do echo "${a}.${b}.${c}.${d}"; done; done; done; done
192.168.0.1
192.168.0.2
192.168.0.3
...
192.168.255.254

You can also use shell brace expansion like so:
echo -e 192.168.{0..255}.{1..254}'\n'

This is simpler than the loop, however, this method should be avoided for very large ranges.

Here’s pointless but fun scripting exercise. First, we generate a directory structure going three levels deep and looking something like this:

level1_{1..10}
  ---level2_{1..10}
    ---level3_{1..10}
      -file{1..10}

So in the end we have 1111 directories and 10000 files.
target_dir="/tmp/source" ; mkdir -p "${target_dir}"
for i in `seq 1 10`
do
  mkdir -p "${target_dir}/level1_${i}"
  for j in `seq 1 10`
  do
    mkdir -p "${target_dir}/level1_${i}/level2_${j}"
    for k in `seq 1 10`
    do
      mkdir -p "${target_dir}/level1_${i}/level2_${j}/level3_${k}"
      for l in `seq 1 10`
      do
        touch "${target_dir}/level1_${i}/level2_${j}/level3_${k}/file_level3_${k}_${l}.file"
      done
    done
  done
done

Next step is to configure an array of some common commands
set -a commands
commands[0]='tar cvf archive_`expr $RANDOM % 1000`.tar'
commands[1]="gzip -fr"
commands[2]="/bin/rm -r"
commands[3]="bzip2"

Now we run a “find” and for every file or directory execute a random command from the array
find "${target_dir}" | while read line ; do eval "${commands[$(echo `expr $RANDOM % 4`)]} `echo ${line}`" ; done

After this whole random mess is done, what we end up with is very different from the original 1111 directories and 10000 files:
[root@wil-dev-l04]# find . -type d | wc -l
530
[root@wil-dev-l04]# find . -type f | wc -l
4552
[root@wil-dev-l04]# find . -type f -name "*.tar" | wc -l
934
[root@wil-dev-l04]# find . -type f -name "*.gz" | wc -l
2562
[root@wil-dev-l04]# find . -type f -name "*.bz2" | wc -l
1056

What is the point of this? No point, just an exercise in randomness to sharpen our scripting skills.

Print Friendly, PDF & Email

One Comment »

Leave a Reply

%d bloggers like this: