How to do X in Bash

I have written various bash scripts throughout my career. This post will act as a living documents of the different things I do in a bash script as a reminder for myself in future and anyone else that may find it useful. I tried to structure this document in a logical manner, I will play with this structure over time and see what works.

Shebang

As per this question I use the following shebang:

#!/usr/bin/env bash

Use a .env file for common variables

This file is simply a .sh file with your environment variables in that you source in one place. For example say we have the following .env file:

SOME_PROGRAM_VERSION=1.1.0

Then to use it in a script simply source it as below:

# ...
# somewhere at the top of your script
. ./.env
# now you can refer to ${SOME_PROGRAM_VERSION} later in your script

Use common.sh for Common Functions

This is similar to .env except that it is used to store common functions. For example if you want to be able to easily echo a row of dashes for easier output I might have the following function in my common.sh file:

#!/usr/bin/env bash
set -e

function horizontal_line() {
    echo "----------------------------------------------------------------------------------------------------------------------------------------"
}

And to use it in another script I would do so as follows:

# ...
# somewhere at the top of your script
. ./.env
. ./common.sh

# ...
# in some function where I want to output a horizontal line simply use
horizontal_line
echo "Foo"
horizontal_line

Ensure functions start in a Common Location

At the top of your bash file before any functions simply cd to/path/you/need.

For example:

cd ~/myapp

function foo() {
     # foo does not need to cd to ~/myapp as it is already here
}

function bar() {
    # Yup you guessed it! bar is also already in myapp (assuming you run this functions in isolation and not after each other)
}

Working with Files and Folders

Check if a File Exists

if [ -f path/to/file ]; then
    echo "File exists"
    # do whatever else you want to with the file
else
    # do whatever you need to if the file does not exist
fi

See more about this test operator here and a stackoverflow question that goes through other approaches.

Check If a CLI Command is Installed

This is generally of the form:

hash toolname 2>/dev/null || {
    # do whatever you want to do if the tool is not present
}

For example to check if docker is present and output an error if it is not:

hash docker 2>/dev/null || {
    echo >&2 "Docker not installed, it is needed to run this script" # by piping to >&2 we are outputting to the error stream
}

Convenience Function - Check if Exists and Suggest Install Step/s

function is_present() {
  hash "$1" 2>/dev/null || {
    echo "You do not have $1 installed"
    echo "Install this using: $2"
    exit 1
  }
}

# Then use it somewhere else for example:
is_present xsv "brew install xsv"

Only Run a CLI Command if it is Installed

This is similar to the previous one except we want to run a CLI command but only if it exists. To do this we use type as below:

if type someCommand > /dev/null; then someCommand; fi

Run the Unaliased / Unshadowed Command

Often times a command may be aliased by a more modern version of another command for example find is aliased to fd: alias find='fd'. But what if at somepoint you want to use the original find command (just once off)? Its simple actually you use the command tool: command find will call the unshadowed find.

Variables

I always wrap variables in " in case the value of the variable expands to something with spaces. For example:

"${num_nodes}"

and to subtract from a numeric variable:

"$((num_nodes -1))"

Iterating

for i in $(seq 0 "$((num_nodes -1))"); do echo ------"${i}";cd someDir"${i}";done

Or over multiple lines:

for i in $(seq 0 "$((num_nodes -1))"); do
    echo ------"${i}"
    cd someDir"${i}"
done

I do not think indentation is required it just helps for readability.

Handing Arguments to a Script

For example if you script needs at least one argument:

  if [ $# -lt 1 ]; then
    echo 1>&2 "$0: not enough arguments."
    echo "Usage: some_command 7"
    exit 2
  fi

Pass All Arguments to Another Command/Script

someCommand "$@" --some-other-flag

For example if the script was called with bash my-awesome-script.sh someFunction abc 2 then the above example would be evaluated as:

someCommand abc 2 --some-other-flag

See this question for more details.

Useful flags

Break on Errors

#!/usr/bin/env bash
set -e

# the rest of your script

Echo Commands Before Running Them

#!/usr/bin/env bash
set -x

# the rest of your script

Combining Multiple Set Options

#!/usr/bin/env bash
set -ex

# the rest of your script

Functions

whatever_name_you_want() {
    foo=${1} # this is the first argument passed for example: `whatever_name_you_want 1` then `foo` is assigned to `1`
}

Calling One Function in a File of Functions

Place the following at the bottom of your bash script:

if declare -f "$1" >/dev/null; then
  "$@"
else
  echo "'$1' is not a known function name" >&2
  exit 1
fi

Then to run just one function pretend it is the one from before whatever_name_you_want 2 and the script it is in is called my_awesome_script.sh then I would run just this function as follows:

bash my_awesome_script whatever_name_you_want 2

Checking What OS The Script is Running In

function get_running_os_name() {
  unameOut="$(uname -s)"
  case "${unameOut}" in
    Linux*)     machine=linux;;
    Darwin*)    machine=darwin;;
    CYGWIN*)    echo "Cygwin not supported" && exit 126;;
    MINGW*)     echo "MinGw not supported" && exit 126;;
    *)          echo "${unameOut} not supported" && exit 126;;
  esac
  echo "${machine}"
}

Making a Function Return a Value

function something_that_returns() {
    MY_VAR="hello"
    # reassign this and do whatever else you need to
    # to return the value of MY_VAR we simply:
    echo "${MY_VAR}"
}

# later if we want to assign the result of the something_that_returns function we do so as follows:

ANOTHER_VAR=$(something_that_returns)

Files

Waiting for Files to be Created

while [ ! -f path/to/someFile.txt ]
do
  echo "Waiting for someFile.txt to be created"
  sleep 1
done

Replacing All Text in a File In-Place

# -i means in-place, leave this out to not replace the text in the file and instead output the replaced file contents
sed -i "s/texttoreplace/newtext/g" path/to/somefile

See this answer for more details.

String Manipulation

Get the Last N Characters of a String

foo="1234567890"
# to get the last 3 characters of this string i.e. 890
echo -n $foo | tail -c 3
# to assign this to another variable
bar=$(echo -n $foo | tail -c 3)