Bash: A Beginner's Tutorial¶

Bash is both the command interpreter you use in a Linux terminal and a scripting language for repeatable automation. On Deceema, Bash connects files, software environments, analysis programs, and Slurm jobs into one workflow.

This tutorial assumes no programming experience. By the end, you will be able to:

Read and write a Bash script.
Store and safely expand variables.
Accept and validate command-line arguments.
Use conditions, loops, arrays, and functions.
Understand exit statuses, redirection, and pipelines.
Diagnose a script with syntax checking and traces.
Run a tested Bash workflow through Slurm.

Before you begin

Complete the Linux (RHEL) tutorial first if commands, paths, files, and permissions are unfamiliar. Type examples inside the disposable workspace created below, and do not type the leading $ shown in console examples.

1. What Bash Does¶

In an interactive terminal, Bash reads one command at a time. In a script, it reads commands from a text file so they can be reviewed, tested, and repeated.

Term	Plain-language meaning
Shell	A program that interprets commands.
Script	A text file containing commands for a shell to execute.
Variable	A named value stored by the shell.
Argument	A value supplied to a command or script.
Expansion	Bash replacing an expression such as `$HOME` with its value.
Exit status	A number describing whether a command succeeded or failed.
Standard output	A command's normal output stream.
Standard error	A separate stream for diagnostics and errors.
Pipeline	Commands connected so output flows into another command.

Confirm Bash is available:

$ bash --version

Check the shell running your current session:

$ printf '%s\n' "$SHELL"

2. Create a Practice Workspace¶

$ mkdir -p "$HOME/training/bash-beginner"/{data,scripts,logs,results}
$ cd "$HOME/training/bash-beginner"
$ pwd

Everything in this tutorial stays inside that directory.

Create a first script without needing a text editor:

$ printf '%s\n' \
    '#!/bin/bash' \
    'printf "Hello from Bash on Deceema\\n"' \
    > scripts/hello.sh

Inspect it, check its syntax, and run it:

$ cat scripts/hello.sh
$ bash -n scripts/hello.sh
$ bash scripts/hello.sh
Hello from Bash on Deceema

bash -n parses a script without executing its commands. No output means Bash found no syntax error.

The interpreter line¶

The first line is called a shebang:

#!/bin/bash

It tells Linux which interpreter should execute the file when it is run directly. Make the script executable and try that form:

$ chmod u+x scripts/hello.sh
$ ./scripts/hello.sh

During development, bash scripts/hello.sh remains explicit and works even without execute permission.

3. Add Comments and Print Output¶

Bash ignores text after #, except for the shebang:

# Explain why the next command is needed.
printf 'Analysis started\n'

Prefer printf for predictable formatted output:

$ printf 'User: %s\nDirectory: %s\n' "$USER" "$PWD"

Each %s is replaced by the corresponding later argument.

Write comments for decisions

A useful comment explains why a setting or workaround exists. Avoid merely repeating an obvious command in English.

4. Store Values in Variables¶

Assign a variable without spaces around =:

project_name="deceema-training"
input_path="data/values.txt"

Expand it with $name or ${name}:

$ project_name="deceema-training"
$ printf 'Project: %s\n' "$project_name"
Project: deceema-training

Use braces when text immediately follows a variable name:

$ run_id=42
$ printf 'Output: run_%s.csv\n' "$run_id"
$ printf 'Output: %s\n' "run_${run_id}.csv"

Environment variables¶

Variables such as HOME, USER, and PWD are already present in your shell:

$ printf 'Home: %s\nUser: %s\nWorking directory: %s\n' \
    "$HOME" "$USER" "$PWD"

export makes a variable available to child processes:

$ export TRAINING_MODE="beginner"
$ bash -c 'printf "Mode: %s\n" "$TRAINING_MODE"'

Do not export or print secrets into a shared environment or job log.

5. Learn Quoting Before Automation¶

Quoting controls how Bash interprets spaces, wildcard characters, and variable expressions.

Form	Behavior
`"$value"`	Expands variables while preserving the result as one argument.
`'$value'`	Preserves every character literally; no variable expansion.
`$value`	Allows word splitting and wildcard expansion; risky for paths.

Compare single and double quotes:

$ language=Bash
$ printf '%s\n' "Learning $language"
Learning Bash
$ printf '%s\n' 'Learning $language'
Learning $language

See why quoted paths matter:

$ path="results/my report.txt"
$ printf 'Result path: %s\n' "$path"
Result path: results/my report.txt

Quote variable expansions by default

"$path" is one argument. Unquoted $path can become multiple arguments when its value contains whitespace, or can expand wildcard characters.

6. Use Command Substitution and Arithmetic¶

Command substitution captures a command's standard output:

$ current_time=$(date '+%Y-%m-%dT%H:%M:%S%z')
$ printf 'Current time: %s\n' "$current_time"

Arithmetic expansion evaluates integer expressions:

$ completed=3
$ failed=1
$ total=$((completed + failed))
$ printf 'Total runs: %d\n' "$total"
Total runs: 4

Keep complex numerical analysis in Python, R, or another suitable language; Bash arithmetic is mainly useful for counters and simple control flow.

7. Accept Command-Line Arguments¶

When a script runs:

Expression	Meaning
`$0`	Script name.
`$1`, `$2`, …	First, second, and later arguments.
`$#`	Number of supplied arguments.
`"$@"`	All arguments, each preserved separately.

Save this as scripts/greet.sh:

scripts/greet.sh
#!/bin/bash
set -u

if (( $# != 1 )); then
  printf 'Usage: %s NAME\n' "$0" >&2
  exit 2
fi

name=$1
printf 'Welcome, %s\n' "$name"

Test both paths:

$ bash scripts/greet.sh
Usage: scripts/greet.sh NAME
$ bash scripts/greet.sh "Ada Lovelace"
Welcome, Ada Lovelace

The quoted argument remains one value even though it contains a space.

Provide an optional default¶

${1:-value} uses a default when the first argument is missing or empty:

name=${1:-Deceema user}

Use defaults only when they are safe and unambiguous. Required input should fail with a clear usage message.

8. Understand Exit Status and Failure¶

Every command returns an exit status. Zero indicates success; non-zero indicates a failure condition.

$ test -d data
$ printf 'Status: %s\n' "$?"
Status: 0

Use && when the second command should run only after success:

$ test -d data && printf 'The data directory exists\n'

Use || for a fallback after failure:

$ test -f data/missing.txt || printf 'The file is missing\n' >&2

Start workflow scripts defensively¶

set -euo pipefail

-e exits after many unhandled command failures.
-u treats expansion of an unset variable as an error.
pipefail makes a pipeline fail when any component fails.

These settings do not understand your intent. Validate expected conditions and handle expected failures explicitly.

9. Test Files, Strings, and Numbers¶

Bash's [[ ... ]] expression evaluates conditions safely.

File tests¶

Test	True when
`-e PATH`	The path exists.
`-f PATH`	It is a regular file.
`-d PATH`	It is a directory.
`-r PATH`	It is readable.
`-w PATH`	It is writable.
`-s PATH`	It is a non-empty file.

if [[ ! -s "$input_path" ]]; then
  printf 'Input is missing or empty: %s\n' "$input_path" >&2
  exit 1
fi

String tests¶

if [[ -z "$project_name" ]]; then
  printf 'Project name cannot be empty\n' >&2
  exit 1
fi

-z means the string is empty.
-n means it is non-empty.
== compares string values inside [[ ... ]].

Integer comparisons¶

Use arithmetic syntax for integers:

if (( total > 0 )); then
  printf 'At least one item was processed\n'
fi

10. Redirect Output and Errors¶

Every command begins with three standard streams:

Stream	Number	Typical destination
Standard input	`0`	Keyboard or another command.
Standard output	`1`	Terminal or result file.
Standard error	`2`	Terminal or diagnostic log.

Common redirections:

command > output.txt       # replace standard output
command >> output.txt      # append standard output
command 2> error.txt       # replace standard error
command > run.log 2>&1     # combine both streams

Create training data and separate output from diagnostics:

$ printf '8\n2\n6\n4\n' > data/values.txt
$ wc -l data/values.txt > results/line-count.txt \
    2> logs/line-count.err
$ cat results/line-count.txt

Check a target before using > because it replaces existing contents.

11. Build Pipelines¶

A pipeline connects standard output from one command to standard input of the next:

$ sort -n data/values.txt | uniq

Calculate a sum with awk:

$ awk '{ total += $1 } END { print total }' data/values.txt
20

Combine filtering, sorting, and counting:

$ grep -v '^#' data/values.txt | sort -n | uniq -c

With set -o pipefail, an early failure is reflected in the pipeline's final status instead of being hidden by a successful last command.

Tip

A pipeline transforms a stream. If intermediate data must be inspected, split the pipeline into named files or smaller commands while debugging.

12. Repeat Work with Loops¶

Create three inputs:

$ printf '1\n2\n' > data/run-a.txt
$ printf '3\n4\n' > data/run-b.txt
$ printf '5\n6\n' > data/run-c.txt

Loop over them:

for input_path in data/run-*.txt; do
  [[ -e "$input_path" ]] || continue
  printf '%s has %s lines\n' \
    "$input_path" "$(wc -l < "$input_path")"
done

The existence check handles the case where the wildcard matches no files.

A counter loop¶

for (( index = 1; index <= 3; index++ )); do
  printf 'Iteration %d\n' "$index"
done

When hundreds of independent iterations need compute resources, use a Slurm job array rather than a long login-node loop.

13. Store Lists in Arrays¶

Bash arrays keep values separate without encoding them into one string:

inputs=(
  "data/run-a.txt"
  "data/run-b.txt"
  "data/run-c.txt"
)

printf 'Input count: %d\n' "${#inputs[@]}"

for input_path in "${inputs[@]}"; do
  printf 'Input: %s\n' "$input_path"
done

Arrays are also safer than constructing a command as a string:

command=(wc -l -- "$input_path")
"${command[@]}"

Avoid eval; it reparses text as Bash code and can turn data into unintended commands.

14. Organize Work with Functions¶

Functions group commands under clear names:

log() {
  printf '%s %s\n' "$(date '+%Y-%m-%dT%H:%M:%S%z')" "$*" >&2
}

require_readable_file() {
  local path=$1

  if [[ ! -r "$path" ]]; then
    log "Input is not readable: $path"
    return 1
  fi
}

log "Starting analysis"
require_readable_file "data/values.txt"

local prevents a function's variable from overwriting one in the rest of the script.
return sets the function's exit status.
$* combines function arguments for this human-readable log message. For forwarding arguments to another command, use "$@" instead.

15. Build a Complete Analysis Script¶

Save this as scripts/summarize.sh:

scripts/summarize.sh
#!/bin/bash
set -euo pipefail

usage() {
  printf 'Usage: %s INPUT OUTPUT [RUN_NAME]\n' "$0" >&2
}

log() {
  printf '%s %s\n' "$(date '+%Y-%m-%dT%H:%M:%S%z')" "$*" >&2
}

if (( $# < 2 || $# > 3 )); then
  usage
  exit 2
fi

input_path=$1
output_path=$2
run_name=${3:-training-run}

if [[ ! -s "$input_path" ]]; then
  log "Input is missing or empty: $input_path"
  exit 1
fi

output_dir=$(dirname -- "$output_path")
mkdir -p -- "$output_dir"

log "Starting $run_name"

awk -v run_name="$run_name" '
  NF { total += $1; count += 1 }
  END {
    if (count == 0) exit 2
    printf "run=%s\ncount=%d\nmean=%.2f\n", \
      run_name, count, total / count
  }
' "$input_path" > "$output_path"

log "Wrote $output_path"

Validate and run it¶

$ bash -n scripts/summarize.sh
$ bash scripts/summarize.sh \
    data/values.txt results/summary.txt first-analysis \
    2> logs/summary.log
$ cat results/summary.txt
$ cat logs/summary.log

Expected result:

run=first-analysis
count=4
mean=5.00

Test failure paths¶

A useful script should fail clearly when used incorrectly:

$ bash scripts/summarize.sh
$ bash scripts/summarize.sh data/missing.txt results/missing.txt

Confirm that each prints a helpful diagnostic and returns non-zero.

16. Debug a Script¶

Check syntax¶

$ bash -n scripts/summarize.sh

Trace expanded commands¶

$ bash -x scripts/summarize.sh \
    data/values.txt results/debug-summary.txt \
    2> logs/debug-trace.log

-x can expose expanded values, so never use a trace where passwords, tokens, private keys, or other secrets could appear.

Add a temporary checkpoint¶

printf 'input=%q output=%q\n' "$input_path" "$output_path" >&2

%q prints a shell-escaped representation helpful for diagnosing spaces and special characters.

If ShellCheck is available in your approved environment, it can identify many common issues:

$ command -v shellcheck
$ shellcheck scripts/summarize.sh

Do not assume ShellCheck is installed; bash -n is still useful everywhere Bash is available.

17. Run the Workflow with Slurm¶

Save this as scripts/bash-job.sh:

scripts/bash-job.sh
#!/bin/bash
#SBATCH --job-name=bash-training
#SBATCH --account=PROJECT_CODE
#SBATCH --qos=QOS_NAME
#SBATCH --time=00:05:00
#SBATCH --ntasks=1
#SBATCH --cpus-per-task=1
#SBATCH --mem=512M
#SBATCH --output=logs/%x-%j.out
#SBATCH --error=logs/%x-%j.err

set -euo pipefail

module purge
module load deceema

printf 'Job %s started on %s at %s\n' \
  "$SLURM_JOB_ID" "$(hostname)" "$(date '+%Y-%m-%dT%H:%M:%S%z')"

bash scripts/summarize.sh \
  data/values.txt \
  "results/slurm-${SLURM_JOB_ID}.txt" \
  "slurm-${SLURM_JOB_ID}"

Replace PROJECT_CODE and QOS_NAME with the matching values from your approved Deceema access. Submit from the practice directory:

$ sbatch scripts/bash-job.sh

Then use the returned ID:

$ squeue --jobs=JOB_ID
$ sacct --jobs=JOB_ID --format=JobID,State,ExitCode,Elapsed

The script uses project-relative paths, so its submission directory is part of the workflow. For deeper scheduling guidance, see Jobs on Deceema.

18. Script Safely on Shared Infrastructure¶

Quote every path and use -- before file operands where supported.
Validate required input before creating or replacing output.
Keep source data separate from generated files.
Avoid eval and commands assembled from untrusted text.
Never store passwords, tokens, private keys, or credentials in scripts.
Remember that bash -x can reveal expanded values in logs.
Never recursively delete an unresolved variable, wildcard, home directory, project root, or unverified path.
Test with disposable data before processing valuable project files.
Run computationally demanding or long-lived work through Slurm.
Preserve the script, input identity, software environment, job ID, and logs needed to reproduce a result.

19. Final Project: Build a Batch Report¶

Create scripts/report.sh that:

Requires an input directory and output file.
Rejects a missing or unreadable input directory.
Finds every run-*.txt file without using eval.
Counts the files and all lines across them.
Writes the start time, input directory, file count, and line count to the output file.
Writes progress messages to standard error.
Passes bash -n.
Succeeds against the three data/run-*.txt files created earlier.

One possible solution:

scripts/report.sh
#!/bin/bash
set -euo pipefail

if (( $# != 2 )); then
  printf 'Usage: %s INPUT_DIRECTORY OUTPUT_FILE\n' "$0" >&2
  exit 2
fi

input_dir=$1
output_path=$2

if [[ ! -d "$input_dir" || ! -r "$input_dir" ]]; then
  printf 'Input directory is not readable: %s\n' "$input_dir" >&2
  exit 1
fi

inputs=("$input_dir"/run-*.txt)
if [[ ! -e "${inputs[0]}" ]]; then
  printf 'No run files found in %s\n' "$input_dir" >&2
  exit 1
fi

mkdir -p -- "$(dirname -- "$output_path")"
printf 'Processing %d files\n' "${#inputs[@]}" >&2

line_count=$(cat -- "${inputs[@]}" | wc -l)

{
  printf 'started=%s\n' "$(date '+%Y-%m-%dT%H:%M:%S%z')"
  printf 'input_directory=%s\n' "$input_dir"
  printf 'file_count=%d\n' "${#inputs[@]}"
  printf 'line_count=%d\n' "$line_count"
} > "$output_path"

printf 'Wrote %s\n' "$output_path" >&2

Validate and run the solution:

$ bash -n scripts/report.sh
$ bash scripts/report.sh data results/report.txt 2> logs/report.log
$ cat results/report.txt
$ cat logs/report.log

The timestamps will vary, but the report should include:

file_count=3
line_count=6

You can now automate a Deceema workflow

You have used variables, quoting, arguments, validation, arrays, pipelines, functions, redirection, diagnostics, and Slurm—the core Bash skills needed for reproducible HPC automation.

Bash Cheat Sheet¶

Goal	Syntax
Assign a variable	`name=value`
Expand safely	`"$name"`
All arguments	`"$@"`
Argument count	`$#`
Default value	`${name:-default}`
Capture output	`value=$(command)`
Integer arithmetic	`$((a + b))`
Test a file	`[[ -f "$path" ]]`
Conditional	`if condition; then ... fi`
Loop over values	`for item in ...; do ... done`
Define a function	`name() { ...; }`
Replace output file	`command > file`
Append output	`command >> file`
Redirect errors	`command 2> file`
Connect commands	`command1 \| command2`
Check syntax	`bash -n script.sh`
Trace execution	`bash -x script.sh`

Common Beginner Mistakes¶

Mistake	Better habit
Writing `name = value`	Assign without spaces: `name=value`.
Expanding a path as `$path`	Quote it: `"$path"`.
Confusing single and double quotes	Use double quotes when variables should expand.
Assuming printed text means success	Check exit status and expected output.
Depending on terminal history or active modules	Recreate the environment inside the script.
Hiding an early pipeline failure	Enable `pipefail` and test the pipeline.
Constructing a command with `eval`	Use an array and `"${command[@]}"`.
Debugging secrets with `bash -x`	Keep secrets out of scripts and disable traces around sensitive operations.
Using a login-node loop for heavy work	Use a Slurm job or array.