2 Navigating the filesystem

Introduction to the Bash Shell

Part I — Orientation Notebook 2

The filesystem as a tree, and how to move through it: where you are, how to get elsewhere with paths, and how to see what is around you.

Raymond Amador v1.0.0 · CC BY 4.0 (text) / MIT (code)

What this notebook is about

In Notebook 1 you learned to run a command and to find help on your own. But a command always acts somewhere: it lists this folder, opens that file. So the next question is where: where you are in the filesystem, and how to get to where the file you want lives.

The filesystem is a tree of folders, and this notebook is about walking it. Our playground is the course’s real data/ folder: .xyz trajectories and simulation inputs from the Molecular and Materials Modelling course. You do not need to know what any of these files are. They are just files in folders; the skill is reaching them, and that skill is identical whether the file holds atomic coordinates or a grocery list.

The filesystem is a tree

Every file on the machine lives in a directory (a “folder”), directories nest inside other directories, and the whole thing hangs off a single top called the root, written /. Follow the branches down and you reach every file; the list of branches you followed, joined by slashes, is the file’s path: its address in the tree.

Two landmarks you will lean on constantly: / is the root of everything, and ~ is your home directory, the branch you start on and where your own files live. Here is the patch of tree we are about to explore:

The course data, as a tree

/                              the root — everything is under here
└── ~  (your home directory)      where you start
    └── data/                  the playground
        ├── trajectories/
        │   ├── lj38-optimization.xyz
        │   └── lj38-relaxed.xyz
        ├── inputs/
        │   ├── geo-opt.inp
        │   ├── production-md.inp
        │   └── replica-exchange.lammps
        ├── results/
        │   ├── sn2-neb.ener
        │   └── pt-slab.xyz
        ├── logs/
        │   ├── gr2hno3-nvt.log
        │   └── gr2hno3-restart.log
        ├── scaling/
        │   ├── strong_scaling.csv
        │   └── weak_scaling.csv
        └── .dataset-notes     (hidden — see ls -a, below)

Where am I — pwd and the prompt

When you are unsure where you stand, ask. pwd (“print working directory”) prints the absolute path of the directory you are currently in:

pwd

/home/runner/work/bash-primer/bash-primer

That is the working directory: the folder every relative command is measured from. There is a quieter way to know it, too: the prompt itself can show your location. In this course’s live terminal the prompt is set to display the working directory, so it reads something like ~/data $, where the $ still means “your turn”, but the part before it tells you where you are.

See it for yourself

The rendered cells on this page keep a plain $ prompt, because a printed page cannot know where you are. To watch the prompt track your location as you move, open the live terminal and run a few cd commands; the path before the $ changes with every step.

Moving around — cd

To go somewhere, use cd (“change directory”) and give it the path of where you want to be.

cd — change the working directory — move around the tree.

<dir>	go into a directory (by absolute or relative path)
~ / cd	go to your home directory (bare cd does this too)
..	go up to the parent directory
-	go back to the previous directory you were in
/abs/path	go to an absolute location, counted from the root /

more: help cd

Watch out: bare cd with no argument jumps to your home directory — surprising the first time

Let’s walk down into the data and back up. cd prints nothing when it succeeds: just like a real terminal, the only sign it worked is that the next pwd reports the new location:

cd data

pwd

/home/runner/work/bash-primer/bash-primer/data

cd ..

pwd

/home/runner/work/bash-primer/bash-primer

The .. meant “up one level”, so we landed back where we started. That little .. is one of a handful of path shorthands worth knowing cold, which brings us to the heart of the notebook.

Absolute vs relative paths

There are two ways to name any place in the tree, and the difference is the single most useful idea in this notebook.

• An absolute path starts from the root / and spells out every branch: /home/you/data/inputs/geo-opt.inp. It means the same thing no matter where you are standing; it is the file’s full address.

• A relative path starts from where you are now (your working directory): from the repo root, the same file is just data/inputs/geo-opt.inp. Shorter, but its meaning depends on where you stand.

Three shorthands appear in relative paths constantly: . is “here” (the current directory), .. is “up one level” (the parent), and ~ is your home directory.

Here is the same file reached both ways. First relatively, from where we are:

ls data/results/sn2-neb.ener

data/results/sn2-neb.ener

Now the absolute path to the very same file: $(pwd) fills in the full address of where we are, and we tack the rest on:

ls "$(pwd)/data/results/sn2-neb.ener"

/home/runner/work/bash-primer/bash-primer/data/results/sn2-neb.ener

Same file, two names. Use a relative path when the thing is near you (less to type); reach for an absolute path when you need to name a place unambiguously, from anywhere.

Listing — ls

You have met ls already; now meet its workhorse flags. They turn a bare list of names into something you can actually read: sizes, dates, hidden files, and more.

ls — list directory contents.

-l	long format: permissions, size, owner, timestamp
-a	include hidden dotfiles
-h	human-readable sizes (with -l)
-t	sort by modification time, newest first
-R	recurse into subdirectories

more: man ls

Watch out: dotfiles stay hidden without -a; flags combine, e.g. ls -lh

A one-line motivation, then the physics-optional release valve: the data/ files are simulation outputs, and a real reason to list them is to see which run produced the most data. You do not need to know what the files are; we are only reading their names, sizes, and dates. Plain ls just names them:

ls data/trajectories

lj38-optimization.xyz  lj38-relaxed.xyz

Add -l for the long form: one file per line, with permissions, size (in bytes), and a timestamp:

ls -l data/trajectories

total 160
-rw-r--r-- 1 runner runner 158697 Apr  1  2023 lj38-optimization.xyz
-rw-r--r-- 1 runner runner   2517 Apr  1  2023 lj38-relaxed.xyz

Those byte counts are hard to eyeball. Flags combine, so add -h for human-readable sizes:

ls -lh data/trajectories

total 160K
-rw-r--r-- 1 runner runner 155K Apr  1  2023 lj38-optimization.xyz
-rw-r--r-- 1 runner runner 2.5K Apr  1  2023 lj38-relaxed.xyz

Now the size difference jumps out: one trajectory is far larger than the other. And remember the gotcha from the card: without -a, the hidden .dataset-notes file is invisible:

ls -a data

.  ..  .dataset-notes  README.md  inputs  logs	results  scaling  trajectories

There it is, alongside . (this directory) and .. (its parent).

Seeing structure — tree

ls shows one directory at a time. To see a whole branch at a glance (folders within folders), use tree:

tree — show a directory and everything beneath it as an indented tree.

-L n	descend at most n levels (keep deep trees readable)
-d	show directories only, not files

more: man tree · tree --help

Watch out: not always preinstalled — it ships in this course's image; elsewhere install it with your package manager

tree -L 2 data

data
├── README.md
├── inputs
│   ├── geo-opt.inp
│   ├── production-md.inp
│   └── replica-exchange.lammps
├── logs
│   ├── gr2hno3-nvt.log
│   └── gr2hno3-restart.log
├── results
│   ├── pt-slab.xyz
│   └── sn2-neb.ener
├── scaling
│   ├── README.md
│   ├── generate.py
│   ├── strong_scaling.csv
│   └── weak_scaling.csv
└── trajectories
    ├── lj38-optimization.xyz
    └── lj38-relaxed.xyz
6 directories, 14 files

That is the same patch of tree from the top of the notebook, now drawn by the machine from the real folders. The machine’s version differs from the hand-drawn sketch in a few small ways: tree lists names alphabetically, it shows the README.md and generate.py files the sketch left out, and without -a it hides the .dataset-notes the sketch called out. Same tree, drawn honestly. -L 2 kept it to two levels deep; drop it on a big tree and you may get more than you bargained for.

Tab completion — the navigation superpower

Everything above you can do with careful typing. This last one you can only do by pressing a key, so it cannot be shown in a printed cell. But it is the single biggest speed-up in the whole notebook, so do not skip it.

Start typing a command or a path and press Tab: the shell completes it for you. Type cd data/tr then Tab, and the shell fills in cd data/trajectories/. Press Tab twice and it lists all the options that match. It cuts typos, saves keystrokes, and means you never again misspell a long filename like lj38-optimization.xyz.

Practise this in your terminal

Tab completion is interactive: there is nothing to render here, only something to do. Open the live terminal, type cd data/ and start pressing Tab, and feel how much faster moving around becomes. This one habit will save you more time than any other in this course.

Exercises

The pattern is the same as before: a task, a place for your answer, and an automatic ✓. Each starts from the repo root, so paths line up.

Exercise 1 (worked) — Where am I, what’s here

Report where you are with pwd, list what is in the current directory with ls, then list your home directory in full, including hidden files, with ls -la ~.

/home/runner/work/bash-primer/bash-primer
CHANGELOG.md	   SERIES_VERSION  data			     postBuild
CITATION.cff	   _config.yml	   environment.yml	     reference
CLAUDE.md	   _ext		   jupyter_server_config.py  references.bib
LICENSE-CODE	   _static	   manifest.yml		     requirements.txt
LICENSE-CONTENT    _toc.yml	   modulefiles		     robots.txt
NOTEBOOK_STYLE.md  apt.txt	   notebooks		     templates
README.md	   commands.yml    opt			     tools
total 80
drwxr-x---  15 runner runner 4096 Jul 20 06:06 .
drwxr-xr-x+  5 root   root   4096 Jul 14 09:17 ..
-rw-------   1 runner runner 1549 Jul 20 06:06 .bash_history
-rw-r--r--   1 runner runner  220 Mar 31  2024 .bash_logout
-rw-r--r--   1 runner runner   67 Jul 14 08:29 .bash_profile

✓ pwd reports where you are and ls runs

Exercise 2 (your turn) — Reach the data directory

Move into the data/ directory with cd, then list it in long, human-readable form with ls -lh. (Afterwards, pwd should end in data.)

total 24K
-rw-r--r-- 1 runner runner 2.9K Jun  1  2023 README.md
drwxr-xr-x 2 runner runner 4.0K Mar  1  2023 inputs
drwxr-xr-x 2 runner runner 4.0K Feb  1  2023 logs
drwxr-xr-x 2 runner runner 4.0K May  1  2023 results
drwxr-xr-x 2 runner runner 4.0K May 15  2023 scaling
drwxr-xr-x 2 runner runner 4.0K Apr  1  2023 trajectories

✓ you are now standing in the data directory

Exercise 3 (your turn) — Same file, two paths

Point at the file data/results/pt-slab.xyz with a relative path, then print its absolute path (hint: $(pwd) is the absolute path of where you are).

data/results/pt-slab.xyz
/home/runner/work/bash-primer/bash-primer/data/results/pt-slab.xyz

✓ the absolute path resolves to the same real file

Exercise 4 (worked) — Visualize the tree

Draw the data/ tree two levels deep with tree -L 2.

data
├── README.md
├── inputs
│   ├── geo-opt.inp
│   ├── production-md.inp
│   └── replica-exchange.lammps
├── logs
│   ├── gr2hno3-nvt.log
│   └── gr2hno3-restart.log
├── results
│   ├── pt-slab.xyz
│   └── sn2-neb.ener
├── scaling
│   ├── README.md
│   ├── generate.py
│   ├── strong_scaling.csv
│   └── weak_scaling.csv
└── trajectories
    ├── lj38-optimization.xyz
    └── lj38-relaxed.xyz
6 directories, 14 files

✓ the tree shows the trajectories/ subdirectory

Exercise 5 (your turn) — The right ls flags

List data/ with one combined set of flags so that you see hidden files, human-readable sizes, and entries sorted newest-first by time. Which entry is newest? (The flags -a, -h, -t, and -l combine into one.)

total 36K
-rw-r--r--  1 runner runner  460 Jun 15  2023 .dataset-notes
-rw-r--r--  1 runner runner 2.9K Jun  1  2023 README.md
drwxr-xr-x  7 runner runner 4.0K May 15  2023 .
drwxr-xr-x 13 runner runner 4.0K May 15  2023 ..
drwxr-xr-x  2 runner runner 4.0K May 15  2023 scaling
drwxr-xr-x  2 runner runner 4.0K May  1  2023 results
drwxr-xr-x  2 runner runner 4.0K Apr  1  2023 trajectories
drwxr-xr-x  2 runner runner 4.0K Mar  1  2023 inputs
drwxr-xr-x  2 runner runner 4.0K Feb  1  2023 logs

✓ sorted newest-first, the hidden .dataset-notes is the top entry

Exercise 6 (terminal-only) — Tab completion

This one has no ✓: it is interactive. In the live terminal, type cd data/ then the first letter or two of a subdirectory, and press Tab. Watch the shell finish the name. Then try it on a long filename inside trajectories/. That reflex (type a little, press Tab) is worth building now.

Outlook

You can now find your way to any file in the tree: you know where you are, how to move, how to name a place two different ways, and how to see what is around you. What you cannot yet do is look inside a file or move one around. That is the next notebook: cat, less, head, tail to peek at contents, and cp, mv, rm, mkdir to organize them. It is the last stretch of groundwork before the text-extraction heart of the course in Part II.

New in the Compendium

• cd — change the working directory — move around the tree.
• ls — list directory contents.
• tree — show a directory and everything beneath it as an indented tree.

See the full Compendium Scriptorum for every command met so far, and where to find it again.

Take this notebook with you

Open a live terminal from the “Practice here” box in any section to run everything yourself; nothing to install. The published notebooks ship without worked solutions; if you would like the reference solutions (to teach from or to check your own work), get in touch: hello@ramador.me.