11 Permissions and execution

11 Permissions and execution#

What this notebook is about#

In Notebook 9 you learned to write a file, and in Notebook 10 you learned how the shell reads the line you type. Put them together and you can author a little script: a shebang line and a couple of commands. Then you type its name, and the shell says:

command not found

The file is right there. You can cat it. Why won’t it run? Because nothing has yet told the system that this file may be run. A file is, by default, just data. This notebook is about the small set of conventions (the permission bits, the executable bit, the shebang, and $PATH) that promote a plain text file to a command. It is the bridge to Notebook 12, where you write real scripts.

(As ever, this is about files and the system, not physics.)

A. Reading permissions#

You have already seen permissions in passing: ls -l (Notebook 2) prints a cryptic string of letters at the start of every line. Here is one for a real file in data/:

ls -l data/trajectories/lj38-relaxed.xyz

-rw-r--r-- 1 runner runner 2517 Apr  1  2023 data/trajectories/lj38-relaxed.xyz

That leading -rw-r--r-- is the permission string, and it is not cryptic once you know it splits into four parts: a type character, then three triads (one each for the owner, the group, and everyone else), where each triad is some combination of r (read), w (write), and x (execute), with - meaning “not allowed.”

Anatomy of a permission string: -rwxr-xr-x

    -   rwx   r-x   r-x
    │    │     │     │
    │    │     │     └─ others : r-x   everyone else may read & execute, not write
    │    │     └─────── group  : r-x   the file's group may read & execute, not write
    │    └───────────── owner  : rwx   you, the owner, may read, write & execute
    └────────────────── type   : -     a regular file   ( d = directory )

The very first character is the file type, not a permission: - for a regular file, d for a directory. Here is that same ls -l, but on a directory (the -d flag means “the directory itself, not its contents”):

ls -ld data/trajectories

drwxr-xr-x 2 runner runner 4096 Apr  1  2023 data/trajectories

The line starts with d, and notice it has x bits, which on a directory mean something special, as we will see next.

B. What r, w, and x actually mean#

For a file, the three are what you would guess:

r: you may read its contents (cat, less, cp it).
w: you may change it (edit it, or overwrite it).
x: you may run it as a program. This is the one that matters for scripts, and the whole point of this notebook.

For a directory, the same three letters mean something slightly different, and the third is the classic source of confusion:

r: you may list what is inside (ls it).
w: you may add, rename, or remove entries in it.
x: you may enter it and reach things inside (cd into it, or name a file within it). On a directory, x is “may traverse,” not “may run.”

That last point trips everyone once: a directory with x is not “executable” in the script sense: it just means you are allowed to pass through it. Keep the two readings of x separate and the rest stays simple.

C. Changing permissions: `chmod`#

To change who may do what, you use chmod (“change mode”). It speaks two notations, and both are worth knowing because you will meet both in the wild.

chmod — change a file's permissions — who may read, write, and execute it.

+x / u+x / go-w / a+r	symbolic: add (`+`) or remove (`-`) a permission for user (`u`) / group (`g`) / others (`o`) / all (`a`) — `+x` is the one you reach for, it makes a file executable
755 / 644 / 700	octal: set all three triads at once — `755` = `rwx` for you and `r-x` for everyone else, `644` = read/write for you and read-only for others, `700` = full access for you and nothing for anyone else
-R	recurse into a directory tree, applying the change to everything inside (powerful — and easy to misfire; see the admonition)

more: man chmod

Watch out: chmod 777 makes a file world-writable — the rm -rf of permissions; never reach for it just to "make something work". And chmod -R aimed at the wrong directory is that same mistake at scale

The symbolic notation reads like a sentence: who (u user, g group, o others, a all), then + or -, then the permission. chmod +x file adds the execute bit; chmod go-w file removes write for group and others. Let’s watch the execute bit go on. Here is a fresh script with the default permissions:

ls -l scratch/greet.sh

-rw-r--r-- 1 runner runner 47 Jul 20 06:08 scratch/greet.sh

No x anywhere: -rw-r--r--. Add the execute bit and look again:

chmod +x scratch/greet.sh && ls -l scratch/greet.sh

-rwxr-xr-x 1 runner runner 47 Jul 20 06:08 scratch/greet.sh

Now x appears in all three triads (-rwxr-xr-x): the file is executable.

The octal notation is the compact idiom you will see in scripts and documentation. Each triad is a single digit, summing r=4, w=2, x=1. So 7 (=4+2+1) is rwx, 6 (=4+2) is rw-, 5 (=4+1) is r-x. The three everyday values:

644 = rw-r--r--: a normal data file (you read/write, others read).
755 = rwxr-xr-x: an executable script or a directory.
700 = rwx------: private to you, no access for anyone else.

Watch 644 strip the execute bit back off, then 755 restore it:

chmod 644 scratch/greet.sh && ls -l scratch/greet.sh

-rw-r--r-- 1 runner runner 47 Jul 20 06:08 scratch/greet.sh

chmod 755 scratch/greet.sh && ls -l scratch/greet.sh

-rwxr-xr-x 1 runner runner 47 Jul 20 06:08 scratch/greet.sh

⚠ Don’t reach for chmod 777

777 grants rwx to everyone, including write. It is the rm -rf of permissions: a file (or worse, a directory) that anyone on the system may rewrite or replace. It is tempting as a “just make it work” hammer when something is denied, but it almost never fixes the real problem and quietly opens a hole. Reach for the narrowest mode that does the job: usually +x, or 755 for a script, 644 for data. And take special care with chmod -R: aimed at the wrong directory, it rewrites the permissions of an entire tree in one stroke.

D. Making a script runnable — the three things#

Here is the whole recipe. A text file becomes a command you can run when three things are true.

1. The shebang. The first line of the file names the interpreter that should run it, introduced by #! (the “shebang”). For a bash script, write it in this portable form:

#!/usr/bin/env bash

You may also see a hard-coded #!/bin/bash. Prefer the env form: instead of demanding bash at one fixed path, it asks env to find bash on the $PATH (§E), so the same script runs on a Mac, a cluster, and a container, wherever bash happens to live.

2. The executable bit. As in §C: chmod +x script.sh. Without it, the system refuses to run the file at all.

3. Running it with ./. This is the step that surprises everyone. With the shebang in place and the file executable, you still cannot run it by its bare name:

greet.sh 2>/dev/null || echo "bare name: command not found"

bare name: command not found

It is executable, yet “command not found.” Why? Because when you type a bare name, the shell looks for it only in the directories on your $PATH, and, for security, the current directory is not on $PATH. (Imagine downloading a folder with a malicious file called ls in it; if . were on the path, a plain ls would run their program instead of the real one.) So you must say explicitly “run the one right here,” and ./ is how: . is the current directory:

./greet.sh

hello from a script

That ./ in front of a script name is the single most common beginner stumble. Now you know exactly what it means: not magic punctuation, just “the file in this directory, not one somewhere on $PATH.”

E. `$PATH` and finding commands#

So what is on the path? $PATH is a plain variable holding a list of directories, separated by colons. When you type a bare command, the shell walks that list left to right and runs the first match it finds. Print it, quoted, of course (Notebook 10):

echo "$PATH"

/opt/hostedtoolcache/Python/3.12.13/x64/bin:/opt/hostedtoolcache/Python/3.12.13/x64:/snap/bin:/home/runner/.local/bin:/opt/pipx_bin:/home/runner/.cargo/bin:/home/runner/.config/composer/vendor/bin:/usr/local/.ghcup/bin:/home/runner/.dotnet/tools:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin

Those are the directories searched for every command you have run all course: ls, grep, awk, and the rest each live in one of them. To find out which (and, more usefully, what a name will actually do) use type:

type — tell you what a name actually runs — a shell builtin, a function, an alias, or the path to a program — by searching exactly as the shell would.

-a	show ALL matches, not just the first one the shell would use
-t	print only the kind (`builtin`, `file`, `alias`, `function`) — handy inside scripts

more: help type; the simpler external tool which finds only programs on $PATH

Watch out: type is bash-aware, so it sees builtins, aliases, and functions that the external which cannot — when the two disagree, type is the one telling the truth about what your shell will actually run

type is bash-aware: it knows the difference between a program on disk, a shell builtin, an alias, and a function. Watch it tell cd (a builtin, part of the shell itself) apart from ls (a program found on $PATH):

type cd

cd is a shell builtin

type ls

ls is aliased to `ls --color=auto'

There is also the simpler, older which, an external tool that only reports programs found on $PATH (it cannot see builtins or aliases):

which grep

/usr/bin/grep

This points at the final idea of the notebook, and the reason $PATH matters for you: if a bare name runs because its directory is on the path, then you can promote your own script to a command by dropping it in such a directory, and then call it by name from anywhere, no ./ needed. Adding a directory to the path for the current session is a one-liner (you will try it in the stretch exercise):

export PATH="$HOME/bin:$PATH"

Making that change permanent, so it survives a new terminal, means editing a startup file like ~/.bashrc, which belongs to Notebook 14, so we only point at it here.

Exercises#

chmod changes real files, so, as always, everything that creates or modifies a file happens in a fresh scratch/; data/ and $PATH we only read.

Warm-up 1 (worked) — Read a permission string#

Make two things in scratch/ (a data file at mode 644 and a sub-directory) and read their ls -l lines: the leading character (- vs d) and the triads.

-rw-r--r-- 1 runner runner 10 Jul 20 06:08 scratch/notes.txt
drwxr-xr-x 2 runner runner 4096 Jul 20 06:08 scratch/results

✓ the file reads as a regular file at mode 644 and the directory reads as a directory

Warm-up 2 (your turn) — `chmod` both ways#

On a scratch file: make it executable with symbolic chmod +x, confirm with ls -l; then set it to 644 and to 755 with octal chmod, confirming each. End at 755.

-rwxr-xr-x 1 runner runner 28 Jul 20 06:08 scratch/task.sh
-rw-r--r-- 1 runner runner 28 Jul 20 06:08 scratch/task.sh
-rwxr-xr-x 1 runner runner 28 Jul 20 06:08 scratch/task.sh

✓ the file ended executable at mode 755

Applied 1 (your turn) — Make a script runnable (the core skill)#

In scratch/, write a tiny script: a #!/usr/bin/env bash shebang followed by a couple of echo lines. Make it executable with chmod +x, then run it with ./. (In your terminal you would type the lines into Vim from Notebook 9; here we drop them into the file with a here-document: cat > file <<'EOF' … EOF writes everything up to the EOF marker into the file.)

report generated
by a script you made runnable

✓ the script is executable and runs with ./, printing its output

Applied 2 (worked) — `$PATH`, `type`, and the `./` lesson#

Print the path, use type/which to locate a real command, then show the contrast that started §D: a bare script name fails, ./script works.

/opt/hostedtoolcache/Python/3.12.13/x64/bin:/opt/hostedtoolcache/Python/3.12.13/x64:/snap/bin:/home/runner/.local/bin:/opt/pipx_bin:/home/runner/.cargo/bin:/home/runner/.config/composer/vendor/bin:/usr/local/.ghcup/bin:/home/runner/.dotnet/tools:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
ls is aliased to `ls --color=auto'
/usr/bin/grep
bare name: not found (current dir isn't on PATH)
I ran

✓ which located grep, ./here.sh ran, and the bare name failed (dir not on PATH)

Composite — putting it together (a runnable analysis script)#

The capstone, and the shape of every script you will write from here on. Build a small executable script that processes a trajectory log: it extracts the energy lines (the grep of Notebook 6), pulls out the numbers, and computes their mean (the awk of Notebook 8). Give it a proper shebang, make it executable, and run it with ./. We copy the log into scratch/ first (data/ stays read-only), so the script can work on a file beside it.

mean energy: -143.4444

✓ the executable script ran with ./ and printed the mean energy -143.4444

Optional stretch (your turn) — Promote a script to a command#

Take a script out of needing ./. Put it in a scratch/bin/ directory, add that directory to $PATH for this session with export PATH=…, and then call the script by its bare name, no ./. (Making such a change permanent is Notebook 14; here it lasts only for the session.)

promoted to a command

✓ the script in scratch/bin ran by bare name once its directory was on PATH

Outlook#

Your file can now run: you can read its permissions, set the executable bit with chmod, give it a shebang, and launch it with ./, or promote it onto $PATH and call it by name. That is everything around a script. Next (Notebook 12): what goes inside it: variables, arguments and flags, functions, and the robustness habits, turning a runnable file into a proper, reusable script.

New in the Compendium

chmod — change a file's permissions — who may read, write, and execute it.
type — tell you what a name actually runs — a shell builtin, a function, an alias, or the path to a program — by searching exactly as the shell would.

See the full Compendium Scriptorum for every command met so far, and where to find it again.