Unix and GNU/Linux - Basic Commands - CrystalLabs

Intro to Unix commands. List of most important ones. Basic info everyone should know. Quiz.

Introduction

This article provides an overview of Unix or GNU/Linux commands:

The first part provides basic theory, terminology, and logic behind Unix programs
The second part provides a listing of important commands, including notes where applicable

You should read this article after you are already interested in working with the Unix shell and knowing which commands exist.

Shell Input

On Unix systems, the shell is a command interpreter which waits for user commands and executes them.

The “command” is determined by the first non-special word on the line. That word can be a program (file), alias, keyword, shell builtin, or a shell function. A wider explanation can be found in , section “Command Line Arguments”.

This article is focused on programs specifically.

Programs in General

Terms binaries, executables, commands, and programs, generally refer to the same thing – Unix programs.

They can be used interchangeably, but there are some nuances. Here is a brief summary of their origin and typical use:

A binary is a Unix term. Programs on Unix are often called binaries because they are compiled to binary form and are not ASCII text. The name has been particularly emphasized on the Unix filesystem where directories containing programs are usually called “[s]bin”, such as /usr/bin/.

Strictly speaking, calling programs binaries is imprecise. First, there are many other file types that are also binary, but aren’t programs. For example, JPEG images. This is particularly noticeable in Usenet where all non-text files, such as pictures, are posted in groups having word “binaries” in their name by convention.

Second, if a program is not compiled, but is a human-readable script that is interpreted and executed via a separate program, then it isn’t a binary. But we may still call it “binary” to refer to its executable nature when its exact type is not important.

An executable is generally more precise and well-known term. It refers to all executable or runnable files, or in other words – programs. It doesn’t matter if they are compiled or interpreted.

A command can also be used as a synonym for program. However, “commands” typically imply a context in which we are typing commands into the command line, as opposed to e.g. using graphics. The term may also be somewhat imprecise because “commands” typed into shell can certainly be programs, but also shell aliases, keywords, functions, or builtins.

A program can also be used as a term, especially when referring to software packages as a whole. The term is mostly used in general IT terminology to name and distinguish between different programs, unless we are specifically referring to one of their executables.

Unix Philosophy in Programs

Unix philosophy is an established term referring to basic principles that became so successful to not only define Unix, but also how we think about computers in general.

One of those principles is “make each program be small and do one thing well”.

That’s why Unix machines typically consist of many commands and programs which can be combined in arbitrary ways. There is a saying, “Unix programs become successful when they are used in ways which their authors did not predict”.

Or as Brian Kernighan and Rob Pike summarize in “The Unix Programming Environment”:

The idea [is] that the power of a system comes more from the relationships among programs than from the programs themselves.

Just basic examples of this principle may be commands sort and uniq, which sort input and remove duplicate lines. Although they are often used in combination, they are separate and each does one specific function.

Where are Executable Files?

There is a standard called the Filesystem Hierarchy Standard which prescribes where files on Unix-like systems should be placed.

That particular standard was first released in 1994 and it became popular because many GNU/Linux distributions chose to follow it. However, it is not new school. More or less this same hierarchy has been in use on Unix since the early days, and was first documented in man page hier(7) in 1979.

Executable files are usually placed in directories called bin/ and sbin/ (“superuser bin”). These subdirectories can be found at various locations on the filesystem, such as:

Typical locations include:

/bin/
/sbin/
/usr/bin/
/usr/sbin/
/usr/local/bin/
/usr/local/sbin/
~/bin/
~/.local/bin/

Sometimes there is also libexec/, to which one puts files that are binaries, but intended to be run by other programs rather than users. There could also be /usr/X11/bin/ or /usr/bin/X11/ for X-specific programs, but that location has been deprecated over time.

Directories named “sbin” refer to “superuser bin”. Commands which require administrative privileges for full operation are placed there. Sbin directories are usually not in PATH of normal users, so that they don’t clutter their lists of available commands.

Show me all Executable Files

We’ve just explained which directories usually contain executables.

So, to see all programs available in them, you could do any one of these things:

Run ls on them:

ls /bin/ /sbin/ /usr/bin/ /usr/sbin/ /usr/local/bin/ /usr/local/sbin/ ~/bin/ ~/.local/bin/ 2>/dev/null

Type the beginning of a command name, such as a, and then press Tab,Tab to see available auto-completion choices

Search the complete disk for directories named bin/ or sbin/, and then list all files in them. This could be done with e.g.:

find / -type d \( -name bin -o -name sbin \) -exec ls {} \; 2>/dev/null

# Or:

find / -type d \( -name bin -o -name sbin \) -exec ls {} \; 2>/dev/null | sort | uniq

Quiz

Apart from reading the FHS standard or knowing the common locations, how could you make an educated guess as to where binaries are found on your system?

Do you have any additional ideas?

The above idea for using find and searching the system for all directories named “bin/” or “sbin/” is a powerful one. It scores high on the list of thoroughness, other than maybe literally searching the whole disk for all files which have the execute permission set.

But there are two other options; one simple, and one resourceful.

The simple one is:

You may recall that when you type just a command name in the shell, such as ls, the system must find where ls is actually located before it can run it.

It does so by searching for ls in directories which are listed in the process’ environment variable PATH. Thus, you could print contents of PATH:
```
echo $PATH

/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
```
That approach works, but as you see, it is not exhaustive – it only shows the current user’s PATH.
A more interesting approach would be to recall that there is a command whereis on Unix, which, when given a name, tries to locate that name among binary, source, and man files on the system. We could assume that it has a list of directories to search embedded somewhere, and in theory we could find out what that list is.

One approach would be to run man whereis and see if the directories are explicitly listed there. In this case, they are not.

But, what we can do is extract string data from the binary itself, and see if it mentions “bin"s anywhere. And indeed, this approach turns out to produce very useful results:
```
strings `which whereis` | grep ^/ | grep bin

/usr/bin
/usr/sbin
/usr/games/bin
/usr/TeX/bin
/usr/tex/bin
/usr/interviews/bin/LINUX
/usr/X11R6/bin
/usr/X386/bin
/usr/bin/X11
/usr/X11/bin
/usr/X11R5/bin
/usr/local/bin
/usr/local/sbin
/usr/local/games/bin
/usr/local/TeX/bin
/usr/local/tex/bin
/usr/local/bin/X11
/opt/*/bin
```
To automatically list all files in those directories, like we’ve done in the case of find above, we could simply run:
```
strings `which whereis` | grep ^/ | grep bin | xargs ls
```

Remember that this was a quiz question, and not something users would routinely do.

Important Commands

Using the above recipes for finding all executables on the system, we can load the list into a text editor and leave only those that are really important and that everyone using GNU/Linux should know about. By doing so, I have trimmed my list from ~4500 entries to 200.

The filtered list is diverse, including primarily commands that are basic, or related to user, process, hardware, and network management. Also, it includes some commands that are important but may be out of your comfort zone (such as, for example, dd). But you have an opportunity to find out about them sooner rather than later.

Instead of describing each one manually, we are going to provide just a list with a one-line description from their official documentation.

Here’s how we can conveniently do this by combining some commands on the command line, and supposing that we have the list of commands filtered and saved to file /tmp/commands:

cat /tmp/commands |xargs whatis

addgroup (8)         - add a user or group to the system
adduser (8)          - add a user or group to the system
aspell (1)           - interactive spell checker
badblocks (8)        - search a device for bad blocks
bc (1)               - An arbitrary precision calculator language
biosdecode (8)       - BIOS information decoder
blkid (8)            - locate/print block device attributes
cat (1)              - concatenate files and print on the standard output
cfdisk (8)           - display or manipulate a disk partition table
chcpu (8)            - configure CPUs
chgrp (1)            - change group ownership
chmem (8)            - configure memory
chmod (1)            - change file mode bits
chmod (2)            - change permissions of a file
chown (1)            - change file owner and group
chown (2)            - change ownership of a file
chroot (8)           - run command or interactive shell with special root directory
clear (1)            - clear the terminal screen
cmp (1)              - (unknown subject)
column (1)           - columnate lists
cp (1)               - copy files and directories
cut (1)              - remove sections from each line of files
date (1)             - print or set the system date and time
dd (1)               - convert and copy a file
delgroup (8)         - remove a user or group from the system
deluser (8)          - remove a user or group from the system
df (1)               - report file system disk space usage
dict (1)             - DICT Protocol Client
diff (1)             - (unknown subject)
dig (1)              - DNS lookup utility
dmesg (1)            - print or control the kernel ring buffer
du (1)               - estimate file space usage
dump_xsettings (1)   - prints X11 application settings from an XSETTINGS provider
echo (1)             - display a line of text
editor (1)           - Nano's ANOther editor, an enhanced free Pico clone
env (1)              - run a program in a modified environment
fdisk (8)            - manipulate disk partition table
file (1)             - determine file type
find (1)             - search for files in a directory hierarchy
free (1)             - Display amount of free and used memory in the system
fsck (8)             - check and repair a Linux filesystem
fuser (1)            - identify processes using files or sockets
getconf (1)          - Query system configuration variables
getent (1)           - get entries from Name Service Switch libraries
grep (1)             - print lines that match patterns
halt (8)             - Halt, power-off or reboot the machine
hdparm (8)           - get/set SATA/IDE device parameters
head (1)             - output the first part of files
HEAD (1p)            - Simple command line user agent
hexdump (1)          - ASCII, decimal, hexadecimal, octal dump
host (1)             - DNS lookup utility
hostid (1)           - print the numeric identifier for the current host
hostname (1)         - show or set the system's host name
htop (1)             - interactive process viewer
id (1)               - print real and effective user and group IDs
ifconfig (8)         - configure a network interface
ifdown (8)           - take a network interface down
ifup (8)             - bring a network interface up
insmod (8)           - Simple program to insert a module into the Linux Kernel
kill (1)             - send a signal to a process
killall (1)          - kill processes by name
last (1)             - show a listing of last logged in users
lastb (1)            - show a listing of last logged in users
lastlog (8)          - reports the most recent login of all users or of a given user
less (1)             - opposite of more
ln (1)               - make links between files
logger (1)           - enter messages into the system log
lp (1)               - print files
ls (1)               - list directory contents
lsblk (8)            - list block devices
lscpu (1)            - display information about the CPU architecture
lsipc (1)            - show information on IPC facilities currently employed in the system
lslocks (8)          - list local system locks
lslogins (1)         - display information about known users in the system
lsmem (1)            - list the ranges of available memory with their online status
lsmod (8)            - Show the status of modules in the Linux Kernel
lsns (8)             - list namespaces
lsof (8)             - list open files
lspci (8)            - list all PCI devices
lsusb (8)            - list USB devices
man (1)              - an interface to the on-line reference manuals
manpath (1)          - determine search path for manual pages
mc (1)               - Visual shell for Unix-like systems.
md5sum (1)           - compute and check MD5 message digest
mdadm (8)            - manage MD devices aka Linux Software RAID
mesg (1)             - display (or do not display) messages from other users
mii-tool (8)         - view, manipulate media-independent interface status
mkdir (1)            - make directories
mkfs (8)             - build a Linux filesystem
modinfo (8)          - Show information about a Linux Kernel module
modprobe (8)         - Add and remove modules from the Linux Kernel
mount (2)            - mount filesystem
mv (1)               - move (rename) files
nice (1)             - run a program with modified scheduling priority
nohup (1)            - run a command immune to hangups, with output to a non-tty
nproc (1)            - print the number of processing units available
nslookup (1)         - query Internet name servers interactively
parted (8)           - (unknown subject)
passwd (1)           - change user password
paste (1)            - merge lines of files
pidof (8)            - find the process ID of a running program.
pidstat (1)          - Report statistics for Linux tasks.
ping (8)             - send ICMP ECHO_REQUEST to network hosts
pkill (1)            - look up or signal processes based on name and other attributes
poweroff (8)         - Halt, power-off or reboot the machine
ps (1)               - report a snapshot of the current processes.
pstree (1)           - display a tree of processes
pwd (1)              - print name of current/working directory
reboot (2)           - reboot or enable/disable Ctrl-Alt-Del
reboot (8)           - Halt, power-off or reboot the machine
reset (1)            - terminal initialization
resize (1)           - set environment and terminal settings to current xterm window size
rm (1)               - remove files or directories
rmdir (1)            - remove empty directories
scp (1)              - secure copy (remote file copy program)
script (1)           - make typescript of terminal session
scriptreplay (1)     - play back typescripts, using timing information
service (8)          - run a System V init script
shuf (1)             - generate random permutations
sort (1)             - sort lines of text files
split (1)            - split a file into pieces
ss (8)               - another utility to investigate sockets
startx (1)           - initialize an X session
stat (1)             - display file or file system status
su (1)               - run a command with substitute user and group ID
sudo (8)             - execute a command as another user
swapoff (8)          - enable/disable devices and files for paging and swapping
swapon (8)           - enable/disable devices and files for paging and swapping
sysctl (8)           - configure kernel parameters at runtime
tabs (1)             - set tabs on a terminal
tac (1)              - concatenate and print files in reverse
tail (1)             - output the last part of files
tcptraceroute (8)    - print the route packets trace to network host
tee (1)              - read from standard input and write to standard output and files
tee (2)              - duplicating pipe content
telinit (8)          - Change SysV runlevel
top (1)              - display Linux processes
touch (1)            - change file timestamps
traceroute (1)       - print the route packets trace to network host
tty (1)              - print the file name of the terminal connected to standard input
tune2fs (8)          - adjust tunable filesystem parameters on ext2/ext3/ext4 filesystems
umount (8)           - unmount file systems
uname (1)            - print system information
uniq (1)             - report or omit repeated lines
uptime (1)           - Tell how long the system has been running.
useradd (8)          - create a new user or update default new user information
userdel (8)          - delete a user account and related files
usermod (8)          - modify a user account
vim (1)              - Vi IMproved, a programmer's text editor
vipw (8)             - edit the password, group, shadow-password or shadow-group file
visudo (8)           - edit the sudoers file
w (1)                - Show who is logged on and what they are doing.
w3m (1)              - a text based web browser and pager
wall (1)             - write a message to all users
watch (1)            - execute a program periodically, showing output fullscreen
wc (1)               - print newline, word, and byte counts for each file
wget (1)             - The non-interactive network downloader.
whatis (1)           - display one-line manual page descriptions
whereis (1)          - locate the binary, source, and manual page files for a command
which (1)            - locate a command
who (1)              - show who is logged on
whoami (1)           - print effective userid
whois (1)            - client for the whois directory service
wipefs (8)           - wipe a signature from a device
write (1)            - send a message to another user
xargs (1)            - build and execute command lines from standard input
xcalc (1)            - scientific calculator for X
xclock (1)           - analog / digital clock for X
xev (1)              - print contents of X events
xeyes (1)            - a follow the mouse X demo
xkill (1)            - kill a client by its X resource
xload (1)            - system load average display for X
xlogo (1)            - X Window System logo
xmag (1)             - magnify parts of the screen
xman (1)             - Manual page display program for the X Window System
xrandr (1)           - primitive command line interface to RandR extension
xterm (1)            - terminal emulator for X

Command Notes

Here is a minimal list of short, useful comments on the above commands.

aspell is a spell checker. Usually, run with aspell -c FILE and use commands printed at bottom-right of the interface
bc is an arbitrary precision calculator language. Usually, run with bc -l to start with sane defaults
clear clears the terminal screen, which can also be done more conveniently by just pressing Ctrl+l. Also, with Ctrl+l, your command line does not have to be empty
cmp compares files
dd is a versatile, low-level command for copying data raw, on a level below filesystem
df, du display disk usage. Use with -h if numbers in bytes confuse you
dict is a dictionary. Use simply as dict WORD
diff shows difference between text files. Typical use is with option -u for a “unified diff” format of output
env runs a program in a modified environment, or prints current environment variables. Also see export and declare -x
hdparm can be used to test disk performance. To run it non-destructively, run hdparm -tT /dev/DISK
kill, despite its name, sends any signals to processes
resize sets environment and terminal settings to current xterm window size. Best used as eval $(resize)
script and scriptreplay replay terminal sessions. For replaying, script must be started with option -t, such as script -ttiming and then scriptreplay timing
uname prints system information. To see all info, run as uname -a
vim is a venerable Unix text editor

Automatic Links

The following links appear in the article:

1.
2. ASCII - https://en.wikipedia.org/wiki/ASCII
3. Filesystem Hierarchy Standard - https://en.wikipedia.org/wiki/Filesystem_Hierarchy_Standard
4. Unix Philosophy - https://en.wikipedia.org/wiki/Unix_philosophy
5. Usenet - https://en.wikipedia.org/wiki/Usenet

Unix and GNU/Linux - Basic Commands

Table of Contents