Learning to Love Your Terminal

Introduction

After reading An Introduction to the Linux Terminal and A Linux Command Line Primer, you should have a good grasp on the fundamentals of working in a modern command line environment. However, many users who are new to the command line may still find it fairly restrictive. This tutorial is designed for these users to provide more background on command line interfaces, as well as advice around customization and cross-platform portability. The goal is to feel just as comfortable in a terminal environment as you are with using a computer in any other way.

Improving your Windows Terminal Experience

On Windows, there are many choices for a Terminal equivalent. Historically, Windows did not use Unix-style command line shells, such as bash, which have been ubiquitous on macOS and Linux since the early 2000s. It also lacked special features for highlighting text, opening multiple tabs, and so on. This is because it did not use common command-line terminal interfaces, sometimes called terminal emulators because they emulate the interfaces of older non-graphical computers.

Instead, Windows offered two of its own native command line interfaces: the Windows Command Prompt, and from Windows 7 onward, the Microsoft PowerShell. The Command Prompt, also called cmd.exe, uses legacy MS-DOS syntax with relatively few additions. PowerShell provides somewhat more modern syntax relative to cmd.exe (where “modern” in this context means “closer to a modern macOS or Linux shell”), as well as signal handling functions specific to certain Windows software, making it useful for Windows administrators.

Neither of these Windows shells include many fundamental features of modern Unix-style shells, and they are generally not well suited to most cloud development. Because of this, users who needed to work on cloud servers from Windows would usually install software like PuTTY (a tty is the historical name of a Unix terminal), mobaXterm, or ConEmu. Each of these applications would usually include their own terminal interface, as well as a built-in SSH client for connecting to remote servers. These two features are not usually thought of as analogous on other platforms, but on Windows, it was usually a safe assumption that if you needed a Unix-style terminal, you were going to be working on a remote server, so they were often packaged together. For this reason, using a dedicated SSH GUI like PuTTY is still a popular way of working with cloud servers from Windows.

On other platforms, ssh is just a command-line program that you can run from a terminal, and it is part of a core group of Linux utilities. It is also possible to install these core Linux utilities on Windows, along with a port of the standard bash shell. Originally, this functionality was provided by the Cygwin project, which includes ports of many other Linux system tools. Installing Git on Windows also provides its own bash shell from the MSYS2 project, which shares some functionality and upstream code with Cygwin. This would allow you to, for example, ssh from the command line on Windows – as well as using common Linux utilities like grep or cat – without needing another program to provide this functionality.

Windows Package Managers

These ports of Linux utilities to Windows are typically robust and well-maintained. However, they have a couple of significant downsides which have made them less popular than using similar functionality on macOS or Linux. One is that they generally do not include full-fledged package managers for installing other command line software as needed, which is a core assumption of most Linux environments (also provided by tools like Homebrew on macOS). In recent years, Windows has gained its own ecosystem of command line package managers, such as Chocolatey and Scoop, that can be used with Git’s bash shell or other environments. Like Homebrew on macOS, these are especially useful on a local machine because they can be used to install desktop software in addition to command line tools. However, because most cloud software is still not designed to run natively on Windows, Chocolatey and Scoop’s package repositories are often less complete than their macOS or Linux equivalents. Using them often requires you to translate install documentation written for more common platforms into a working Windows equivalent, making them unintuitive for beginner users.

The other downside is that PuTTy, Git Bash, and other all-in-one Windows terminal environments usually have very barebones UI features, lacking support for syntax highlighting or tabs. Because the Windows Command Prompt has not visibly improved for some time, Windows users may be pushed toward more awkward workflows than users of a modern terminal like iTerm2 on macOS. To approximate an environment like this, you could instead combine Git’s bash shell, Chocolatey’s package manager, and ConEmu’s terminal customization. This produces a very usable result, but requires a unique configuration that can be considered brittle or less reproducible than other platforms.

Nowadays, you can use the new Windows Terminal along with WSL2, also called the Windows Subsystem for Linux. These two projects substantially solve many outstanding issues. The Windows Terminal provides similar functionality to ConEmu, such as tabs, highlighting, and modern text rendering, and it can be used with any installed shell. It is also installed by default on Windows 11, significantly lowering the barrier to access and the need for third-party tools to do this work. WSL2 accomplishes a related goal: by allowing users to install a Linux compatibility environment that runs within Windows and is directly supported by Microsoft, they (mostly) no longer need to consider the entire set of other Windows command line tools.

WSL2 is a full Linux environment that runs within Windows. Because it is not a port of Linux tools to Windows, it comes with significant advantages. When working within a WSL2 environment, you can use apt or Homebrew or any other native Linux tools to install and run software exactly as you would on a cloud server. This also has some downsides. Unlike with a macOS or Linux terminal, you can’t run native desktop software from WSL2, only Linux software that is installed into your Linux environment. In many cases this will be sufficient, especially if you are mostly using your terminal to deploy software to remote servers or make small configuration changes. However, you may still want to configure Windows Terminal to launch multiple different shell environments depending on your needs: for example, one using Git Bash and Chocolatey, which works with your native Windows software, and one using WSL2, which provides a full package manager and allows you to follow Linux documentation as-is. Many of these tools now have mutual support for one another built in, making this relatively straightforward, and very powerful.

zsh and other Shell Interpreters

Although bash is the most common shell on modern Linux distributions, and its syntax is considered the most widely compatible with most environments, it is not the only one. Bourne shell syntax, or /bin/sh, is a subset of bash, and is sometimes still used in minimal environments such as containers. There is also the Z shell, or zsh, which is becoming more popular due to its more flexible MIT license and its configurability. As of 2020, it is the default Terminal shell on macOS, and it is widely available in other environments.

Zsh provided earlier and more widespread support for theming, text highlighting, and rendering of non-text characters (also called glyphs, essentially an earlier form of Emoji) than the default bash shell in many environments. Because of this, there is a larger ecosystem of terminal customization tools for Zsh, such as Oh My Zsh. More importantly, Zsh tools and documentation usually prioritize installing fonts with supports for complex glyphs, such as the Powerline fonts, which is helpful for solving text rendering problems in other environments.

One downside that potential Zsh users quickly realize is that zsh and bash do not have strictly identical syntax, and bash shell scripts are the most common by far. While most everyday shell conventions for navigating directories are the same, some differences arise. This includes testing equivalency with comparison operators, complex filesystem search strings, and so on.

As a general rule, if you are writing a shell script with syntax more complex than bash accommodates, you should consider a different scripting language. Shell syntax is powerful, but it can also be unwieldy. The Go language, for example, has become popular for writing command line tools that use more advanced flow control than is appropriate for shell scripts. With some exceptions, even dedicated users of zsh do not usually write and maintain standalone zsh scripts.

With that in mind, you should not worry about any compatibility issues arising from using zsh as your default, interactive shell. Virtually all environments where you can run zsh will also have the bash interpreter installed to run any bash scripts as needed. When you are writing a standalone shell script, or running a shell script that you downloaded from elsewhere, the first line will normally contain #!/bin/bash. This is known as a shebang, or an interpreter directive, which tells the computer which program, or interpreter, to use to run the script by default. This is especially important for shell scripts which cannot otherwise be distinguished by their file extension: both bash scripts and other shell scripts all end in .sh. Because bash is always installed at /bin/bash in compatible environments (including Git Bash on Windows), providing a complete path here is a widespread and safe convention.

Improving Search with Globstars

An example of advanced shell syntax that is supported in zsh by default, but has to be enabled manually in bash, is the globstar, or ** pattern. Globbing is another name for performing fuzzy-match searching, i.e., searching for files using wildcard * characters. The ** globstar allows you to use wildcard substitution not only within a single filename, but for entire directories as well. For example, if you were searching for a file named config.txt somewhere in your home directory, but you didn’t know where to look, you might need to use a command like find, which is designed to recursively search through nested directories. Even if you already know find syntax, this would add an additional step to your process where you might otherwise have used a wildcard.

By using a globstar, you could instead provide a path like ~/**/config.txt to any other command. The shell would automatically expand the search for you, without needing to invoke find directly. Shells are good at providing this kind of functionality — what would normally require an entire other dependency can be incorporated into a single additional character substitution. To enable globstar use in bash, you can run shopt -s globstar.

shopt -s globstar

You can also experiment with glob search strings using DigitalOcean’s Glob Tool.

Improving Your Terminal’s Core Utilities with Rust

As mentioned, the Go language has become very powerful for building modern command line tools. Another relatively new programming language that is especially relevant to command line environments is Rust. Rust is a low-level language that is generally considered similar to C++ and other C-likes, but with much more modern syntax and less of C’s accumulated baggage. Rust is popular for many use cases, including WebAssembly, but it is particularly useful for rewriting C code.

Nearly all of the core utilities that are thought of as essential to command line environments, such as ssh, curl, and cat, are written in the C language, in many cases going back multiple decades. This is why many of them have so many dozens of options that have been added over years of active maintenance. In many ways, Linux has been built up around these specific tools, and they are unlikely ever to be officially replaced. However, there have been recent efforts to develop improved versions of each of them using Rust.

For example, bat provides more syntax highlighting and similar features to cat, dust is similar to using du to check disk usage but with more sophisticated graphical output, and ripgrep, or rg, is a much faster implementation of grep.

The relative advantages of these new Rust tools should not be taken as a judgment on C or Rust themselves, or on their relative performance. Rather, it is a consequence of maintaining the same codebases for a long time. Many core Linux utilities are not prioritizing getting faster, and in most cases, maintaining these utilities is about ensuring that they do not break compatibility with any legacy functionality. Most of these Rust utilities have been added to different platforms’ package managers, but they are still treated as new and optional.

Profiles and Aliases

The most significant downside to these new Rust tools is that many people may actually forget to use them, since their habits of using cat or grep will quickly become ingrained. To work around this, or to further customize your shell environment in general, you can use a profile file.

Both bash and zsh support profile files. A profile file is a file in your home directory called .bash_profile or .bashrc for bash, or .zshrc for zsh. Often, they are created automatically when you first launch your shell, but because they start with a . character, they are treated by the system as hidden, and remain invisible unless edited directly. Profile files can contain a number of settings that help to initialize your shell environment, such as aliases from one command to another.

With this method, you can create an alias in your ~/.bash_profile to make the grep command always run rg instead of your system grep. As mentioned, these Rust tools are not usually installed by default, so you would first need to install rg. If you are using Homebrew, you can install it from the provided ripgrep package.

brew install ripgrep

Next, open ~/.bash_profile using nano or your favorite text editor. This file may or may not already exist if you already have some profile settings configured.

nano ~/.bash_profile

Then, add a line to the end of the file that includes the alias command:

…
alias grep='rg'

Save and close the file, then close and open a new bash terminal. From now on, when you run grep, you’ll get rg instead:

grep --version

Output
ripgrep 12.1.1
-SIMD -AVX (compiled)
+SIMD +AVX (runtime)

You should be aware that if you copy and paste a grep command that you find elsewhere, and it uses functionality that isn’t present in rg, it may not work correctly. For this reason, you should use aliases sparingly. However, they are useful for creating shortcuts that do not outright replace other commands. For example, the Python programming environment comes with a built-in webserver that can serve any small static sites by running python -m http.server port_number. If you use this feature often, you may want to create an alias along the lines of alias pyserver="python -m http.server 8000".

Each terminal session is configured with a number of environment variables by default. Many of these are set automatically by the system, but you can provide additional overrides in ~/bash_profile, or interactively within a single shell session. To view your current environment variables, run env:

env

Output
SHELL=/usr/local/bin/bash
ITERM_PROFILE=bash
COLORTERM=truecolor
XPC_FLAGS=0x0
TERM_PROGRAM_VERSION=3.4.15
…

Many of these contain information about your shell itself, which will not be particularly useful. One exception to this is the PATH variable, which contains a list of all the directories that are automatically checked to run command-line programs. You can pipe with grep to output only the line containing the PATH variable from the env command:

env | grep PATH

Output
PATH=/usr/local/opt/mysql-client/bin:/Users/sammy/.gem/ruby/3.0.0/bin:/usr/local/opt/ruby/bin:/usr/local/lib/ruby/gems/3.0.0/bin:/Users/sammy/.cargo/bin:/Users/sammy/.rbenv/shims:/Users/sammy/.pyenv/shims:/Users/sammy/.pyenv/bin:/Users/sammy/Library/Python/3.9/bin:/usr/local/opt/gnu-sed/libexec/gnubin:/usr/local/sbin:/usr/local/opt/libpq/bin:/usr/local/opt/coreutils/libexec/gnubin:/usr/local/opt/findutils/libexec/gnubin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Library/TeX/texbin:/opt/X11/bin:/Library/Apple/usr/bin:/Library/Frameworks/Mono.framework/Versions/Current/Commands

Note the directories in this list are separated by a : character. In general, your PATH variable will be longer on Windows or macOS than on Linux. Linux tries to enforce installing command-line programs only into directories like bin or /usr/local/bin, whereas Windows and Mac software is often installed to other directories that need to be added to your PATH for those commands to be available on the command line. Installing software via a package manager usually takes care of this. While you can manually move programs into /usr/local/bin or another directory on your path, be aware that package managers expect to be able to manipulate the contents of these directories. Any programs that you install manually could be overwritten, or cause an error when attempting to be overwritten.

You can check the installed location of a command line program by using the which command:

which python

Output
/Users/sammy/.pyenv/shims/python

This is useful for verifying which version of a program will run by default when, for example, python is run from the command line. If a program is present in multiple directories on your PATH, the directory that is listed first in your PATH variable will take precedence. Python is a notorious example. Because macOS and Linux both include a version of Python with the system that is sometimes updated either too frequently or too infrequently, tools like pyenv are designed to register a separate installation of Python as early on your path as possible. This ensures that you are always working directly with and installing additional libraries for the pyenv-provided Python.

Any time you do not get the expected result from a command, ask yourself these questions:

• Is the program that you want to run on your path?
• Was a package manager supposed to put it on your path when you installed it, or do you need to put it there manually?
• Is this the only version of this program with this exact name on your computer?

The variable inheritance behavior of your PATH and your shell environment is usually straightforward — you may just have multiple programs making other, conflicting assumptions. Knowing how to check and configure your environment will go a long way toward helping you be comfortable on the command line.

Conclusion

In this tutorial, you reviewed many nuances of terminal environments, including configuration on Windows, differences between bash and other shells, modern command-line utilities, and environment variables including paths and aliases. Many of these are overlooked by new developers, and each of them can make working either locally or in the cloud much more pleasant and effective.

Next, you may want to learn to work with DigitalOcean’s command line client, doctl.