Writing Command Line Tools in Python

Python is an incredibly flexible programming language that integrates well with existing programs. A lot of Python code is written as scripts and command-line interfaces (CLIs).

Command line tools and interfaces are an effective thing because they allow you to automate almost anything. As a consequence, these interfaces can become quite complex over time.

It usually starts with a simple Python script that does something. For example, it accesses a webAPI and outputs the result to the console:

# print_user_agent.py
import requests

json = requests.get('http://httpbin.org/user-agent').json()
print(json['user-agent'])

You can run this script with the command python3 print_user_agent.py and it will output the name of the user-agent used to call the API.

As said, a pretty simple script.

But what to do when such a program grows and becomes more and more complex?

This is the question we are going to solve today. You’ll learn the basics of writing command line interfaces in Python and how click makes it easy.

Using this knowledge, we’ll take you step by step from a simple script to a command line interface with arguments, options, and helpful usage instructions. We’ll do all this with the help of the click library.

Why would you want to write scripts and command line tools in Python?
The code above is just an example, not very useful in real life. In fact, there are much more complex scripts. You may have had experience with them and know that they can be an important part of our daily work: some scripts stick around for the life of the project they were written for. Some begin to benefit other teams or projects. They may even expand their functionality.

In these cases, it’s important to make the scripts more flexible and customizable by using command line options. They allow you to specify the server name, credentials, or any other information to the script.

This is where modules like optparse and argparse come in, which make our lives a lot easier. But before we get acquainted with them, let’s go over the terminology.

Fundamentals of the command line interface
The command line interface (CLI) starts with the name of the executable. You type the name into the console and access the main entry point of a script, such as pip.

Depending on the complexity of the CLI, there are usually certain parameters that you can pass to the script:

An argument, which is a required parameter. If it is not passed, the CLI will return an error. For example, in the following command click is an argument: pip install click.
An option is an optional parameter that combines a name and a value, such as –cache-dir ./my-cache. You are telling the CLI that the value ./my-cache should be used as the cache directory.
A flag that enables or disables a particular script. Probably the most common is –help. You only specify a name, and the CLI interprets the value independently.
With more complex CLIs, such as pip or Heroku CLI, you access a set of functions that are collected under the main entry point. These are usually called commands or subcommands.

You may have already used the CLI when you installed the Python library using the pip install command. The install command tells the CLI that you want to use the package install function, and gives you access to the parameters specific to that function.

Command line packages available in the standard Python 3.x library
Adding commands and parameters to your scripts can make them much better, but parsing the command line is not as easy as it may seem. However, instead of trying to solve this problem yourself, it is better to use one of the many packages that have done it for you.

The two best known packages for this are optparse and argparse. They are part of the standard Python library and are added there on an all-inclusive basis.

For the most part, they do the same thing and work in a similar way. The main difference is that optparse has not been used since Python 3.2, and argparse is considered the standard for creating CLIs in Python.

You can learn more about them in the Python documentation, but to get an idea of what a script with argparse looks like, take a look at the example below:

import argparse

parser = argparse.ArgumentParser(description='Process some integers.')
parser.add_argument('integers', metavar='N', type=int, nargs='+',
                    help='an integer for the accumulator')
parser.add_argument('--sum', dest='accumulate', action='store_const',
                    const=sum, default=max,
                    help='sum the integers (default: find the max)')

args = parser.parse_args()
print(args.accumulate(args.integers))

click vs. argparse: a better alternative?
You’re probably looking at this code and thinking, “What does it all mean?” And that’s one of the problems with argparse: code with it is unintuitive and hard to read.

That’s why you might like click.

Click solves the same problem as optparse and argparse, but in a slightly different way. It uses decorators, so your commands have to be functions that can be wrapped with these decorators.

With click, it’s easy to create a feature-rich CLI with little code. And that code will be easy to read even as your CLI grows and becomes more complex.