Table of contents:

Introduction

Have you ever wondered how processes communicate with you and each other e.g. when you run them in a terminal or command prompt?

Well, as is generally the case there are many answers to this question but let’s explore one of the most common and important mechanisms, the standard streams.

The standard streams are made up of three essential communication channels, standard input (stdin), standard output (stdout), and standard error (stderr).

drawing

By design, stdin, stdout, and stderr enable bidirectional communication between processes and the user. In an interactive process execution context, stdin serves as the input channel for the process, allowing the user to provide data or instructions. Simultaneously, stdout and stderr act as output channels, providing normal output and error messages, respectively, to the user. This bidirectional nature allows for interactive and dynamic interactions between the process and the user, creating a seamless flow of data and information. Users can actively participate, providing input through stdin and receiving process responses or outputs through stdout and stderr. This bidirectional communication ensures an engaging and interactive experience, enabling real-time interaction and collaboration between users and processes.

To illustrate their bidirectional communication, let’s consider an interactive terminal session example:

$ run_calculator
Enter two numbers:
<---[User input]
5
<---[User input]
ten
Invalid input! Please enter a numeric value.
<---[User input]
10
The sum of the numbers is: 15

In this example, the run_calculator process prompts the user to input two numbers. The user provides these numbers, 5 and 10, via stdin, followed by hitting the Enter key. The process retrieves this input from stdin, executes a calculation (in this scenario, an addition), and subsequently displays the result through stdout as The sum of the numbers is: 15. It’s important to note that the user initially typed ten instead of 10 and as a result received an error message Invalid input! Please enter a numeric value. via stderr. This interactive session demonstrates how stdin, stdout, and stderr enable real-time input and output interaction between the user and the process.

Overview

Let’s explore the standard streams a bit further.

  1. stdin stands for standard input. It represents the input stream where processes can read data from the user or from another process. For example, when you type text or provide input to a process while it’s running, you’re supplying data through stdin. Processes can read this input and process it accordingly.

    Here’s a simple example where the read command receives input from the user via stdin:

     $ echo "Please enter your name:" && read name && echo "Hello, $name!"
 # Please enter your name:
 # <---[User input]
 # asideofcode
 # Hello, asideofcode!

  2. stdout stands for standard output. It represents the output stream where processes can write their normal or expected output. This can include text, results, or any other relevant information. When you see process output displayed in the terminal or command prompt, it’s typically directed to stdout. For instance, the results of a calculation, a list of files, or log messages are commonly sent to stdout.

    Here’s a simple example where the ls command outputs a list of the contents of a directory to stdout:

     $ ls
 # example.js package.json

  3. stderr stands for standard error. It represents the output stream where processes can write their error messages or any other unexpected or exceptional output. Unlike stdout, stderr is specifically intended for error-related information. If a process encounters an issue, it can send error messages to stderr so that they can be distinguished from normal output. Error messages are often displayed separately or captured for troubleshooting purposes.

    Here’s a simple example where the ls command attempts to list the contents of a non-existent directory which results in an error message being output to stderr:

     $ ls /non_existent_directory
 # ls: /non_existent_directory: No such file or directory

These standard streams exhibit various characteristics that make them versatile and adaptable to different scenarios. Let’s examine some key characteristics:

  • Open and Closed States: These communication channels can be in an open or closed state, signaling their readiness to read or write data. Typically, stdin remains open to accept input from the user or other processes, while stdout and stderr are open for outputting regular data and error messages, respectively. Once a process concludes its execution or a specific channel is intentionally closed, it becomes unavailable for further use.

  • Redirection Capability: The capability to redirect data flow from one channel to another is one of the powerful features these channels offer. Redirection lends tremendous flexibility in managing input and output. For instance, with redirection, one can use the output of one process as the input to another, or consolidate error messages from stderr into stdout.

  • Standardization: stdin, stdout, and stderr are standardized across all Unix-based systems. This standardization ensures that these communication channels behave predictably, regardless of the specific system the program is running on. It also facilitates the sharing and reuse of code across different projects and systems.

Here are a few examples to illustrate redirection:

# Redirect stdout from one process to stdin for another process
command_with_output | command_that_consumes_input

# Redirect stderr to stdout
command_that_generates_error 2>&1
  
# Redirect stdout and stderr to separate files
command_with_output > output.txt 2> errors.txt
  
# Redirect stdout and stderr to the same file
command_with_output > output_and_errors.txt 2>&1

What makes standard input special?

The output side of things is relatively straightforward. You write to one of the output streams, stdout and stderr, and that’s it. The topic of input is a bit more interesting.

stdin, the standard input channel, is a versatile communication channel that allows processes to receive data. It can receive input in different ways, such as through piping or user input from the keyboard.

When data is piped into a process, it is read from stdin until the pipe is closed, which typically occurs immediately after the data is provided. For example, consider the following command:

echo "Hello, World!" | grep "Hello"

In this case, the output of the echo command is piped to the grep command. The grep command reads the data from stdin and searches for the string “Hello”. Note that the stdout of one process is piped into the stdin of another.

On the other hand, when reading from the keyboard, the behaviour of stdin is slightly different. Users can type input, and stdin captures that input until a specific signal is sent. In most systems, pressing Ctrl+D on an empty line signals the end-of-file (EOF) condition, indicating that no more data will be provided. However, it’s worth noting that Ctrl+D does not immediately close stdin; it signifies the end of user input.

To handle multiple lines of input from the keyboard, users can enter Ctrl+D twice consecutively or send an empty line to signal the end of input. This behaviour allows processes to process the input received through stdin effectively.

Standard streams through pipes

drawing

The fact that stdin can accept input from pipes contributes significantly to the versatility and composability in programming. Pipes, represented as | in most shell scripting languages, are a powerful feature that enable the output of one program to be used as the input of another. This chaining or ‘piping’ of commands allows for the creation of sophisticated data processing streams. By directing output from one process directly into stdin of another, developers can build complex programs from simpler, independent parts. This ability to compose larger functionality from small, focused programs forms the essence of the Unix philosophy, and continues to be a vital feature in modern programming paradigms; see the example below.

$ cat data.txt | grep 'example' | wc -l
1153

This example demonstrates how to count the number of lines in a text file (data.txt) that contain a specific word, “example”. The command sequence utilizes the Unix philosophy of “piping” where the output of one program becomes the input to another, thus chaining commands together. The cat command reads the file content, then grep filters lines containing the word “example”. The filtered output is then piped into wc -l which counts the number of lines. Each of these individual commands communicate via stdin and stdout, effectively illustrating their combined utility in a practical application.

Standard streams in programming

Here’s an example showcasing stdin usage in a Node.js process:

// ./example.js

const fs = require('fs');

// Synchronously read from stdin. 
// + When stdin is from a pipe e.g. `echo "hello world" | node example.js`,
//   then this line is executed immediatly
// + When stdin is from a interactive source, then this
//   line blocks until we encounter the first signal for
//   end of user input (i.e. Ctrl+D)
console.log(
    "First synchronous read from stdin:",
    fs.readFileSync('/dev/stdin').toString(),
);

// When stdin is from a pipe, this never runs because the pipe is immediately closed when the process runs
// This is only good for an interactive session, and this callback is invoked everytime the end of user input signal is dispatched (e.g. Ctrl+D or carriage return)
process.stdin.on('data', (data) => {
    const input = data.toString().trim();
    console.log(`Received input: ${input}`);
});

console.log('Node.js process started. Please enter some input:');

Here are some example terminal sessions.

With stdin from a pipe: where the process exits immediately because the pipe is closed, the callback to process.stdin.on('data', cb) is never invoked.

$ echo "meow" | node ./example.js
First synchronous read from stdin: meow

Node.js process started. Please enter some input:

With stdin during an interactive session: where the process waits for and consumes input in a loop until an end-of-file signal is dispatched.

$ node ./example.js
hello world
<---[Ctrl+D]
First synchronous read from stdin: hello world

Node.js process started. Please enter some input:
that was cool<---[Enter]
Received input: that was cool
meow<---[Ctrl+D]Received input: meow
<---[Ctrl+D]

When it comes to stdout and stderr, you can conveniently make use of console.log and console.error in Node.js. These functions act as wrappers around process.stdout.write and process.stderr.write, allowing you to easily write output or error messages to the corresponding streams.

However, it’s worth noting that the concept of standard streams applies to other programming languages as well. Regardless of the language used, you’ll typically find similar constructs to handle standard input, output and error streams when running a process.

Conclusion

Standard streams, stdin, stdout, and stderr, provide a powerful mechanism for allowing processes to receive input from a variety of sources, including from the user, and to provide output. Remember, stdin is for receiving input, stdout is for regular output, and stderr is for error-related output.

Standard streams enable real-time input and output interaction between the user and the process.

By directing output from one process directly into stdin of another, developers can build complex programs from simpler, independent parts. This ability to compose larger functionality from small, focused programs forms the essence of the Unix philosophy, and continues to be a vital feature in modern programming paradigms; see the example below.

Knowledge of standard streams comes in handy when you need to interact with processes, analyse their output, or troubleshoot any issues.

Happy programming and exploring the world of process communication!