Piping and Directing Output

This article explains how to chain various commands together, so that the output of one becomes the input of another — called piping — as well as how to direct output to and from files, instead of always from the keyboard and to the console.

Basic Concepts

In almost all cases, when you execute a program, that program needs some data to do its work, and it then produces some kind of result. The data we feed to a program is called input, and the result is called output.

inputoutput.png

In most cases, when we're running programs from the command line, the input comes from the keyboard, and the output goes to the monitor.

keyboardtomonitor.png

Let's give an example; suppose I am using my command prompt, and I type the following command:

$ ls -l

In this case:

  • The input is the keyboard — the -l parameter I typed, which configures how ls works
  • The program is the ls command
  • The output is displayed on the monitor

This is how most programs work, when you use them from the command line. You type the command, and the results (if any) are displayed on the screen.

The Pipe

The pipe allows us to change this paradigm, whereby the output of one program becomes the input of another program.

For example, in the command above, there could be quite a number of files listed, and it's quite possible that some of the results might have scrolled off the top of the screen, before I could see them. But I could fix that by sending the output of the ls command to the less command, which would allow me to scroll through the results, just as if I were using less to read a text file.

lstoless2.png

I would do that by using the | character — which is called, appropriately enough, the pipe — as follows:

$ ls -l | less

This tells BASH to do the following:

  1. Execute the ls command, with the parameter -l as input
  2. Take the results of executing that command — the output — and pass them as input to the less command
  3. Send the output of the less command to the monitor as usual

And here's the cool part: You're not limited to just one pipe; you could take the output of one program and send it as input to a second, and then send the output of that command to a third command! For example, suppose I want to do the same thing as above, but I only want to show files that match the string 'init'. I could use the grep command to do that; if I send the output of the ls command to grep, and tell it to look only for lines that contain 'init', and then send the output of that to the less command, then I'll be able to easily scroll through all of the files that are named init or contain the characters init.

threecommands.png

To do this, I would use this command:

$ ls -l | grep 'init' | less
  • List all of the files, using the -l option
  • Search the results of that file listing for the string init
  • Send the results of that search to less

Directing Output

The pipe is very handy, in that it allows us to “chain” various commands together, so that the output of one command becomes the input for the next. However, it still ties us to the basic

keyboardtomonitor.png

paradigm; the input comes from the keyboard, there can be one or many programs, and the output ends up on the monitor. Linux also provides us with a way to alter where we get our input, or where we send the output.

Consider this example:

$ ls -l > temp.txt

If you were to type this command at the command prompt, you would see no result on the screen. This is because I have redirected the output from the monitor to a text file, called temp.txt, by using the > character.

It's pretty easy to remember how the > character works, because it looks like an arrow; you can visualize the output of the ls command being sent into the temp.txt file.

Be careful when using this character, though! If there is already a file named temp.txt, this command will replace that file with the new one. If you wish, you can append data to the file, instead of replacing it, by using >> instead of >. For example,

$ ls -l >> temp.txt

will take the result of the ls command, and add it to the end of the temp.txt file, instead of replacing it.

In this way we have changed our paradigm; we're now taking input from the keyboard, but sending the output to a file.

outputtofile.png

If we wish, we can also do the reverse, using the < character, to take our input from a file. For example, if still wish to search for files/directories that contain “init,” we can use grep to search the file, like this:

$ grep 'init' < temp.txt

In this case, the example is facetious, because you can specify a filename for grep to search in its parameters. However, it helps the example flow along to keep using it.

The output of the search — as usual — is shown on the screen. If we want to get really crazy, we can take the result of our search, and send it to a second file, called temp2.txt, like this:

$ grep 'init' < temp.txt > temp2.txt

This will create a second file, called temp2.txt, which will contain the output from grep.

2files.png

You can, of course, combine pipes and redirects in one command. For example:

$ ls -l | grep 'init' > temp.txt

which works like this:

lsgrepfile.png

In other words:

  1. Use the ls program, with -l as input, to list the files in the current directory
  2. Pass the output from ls as input to grep, and search for entries that match 'init'
  3. Take the output from grep and put it in a file called temp.txt.
Unless otherwise stated, the content of this page is licensed under Creative Commons Attribution-Share Alike 2.5 License.