Standard Streams

• The three standard streams are communication channels between a computer program and its environment.

• They are:

  • Standard Input

  • Standard Output

  • Standard Error

Standard Output

• Standard output is a place to which a program or command can send information.

• The information could go to a screen to be displayed, to a file, or even to a printer or other devices.

  

Standard Error

• Commands and programs also have a destination to send error messages: standard error.

• By default, the shell directs standard error information to the screen for us to read, but we can change that destination if needed!

Standard Input

• Standard input is where a program or command gets its input information from. By default, the shell directs standard input from the keyboard.

• The input information could come from a keyboard, a file, or even from another command!

  

Redirection

• "redirection" describes the ways we can alter the source of standard input and the destinations for standard output and standard error.

  Redirecting Output

  • The redirect output symbol (\>) tells the shell to redirect the output of a command to a specific file instead of the screen.

  • By default, the date command will print the current date to the screen. If we instead run date > output.txt the output will be redirected to a file called output.txt

  • If the file does not exist, it will be automatically created.

  • Note: the > symbol needs to occur after any options and arguments!

      command > filename
      echo "moo" > cow.js # redirects the output of echo
      ls -l > files.txt # saves the output of ls -l to a file.
    

Appending

• When we redirect the output into a file using \> any existing contents in the file are overwritten. Sometimes this is not what we want!

• Instead, keep the existing contents in the file and add new content to the end of the file, use \>> when redirecting.

    echo "hello" >> greeting.txt
    echo "world" >> greeting.txt
    cat greeting.txt
    #hello
    #world
  

Redirecting Input

• To pass the contents of a file to standard input, use the < symbol followed by the filename.

• For example, we could pass the contents of the chickens.txt file to the cat command using cat < chickens.txt

• cat (and many other commands) are set up to accept filenames as arguments directly, but we can also redirect to standard input manually.

    command < filename
    cat < chickens.txt
  

• We can redirect standard input and output at the same time! In this example, we are using cat to read in the contents of original.txt and then redirecting the output to a file called output.txt

• In this example, we are redirecting the names.txt file to the sort command. We are also redirecting the output of sort to a file called sorted.txt

    cat < original.txt > output.txt
    sort < names.txt > sorted.txt
  

Redirecting Standard Error

• By default, error messages are output to the screen, but we can change this by redirecting standard error.

• The standard error redirection operator is 2>

• If we ran a command like cat nonexistentfile (where the file does not exist) we would see an error printed to the screen. We can instead redirect standard error to a file with cat nonexistentfile 2> error.txt

We can use the same >> syntax to append when redirecting standard error.

cat nonexistentfile 2> error.txt
ls -zzz 2>> error.txt # ls with illegal option

Why 2> ?

• Each stream gets its own numeric file descriptor, and for standard error, the number is 2.

• The > operator defaults to using 1 as the file descriptor number, which is why we didn't need to specify 1> to redirect the standard output

• Similarly, the < operator uses a default file descriptor number of 0, so we don't need to specify 0< to redirect to standard input (though we can!)

  

All Together now

• We can redirect multiple streams at once! In this example, we are concatenating two files, redirecting standard output to a file called insects.txt, and redirecting standard error to a file called error.txt

• When redirecting both standard output and standard error, make sure standard output comes FIRST. Always redirect standard error after standard output.

    cat bees.txt ants.txt > insects.txt 2> error.txt
  

  

Pipes

• Pipes are used to redirect a stream from one program to another program. We can take the output of one command and redirect it to the input of another.

• We use the pipe character ( | ) to separate two commands. The output of the first command will be passed to the standard input of the second command.

• This example shows the output of the date command being piped to the rev command. The result is the reverse of the current date!

    command1 | command2
    date | rev
  

  

• This example pipes the output of ls to less. The /usr/bin directory typically contains a bunch of stuff, so it can be nice to use less to read the results in a more manageable way.

    ls -l /usr/bin | less
  

• This example counts the number of files (non hidden files) in a directory. We pipe the output of ls to the word count command. The -l option tells wc to count the number of lines.

    ls | wc -l
  

  

• In this example, we are calling tac with a file and then piping the output to rev. The final result is the content of file.txt printed "horizontally" and "vertically" reversed

    tac file.txt | rev
  

  

• This example concatenates two files using cat and then sorts them alphabetically

    cat colors.txt pets.txt | sort
  

\> vs |

• Though both the > character and the | character are used to redirect output, they do it in very different ways.

• \> connects a command to some file.

• | connects a command to another command.

    ls -l /usr/bin > list.txt
    ls -l /usr/bin | less
  

  

tee

• What if I wanted to create a file with the output of cat? Enter the tee command

• The tee program reads standard input and copies it both to standard output AND to a file. This allows us to capture information part of the way through a pipeline, without interrupting the flow.

• It takes some information coming from one command and gives it to a file we specify and also passes it to the next command. Hence this allows us to take information in the middle of a pipeline and save it to a file without stopping.

    command1 | tee file.txt | command2
    cat file1 file2 file3 | tee combo.txt | wc -w