Running other programs from Tcl - exec, open

• open ...... run a new program with I/O connected to a file descriptor
• exec ...... run a new program as a subprocess

The open call is the same call that is used to open a file. If the first character in the file name argument is a "pipe" symbol (|), then open will treat the rest of the argument as a program name, and will run that program with the standard input or output connected to a file descriptor. This "pipe" connection can be used to read the output from that other program or to write fresh input data to it or both.

If the "pipe" is opened for both reading and writing you must be aware that the pipes are buffered. The output from a puts command will be saved in an I/O buffer until the buffer is full, or until you execute a flush command to force it to be transmitted to the other program. The output of this other program will not be available to a read or gets until its output buffer is filled up or flushed explicitly.

(Note: as this is internal to this other program, there is no way that your Tcl script can influence that. The other program simply must cooperate. Well, that is not entirely true: the expect extension actually works around this limitation by exploiting deep system features.)

The exec call is similar to invoking a program (or a set of programs piped together) from the prompt in an interactive shell or a DOS-box or in a UNIX/Linux shell script. It supports several styles of output redirection, or it can return the output of the other program(s) as the return value of the exec call.

open |progName ?access?

Returns a file descriptor for the pipe. The progName argument must start with the pipe symbol. If progName is enclosed in quotes or braces, it can include arguments to the subprocess.

exec ?switches? arg1 ?arg2? ... ?argN?

exec treats its arguments as the names and arguments for a set of programs to run. If the first args start with a "-", then they are treated as switches to the exec command, instead of being invoked as subprocesses or subprocess options.

switches are:

-keepnewline

Retains a trailing newline in the pipeline's output. Normally a trailing newline will be deleted.

--

Marks the end of the switches. The next string will be treated as arg1, even if it starts with a "-"

arg1 ... argN can be one of:

• the name of a program to execute
• a command line argument for the subprocess
• an I/O redirection instruction.
• an instruction to put the new program in the background:

  exec myprog &
  

  will start the program myprog in the background, and return immediately. There is no connection between that program and the Tcl script, both can run on independently.
  The & must be the last argument - you can use all other types of arguments in front of it.

  [NOTE: add information on how to wait for the program to finish?]
  

There are many I/O redirection commands. The main subset of these commands is:

|

Pipes the standard output of the command preceding the pipe symbol into the standard input of the command following the pipe symbol.

< fileName

The first program in the pipe will read input from fileName.

<@ fileID

The first program in the pipe will read input from the Tcl descriptor fileID. fileID is the value returned from an open ... "r" command.

<< value

The first program in the pipe will read value as its input.

> fileName

The output of the last program in the pipe will be sent to fileName. Any previous contents of fileName will be lost.

>> fileName

The output of the last program in the pipe will be appended to fileName.

2> fileName

The standard error from all the programs in the pipe will be sent to fileName. Any previous contents of fileName will be lost.

2>> fileName

The standard error from all the programs in the pipe will be appended to fileName.

>@ fileID

The output from the last program in the pipe will be written to fileID. fileID is the value returned from an open ... "w" command.

If you are familiar with shell programming, there are a few differences to be aware of when you are writing Tcl scripts that use the exec and open calls.

• You don't need the quotes that you would put around arguments to escape them from the shell expanding them. In the example, the argument to the sed command is not put in quotes. If it were put in quotes, the quotes would be passed to sed, instead of being stripped off (as the shell does), and sed would report an error.
• If you use the open |cmd "r+" construct, you must follow each puts with a flush to force Tcl to send the command from its buffer to the program. The output from the program itself may be buffered in its output buffer.

  You can sometimes force the output from the external program to flush by sending an exit command to the process.

  You can also use the fconfigure command to make a connection (channel) unbuffered.

  As already mentioned, expect extension to Tcl provides a much better interface to other programs, which in particular handles the buffering problem.
  [NOTE: add good reference to expect]
  

• If one of the commands in an open |cmd fails the open does not return an error. However, attempting to read input from the file descriptor with gets $file will return an empty string. Using the gets $file input construct will return a character count of -1.
• Tcl does not expand file names like the UNIX/Linux shells do. So:

  exec ls *.tcl
  

  will fail - there is most probably no file with the literal name "*.tcl".

  If you need such an expansion, you should use the glob command:

  eval exec ls [glob *.tcl]
  

  or, from Tcl 8.5 onwards:

  exec ls {expand}[glob *.tcl]
  

  where the {expand} prefix is used to force the list to become individual arguments.
• If one of the commands in an exec call fails to execute, the exec will return an error, and the error output will include the last line describing the error.

The exec treats any output to standard error to be an indication that the external program failed. This is simply a conservative assumption: many programs behave that way and they are sloppy in setting return codes.

Some programs however write to standard error without intending this as an indication of an error. You can guard against this from upsetting your script by using the catch command:

if { [catch { exec ls *.tcl } msg] } {
   puts "Something seems to have gone wrong but we will ignore it"
}

To inspect the return code from a program and the possible reason for failure, you can use the global errorInfo variable:

if { [catch { exec ls *.tcl } msg] } {
   puts "Something seems to have gone wrong:"
   puts "Information about it: $::errorInfo"
}

---

Example

#
# Write a Tcl script to get a platform-independent program:
#
# Create a unique (mostly) file name for a Tcl program
set TMPDIR "/tmp"
if { [info exists ::env(TMP)] } {
    set TMPDIR $::env(TMP)
}
set tempFileName "$TMPDIR/invert_[pid].tcl"

# Open the output file, and
# write the program to it

set outfl [open $tempFileName w]

puts $outfl {
    set len [gets stdin line]
    if {$len < 5} {exit -1}

    for {set i [expr {$len-1}]} {$i >= 0} {incr i -1} {
        append l2 [string range $line $i $i]
    }
    puts $l2
    exit 0
}

# Flush and close the file
flush $outfl
close $outfl

#
# Run the new Tcl script:
#
# Open a pipe to the program (for both reading and writing: r+)
#
set io [open "|[info nameofexecutable] $tempFileName" r+]

#
# send a string to the new program
#     *MUST FLUSH*
puts $io "This will come back backwards."
flush $io

# Get the reply, and display it.
set len [gets $io line]

puts  "To reverse: 'This will come back backwards.'"
puts "Reversed is: $line"
puts "The line is $len characters long"

# Run the program with input defined in an exec call

set invert [exec [info nameofexecutable] $tempFileName << \
       "ABLE WAS I ERE I SAW ELBA"]

# display the results
puts "The inversion of 'ABLE WAS I ERE I SAW ELBA' is \n $invert"

# Clean up
file delete $tempFileName

Previous lesson | Index | Next lesson