pipes Namespace Reference

Data Structures
class	Template

Functions
def	makepipeline
def	quote
def	test

Variables
list	__all__ = ["Template"]
string	FILEIN_FILEOUT = 'ff'
string	STDIN_FILEOUT = '-f'
string	FILEIN_STDOUT = 'f-'
string	STDIN_STDOUT = '--'
string	SOURCE = '.-'
string	SINK = '-.'
list	stepkinds
string	_safechars = string.ascii_letters+string.digits+'!@%_-+=:,./'
string	_funnychars = '"`$\\'

Detailed Description

Conversion pipeline templates.

The problem:
------------

Suppose you have some data that you want to convert to another format,
such as from GIF image format to PPM image format.  Maybe the
conversion involves several steps (e.g. piping it through compress or
uuencode).  Some of the conversion steps may require that their input
is a disk file, others may be able to read standard input; similar for
their output.  The input to the entire conversion may also be read
from a disk file or from an open file, and similar for its output.

The module lets you construct a pipeline template by sticking one or
more conversion steps together.  It will take care of creating and
removing temporary files if they are necessary to hold intermediate
data.  You can then use the template to do conversions from many
different sources to many different destinations.  The temporary
file names used are different each time the template is used.

The templates are objects so you can create templates for many
different conversion steps and store them in a dictionary, for
instance.


Directions:
-----------

To create a template:
t = Template()

To add a conversion step to a template:
   t.append(command, kind)
where kind is a string of two characters: the first is '-' if the
command reads its standard input or 'f' if it requires a file; the
second likewise for the output. The command must be valid /bin/sh
syntax.  If input or output files are required, they are passed as
$IN and $OUT; otherwise, it must be  possible to use the command in
a pipeline.

To add a conversion step at the beginning:
   t.prepend(command, kind)

To convert a file to another file using a template:
  sts = t.copy(infile, outfile)
If infile or outfile are the empty string, standard input is read or
standard output is written, respectively.  The return value is the
exit status of the conversion pipeline.

To open a file for reading or writing through a conversion pipeline:
   fp = t.open(file, mode)
where mode is 'r' to read the file, or 'w' to write it -- just like
for the built-in function open() or for os.popen().

To create a new template object initialized to a given one:
   t2 = t.clone()

For an example, see the function test() at the end of the file.

Function Documentation

def pipes.makepipeline	(		infile,
			steps,
			outfile
	)

Definition at line 196 of file pipes.py.

References tempfile.mktemp(), and quote().

def makepipeline(infile, steps, outfile):
    # Build a list with for each command:
    # [input filename or '', command string, kind, output filename or '']

    list = []
    for cmd, kind in steps:
        list.append(['', cmd, kind, ''])
    #
    # Make sure there is at least one step
    #
    if not list:
        list.append(['', 'cat', '--', ''])
    #
    # Take care of the input and output ends
    #
    [cmd, kind] = list[0][1:3]
    if kind[0] == 'f' and not infile:
        list.insert(0, ['', 'cat', '--', ''])
    list[0][0] = infile
    #
    [cmd, kind] = list[-1][1:3]
    if kind[1] == 'f' and not outfile:
        list.append(['', 'cat', '--', ''])
    list[-1][-1] = outfile
    #
    # Invent temporary files to connect stages that need files
    #
    garbage = []
    for i in range(1, len(list)):
        lkind = list[i-1][2]
        rkind = list[i][2]
        if lkind[1] == 'f' or rkind[0] == 'f':
            temp = tempfile.mktemp()
            garbage.append(temp)
            list[i-1][-1] = list[i][0] = temp
    #
    for item in list:
        [inf, cmd, kind, outf] = item
        if kind[1] == 'f':
            cmd = 'OUT=' + quote(outf) + '; ' + cmd
        if kind[0] == 'f':
            cmd = 'IN=' + quote(inf) + '; ' + cmd
        if kind[0] == '-' and inf:
            cmd = cmd + ' <' + quote(inf)
        if kind[1] == '-' and outf:
            cmd = cmd + ' >' + quote(outf)
        item[1] = cmd
    #
    cmdlist = list[0][1]
    for item in list[1:]:
        [cmd, kind] = item[1:3]
        if item[0] == '':
            if 'f' in kind:
                cmd = '{ ' + cmd + '; }'
            cmdlist = cmdlist + ' |\n' + cmd
        else:
            cmdlist = cmdlist + '\n' + cmd
    #
    if garbage:
        rmcmd = 'rm -f'
        for file in garbage:
            rmcmd = rmcmd + ' ' + quote(file)
        trapcmd = 'trap ' + quote(rmcmd + '; exit') + ' 1 2 3 13 14 15'
        cmdlist = trapcmd + '\n' + cmdlist + '\n' + rmcmd
    #
    return cmdlist


# Reliably quote a string as a single argument for /bin/sh