deflate, inflate - Routines for working with piped variable data

SYNOPSIS

deflate  [ input-dataset deflated-dataset ]
inflate  [ deflated-dataset piped-datafile restored-dataset ]

There are no parameters.

DESCRIPTION

deflate and inflate are two very unusual functions for working with piped, serialized variable data. Currently, only two ingesters, hrptin and metin have options that allow variable data to be serialized and piped to UNIX stdout. deflate and inflate are provided as a mechanism for efficiently transferring ingested datasets from a reception machine to a remote processing machine over a low-bandwidth network.

The following shows the intended use of deflate and inflate in conjunction with metin:

On reception machine
metin pipe_output=yes metin-output | compress > comp-output
deflate metin-output deflate-output
rcp or ftp comp-output and deflate-output to processing machine
On processing machine
uncompress comp-output > uncomp-output
inflate deflate-output uncomp-output restored-metin-output

In the above sequence, compress could be any serial compressor, such as UNIX compress or GNU gzip. (The corresponding uncompress would be zcat and gunzip respectively.)

Also, instead of using rcp or ftp to transfer the compressed data from the reception machine to the processing machine, the compressed data could be written directly to a disk on the processing machine using NFS. Or, the compressed data could be piped into a remote uncompress process on the processing machine.

SEE ALSO

exppipe

NOTES

In general, variable data is written to output datasets using direct access I/O rather than sequential access I/O. Serialization adds information to the stream of piped variable data that allows it to be properly located in the reconstructed dataset created by inflate. Each piece of variable data is prefixed with its length and position in the reconstructed dataset.


Last Update: $Date: 1998/05/28 18:14:49 $