User defined pipe and redirection operators

From Rosetta Code
User defined pipe and redirection operators is a draft programming task. It is not yet considered ready to be promoted as a complete task, for reasons that should be found in its talk page.

If the language supports operator definition, then:

  • create "user defined" the equivalents of the Unix shell "<", "|", ">", "<<", ">>" and $(cmd) operators.
  • Provide simple equivalents of: cat, tee, grep, & uniq, but as filters/procedures native to the specific language.
  • Replicate the below sample shell script, but in the specific language
  • Specifically do not cache the entire stream before the subsequent filter/procedure starts. Pass each record on as soon as available through each of the filters/procedures in the chain.

Alternately: if the language does not support operator definition then replace with:

  • define the procedures: input(cmd,stream), pipe(stream,cmd), output(stream, stream), whereis(array), append(stream)
For bonus Kudos: Implement the shell "&" concept as a dyadic operator in the specific language. e.g.:
( head x & tail x & wait ) | grep test

Sample shell script: ¢ draft - pending a better (more interesting) suggestion ¢

aa="$(
(
head -4 < List_of_computer_scientists.lst;
cat List_of_computer_scientists.lst | grep ALGOL | tee ALGOL_pioneers.lst;
tail -4 List_of_computer_scientists.lst
) | sort | uniq | tee the_important_scientists.lst | grep aa
);
echo "
Pioneer: $aa"

Input Records:

A test sample of scientists from wikipedia's "List of computer scientists"
Name Areas of interest
Wil van der Aalst business process management, process mining, Petri nets
Hal Abelson intersection of computing and teaching
Serge Abiteboul database theory
Samson Abramsky game semantics
Leonard Adleman RSA, DNA computing
Manindra Agrawal polynomial-time primality testing
Luis von Ahn human-based computation
Alfred Aho compilers book, the 'a' in AWK
Stephen R. Bourne Bourne shell, portable ALGOL 68C compiler
Kees Koster ALGOL 68
Lambert Meertens ALGOL 68, ABC (programming language)
Peter Naur BNF, ALGOL 60
Guido van Rossum Python (programming language)
Adriaan van Wijngaarden Dutch pioneer; ARRA, ALGOL
Dennis E. Wisnosky Integrated Computer-Aided Manufacturing (ICAM), IDEF
Stephen Wolfram Mathematica
William Wulf compilers
Edward Yourdon Structured Systems Analysis and Design Method
Lotfi Zadeh fuzzy logic
Arif Zaman Pseudo-random number generator
Albert Zomaya Australian pioneer of scheduling in parallel and distributed systems
Konrad Zuse German pioneer of hardware and software

These records can be declared in any format appropriate to the specific language. eg table, array, list, table or text file etc.

Output:

Pioneer: Adriaan van Wijngaarden - Dutch pioneer; ARRA, ALGOL

ALGOL 68[edit]

See User defined pipe and redirection operators/ALGOL 68

Works with: ALGOL 68 version Revision 1; one minor extension - PRAGMA READ; one major extension - Algol68G's Currying.
Works with: ALGOL 68G version tested with release 1.18.0-9h.tiny.

File: Iterator_pipe_operators.a68

MODE
PAGEIN = PAGE,
PAGEAPPEND = REF PAGE,
PAGEOUT = REF PAGE;
 
MODE
MOID = VOID,
YIELDLINE = PROC(LINE)VOID,
GENLINE = PROC(YIELDLINE)VOID,
FILTER = PROC(GENLINE)GENLINE, # the classic shell filter #
MANYTOONE = PROC([]GENLINE)GENLINE; # eg cat, as in con[cat]enate #
 
PRIO =: = 5, << = 5, >> = 5;
 
OP < = (FILTER filter, PAGEIN page)GENLINE: filter(READ page),
< = (MANYTOONE cmd, PAGEIN page)GENLINE: cmd(READ page),
<< = (FILTER filter, PAGEIN page)GENLINE: filter(READ page),
> = (GENLINE gen, PAGEOUT page)VOID: gen(WRITE page),
>> = (GENLINE gen, PAGEAPPEND page)VOID: gen(APPEND page),
=: = (GENLINE gen, FILTER filter)GENLINE: filter(gen),
=: = (GENLINE gen, MANYTOONE cmd)GENLINE: cmd(gen);

File: Iterator_pipe_utilities.a68

 # Sample ''utilities'' PROCedure declarations #
PROC cat = ([]GENLINE argv)GENLINE:~;
PROC tee = ([]YIELDLINE args filter)FILTER:~;
PROC grep = (STRING pattern, []GENLINE argv)GENLINE:~;
PROC uniq = (GENLINE arg)GENLINE:~;
PROC sort = (GENLINE arg)GENLINE:~;
PROC head = (INT n, []GENLINE args)GENLINE:~;
PROC tail = (INT n, []GENLINE args)GENLINE:~;

File: Iterator_pipe_page.a68

# Sample ''pipe I/O'' OPerator declarations #
OP READ = (PAGEIN page)GENLINE:~;
OP WRITE = (PAGEOUT page)YIELDLINE: ~;
OP APPEND = (PAGEAPPEND page)YIELDLINE:~;

File: test_Iterator_pipe_page.a68

#!/usr/local/bin/a68g --script #
# First define what kind of record (aka LINE) we are piping and filtering #
FORMAT line fmt = $xg$;
MODE
LINE = STRING,
PAGE = FLEX[0]LINE,
BOOK = FLEX[0]PAGE;
 
PR READ "Iterator_pipe_page.a68" PR
PR READ "Iterator_pipe_operators.a68" PR
PR READ "Iterator_pipe_utilities.a68" PR
 
PAGE list of computer scientists = (
"Wil van der Aalst - business process management, process mining, Petri nets",
"Hal Abelson - intersection of computing and teaching",
"Serge Abiteboul - database theory",
"Samson Abramsky - game semantics",
"Leonard Adleman - RSA, DNA computing",
"Manindra Agrawal - polynomial-time primality testing",
"Luis von Ahn - human-based computation",
"Alfred Aho - compilers book, the 'a' in AWK",
"Stephen R. Bourne - Bourne shell, portable ALGOL 68C compiler",
"Kees Koster - ALGOL 68",
"Lambert Meertens - ALGOL 68, ABC (programming language)",
"Peter Naur - BNF, ALGOL 60",
"Guido van Rossum - Python (programming language)",
"Adriaan van Wijngaarden - Dutch pioneer; ARRA, ALGOL",
"Dennis E. Wisnosky - Integrated Computer-Aided Manufacturing (ICAM), IDEF",
"Stephen Wolfram - Mathematica",
"William Wulf - compilers",
"Edward Yourdon - Structured Systems Analysis and Design Method",
"Lotfi Zadeh - fuzzy logic",
"Arif Zaman - Pseudo-random number generator",
"Albert Zomaya - Australian pioneer of scheduling in parallel and distributed systems",
"Konrad Zuse - German pioneer of hardware and software"
);
 
PAGE algol pioneers list, the scientists list;
PAGE aa;
 
# Now do a bit of plumbing: #
cat((
head(4, ) < list of computer scientists,
cat(READ list of computer scientists) =: grep("ALGOL", ) =: tee(WRITE algol pioneers list),
tail(4, READ list of computer scientists)
)) =: sort =: uniq =: tee(WRITE the scientists list) =: grep("aa", ) >> aa;
 
# Finally check the result: #
printf((
$"Pioneer: "$, line fmt, aa, $l$,
$"Number of Algol pioneers: "g(-0)$, UPB algol pioneers list, $l$,
$"Number of scientists: "g(-0)$, UPB the scientists list, $l$
))

Output:

Pioneer:  Adriaan van Wijngaarden - Dutch pioneer; ARRA, ALGOL
Number of Algol pioneers: 6
Number of scientists: 15

Go[edit]

package main
 
import (
"bufio"
"fmt"
"io"
"io/ioutil"
"regexp"
"sort"
"strings"
)
 
// fake file system
var fs = make(map[string]string)
 
type file struct{ name string }
 
func (f file) Write(p []byte) (int, error) {
fs[f.name] += string(p)
return len(p), nil
}
 
// role of > operator
func toName(name string, src io.Reader) {
io.Copy(file{name}, src)
}
 
// role of < operator
func fromName(name string) io.Reader {
return strings.NewReader(fs[name])
}
 
func tee(in io.Reader, name string) io.Reader {
return io.TeeReader(in, file{name})
}
 
func grep(in io.Reader, pat string) io.Reader {
pr, pw := io.Pipe()
go func() {
bf := bufio.NewReader(in)
for {
line, readErr := bf.ReadString('\n')
if match, _ := regexp.MatchString(pat, line); match {
_, writeErr := io.WriteString(pw, line)
if writeErr != nil {
return
}
}
if readErr != nil {
pw.CloseWithError(readErr)
return
}
}
pw.Close()
}()
return pr
}
 
func uniq(in io.Reader) io.Reader {
pr, pw := io.Pipe()
go func() {
bf := bufio.NewReader(in)
var last string
for {
s, readErr := bf.ReadString('\n')
switch readErr {
case nil:
if s != last {
_, writeErr := io.WriteString(pw, s)
if writeErr != nil {
return
}
last = s
}
continue
case io.EOF:
if s > "" && s+"\n" != last {
_, writeErr := io.WriteString(pw, s+"\n")
if writeErr != nil {
return
}
}
pw.Close()
default:
pw.CloseWithError(readErr)
}
return
}
}()
return pr
}
 
func head(in io.Reader, lines int) io.Reader {
pr, pw := io.Pipe()
go func() {
if lines <= 0 {
lines = 10
}
for bf := bufio.NewReader(in); lines > 0; lines-- {
s, readErr := bf.ReadString('\n')
_, writeErr := io.WriteString(pw, s)
if writeErr != nil {
return
}
if readErr == nil {
continue
}
if readErr == io.EOF {
if s > "" {
io.WriteString(pw, "\n")
}
}
pw.CloseWithError(readErr)
return
}
pw.Close()
}()
return pr
}
 
func tail(in io.Reader, lines int) io.Reader {
pr, pw := io.Pipe()
go func() {
if lines <= 0 {
lines = 10
}
ring := make([]string, lines)
rn := 0
full := false
bf := bufio.NewReader(in)
var readErr error
var s string
for {
s, readErr = bf.ReadString('\n')
if readErr == nil {
ring[rn] = s
rn++
if rn == lines {
rn = 0
full = true
}
continue
}
if readErr == io.EOF && s > "" {
ring[rn] = s + "\n"
rn++
}
break
}
writeLines := func(start, end int) {
for i := start; i < end; i++ {
if _, err := io.WriteString(pw, ring[i]); err != nil {
return
}
}
}
if full {
writeLines(rn, lines)
}
writeLines(0, rn)
pw.CloseWithError(readErr)
}()
return pr
}
 
func sortPipe(in io.Reader) io.Reader {
pr, pw := io.Pipe()
go func() {
b, err := ioutil.ReadAll(in)
if len(b) > 0 {
if b[len(b)-1] == '\n' {
b = b[:len(b)-1]
}
list := strings.Split(string(b), "\n")
sort.Strings(list)
for _, s := range list {
if _, err := io.WriteString(pw, s+"\n"); err != nil {
return
}
}
}
pw.CloseWithError(err)
}()
return pr
}
 
func main() {
fs["List_of_computer_scientists.lst"] =
`Wil van der Aalst business process management, process mining, Petri nets
Hal Abelson intersection of computing and teaching
Serge Abiteboul database theory
Samson Abramsky game semantics
Leonard Adleman RSA, DNA computing
Manindra Agrawal polynomial-time primality testing
Luis von Ahn human-based computation
Alfred Aho compilers book, the 'a' in AWK
Stephen R. Bourne Bourne shell, portable ALGOL 68C compiler
Kees Koster ALGOL 68
Lambert Meertens ALGOL 68, ABC (programming language)
Peter Naur BNF, ALGOL 60
Guido van Rossum Python (programming language)
Adriaan van Wijngaarden Dutch pioneer; ARRA, ALGOL
Dennis E. Wisnosky Integrated Computer-Aided Manufacturing (ICAM), IDEF
Stephen Wolfram Mathematica
William Wulf compilers
Edward Yourdon Structured Systems Analysis and Design Method
Lotfi Zadeh fuzzy logic
Arif Zaman Pseudo-random number generator
Albert Zomaya Australian pioneer of scheduling in parallel and distributed systems
Konrad Zuse German pioneer of hardware and software
`

toName("aa", grep(tee(uniq(sortPipe(io.MultiReader(
head(fromName("List_of_computer_scientists.lst"), 4),
tee(grep(fromName("List_of_computer_scientists.lst"), "ALGOL"),
"ALGOL_pioneers.lst"),
tail(fromName("List_of_computer_scientists.lst"), 4)))),
"the_important_scientists.lst"), "aa"))
 
fmt.Print("Pioneer: ", fs["aa"])
showCount("Number of ALGOL pioneers", "ALGOL_pioneers.lst")
showCount("Number of scientists", "the_important_scientists.lst")
}
 
func showCount(heading, name string) {
if data, ok := fs[name]; ok {
lines := strings.Split(data, "\n")
n := len(lines)
if lines[n-1] == "" {
n--
}
fmt.Printf("%s: %v\n", heading, n)
} else {
fmt.Println(name, "not found")
}
}

Output:

Pioneer: Adriaan van Wijngaarden  Dutch pioneer; ARRA, ALGOL
Number of ALGOL pioneers: 5
Number of scientists: 13

J[edit]

If we ignore the gratuitous complexity requirements of this task, it boils down to this:

Step 0: get the data. The task does not specify how to get the data, so here I use lynx, which is readily available on most unix-like systems, including cygwin. Note that lynx needs to be in the OS PATH when running j.

require 'task'
data=:<;._2 shell 'lynx -dump -nolist -width=999 http://en.wikipedia.org/wiki/List_of_computer_scientists'

Step 1: define task core algorithms:

grep=: [email protected]:0 # ]

Step 2: select and display the required data:

   ;'aa' grep 'ALGOL' grep data
* Adriaan van Wijngaarden - Dutch pioneer; ARRA, ALGOL
 

As for the concept of a pipe that presents data one record at a time to a downstream function, that corresponds to the J operator @ and we could achieve the "left to right" syntax mechanism by explicitly ordering its arguments 2 :[email protected]' but it's not clear how to demonstrate that usefully, in this task. (And, I could write a lot of code, to accomplish what's being accomplished here with the two successive greps, but I find that concept distasteful and tedious.)

However, note also that J's sort (/:~) and uniq (~.) operations would work just fine on this kind of data. For example:

   ;'aa' grep 'ALGOL' grep data,data
* Adriaan van Wijngaarden - Dutch pioneer; ARRA, ALGOL
* Adriaan van Wijngaarden - Dutch pioneer; ARRA, ALGOL
 
 ;'aa' grep ~. 'ALGOL' grep data,data
* Adriaan van Wijngaarden - Dutch pioneer; ARRA, ALGOL
 

That said, this implements most (perhaps all) of the required complexities:

declare=: [email protected]
tee=: 4 :0
if._1=nc boxopen x do.(x)=: '' end.
(x)=: (do x),y
y
)
grep=: 4 :'x ([email protected]:0 # ]) y'
pipe=:2 :'v@(u"0)' NB. small pipe -- spoon feed one record at a time
PIPE=:2 :0 NB. big pipe -- feed everything all together
v u y
:
v (,x)"_ y NB. syntactic sugar, beware of tooth decay
)
head=: {.
tail=: -@[ {. ]
sort=: /:~
uniq=: ~.
cat=: ]
echo=: smoutput@;
 
declare;:'ALGOL_pioneers the_important_scientists'
aa=: ;do TXT=:0 :0 -.LF
(
(
4 head data
),(
cat pipe
('ALGOL'&grep) pipe
('ALGOL_pioneers'&tee)
data
),(
4 tail data
)) ''PIPE
sort PIPE
uniq PIPE
('the_important_scientists'&tee) PIPE
('aa'&grep)
''
)
 
echo 'Pioneer:';aa

This produces the result:

Pioneer:     * Adriaan van Wijngaarden - Dutch pioneer; ARRA, ALGOL

Perl[edit]

Implementing only stream chaining, cat, grep and tee. Oddly enough, I don't feel the urge to implement all of the more-or-less-the-same features asked for by the task.

use strict;
use 5.10.0;
 
package IO::File;
sub readline { CORE::readline(shift) } # icing, not essential
 
package Stream;
use Exporter 'import';
 
# Only overload one operator. "file | stream" and "stream | stream"
# are not ambiguous like with shell commands.
use overload '|' => \&chain;
sub new {
my $cls = shift;
bless { args => [@_] }, ref $cls || $cls;
}
 
sub chain {
my ($left, $right, $swap) = @_;
($left, $right) = ($right, $left) if $swap;
 
if (!ref $left) {
my $h;
open $h, $left and $left = $h or die $left
}
 
if (!ref $right) {
# output file not implemented: don't know where I'd ever use it
my $h;
open $h, '>', $right and $right = $h or die $right
}
 
if (ref $left and $left->isa(__PACKAGE__)) {
$left->{output} = $right;
}
 
if (ref $right and $right->isa(__PACKAGE__)) {
$right->{input} = $left;
}
$right;
}
 
# Read a line and do something to it. By default it's this dummy
# pass-through function. Overriding it defines a subclass' behavior
sub transform { shift; shift }
 
sub readline {
my $obj = shift;
my $line;
return $line = <STDIN> unless defined $obj->{input};
 
while (1) {
$line = $obj->{input}->readline or return;
return $line if $line = $obj->transform($line);
}
}
 
package Cat;
use parent -norequire, 'Stream';
# Dummy, exactly the same as Stream. Except now we can invoke
# as Cat::ter, instead of Stream::ter, which is not even a word
sub ter { Cat->new(@_) }
 
package Grep;
use parent -norequire, 'Stream';
 
sub transform {
my ($obj, $line) = @_;
for (@{$obj->{args}}) {
return $line if ($line =~ $_)
}
return;
}
 
sub per { Grep->new(@_) }
 
package Tee;
use parent -norequire, 'Stream';
sub er{
my $obj = Tee->new(@_);
@{$obj->{tees}} =
map { open my $h, '>', $_ or die $_; $h }
@{$obj->{args}};
delete $obj->{args};
$obj
}
 
sub transform {
my ($obj, $line) = @_;
print $_ $line for @{$obj->{tees}};
$line;
}
 
package main;
my $chain =
'/etc/services' # head of chain; omit to use STDIN
| Cat::ter # don't really need this line
| Grep::per(qr/tcp/)
| Tee::er('/tmp/t1', '/tmp/t2')
| Grep::per(qr/170/)
| Tee::er('/tmp/t3')
;
 
print while $_ = $chain->readline;

Racket[edit]

 
#lang racket
 
(module racksh racket
 
(require (for-syntax racket/syntax))
 
(provide (except-out (all-from-out racket) #%app sort)
(rename-out [shell-app #%app]))
 
(define-syntax (shell-app stx)
(define (=? x y)
(eq? (if (syntax? x) (syntax-e x) x) (if (syntax? y) (syntax-e y) y)))
(define (err msg) (raise-syntax-error 'shell msg stx))
(define (make-call xs)
(cond [(null? xs) (err "empty form")]
[(string? (syntax-e (car xs))) #`(shell #,(car xs) #,@(cdr xs))]
[(eq? #\{ (syntax-property (car xs) 'paren-shape)) (cons #'void xs)]
[else xs]))
(syntax-case stx ()
[(_ x ...) (eq? #\{ (syntax-property stx 'paren-shape))
(let loop ([xs (reverse (syntax->list #'(\; x ...)))]
[form '()] [thunks '()] [I #f] [O #f] [seq '()])
(cond [(null? xs) #`(begin #,@seq)]
[(=? '\; (car xs))
(loop (cdr xs) '() '() #f #f
(if (and (null? form) (null? thunks)) seq
(let* ([form (make-call form)]
[r (if (null? thunks) form
#`(pipe (list (λ() #,form) #,@thunks)))]
[r (if I #`(#,@I (λ() #,r)) r)]
[r (if O #`(#,@O (λ() #,r)) r)])
(cons r seq))))]
[(=? '\| (car xs))
(loop (cdr xs) '() (cons #`(λ() #,(make-call form)) thunks) I O
seq)]
[(or (=? '< (car xs)) (=? '<< (car xs)))
(cond [(null? form) (err "missing expression after < or <<")]
[I (err "duplicate < or << specified")]
[else (loop (cdr xs) (cdr form) thunks
(if (=? '< (car xs))
#`(with-input-from-file #,(car form))
#`(with-input-from-string #,(car form)))
O seq)])]
[(or (=? '> (car xs)) (=? '>> (car xs)))
(cond [(null? form) (err "missing expression after > or >>")]
[O (err "duplicate > or >> specified")]
[else (loop (cdr xs) (cdr form) thunks I
#`(with-output-to-file #,(cadr xs) #:exists
'#,(if (=? '> (car xs)) 'truncate 'append))
seq)])]
[else (loop (cdr xs) (cons (car xs) form) thunks I O seq)]))]
[(_ x ...) #'(x ...)]))
 
(define (pipe thunks)
(if (null? (cdr thunks)) ((car thunks))
(let-values ([(I O) (make-pipe)])
(parameterize ([current-output-port O])
(thread (λ() (dynamic-wind void (car thunks)
(λ() (close-output-port O))))))
(parameterize ([current-input-port I]) (pipe (cdr thunks))))))
 
(define (shell cmd . args)
(apply system* (find-executable-path cmd)
(map (λ(x) (if (string? x) x
(with-output-to-string (λ() (display x)))))
args)))
 
 ;; implements a common interface of reading a bunch of files; '- means
 ;; stdin; no files means just stdin
(define (call/files files proc)
(if (null? files) (proc (current-input-port))
(let-values ([(I O) (make-pipe)])
(thread
(λ() (for ([file (in-list files)])
(if (eq? '- file)
(copy-port (current-input-port) O)
(call-with-input-file file (λ(i) (copy-port i O)))))
(close-output-port O)))
(proc I))))
 
(define-syntax (define-io stx)
(syntax-case stx ()
[(_ (name . xs) E ...)
(with-syntax ([io-name (format-id #'name "io-~a" #'name)])
#'(begin (provide (rename-out [io-name name]))
(define (io-name . xs) E ...)))]))
 
(define-io (echo . xs)
(for-each display (add-between xs " "))
(newline))
(define-io (cat . files)
(call/files files (λ(I) (copy-port I (current-output-port)))))
(define-io (sort . files)
(display-lines (sort (call/files files port->lines) string<?)))
(define-io (head n . files)
(call/files files
(λ(I) (for ([l (in-lines I)] [i (in-range n)]) (displayln l)))))
(define-io (tail n . files)
(display-lines (take-right (call/files files port->lines) n)))
(define-io (grep rx . files)
(call/files files
(λ(I) (for ([l (in-lines I)] #:when (regexp-match? rx l)) (displayln l)))))
(define-io (uniq . files)
(call/files files
(λ(I) (let loop ([last #f])
(define line (read-line I))
(unless (eof-object? line)
(unless (equal? line last) (displayln line))
(loop line))))))
(define-io (tee file)
(call-with-output-file file #:exists 'truncate
(λ(O) (for ([l (in-lines (current-input-port))])
(displayln l O) (displayln l)))))
 
(provide $)
(define-syntax-rule ($ E ...)
(with-output-to-string (λ() E ...))))
 
(module sample (submod ".." racksh)
{\;
define file "List_of_computer_scientists.lst" \;
define aa ($
{{ head 4 < file \;
cat file \| grep "ALGOL" \;
tail 4 < file \;
} \| sort \| uniq \| tee "the_important_scientists.lst" \| grep "aa"
}) \;
echo "Pioneer:" aa}
)
 
(require 'sample)
 

Tcl[edit]

The syntax of redirections is slightly out, as they're inserted as explicit pipeline elements, and standard Tcl syntax is used to pull in results from sub-pipelines (because it is vastly simpler):

package require Tcl 8.6
 
# Helpers
proc aspipe {input cmd args} {
tailcall coroutine pipe[incr ::pipes] eval {yield [info coroutine];} \
[list $cmd $input {*}$args] {;break}
}
proc forpipe {input var body} {
upvar 1 $var v
while {[llength [info commands $input]]} {
set v [$input]
uplevel 1 $body
}
}
 
# Pipeline framework; parses, collects results as newline-separated lines
proc pipeline args {
if {![llength $args]} {error "no pipeline components"}
set p [aspipe {} eval {while {[gets stdin line]>=0} {yield $line}}]
set oi -1
foreach ni [lsearch -all [lappend args "|"] "|"] {
set cmd [lrange $args [expr {$oi+1}] [expr {$ni-1}]]
set p [aspipe $p {*}$cmd]
set oi $ni
}
set accum {}
forpipe $p line {
lappend accum $line
}
return [join $accum \n]
}
 
# Pipeline implementations - redirections
proc << {in args} {
foreach string $args {
foreach line [split $string "\n"] {
yield $line
}
}
}
proc < {in filename} {
set f [open $filename]
while {[gets $f line] >= 0} {
yield $line
}
close $f
}
proc > {in filename} {
set f [open $filename w]
forpipe $in line {
puts $f $line
}
close $f
}
proc >> {in filename} {
set f [open $filename a]
forpipe $in line {
puts $f $line
}
close $f
}
 
# Pipeline implementations - "commands"
proc cat {in args} {
foreach filename $args {
if {$filename eq "-"} {
forpipe $in line {
yield $line
}
} else {
set f [open $filename]
while {[gets $f line] >= 0} {
yield $line
}
close $f
}
}
}
proc head {in count} {
forpipe $in line {
if {[incr i] <= $count} {
yield $line
}
}
}
proc tail {in count} {
incr count -1
set accum {}
forpipe $in line {
set accum [lrange [lappend accum $line] end-$count end]
}
foreach item $accum {yield $item}
}
proc grep {in RE} {
forpipe $in line {
if {[regexp $RE $line]} {yield $line}
}
}
proc sort {in} {
set accum {}
forpipe $in line {
lappend accum $line
}
foreach line [lsort $accum] {yield $line}
}
proc uniq {in} {
forpipe $in line {
if {![info exists prev] || $prev ne $line} {
yield $line
}
set prev $line
}
}
proc wc {in {type "words"}} {
set count 0
switch $type {
words { set RE {\S+} }
lines { set RE {.*} }
}
forpipe $in line {
incr count [regexp -all $RE $line]
}
yield $count
}
proc tee {in filename} {
set f [open $filename w]
forpipe $in line {
puts $f $line
yield $line
}
close $f
}

Sample pipeline:

set file "List_of_computer_scientists.lst"
set aa [pipeline \
<< [pipeline < $file | head 4] [pipeline < $file | grep ALGOL | tee "ALGOL_pioneers.txt"] [pipeline < $file | tail 4] \
| sort | uniq | tee "the_important_scientists.lst" | grep aa]
puts "Pioneer: $aa"