Talk:User defined pipe and redirection operators: Difference between revisions
Talk:User defined pipe and redirection operators (view source)
Revision as of 19:41, 16 September 2011
, 12 years ago→comments after writing the J implementation
Line 213:
::: I disagree about the pipe-like dataflow being not useful. For example, when reading or writing a PDF, a PDF stream object may go through multiple encoders/filters in sequence, e.g. PNG, then hexencoded, then RLE, then flate, and each stream itself may contain multiple streams inside (not unlike tar xf). When dealing with one of these, it's probably a good idea to use filters that pass on partial data as soon as possible so that total memory usage doesn't get blown out of proportion. It has ''nothing'' to do with OS pipes, either.
::: The beef you have is probably more of a J thing: assume we'll all have massively parallel computers in the near future; assume we'll always have enough memory; thus always deal with the full extent of available data because it's the 'right thing' and will naturally lead to better-looking, more concise code. Which is probably ok for academia, but it's not fair to blame everything else that fall outside of academic scope. --[[User:Ledrug|Ledrug]] 19:27, 16 September 2011 (UTC)
:::: I do not know enough about PDF internals to contemplate that one in any depth. But my impression of PDF is that it represents a sequence of page. So I think the logical architecture there would be to produce each page independently. Or, if there is intermediate state on one page that's needed for a subsequent page, the logical architecture would be to produce each page sequentially. Or perhaps you split that in two and build up a document structure that's independent of the pages and then build the pages independently... But I am not aware of any requirement for an arbitrarily sized buffer which has nothing to do with the structure of the data being processed. That's what pipes do -- and it has no algorithmic value that I'm aware of. Mind you, I have seen PDF implementations which produce arbitrary rectangles on a page rather than the whole page. But that does not seem like much of a win when I am reading the thing -- usually it just means extra waits imposed on me before I can finish reading a page, and even when I am reading a PDF on my phone there seems to be enough memory to cache a complete image of the page at least for a short while. And, even if "rectangle smaller than a page" was an algorithmic requirement, that's still different from a pipe -- your page winds up being a collection of rectangles and your implementation knows how big they are. With pipes, the buffer size is imposed by the OS and can be subject to change for reasons that have nothing to do with your implementation. --[[User:Rdm|Rdm]] 19:41, 16 September 2011 (UTC)
| |||