Text processing/1: Difference between revisions

<langsyntaxhighlight lang="ada">with Ada.Text_IO; use Ada.Text_IO;

end Data_Munging;</langsyntaxhighlight>

<langsyntaxhighlight lang="algol68">INT no data := 0; # Current run of consecutive flags<0 in lines of file #

upb list = 1, no data max, no data max = 0, upb list = 1, list repr, no data max line, $l$))</langsyntaxhighlight>

<langsyntaxhighlight AutoHotkeylang="autohotkey"># Author AlephX Aug 17 2011

FileAppend, Valid readings: %TotValid%`nTotal Value: %TotSUm%`nAverage: %TotAvg%, %result%</langsyntaxhighlight>

<langsyntaxhighlight lang="awk">BEGIN{

}</langsyntaxhighlight>

<langsyntaxhighlight lang="dos">@echo off

goto :EOF</langsyntaxhighlight>

<langsyntaxhighlight lang="bbcbasic"> file% = OPENIN("readings.txt")

PRINT '"Longest run of bad readings = " ; BadMax% " ending " BadDate$</langsyntaxhighlight>

<langsyntaxhighlight lang="c">#include <stdio.h>

}</langsyntaxhighlight>

<langsyntaxhighlight Cpplang="cpp">#include <iostream>

}</langsyntaxhighlight>

<langsyntaxhighlight lang="cobol"> IDENTIFICATION DIVISION.

.</langsyntaxhighlight>

<langsyntaxhighlight lang="lisp">(defvar *invalid-count*)

*max-invalid* *max-invalid-date*)))</langsyntaxhighlight>

<langsyntaxhighlight lang="d">void main(in string[] args) {

}</langsyntaxhighlight>

<langsyntaxhighlight lang="forth">\ data munging

main bye</langsyntaxhighlight>

So, to handle this, the plan becomes to read the record into a CHARACTER variable, read the date part with a FORMAT statement working on the first ten characters, and then read the rest of the input via free-format. If the data field is corrupt (say, a letter in place of a digit) the ERR label will be selected. Similarly, when reading from a CHARACTER variable there is no read-next-record available, so if there is a shortage of data (a number is missing) the END label will be selected. A complaint is printed, a HIC counted, and if HIC is not too large, go back and try for the next record. This means that record count no longer relates to the number of values read, so a count of bad values is maintained as well. In the event, no such troublesome records were encountered in this example. An advantage of reading the record into a scratchpad is that any error messages can quote the text of the record. Similarly, tabs could be converted to spaces, etc. but this isn't needed as the free-format reading allows either, and commas. More generally, a much more comprehensive analysis of the record's text could be made should the need arise, most obviously that the date is a valid one and, its hyphens are placed as expected, and the dates are in sequence without gaps. Another useful check is for the appearance of additional text beyond the twenty-four pairs specified - for example, a CRLF has been "lost" and the next record's content follows on the same line. Somehow. I suspect bungles with "cut&paste" hand editing, but the data suppliers never even apologised.

In decades of working with (half-)hourly data on the generation and consumption of electricity, it has been demonstrated that modern data suppliers endlessly demonstratemanifest an inability to stick with a particular format, even one of their own design, and that the daylight savings changeover days (where there are ''not'' twenty-four hours in a day) surprass their competence annually. Persons from a mainframe background do a much more reliable job than those who have only tinkered with spreadsheets.

<langsyntaxhighlight lang="go">package main

}</langsyntaxhighlight>

<langsyntaxhighlight Haskelllang="haskell">import Data.List

. (\(a,b)-> (a,(fst.(dat!!).(`div`24))b,(fst.(dat!!).(`div`24))(a+b)))) $ snd summ</langsyntaxhighlight>

<langsyntaxhighlight Haskelllang="haskell">*Main> :main ["./RC/readings.txt"]</langsyntaxhighlight>

<langsyntaxhighlight Iconlang="icon">record badrun(count,fromdate,todate) # record to track bad runs

end</langsyntaxhighlight>

<langsyntaxhighlight lang="j"> load 'files'

1993-03-05</langsyntaxhighlight>

<langsyntaxhighlight lang="j">formatDailySumry=: dyad define

)</langsyntaxhighlight>

<langsyntaxhighlight lang="j"> (_4{.Dates) formatDailySumry _4{. DailySummary

Maximum run(s) of 589 consecutive false readings ends at line(s) starting with date(s): 1993-03-05</langsyntaxhighlight>

<langsyntaxhighlight lang="java">import java.io.File;

}</langsyntaxhighlight>

<langsyntaxhighlight lang="javascript">var filename = 'readings.txt';

}</langsyntaxhighlight>

<langsyntaxhighlight lang="jq">foreach STREAM as $row ( INITIAL; EXPRESSION; VALUE ).</langsyntaxhighlight>

<langsyntaxhighlight lang="jq">foreach (inputs | split("\t")) as $line (INITIAL; EXPRESSION; VALUE)</langsyntaxhighlight>

<langsyntaxhighlight lang="jq">foreach ((inputs | split("\t")), null) as $line (INITIAL; EXPRESSION; VALUE)</langsyntaxhighlight>

<langsyntaxhighlight lang="jq"># Input: { "max": max_run_length,

process</langsyntaxhighlight>

<langsyntaxhighlight lang="sh">$ jq -c -n -R -r -f Text_processing_1.jq readings.txt

The maximal run has length 93 and starts at line 5378 and has start date 2004-09-30.</langsyntaxhighlight>

<langsyntaxhighlight Lualang="lua">filename = "readings.txt"

print( string.format( "Maximum %d consecutive false readings starting at %s.", max_rejected, max_rejected_date ) )</langsyntaxhighlight>

=={{header|Mathematica}}/{{header|Wolfram Language}}==

<langsyntaxhighlight Mathematicalang="mathematica">FileName = "Readings.txt"; data = Import[FileName,"TSV"];

" consecutive false readings ends at line starting with date(s):",MaxRunTime]</langsyntaxhighlight>

<langsyntaxhighlight lang="nim">import os, sequtils, strutils, sequtilsstrformat

nodataMaxLine: seq[string] = @[]

varfor fline =in open(filename).lines:

fielddata: = line.split()seq[float]

dateflags: = fieldseq[0int]

let flags: seq[int]fields = @[]line.split()

for i, ffield in fieldfields[1 .. -^1]:

if i mod 2 == 0: data.add parseFloat(ffield)

else: flags.add parseInt(ffield)

for datum, flag in items(zip(data, flags)).items:

echolet "Line:average $#= if Reject:numLine $#> Accept0: $#totLine / LineTot: $#float(numLine) LineAvgelse: $#"0.0

echo &"Line: .format({date,} Reject: {data.len - numLine,:2} Accept: {numLine:2} ",

&"LineTot: formatFloat({totLine, precision =:6.2f} 0),LineAvg: formatFloat({average:4.2f}"

Total = 11358393.35839e+0640

Average = 10.497450

Maximum run(s) of 589 consecutive false readings ends at line starting with date(s): 1993-03-05.</pre>

<langsyntaxhighlight lang="ocaml">let input_line ic =

nodata_max (String.concat ", " nodata_maxline);</langsyntaxhighlight>

<langsyntaxhighlight lang="perl">use strict;

$nodata_max, $nodata_maxline;</langsyntaxhighlight>

<langsyntaxhighlight lang="perl">use strict;

}</langsyntaxhighlight>

=={{header|Perl 6Phix}}==

date:1990-01-01 accept:22 reject:2 26.818 mean from 22 validsum:590.0

date:1990-01-02 accept:24 reject:0 17.083 mean from 24 validsum:410.0

date:1990-01-03 accept:24 reject:0 58.958 mean from 24 validsum:1415.0

<langsyntaxhighlight PicoLisplang="picolisp">#!/usr/bin/picolisp /usr/lib/picolisp/lib.l

(bye)</langsyntaxhighlight>

<langsyntaxhighlight lang="pli">text1: procedure options (main); /* 13 May 2010 */

end text1;</langsyntaxhighlight>

<langsyntaxhighlight PureBasiclang="purebasic">#TASK="Text processing/1"

EndIf</langsyntaxhighlight>

<langsyntaxhighlight lang="python">import fileinput

nodata_max, ", ".join(nodata_maxline))</langsyntaxhighlight>

<langsyntaxhighlight Rlang="r">#Read in data from file

hours.between.good.measurements <- diff(times[t(flags)])/3600</langsyntaxhighlight>

<langsyntaxhighlight lang="racket">#lang racket

max-consecutive-false (string-join max-false-tags))))</langsyntaxhighlight>

<langsyntaxhighlight lang="rexx">/*REXX program to process instrument data from a data file. */

sy: say arg(1); call lineout ofid,arg(1); return</langsyntaxhighlight>

<langsyntaxhighlight lang="ruby">filename = "readings.txt"

puts "Maximum run(s) of #{max_invalid_count} consecutive false readings ends at #{invalid_run_end}"</langsyntaxhighlight>

<langsyntaxhighlight lang="ruby">Reading = Struct.new(:date, :value, :flag)

<langsyntaxhighlight lang="scala">object DataMunging {

}</langsyntaxhighlight>

<langsyntaxhighlight lang="scala">object AltDataMunging {

}</langsyntaxhighlight>

<langsyntaxhighlight lang="tcl">set max_invalid_run 0

puts "Maximum run(s) of $max_invalid_run consecutive false readings ends at $max_invalid_run_end"</langsyntaxhighlight>

<langsyntaxhighlight Ursalalang="ursala">#import std

main = ^T(daily_stats^lrNCT/~& @mSL 'summary ':,long_run) parsed_data</langsyntaxhighlight>

<langsyntaxhighlight lang="vb">Set objFSO = CreateObject("Scripting.FileSystemObject")

Set objFSO = Nothing</langsyntaxhighlight>

<langsyntaxhighlight lang="vedit">#50 = Buf_Num // Current edit buffer (source data)

IN</langsyntaxhighlight>