Text processing/1: Difference between revisions

 

The data is [http://www.eea.europa.eu/help/eea-help-centre/faqs/how-do-i-obtain-eea-reports free to download and use] and is of this format:

<pre style="overflow:scroll">

1991-03-30 10.000 1 10.000 1 10.000 1 10.000 1 10.000 1 10.000 1 10.000 1 10.000 1 10.000 1 10.000 1 10.000 1 10.000 1 10.000 1 10.000 1 10.000 1 10.000 1 10.000 1 10.000 1 10.000 1 10.000 1 10.000 1 10.000 1 10.000 1 10.000 1

 

Structure your program to show statistics for each line of the file, (similar to the original Python, Perl, and AWK examples below), followed by summary statistics for the file. When showing example output just show a few line statistics and the full end summary.

 

=={{header|Ada}}==

{{libheader|Simple components for Ada}}

with Strings_Edit; use Strings_Edit;

with Strings_Edit.Floats; use Strings_Edit.Floats;

Close (File);

Put_Line ("Syntax error at " & Image (Current.Line) & ':' & Image (Max.Pointer));

The implementation performs minimal checks. The average is calculated over all valid data. For the maximal chain of consequent invalid data, the source line number, the column number, and the time stamp of the first invalid data is printed.

{{out|Sample output}}

Max. 589 false readings start at 1136:20 stamped 1993-2-9

</pre>

 

=={{header|ALGOL 68}}==

{{works with|ALGOL 68G|Any - tested with release mk15-0.8b.fc9.i386}}

<!--{{does not work with|ELLA ALGOL 68|Any (with appropriate job cards) - argc and argv are extensions}} -->

INT no data max := -1; # Max consecutive flags<0 in lines of file #

FLEX[0]STRING no data max line; # ... and line number(s) where it occurs #

upb list := UPB no data max line;

printf(($l"Maximum run"f(p)" of "g(-0)" consecutive false reading"f(p)" ends at line starting with date"f(p)": "$,

Command:

$ a68g ./Data_Munging.a68 - data

Maximum run of 24 consecutive false readings ends at line starting with date: 1991-04-01

</pre>

 

=={{header|AutoHotkey}}==

 

SetFormat, float, 4.2

Totavg := TotSum / TotValid

FileAppend, `n`nDays %Lines%`nMaximal wrong readings: %maxwrong% from %startwrongdate% at %startoccurrence% to %lastwrongdate% at %lastoccurrence%`n`n, %result%

{{out|Sample output}}

<pre>Day: 1990-01-01 sum: 590.00 avg: 26.82 Readings: 22/24.00

 

=={{header|AWK}}==

nodata = 0; # Current run of consecutive flags<0 in lines of file

nodata_max=-1; # Max consecutive flags<0 in lines of file

 

printf "\nMaximum run(s) of %i consecutive false readings ends at line starting with date(s): %s\n", nodata_max, nodata_maxline

{{out|Sample output}}

<pre>bash$ awk -f readings.awk readings.txt | tail

 

=={{header|Batch File}}==

setlocal ENABLEDELAYEDEXPANSION

set maxrun= 0

echo Line: %date% Accept: %count:~-3% tot: %sum:~-8% avg: %mean:~-8%

 

{{out}}

<pre>

 

=={{header|BBC BASIC}}==

IF file% = 0 THEN PRINT "Could not open test data file" : END

PRINT "Overall mean = " ; Total / Count%

@% = &90A

{{out}}

<pre>

 

=={{header|C}}==

#include <stdlib.h>

#include <string.h>

fclose(outfile);

return 0;

{{out|Sample output}}

<pre>1990-01-01 Reject: 2 Accept: 22 Average: 26.818

 

=={{header|C++}}==

#include <fstream>

#include <string>

cout << "Maximum number of consecutive bad readings is " << badCountMax << endl;

cout << "Ends on date " << badDate << endl;

{{out}}

<pre>1990-01-01 Reject: 2 Accept: 22 Average: 26.818

Maximum number of consecutive bad readings is 589

Ends on date 1993-03-05</pre>

 

=={{header|COBOL}}==

PROGRAM-ID. data-munging.

 

 

GOBACK

 

{{Out|Example output}}

 

=={{header|Common Lisp}}==

(defvar *max-invalid*)

(defvar *max-invalid-date*)

(format t "~%Maximum run(s) of ~a consecutive false readings ends at ~

line starting with date(s): ~a~%"

{{out|Example output}}

<pre>...

=={{header|D}}==

{{trans|Python}}

import std.stdio, std.conv, std.string;

 

"readings ends at line starting with date(s): %-(%s, %)",

noDataMax, noDataMaxLine);

The output matches that of the [[#Python|Python]] version.

 

=={{header|Eiffel}}==

class

APPLICATION

 

end

{{out}}

Only the last three lines of the per line summary statistics are shown.

The function file_contents/1 is used by [[Text_processing/2]]. Please update the user if you make any interface changes.

 

-module( text_processing ).

 

{_Previous, Value_flags} = lists:foldr( fun file_content_line_value_flag/2, {[], []}, Rest ), % Preserve order

{binary:bin_to_list( Date_binary ), Value_flags}.

{{out}}

<pre>

=={{header|Forth}}==

{{works with|GNU Forth}}

 

\ 1991-03-30[\t10.000\t[-]1]*24

total-sum f@ total-n @ .mean cr ;

 

 

=={{header|Fortran}}==

Incidentally, a daily average of a set of measurements may be unsuitable when data are missing, as when there is a regular pattern over a day. The N.Z. electricity supply association ruled that in calculating the ratio of daytime to nighttime usage, should there be four or more missing data in a day, then the entire day's data were to be rejected when computing the monthly or quarterly ratio.

 

Crunches a set of hourly data. Starts with a date, then 24 pairs of value,indicator for that day, on one line.

INTEGER Y,M,D !Year, month, and day.

900 CLOSE(IN) !Done.

END !Spaghetti rules.

 

Output:

 

=={{header|Go}}==

 

import (

maxRun, maxDate)

}

{{out}}

<pre>

 

=={{header|Haskell}}==

import Numeric

import Control.Arrow

(\(t,n) -> printf totalFmt n t (t/fromIntegral n)) $ fst summ

mapM_ ((\(l, d1,d2) -> printf maxFmt l d1 d2)

{{out}}

<pre>Some lines:

 

 

=={{header|Icon}} and {{header|Unicon}}==

 

procedure main()

else

write(fout,"No bad runs of data")

{{out|Sample output}}

<pre>...

=={{header|J}}==

'''Solution:'''

parseLine=: 10&({. ,&< (_99&".;._1)@:}.) NB. custom parser

summarize=: # , +/ , +/ % # NB. count,sum,mean

589

]StartDates=: Dates {~ (>:@I.@e.&MaxRun (24 <.@%~ +/)@{. ]) RunLengths

'''Formatting Output'''<br>

Define report formatting verbs:

labels=. , ];.2 'Line: Accept: Line_tot: Line_avg: '

labels , x ,. 7j0 10j3 10j3 ": y

'maxrun dates'=. x

out=. out,LF,'Maximum run(s) of ',(": maxrun),' consecutive false readings ends at line(s) starting with date(s): ',dates

{{out|Show output}}

Line: Accept: Line_tot: Line_avg:

2004-12-28 23 77.800 3.383

Average: 10.497

 

 

=={{header|Java}}==

{{works with|Java|7}}

import java.util.*;

import static java.lang.System.out;

}

}

 

<pre>1990-01-01 out: 2 in: 22 tot: 590.000 avg: 26.818

=={{header|JavaScript}}==

{{works with|JScript}}

var show_lines = 5;

var file_stats = {

function dec3(value) {

return Math.round(value * 1e3) / 1e3;

{{out}}

<pre>Line: 1990-01-01 Reject: 2 Accept: 22 Line_tot: 590 Line_avg: 26.818

 

The "foreach" syntax is:

The basic idea is that for each $row in STREAM, the value specified by VALUE is emitted.

 

If we wished only to produce per-line synopses of the "readings.txt"

file, the following pattern could be used:

In order to distinguish the single-line synopsis from the whole-file synopsis, we will use the following pattern instead:

The "null" is added so that the stream of per-line values can be distinguished from the last value in the stream.

 

 

One point of interest in the following program is the use of JSON objects to store values. This allows mnemonic names to be used instead of local variables.

# "starts": array_of_start_line_values, # of all the maximal runs

# "start_dates": array_of_start_dates # of all the maximal runs

end;

 

 

{{out}}

[22,590,2]

[24,410,0]

[23,47.3,1]

There is one maximal run of lines with flag<=0.

 

=={{header|Julia}}==

using DataFrames

 

maxbaddate = replace("$(df[:Date][maxbadline])", r"T.+$", "")

println("The largest run of bad values is $(maxbadval), on $(maxbaddate) beginning at $(maxbadtime):00 hours.")

{{output}}

<pre>

The overall mean is 18.241666666666667

The largest run of bad values is 23, on 1991-03-31 beginning at 1:00 hours.

</pre>

 

=={{header|Lua}}==

io.input( filename )

 

print( string.format( "Readings: %d", file_lines ) )

print( string.format( "Average: %f", file_sum/file_cnt_data ) )

<pre>Output:

File: readings.txt

Maximum 589 consecutive false readings starting at 1993-02-09.</pre>

 

Scan[(a=Position[#[[3;;All;;2]],1];

Print["Line:",#[[1]] ,"\tReject:", 24 - Length[a], "\t Accept:", Length[a], "\tLine_tot:",

Print["\nFile(s) : ",FileName,"\nTotal : ",AccountingForm@GlobalSum,"\nReadings : ",Nb,

"\nAverage : ",GlobalSum/Nb,"\n\nMaximum run(s) of ",MaxRunRecorded,

 

<pre>Line:1990-01-01 Reject:2 Accept:22 Line_tot:590. Line_avg:26.8182

=={{header|Nim}}==

{{trans|Python}}

 

var

nodata = 0

nodataMax = -1

 

totFile = 0.0

 

for filename in commandLineParams():

var

totLine = 0.0

numLine = 0

 

 

if flag < 1:

inc nodata

numFile += numLine

 

 

echo ""

 

File(s) = readings.txt

Readings = 129403

 

 

=={{header|OCaml}}==

try Some(input_line ic)

with End_of_file -> None

Printf.printf "Maximum run(s) of %d consecutive false readings \

ends at line starting with date(s): %s\n"

 

=={{header|Perl}}==

===An AWK-like solution===

use warnings;

 

printf "\nMaximum run(s) of %i consecutive false readings ends at line starting with date(s): %s\n",

{{out|Sample output}}

<pre>bash$ perl -f readings.pl readings.txt | tail

 

===An object-oriented solution===

use warnings;

 

$parser->_push_bad_range_if_necessary

}

{{out|Sample output}}

<pre>$ perl readings.pl < readings.txt | tail

$</pre>

 

 

 

 

{{out}}

<pre>

 

=={{header|PicoLisp}}==

{{trans|AWK}}

Put the following into an executable file "readings":

 

(let (NoData 0 NoDataMax -1 NoDataMaxline "!" TotFile 0 NumFile 0)

" consecutive false readings ends at line starting with date(s): " NoDataMaxline ) ) )

 

Then it can be called as

<pre>$ ./readings readings.txt |tail

 

=={{header|PL/I}}==

 

declare line character (2000) varying;

finish_up:

 

=={{header|PureBasic}}==

Define File$, InLine$, Part$, i, Out$, ErrEnds$, Errcnt, ErrMax

Define lsum.d, tsum.d, rejects, val.d, readings

;

Print("Press ENTER to exit"): Input()

{{out|Sample output}}

<pre>...

 

=={{header|Python}}==

import sys

 

 

print "\nMaximum run(s) of %i consecutive false readings ends at line starting with date(s): %s" % (

{{out|Sample output}}

<pre>bash$ /cygdrive/c/Python26/python readings.py readings.txt|tail

 

=={{header|R}}==

dfr <- read.delim("readings.txt")

#Calculate daily means

#Calculate time between good measurements

times <- strptime(dfr[1,1], "%Y-%m-%d", tz="GMT") + 3600*seq(1,24*nrow(dfr),1)

 

=={{header|Racket}}==

;; Use SRFI 48 to make %n.nf formats convenient.

(require (prefix-in srfi/48: srfi/48)) ; SRFI 48: Intermediate Format Strings

(unless (zero? N) (srfi/48:format #t "Average = ~10,3F~%" (/ sum N)))

(srfi/48:format #t "~%Maximum run(s) of ~a consecutive false readings ends at line starting with date(s): ~a~%"

{{out|Sample run}}

<pre>$ racket 1.rkt readings/readings.txt | tail

 

Maximum run(s) of 589 consecutive false readings ends at line starting with date(s): 1993-03-05</pre>

 

=={{header|REXX}}==

numeric digits 20 /*allow for bigger (precision) numbers.*/

ifid='READINGS.TXT' /*the name of the input file. */

do j=e to b by -3; _=insert(',',_,j); end /*j*/; return _

/*────────────────────────────────────────────────────────────────────────────*/

'''output''' &nbsp; when using the default input file:

<pre style="height:40ex">

 

=={{header|Ruby}}==

total = { "num_readings" => 0, "num_good_readings" => 0, "sum_readings" => 0.0 }

invalid_count = 0

printf "Average = %.3f\n", total['sum_readings']/total['num_good_readings']

puts ""

 

Alternate implementation:

 

DailyReading = Struct.new(:date, :readings) do

puts

puts "Max run of #{worst_streak.count} consecutive false readings from #{worst_streak.first.date} until #{worst_streak.last.date}"

 

=={{header|Scala}}==

 

A fully functional solution, minus the fact that it uses iterators:

import scala.io.Source

println(report format (files mkString " ", totalSum, totalSize, totalSum / totalSize, invalidCount, startDate))

}

A quick&dirty solution:

def main(args: Array[String]) {

var totalSum = 0.0

println(report format (files mkString " ", totalSum, totalSize, totalSum / totalSize, maxInvalidCount, maxInvalidDate))

}

Last few lines of the sample output (either version):

<pre>

 

=={{header|Sidef}}==

var previous = :valid;

 

var valid = [];

var hour = 0;

if (flag > 0) {

valid << reading;

 

say ("Longest period of invalid readings was #{longest{:count}} hours,\n",

{{out}}

<pre>

</pre>

''Output is from the sample of the task.''

 

=={{header|Tcl}}==

set max_invalid_run_end ""

set tot_file 0

puts "Average = [format %.3f [expr {$tot_file / $num_file}]]"

puts ""

 

=={{header|Ursala}}==

and booleans (type <code>%ebXLm</code>) in the parsed data. The same function is used to compute the daily and the

cumulative statistics.

#import nat

#import flo

@nmrSPDSL -&~&,leql$^; ^/length ~&zn&-@hrZPF+ rlc both ~&rZ+-

 

last few lines of output:

<pre>

 

=={{header|VBScript}}==

Set objFile = objFSO.OpenTextFile(objFSO.GetParentFolderName(WScript.ScriptFullName) &_

"\data.txt",1)

WScript.StdOut.WriteLine

objFile.Close

{{Out}}

<pre>

{{trans|AWK}}

Vedit does not have floating point data type, so fixed point calculations are used here.

File_Open("output.txt")

#51 = Buf_Num // Edit buffer for output file

IT("Maximum run(s) of ") Num_Ins(#13, LEFT+NOCR)

IT(" consecutive false readings ends at line starting with date(s): ") Reg_Ins(15)

{{out|Sample output}}

<pre>

Maximum run(s) of 589 consecutive false readings ends at line starting with date(s): 1993-03-05

</pre>

 

{{omit from|Openscad}}

{{omit from|PARI/GP|No real capacity for string manipulation}}