Entropy: Difference between revisions

=={{header|11l}}==

DefaultDict[Char, Int] hist

L(c) source

R r

{{out}}

<pre>

=={{header|Ada}}==

Uses Ada 2012.

procedure Count_Entropy is

Put(Result, Fore => 1, Aft => 5, Exp => 0);

end;

=={{header|Aime}}==

real h, v;

index x;

}

Examples:

<pre>$ aime -a tmp/entr 1223334444

=={{header|ALGOL 68}}==

# calculate the shannon entropy of a string #

PROC shannon entropy = ( STRING s )REAL:

print( ( shannon entropy( "1223334444" ), newline ) )

{{out}}

<pre>

=={{header|ALGOL W}}==

{{trans|ALGOL 68}}

% calculates the shannon entropy of a string %

% strings are fixed length in algol W and the length is part of the %

write( shannon_entropy( "1223334444", 10 ) )

{{out}}

<pre>

=={{header|APL}}==

ENTROPY←{-+/R×2⍟R←(+⌿⍵∘.=∪⍵)÷⍴⍵}

-+/RATIO×2⍟RATIO

1.846439345

{{out}}

<pre>

=={{header|Arturo}}==

t: #[]

loop s 'c [

]

{{out}}

=={{header|AutoHotkey}}==

Entropy(n)

}

return, e

{{out}}

<pre>1.846440</pre>

=={{header|AWK}}==

{

for (i=1; i<= length($0); i++) {

}

print E/log(2);

{{out|Usage}}

=={{header|BASIC}}==

Works with older (unstructured) Microsoft-style BASIC.

20 S$="1223334444"

30 U$=""

210 E=E-R(I)

220 NEXT I

{{out}}

<pre>1.84643935</pre>

==={{header|QBasic}}===

L = LOG(X) / LOG(2)

END FUNCTION

NEXT I

PRINT E

==={{header|Sinclair ZX81 BASIC}}===

Works with 1k of RAM.

20 LET U$=""

30 FOR I=1 TO LEN X$

200 LET E=E-R(I)

210 NEXT I

{{out}}

<pre>1.8464393</pre>

=={{header|BBC BASIC}}==

{{trans|APL}}

PRINT FNentropy("1223334444")

END

:

DEF FNlogtwo(n)

{{out}}

<pre>1.84643934</pre>

=={{header|BQN}}==

{{out}}

<pre>1.8464393446710154</pre>

=={{header|Burlesque}}==

=={{header|C}}==

#include <stdlib.h>

#include <stdbool.h>

printf("%lf\n",H);

return 0;

Examples:

1223334444

1.846439

$ ./entropy

Rosetta Code is the best site in the world!

=={{header|C sharp|C#}}==

Translation of C++.

using System;

using System.Collections.Generic;

}

}

{{out}}

<pre>The Entropy of 1223334444 is 1.84643934467102</pre>

Without using Hashtables or Dictionaries:

namespace Entropy

{

}

}

=={{header|C++}}==

#include <map>

#include <iostream>

<< " is " << infocontent << std::endl ;

return 0 ;

{{out}}

<pre>(entropy "1223334444")

=={{header|Clojure}}==

(let [len (count s), log-2 (Math/log 2)]

(->> (frequencies s)

(let [rf (/ v len)]

(-> (Math/log rf) (/ log-2) (* rf) Math/abs))))

{{out}}

=={{header|CLU}}==

% this program needs to be merged with 'useful.lib' to get log()

%

po: stream := stream$primary_output()

stream$putl(po, f_form(shannon("1223334444"), 1, 6))

{{out}}

<pre>1.846439</pre>

=={{header|CoffeeScript}}==

freq = (s) ->

result = {}

((frq[f]/n for f of frq).reduce ((e, p) -> e - p * Math.log(p)), 0) * Math.LOG2E

{{out}}

<pre>The entropy of the string '1223334444' is 1.8464393446710157</pre>

Not very Common Lisp-y version:

(let ((table (make-hash-table :test 'equal))

(entropy 0))

(log (/ v (length input-string)) 2))))

table)

More like Common Lisp version:

(declare (type string string))

(let ((table (make-hash-table)))

(- (loop for freq being each hash-value in table

for freq/length = (/ freq length)

=={{header|Crystal}}==

def sum(array : Array(Float64))

res = 0

puts ".[Results]."

puts "Length: " + l.to_s

=={{header|D}}==

double entropy(T)(T[] s)

void main() {

"1223334444"d.dup.entropy.writeln;

{{out}}

<pre>1.84644</pre>

{{Trans|Pascal}}

Just fix Pascal code to run in Delphi.

program Entropytest;

Writeln('Entropy of "', strng, '" is ', entropy(strng): 2: 5, ' bits.');

readln;

=={{header|EchoLisp}}==

(lib 'hash)

;; counter: hash-table[key]++

(for/sum ((s (make-set S))) (hi (hash-ref ht s) n)))

{{out}}

;; by increasing entropy

(H (for/list ((i 1000_000)) (random 1000_000))) → 19.104028424596976

{{trans|C#}}

ELENA 5.0 :

import system'collections;

import system'routines;

console.printLine("The Entropy of ", input, " is ", infoC)

{{out}}

<pre>

{{works with|Erlang/OTP|18}}

<code>:math.log2</code> was added in OTP 18.

def entropy(str) do

leng = String.length(str)

end

{{out}}

=={{header|Emacs Lisp}}==

(let ((freq-table (make-hash-table))

(entropy 0)

(log (/ v length) 2)))))

freq-table)

{{out}}

as shown below (type ctrl-j at the end of the first line

and the output will be placed by emacs on the second line).

=={{header|Erlang}}==

-module( entropy ).

Frequency = How_many / String_length,

{String_length, Acc - (Frequency * math:log(Frequency))}.

{{out}}

=={{header|Euler Math Toolbox}}==

$ v=strtochar(s);

$ m=getmultiplicities(unique(v),v);

$endfunction

>entropy("1223334444")

=={{header|F_Sharp|F#}}==

let ld x = Math.Log x / Math.Log 2.

|> Seq.fold (fun e p -> e - p * ld p) 0.

{{out}}

<pre>1.846439</pre>

=={{header|Factor}}==

prettyprint sequences ;

IN: rosetta-code.entropy

"1223334444" shannon-entropy .

{{out}}

<pre>

=={{header|Forth}}==

create freq 256 cells allot

s" 1223334444" entropy f. \ 1.84643934467102 ok

=={{header|Fortran}}==

Please find the GNU/linux compilation instructions along with sample run among the comments at the start of the FORTRAN 2008 source. This program acquires input from the command line argument, thereby demonstrating the fairly new get_command_argument intrinsic subroutine. The expression of the algorithm is a rough translated of the j solution. Thank you.

!-*- mode: compilation; default-directory: "/tmp/" -*-

!Compilation started at Tue May 21 21:43:12

end program shannonEntropy

=={{header|FreeBASIC}}==

' compile with: fbc -s console

Print : Print "hit any key to end program"

Sleep

{{out}}

<pre>Char count

Sort of hacky, but friendly interactive shell isn't really optimized for mathematic tasks (in fact, it doesn't even have associative arrays).

for arg in $argv

set name count_$arg

echo "$entropy / l(2)" | bc -l

end

{{out}}

<pre>1.84643934467101549345</pre>

=={{header|Go}}==

===Go: Slice version===

import (

}

return entropy

{{out}}

<pre>

===Go: Map version===

import (

const l = float64(len(s))

fmt.Println(math.Log2(l) - hm/l)

{{out}}

<pre>

=={{header|Groovy}}==

-delegate.inject([:]) { map, v -> map[v] = (map[v] ?: 0) + 1; map }.values().inject(0.0) { sum, v ->

def p = (BigDecimal)v / delegate.size()

sum + p * Math.log(p) / Math.log(2)

}

Testing

'1223334444555555555': '1.969811065121',

'122333': '1.459147917061',

println "Checking $s has a shannon entrophy of $expected"

assert sprintf('%.12f', s.shannonEntrophy) == expected

{{out}}

<pre>Checking 1223334444 has a shannon entrophy of 1.846439344671

=={{header|Haskell}}==

main = print $ entropy "1223334444"

entropy = sum . map lg . fq . map genericLength . group . sort

where lg c = -c * logBase 2 c

Or, inlining with an applicative expression (turns out to be fractionally faster):

entropy

main :: IO ()

{{out}}

description.

s := !a | "1223334444"

write(H(s))

every (h := 0.0) -:= P[c := key(P)] * log(P[c],2)

return h

{{out}}

=={{header|J}}==

{{out|Example}}

1.84644

entropy i.256

1

entropy 256$0 1 2 3

So it looks like entropy is roughly the number of bits which would be needed to ''distinguish between'' each item in the argument (for example, with perfect compression). Note that in some contexts this might not be the same thing as information because the choice of the items themselves might matter. But it's good enough in contexts with a fixed set of symbols.

{{trans|REXX}}

{{works with|Java|7+}}

import java.util.Map;

import java.util.HashMap;

return;

}

{{out}}

<pre>

{{works with|ECMAScript 2015}}

Calculate the entropy of a string by determining the frequency of each character, then summing each character's probability multiplied by the log base 2 of that same probability, taking the negative of the sum.

function entropy(str) {

const len = str.length

console.log(entropy('0123')) // 2

console.log(entropy('01234567')) // 3

{{out}}

<pre>1.8464393446710154

3

4</pre> =={{header|JavaScript}}==

const split = s.split('');

const counter = {};

.map(count => count / lengthf * Math.log2(count / lengthf))

.reduce((a, b) => a + b);

{{out}}

<pre>console.log(entropy("1223334444")); // 1.8464393446710154</pre>

The helper function, ''counter'', could be defined as an inner function of ''entropy'', but for the sake of clarity and because it is independently useful,

it is defined separately.

# Output: an object with the strings as keys, the values of which are the corresponding frequencies.

def counter:

(explode | map( [.] | implode ) | counter

| [ .[] | . * log ] | add) as $sum

{{out|Example}}

=={{header|Jsish}}==

From Javascript entry.

function values(obj:object):array {

; entropy('Rosetta Code');

; entropy('password');

{{out}}

{{works with|Julia|0.6}}

@show entropy("1223334444")

{{out}}

=={{header|Kotlin}}==

fun log2(d: Double) = Math.log(d) / Math.log(2.0)

println("------------------------------------ ------------------")

for (sample in samples) println("${sample.padEnd(36)} -> ${"%18.16f".format(shannon(sample))}")

{{out}}

=={{header|Lambdatalk}}==

{def entropy

-> 4.70043971814109

=={{header|Lang5}}==

set execute __A -1 extract nip ; : nip swap drop ; : sum '+ reduce ;

: 2array 2 compress ; : comb "" split ; : lensize length nip ;

dup neg swap log * 2 log / sum ;

=={{header|Liberty BASIC}}==

dim countOfChar( 255) ' all possible one-byte ASCII chars

logBase =log( x) /log( 2)

end function

{{Out}}

<pre> Characters used and the number of occurrences of each

=={{header|Lua}}==

function entropy (X)

end

=={{header|Mathematica}} / {{header|Wolfram Language}}==

1.84644

shE["Rosetta Code"]

=={{header|MATLAB}} / {{header|Octave}}==

This version allows for any input vectors,

including strings, floats, negative integers, etc.

if ischar(d), d=abs(d); end;

[Y,I,J] = unique(d);

p = full(H(H>0))/length(d);

E = -sum(p.*log2(p));

{{out|Usage}}

=={{header|MiniScript}}==

count = {}

for c in s

end function

{{out}}

=={{header|Modula-2}}==

FROM InOut IMPORT WriteString, WriteLn;

FROM RealInOut IMPORT WriteReal;

WriteReal(entropy("1223334444"), 14);

WriteLn;

{{out}}

<pre> 1.8464394E+00</pre>

=={{header|NetRexx}}==

{{trans|REXX}}

options replace format comments java crossref savelog symbols

end report

return

{{out}}

<pre>

=={{header|Nim}}==

proc entropy(s: string): float =

for x in t.values: result -= x/s.len * log2(x/s.len)

=={{header|Objeck}}==

class Entropy {

};

}

Output:

=={{header|OCaml}}==

(* pre-bake & return an inner-loop function to bin & assemble a character frequency map *)

-1.0 *. List.fold_left (fun b x -> b +. calc x) 0.0 relative_probs

'''output:'''

=={{header|Oforth}}==

| freq sz |

s size dup ifZero: [ return ] asFloat ->sz

0.0 freq applyIf( #[ 0 <> ], #[ sz / dup ln * - ] ) Ln2 / ;

{{out}}

=={{header|ooRexx}}==

{{trans|REXX}}

Numeric Digits 16

Parse Arg s

Exit

{{out}}

<pre>1223334444 Entropy 1.846439344671</pre>

=={{header|PARI/GP}}==

<pre>>entropy("1223334444")

%1 = 1.8464393446710154934341977463050452232</pre>

Free Pascal (http://freepascal.org).

PROGRAM entropytest;

Writeln('Entropy of "',strng,'" is ',entropy(strng):2:5, ' bits.');

END.

{{out}}

=={{header|Perl}}==

my %count; $count{$_}++ for @_;

my $entropy = 0;

}

=={{header|Phix}}==

<span style="color: #008080;">with</span> <span style="color: #008080;">javascript_semantics</span>

<span style="color: #008080;">function</span> <span style="color: #000000;">entropy</span><span style="color: #0000FF;">(</span><span style="color: #004080;">sequence</span> <span style="color: #000000;">s</span><span style="color: #0000FF;">)</span>

<span style="color: #0000FF;">?</span><span style="color: #000000;">entropy</span><span style="color: #0000FF;">(</span><span style="color: #008000;">"1223334444"</span><span style="color: #0000FF;">)</span>

{{out}}

<pre>

=={{header|PHP}}==

function shannonEntropy($string) {

number_format(shannonEntropy($string), 6)

);

{{out}}

=={{header|Picat}}==

["1223334444",

"Rosetta Code is the best site in the world!",

Occ.put(E, Occ.get(E,0) + 1)

end,

{{out}}

=={{header|PicoLisp}}==

PicoLisp only supports fixed point arithmetic, but it does have the ability to call libc transcendental functions (for log)

(scl 8)

(load "@lib/math.l")

1. LN2)))

{{Out}}

<pre>

=={{header|PL/I}}==

/*--------------------------------------------------------------------

* 08.08.2014 Walter Pachl translated from REXX version 1

End;

Put Edit('s='''!!s!!''' Entropy=',-e)(Skip,a,f(15,12));

{{out}}

<pre>s='1223334444' Entropy= 1.846439344671</pre>

=={{header|PowerShell}}==

function entropy ($string) {

$n = $string.Length

}

entropy "1223334444"

<b>Output:</b>

<pre>

This solution calculates the run-length encoding of the input string to get the relative frequencies of its characters.

%! shannon_entropy(+String, -Entropy) is det.

,must_be(number, B)

,Sum is A + B.

Example query:

=={{header|PureBasic}}==

NewMap uchar.i() : Define.d e

OpenConsole()

Print("Entropy of ["+#TESTSTR+"] = "+StrD(e,15))

{{out}}

<pre>Entropy of [1223334444] = 1.846439344671015</pre>

=={{header|Python}}==

===Python: Longer version===

import math

print 'Length',l

print 'Entropy:', entropy(h, l)

{{out}}

The <tt>Counter</tt> module is only available in Python >= 2.7.

>>> from collections import Counter

>>>

>>> entropy("1223334444")

1.8464393446710154

===Uses Python 2===

import math

log2=lambda x:math.log(x)/math.log(2)

while True:

=={{header|R}}==

entropy <- function(str) {

vec <- strsplit(str, "")[[1]]

-sum(p_xi * log(p_xi, 2))

}

{{out}}

=={{header|Racket}}==

(require math)

(provide entropy hash-entropy list-entropy digital-entropy)

(check-= (entropy "xggooopppp") 1.8464393446710154 1E-8))

{{out}}

<pre> 1.8464393446710154</pre>

(formerly Perl 6)

{{works with|rakudo|2015-09-09}}

[+] map -> \p { p * -log p }, bag(@a).values »/» +@a;

}

{{out}}

<pre>1.84643934467102</pre>

In case we would like to add this function to Raku's core, here is one way it could be done:

augment class Bag {

method entropy {

}

=={{header|REXX}}==

===version 1===

* 28.02.2013 Walter Pachl

* 12.03.2013 Walter Pachl typo in log corrected. thanx for testing

Numeric Digits (prec)

r=r+0

<!-- these types of comparisons are not part of this Rosetta Code task, and

* Test program to compare Versions 1 and 2

* (the latter tweaked to be acceptable by my (oo)Rexx

Say ' '

Return

{{out}}

<pre>Version 1: 1223334444 Entropy 1.846439344671

The &nbsp; '''LOG2''' &nbsp; subroutine is only included here for functionality, not to document how to calculate &nbsp; LOG₂ &nbsp; using REXX.

numeric digits length( e() ) % 2 - length(.) /*use 1/2 of the decimal digits of E. */

parse arg $; if $='' then $= 1223334444 /*obtain the optional input from the CL*/

ox=izz; ii=ii+is*2**j; end /*while*/; x= x * e** -ii -1; z= 0; _= -1; p= z

do k=1; _= -_ * x; z= z+_/k; if z=p then leave; p= z; end /*k*/

{{out|output|text=&nbsp; when using the default input of: &nbsp; &nbsp; <tt> 1223334444 </tt>}}

<pre>

=={{header|Ring}}==

decimals(8)

entropy = 0

logBase =log( x) /log( 2)

return logBase

Output:

<pre>

=={{header|Ruby}}==

{{works with|Ruby|1.9}}

counts = Hash.new(0.0)

s.each_char { |c| counts[c] += 1 }

end

{{out}}

<pre>

</pre>

One-liner, same performance (or better):

s.each_char.group_by(&:to_s).values.map { |x| x.length / s.length.to_f }.reduce(0) { |e, x| e - x*Math.log2(x) }

=={{header|Run BASIC}}==

source$ = "1223334444"

print " Entropy of '"; source$; "' is "; entropy; " bits."

Characters used and times used of each

'1' 1

=={{header|Rust}}==

let mut histogram = [0u64; 256];

let arg = std::env::args().nth(1).expect("Need a string.");

println!("Entropy of {} is {}.", arg, entropy(arg.as_bytes()));

{{out}}

<pre>$ ./entropy 1223334444

=={{header|Scala}}==

def entropy( v:String ) = { v

// Confirm that "1223334444" has an entropy of about 1.84644

=={{header|scheme}}==

A version capable of calculating multidimensional entropy.

(define (entropy input)

(define (close? a b)

(entropy (list 1 2 2 3 3 3 4 4 4 4))

(entropy (list (list 1 1) (list 1.1 1.1) (list 1.2 1.2) (list 1.3 1.3) (list 1.5 1.5) (list 1.6 1.6)))

{{out}}

=={{header|Scilab}}==

d=strsplit(d);

n=unique(string(d));

word ='1223334444';

E = entropy(word);

{{out}}

=={{header|Seed7}}==

include "float.s7i";

include "math.s7i";

begin

writeln(entropy("1223334444") digits 5);

{{out}}

=={{header|Sidef}}==

var counts = Hash.new;

s.each { |c| counts{c} := 0 ++ };

}

 

{{out}}

<pre>1.846439344671015493434197746305045223237</pre>

=={{header|Standard ML}}==

let

val N = Real.fromInt (String.size input) ;

~ (Vector.foldr (fn (a,s) => if a > 0.0 then term a + s else s) 0.0 freq )

Entropy "1223334444" ;

val it = 1.846439345: real

=={{header|Swift}}==