Unit testing

From Rosetta Code
Unit testing is a draft programming task. It is not yet considered ready to be promoted as a complete task, for reasons that should be found in its talk page.

Demonstrate any support for Unit testing built into the language implementation.

See https://en.wikipedia.org/wiki/Unit_testing and Test a function for more information.

Clearly state whether this support is internal, or external to the interpreter or compiler used.

FreeBASIC

fbcunit - FreeBASIC Compiler Unit Testing Component
Copyright (C) 2017-2020 Jeffery R. Marshall (coder[at]execulink[dot]com)

Unit testing component for fbc compiler. Provides macros and common code for unit testing fbc compiled sources.

See https://github.com/jayrm/fbcunit.

Go

Go has good support for unit testing provided by the "testing" package in its standard library in conjunction with the 'go test' command. This is separate from the Go compiler itself which is invoked with the 'go build' (or 'go run') command.

For a description of the facilities provided, interested readers should consult the documentation for the "testing" package.

For a simple example of the testing process, check out the Test_a_function#Go task.

Insitux

Alongside assertions and mocking available in pure Insitux (see Test a function), the Node.js REPL has the ability to generate a code coverage report of unvisited lines and columns, useful for writing thorough unit tests. Here is a basic demonstration:

(function subject x y z
  (return-unless (num? x) x)
  (return-unless (num? y) y)
  (if x y z))


(assert (= (subject :a 2 3) :a))
(assert (= (subject 1 :b 3) :b))
(assert (= (subject 1 2 3)   2))
Output:

Invoked from the terminal with npx ix . -unv.

unvisited.txt generated: 1 unvisited (97% coverage)
true

unvisited.txt:

entry.ix	4	11

We're being told here that line 4, column 11 has an expression which was not visited at runtime. Through investigation we see that it's impossible for z to be returned, as x must always be a number - an inherently truthy value.

Java

Sjharper79 (talk)

JUnit 5

Java has a testing library called JUnit. It's current version is 5. Using JUnit, one can develop unit tests for any functionality. I wrote an implementation for the Distance and Bearing page and use JUnit testing throughout. Below is one of the tests I created to test the ability of the program to properly list the 20 closest airports.

Basically it sets up a List with the 20 airport names expected in the results. Next it executes the method that caculates said result. It compares the result received with the result expected. AssertTrue(correctResults.contains(airport)) tests whether each aiport received in the results is in the list of expected airports. If it is not, the test fails.

package distanceAndBearing;

import static org.junit.jupiter.api.Assertions.assertTrue;

import java.util.ArrayList;
import java.util.List;
import java.util.stream.Collectors;

import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.Test;

@DisplayName("Distance and Bearing")
public class DistanceAndBearingTest {

	private Airport ap;
    @Test
	@DisplayName("Should list top 20 airports when searched")
	void shouldListTop20AirportsWhenSearched() {
		ArrayList<String> correctResults = new ArrayList<>();
		correctResults.add("Koksijde Air Base");
		correctResults.add("Ostend-Bruges International Airport");
		correctResults.add("Kent International Airport");
		correctResults.add("Calais-Dunkerque Airport");
		correctResults.add("Westkapelle heliport");
		correctResults.add("Lympne Airport");
		correctResults.add("Ursel Air Base");
		correctResults.add("Southend Airport");
		correctResults.add("Merville-Calonne Airport");
		correctResults.add("Wevelgem Airport");
		correctResults.add("Midden-Zeeland Airport");
		correctResults.add("Lydd Airport");
		correctResults.add("RAF Wattisham");
		correctResults.add("Beccles Airport");
		correctResults.add("Lille/Marcq-en-Baroeul Airport");
		correctResults.add("Lashenden (Headcorn) Airfield");
		correctResults.add("Le Touquet-Côte d'Opale Airport");
		correctResults.add("Rochester Airport");
		correctResults.add("Lille-Lesquin Airport");
		correctResults.add("Thurrock Airfield");
	
		List<Airport> results;
		DistanceAndBearing dandb = new DistanceAndBearing();
		boolean success = dandb.readFile("airports.txt");
		results = dandb.findClosestAirports(this.ap.getLat(), this.ap.getLon());
		List<String> airports = results.stream().map(ap -> ap.getAirportName()).collect(Collectors.toList());
		
		airports.stream().forEach(airport ->{
			assertTrue(correctResults.contains(airport));
		});
		
	}

Jsish

Jsish supports an internal -u command line option to run unit tests, as part of the base implementation. These tests can include trial inputs and expected outputs in special comment blocks within a source file. Other command line options determine the level of verbosity and logging used when running tests. Being inside comment sections, unit tests do not affect the normal operation of a script (besides the negligible time taken for the interpreter to read in and skip over the comment sections). When in -u test-mode, source lines starting and ending with semi-colons ; are echoed using a special output format that also evaluates the enclosed expression and captures the result.

/* A+B in Jsish */
var line = console.input();
var nums = line.match(/^\s*([+-]?[0-9]+)\s+([+-]?[0-9]+)\s*/);
if (nums) {
    var A = Number(nums[1]);
    var B = Number(nums[2]);
    if (A <= 1000 && A >= -1000 && B <= 1000 && B >= -1000) {
        printf("%d\n", A + B);
    } else {
        puts("error: A and B both need to be in range -1000 thru 1000 inclusive");
    }
} else {
    puts("error: A+B requires two numbers separated by space");
}

/*
=!INPUTSTART!=
a b
1234 123
-1000 +1000
123 -234
=!INPUTEND!=
*/
/*
=!EXPECTSTART!=
error: A+B requires two numbers separated by space
error: A and B both need to be in range -1000 thru 1000 inclusive
0
-111
=!EXEPECTEND!=
*/
Output:
prompt$ jsish -u A+B.jsi
[PASS] A+B.jsi

prompt$ jsish A+B.jsi
123 -234
-111

jq

jq was developed with unit testing, and the jq source includes a suite of unit tests that utilize the --run-tests option of the jq command. This option expects unit tests to be presented in the form of "paragraphs" that may be specified in one or more files, which by convention have ".test" as the filename suffix.

Here is an example of such a .test file with explanatory comments, followed by a transcript showing how it is used and the output that is produced.

Unit Test File

# unit-tests.test
# Basic tests are groups of three lines: program, input, expected output
# Blank lines and lines starting with # are ignored
# For example:

true
null
true

# Failure tests begin with a %%FAIL line, followed by the program and the expected error message.
# %%FAIL tests have no input line.
%%FAIL
{(0):1}
jq: error: Cannot use number (0) as object key at <top-level>, line 1:

# Testing the testing apparatus:
length
"This should generate an error"
0

Transcript

$ jq --run-tests unit-tests.test
Testing 'true' at line number 5
Testing '{(0):1}' at line number 12
Testing 'length' at line number 16
*** Expected 0, but got 29 for test at line number 18: length
2 of 3 tests passed (0 malformed)

Julia

Julia was developed with unit testing, and testing with macros such as @test are part of the language specification. See https://docs.julialang.org/en/v1/stdlib/Test/index.html.

Nim

Nim provides two mechanisms for unit testing.

The first one is the module unittest which allows to define tests and test suites. The documentation for this the module can be found here: https://nim-lang.org/docs/unittest.html

The second one is a more advanced tool named testament which is used for the development of Nim itself. It offers process isolation for the tests, can generate statistics about test cases, supports multiple targets (C, C++, ObjectiveC, JavaScript, etc), can do simulated Dry-Runs, has logging, can generate HTML reports, can skip tests from a file, and more.

Its documentation can be found here: https://nim-lang.github.io/Nim/testament.html

Phix

Phix 0.8.2+ implements unit_testing via the builtins/unit_tests.e autoinclude, as shown in Test_a_function#Phix.

Racket

One could say that Racket has no unit testing built into the language. The most popular unit-testing framework, rackunit, is just a library. However, the default Racket installation does include rackunit out of the box.

The Racket command line tool, raco, has a command raco test. The command will look for a submodule test and run it. Users can choose any testing framework they want, with rackunit being the most popular one as mentioned above. Again, raco test is not really built into the language either.

Raku

(formerly Perl 6) Raku does not really have a mechanism built into the compiler to do unit testing, (It does have design-by-contract capabilities built in, but that is more for run-time, not really for unit testing.) Raku, and Perl in general does have unit testing built into the community. The Test-Anything-Protocol was invented and developed specifically to do unit testing on the original version of Perl, is still heavily used for modern versions, and has become a major standard for unit testing in many different languages other than Perl.

Testing is such a basic value in Raku, that a suite of unit tests is the language specification. As decreed by Larry Wall, the original author of Perl and the lead designer of Raku, anything that can pass "roast", the official Raku test suite, or at least majority portions of it, can call itself Raku.

Unit testing tools are not built in to the base compiler. Basic tools are distributed with the compiler as loadable modules. Many more complex toolkits and test harnesses are available through the Raku ecosystem. In general, it is uncommon to include testing code in with run-time code; not unheard-of and not forbidden, but uncommon. Instead Raku modules by convention and community pressure tend to have a test suite that is automatically run when the module is installed. Again, there is no rule that a module that is released for public consumption must have a test suite, but the community tends to favour modules that have them and discourage/avoid those without.

REXX

The REXX language does not really have a mechanism built into the interpreter to do unit testing.   (REXX is an interpretive language,   but some REXXes also have a compiler available.)   As it's an interpretive language,   it's fairly easy to create programs that stress the language's specifications.

One helpful feature of the   REXX   language is it's quite stable (as per additions/specifications/enhancements),   and has been around since it was developed at IBM in the early 1980's.

Also, most REXXes authors/creators/programmers/development teams have accumulated a suite of test programs that exercise a major part of the aspects and nuances of the REXX language and can usually find defects in the interpreter.   Another method is to run/execute   A/B   type of testing,   that is,   run/execute REXX programs with two (or more) different REXX interpreters and compare results.   Since REXX does arithmetic using decimal numbers (including "floating point"), errors (even minute ones) are easy to identify.   Indeed, some (if not many) of the programs entered here on Rosetta Code have provided some REXX developers a very good starting base for a comprehensive test suite.

Ruby

Minitest and test::Unit are the testing libraries which ship with Ruby.

Scala

The main Scala testing frameworks ( ScalaCheck, ScalaTest, and specs2) provide an implementation of the common test interface and only need to be added to the classpath to work with sbt.

Smalltalk

All Smalltalk dialects have a testing framework included, which is typically based on the original SUnit test framework. Tests are subclasses of TestCase, and define test methods. Execution and visual presentation can be within the class browser or a UnitTest runner (which are part of the system), but tests can also be executed "headless" (i.e. without UI).

Obviously, this is defined in the class library, because in Smalltalk everything comes from the class library (there is no distinction between "internal" and "external")

Typical test cases look like:

test06b_MiscMath_errors
    self should:[ 0 ln ] raise:DomainError.
    self should:[ 0 log2 ] raise:DomainError.
    self should:[ 0 log10 ] raise:DomainError.
    self should:[ -1 ln ] raise:DomainError.
    self should:[ -1 log2 ] raise:DomainError.
    self should:[ -1 log10 ] raise:DomainError.

test_integerSqrt
    |n sqrt f|

    n := 1000 factorial.
    sqrt := n integerSqrt.
    self assert:(sqrt squared <= n).
    self assert:((sqrt+1) squared > n).
    n := sqrt := nil.

    self should:[-10 integerSqrt] raise:DomainError.

When executed headless, a result object is returned from a run, which can be printed or further analyzed.
For example:

outcome := IntegerTest run.
outcome printCR.
(JSONPrinter encode:outcome) printCR
Output:
97 run, 97 passed, 0 skipped, 0 failed, 0 errors

{
  "name":"RegressionTests::IntegerTest",
  "timestamp":"2020-12-09T04:38:58.185",
  "failures": []
}, "errors":[], "passed":[
  {
    "testCase":{
        "testSelector":"testAnyBit"
    },
    "result":"pass",
    "properties":{
      "endTime":"2020-12-09T04:38:58.185",
      "startTime":"2020-12-09T04:38:58.185",
      "collectedOutput":""
    }
... 
hundreds of more lines

Wren

As a language chiefly intended for embedding, Wren's standard library is quite small and there is no built-in support for unit testing as such.

However, there is an external module called Wren-test which can be used for this purpose and whose documentation should be consulted for the features it contains.

zkl

zkl has built-in unit testing as shown in Test_a_function#zkl. It can test source code ( UnitTester.testSrc("x:=1") ) or compiled code ( UnitTester.testRun(fcn{ x:=1 } ).