Remove duplicate elements: Difference between revisions

{{trans|Python}}

 

V unique = Array(Set(items))

 

{{out}}

=={{header|360 Assembly}}==

{{trans|PL/I}}

REMDUP CSECT

USING REMDUP,R15 set base register

PG DC CL92' '

YREGS

{{out}}

<pre> 6 1 5 2 7 22 4 19 8 9 10 11 12</pre>

This routine works on arrays of bytes, and keeps track of which bytes it has seen in a 256-byte array.

 

jmp test

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

nl: db 13,10,'$'

bufdef: db 127,0

 

 

=={{header|ACL2}}==

 

=={{header|Action!}}==

{{libheader|Action! Tool Kit}}

 

PROC PrintArray(INT ARRAY a INT size)

Test(src2,26)

Test(src3,1)

{{out}}

[https://gitlab.com/amarok8bit/action-rosetta-code/-/raw/master/images/Remove_duplicate_elements.png Screenshot from Atari 8-bit computer]

=={{header|Ada}}==

{{works with|GNAT|GPL 2018}}

use Ada.Text_IO;

Put (e'img);

end loop;

 

=={{header|Aime}}==

Using an index:

 

list(1, 2, 3, 1, 2, 3, 4, 1).ucall(i_add, 1, x, 0);

x.i_vcall(o_, 1, " ");

{{out}}

<pre> 1 2 3 4</pre>

Order preserving solution:

 

for (, integer a in list(8, 2, 1, 8, 2, 1, 4, 8)) {

}

}

{{out}}

<pre> 8 2 1 4</pre>

=={{header|ALGOL 68}}==

Using the associative array code from [[Associative_array/Iteration#ALGOL_68]]

# this example uses strings - for other types, the associative #

# array modes AAELEMENT and AAKEY should be modified as required #

# test the duplicate removal #

print( ( remove duplicates( ( "A", "B", "D", "A", "C", "F", "F", "A" ) ), newline ) )

=={{header|Amazing Hopper}}==

#include <hopper.h>

 

{"Final array:\n",y}, println

exit(0)

{{out}}

<pre>

{{works with|GNU APL}}

The primitive monad ∪ means "unique", so:

 

{{works with|APL2}}

{{works with|GNU APL}}

((⍳⍨w)=⍳⍴w)/w

 

=={{header|AppleScript}}==

===Idiomatic===

 

on unique(x)

end repeat

return R

----

===Idiomatic 2 (more robust)===

While it's quite common to see 'is in', 'is not in', 'contains', and 'does not contain' used in the above way for convenience when the scripter knows the code will only be used to check for simple objects, the commands actually compare ''sections of list'' rather than individual elements.

 

{1, 2, 3, {4, 5}} contains {4, 5} --> false

 

If the search term presented is anything other than a list, it's silently coerced to one for the search.

 

 

So for robustness in case the elements sought may already be lists, or records, it's best to wrap them explicitly in list braces.

 

 

on unique(x)

end repeat

return R

 

{{output}}

----

===Functional===

{{trans|Haskell}}

 

 

-- nub :: [a] -> [a]

set my text item delimiters to dlm

return lstParts

{{Out}}

 

----

===AppleScriptObjC===

use framework "Foundation"

 

set aList to {1, 2, 3, "a", "b", "c", 2, 3, 4, "c", {b:"c"}, {"c"}, "c", "d"}

set orderedSet to current application's class "NSOrderedSet"'s orderedSetWithArray:(aList)

 

{{output}}

 

=={{header|Applesoft BASIC}}==

110 L$(0) = "NOW"

120 L$(1) = "IS"

520 L$(N) = ""

530 N = N - 1

 

=={{header|Arturo}}==

 

 

{{out}}

=={{header|AutoHotkey}}==

Built in Sort has an option to remove duplicates

Sort, a, a, NUD`,

 

=={{header|AWK}}==

then produce a string with the keys of b,

which is finally output.

 

 

=={{header|BASIC256}}==

{{trans|True BASIC}}

arraybase 1

max = 10

next i

end

 

 

=={{header|BBC BASIC}}==

{{works with|BBC BASIC for Windows}}

list$() = "Now", "is", "the", "time", "for", "all", "good", "men", \

\ "to", "come", "to", "the", "aid", "of", "the", "party."

IF j%>=i% l$(n%) = i$ : n% += 1

NEXT

{{out}}

<pre>

 

The list contains atoms and also a few non-atomic expressions. The hash table needs atomic keys, so we apply the <code>str</code> function when searching and inserting elements.

 

(A=

& !C

&

Only solution B produces a list with the same order of elements as in the input.

<pre>Solution A: 19 (4.4) 17 11 13 (1.1) (7.) 222 +11 7 5 3 2 0 cats (-100.2) +7

 

=={{header|Brat}}==

 

 

=={{header|C}}==

Since there's no way to know ahead of time how large the new data structure will need to be, we'll return a linked list instead of an array.

 

#include <stdlib.h>

 

printf("%d ", n->x);

puts("");

 

{{out}}

===O(n^2) version, pure arrays===

 

#include <stdlib.h>

#include <stdbool.h>

free(b);

return 0;

 

{{out}}

===Sorting method===

 

#include <stdlib.h>

 

 

return 0;

 

{{out}}

=={{header|C sharp}}==

{{works with |C sharp|C#|2+}}

List<int> unique = new List<int>();

foreach (int n in nums)

if (!unique.Contains(n))

 

{{works with |C sharp|C#|3+}}

 

=={{header|C++}}==

This version uses <tt>std::set</tt>, which requires its element type be comparable using the < operator.

#include <iostream>

using namespace std;

cout << *iter << " ";

cout << endl;

 

This version uses <tt>hash_set</tt>, which is part of the SGI extension to the Standard Template Library. It is not part of the C++ standard library. It requires that its element type have a hash function.

 

{{works with|GCC}}

#include <iostream>

using namespace std;

cout << *iter << " ";

cout << endl;

 

This version uses <tt>unordered_set</tt>, which is part of the TR1, which is likely to be included in the next version of C++. It is not part of the C++ standard library. It requires that its element type have a hash function.

 

{{works with|GCC}}

#include <iostream>

using namespace std;

cout << *iter << " ";

cout << endl;

 

Alternative method working directly on the array:

 

#include <iterator>

#include <algorithm>

std::copy(data, new_end, std::ostream_iterator<int>(std::cout, " ");

std::cout << std::endl;

 

Using sort, unique, and erase on a vector.

{{works with|C++11}}

#include <iostream>

#include <vector>

std::cout << std::endl;

return 0;

 

=={{header|CafeOBJ}}==

 

=={{header|Ceylon}}==

 

=={{header|Clojure}}==

 

(1 3 2 9 8 0)

 

=={{header|CoffeeScript}}==

{{trans|Kotlin}}

set = []

set.push i for i in data when not (i in set)

 

console.log data

{{out}}

<pre>[ 1, 2, 3, 'a', 'b', 'c', 2, 3, 4, 'b', 'c', 'd' ]

To remove duplicates non-destructively:

 

 

Or, to remove duplicates in-place:

 

 

=={{header|Crystal}}==

Copied and modified from the Ruby version.

 

 

<pre>[1, 2, "a", [1, 2, 3]]</pre>

 

=={{header|D}}==

import std.stdio, std.algorithm;

 

.uniq

.writeln;

{{out}}

<pre>[0, 1, 2, 3, 8, 9]</pre>

Using an associative array:

import std.stdio;

 

hash[el] = true;

hash.byKey.writeln;

{{out}}

<pre>[8, 0, 1, 9, 2, 3]</pre>

Like code D#1, but with an array returned:

{

import std.stdio, std.algorithm, std.array;

auto a = [5,4,32,7,6,4,2,6,0,8,6,9].sort.uniq.array;

a.writeln;

{{out}}

<pre>[0, 2, 4, 5, 6, 7, 8, 9, 32]</pre>

Generics were added in Delphi2009.

 

 

{$APPTYPE CONSOLE}

lIntegerList.Free;

end;

 

{{out}}

=={{header|Déjà Vu}}==

 

for item in [ 1 10 1 :hi :hello :hi :hi ]:

@item

{{out}}

<pre>[ 1 :hello 10 :hi ]</pre>

=={{header|E}}==

 

 

=={{header|ECL}}==

inNumbers := DATASET([{1},{2},{3},{4},{1},{1},{7},{8},{9},{9},{0},{0},{3},{3},{3},{3},{3}], {INTEGER Field1});

DEDUP(SORT(inNumbers,Field1));

{{out}}

<pre>0

=={{header|Elena}}==

ELENA 5.0 :

import system'collections;

import system'routines;

console.printLine(unique.MapValues.asEnumerable())

{{out}}

<pre>

Elixir has an <code>Enum.uniq</code> built-in function.

{{works with|Elixir|1.2}}

# Set approach

def uniq1(list), do: MapSet.new(list) |> MapSet.to_list

end)

|> fn{t,_} -> IO.puts "#{inspect fun}:\t#{t/1000000}\t#{inspect result}" end.()

 

{{out}}

=={{header|Erlang}}==

 

UniqueList = gb_sets:to_list(gb_sets:from_list(List)).

% Alternatively the builtin:

Unique_list = lists:usort( List ).

 

=={{header|Euphoria}}==

 

function uniq(sequence s)

constant s = {1, 2, 1, 4, 5, 2, 15, 1, 3, 4}

? s

 

{{out}}

=={{header|F Sharp|F#}}==

The simplest way is to build a set from the given array (this actually works for any enumerable input sequence type, not just arrays):

set [|1;2;3;2;3;4|]

gives:

val it : Set<int> = seq [1; 2; 3; 4]

 

=={{header|Factor}}==

 

V{ 1 2 1 3 2 4 5 } members .

 

 

=={{header|Forth}}==

The input data is assumed to be sorted.

 

\ a3 is the address beyond the data a2 is traversing.

: skip-dups ( a1 a2 a3 -- a1 a2+n )

cell+ r@ skip-dups

cell 0 d+ \ increment write ptr only

 

Here is another implementation of "uniq" that uses a popular parameters and local variables extension words. It is structurally the same as the above implementation, but uses less overt stack manipulation.

 

a n cells+ to e

a dup to r

cell+

repeat

 

To test this code, you can execute:

 

here test - cell / constant ntest

: .test ( n -- ) 0 ?do test i cells + ? loop ;

 

 

{{out}}

Fortran has no built-in hash functions or sorting functions but the code below implements the compare all elements algorithm.

 

 

program remove_dups

end program remove_dups

 

 

Same as above but using 'ANY' to check if the input number already exists in the array of unique elements:

 

program remove_dups

implicit none

end program remove_dups

 

 

{{out}}

 

=={{header|FreeBASIC}}==

 

Sub removeDuplicates(a() As Integer, b() As Integer)

Print

Print "Press any key to quit"

 

{{out}}

=={{header|Frink}}==

The following demonstrates two of the simplest ways of removing duplicates.

b = [1, 5, 2, 6, 6, 2, 2, 1, 9, 8, 6, 5]

 

// Another way, using the "set" datatype and back to an array.

println[toArray[toSet[b]]

 

{{out}} Note that sets are not guaranteed to be printed in any specific order.

 

=={{header|Futurebasic}}==

include "NSLog.incl"

 

 

HandleEvents

{{output}}

<pre>

=={{header|Gambas}}==

'''[https://gambas-playground.proko.eu/?gist=1e2bb524d2278cd88bccdc21a1683296 Click this link to run this code]'''

Dim sString As String[] = Split("Now is the time for all the good men to come to the aid of the good party 1 2 1 3 3 3 2 1 1 2 3 4 33 2 5 4 333 5", " ")

Dim sFix As New String[]

Print sFix.Join(" ")

 

Output:

<pre>

 

=={{header|GAP}}==

a := [ 1, 2, 3, 1, [ 4 ], 5, 5, [4], 6 ];

# [ 1, 2, 3, 1, [ 4 ], 5, 5, [ 4 ], 6 ]

# true

IsList(b);

 

=={{header|Go}}==

===Map solution===

 

import "fmt"

func main() {

fmt.Println(uniq([]int{1, 2, 3, 2, 3, 4})) // prints: [3 4 1 2] (but in a semi-random order)

===Map preserving order===

It takes only small changes to the above code to preserver order. Just store the sequence in the map:

 

import "fmt"

func main() {

fmt.Println(uniq([]int{1, 2, 3, 2, 3, 4})) // prints: [1 2 3 4]

===Float64, removing duplicate NaNs===

In solutions above, you just replace <code>int</code> with another type to use for a list of another type. (See [[Associative_arrays/Creation#Go]] for acceptable types.) Except a weird thing happens with NaNs. They (correctly) don't compare equal, so you have to special case them if you want to remove duplicate NaNs:

 

import (

func main() {

fmt.Println(uniq([]float64{1, 2, math.NaN(), 2, math.NaN(), 4})) // Prints [1 2 NaN 4]

===Any type using reflection===

Go doesn't have templates or generics, but it does have reflection.

 

Note: due to details with how Go handles map keys that contain a NaN somewhere (including within a complex or even within a sub struct field) this version simply omits any NaN containing values it comes across and returns a bool to indicate if that happened. This version is otherwise a translation of the above order preserving map implementation, it does not for example call reflect.DeepEqual so elements with pointers to distinct but equal values will be treated as non-equal.

 

import (

//uniq(a)

//uniq(nil)

{{out}}

<pre>

 

=={{header|Groovy}}==

assert list.size() == 12

println " Original List: ${list}"

def set = list as Set

assert set.size() == 8

 

{{out}}

{{trans|Modula-2}}

{{works with|PC-BASIC|any}}

10 ' Remove Duplicates

20 OPTION BASE 1

260 END

1000 DATA 1, 2, 2, 3, 4, 5, 5

{{out}}

<pre>

===Usage===

 

 

 

===Sorted result using Set===

O(n ln(n)). Requires there is a partial ordering of elements.

 

 

unique :: Ord a => [a] -> [a]

 

===Unsorted result using Set===

O(n ln(n)). Retains original order. Requires there is a partial ordering of elements.

 

 

unique :: Ord a => [a] -> [a]

loop s [] = []

loop s (x : xs) | member x s = loop s xs

 

===Using filter===

O(n^2). Retains original order. Only requires that elements can be compared for equality.

 

 

unique :: Eq a => [a] -> [a]

unique [] = []

 

===Standard Library===

 

Data.List.nub :: Eq a => [a] -> [a]

 

=={{header|HicEst}}==

CHARACTER :: workspace*100

 

READ(Text=workspace, ItemS=individuals) nums ! convert to numeric

 

 

=={{header|Icon}} and {{header|Unicon}}==

This solution preserves the original order of the elements.

every write(!noDups(args))

end

return a

}

A sample run is:

<pre>

=={{header|IDL}}==

 

 

=={{header|Inform 7}}==

 

let result be a list of Ks;

repeat with X running through L:

add X to result, if absent;

 

=={{header|IS-BASIC}}==

110 RANDOMIZE

120 NUMERIC ARR(1 TO 20),TOP

340 NEXT

350 LET REMOVE=UBOUND(A)-ST

 

{{out}}

=={{header|J}}==

The verb<code> ~. </code>removes duplicate items from ''any'' array (numeric, character, or other; vector, matrix, rank-n array). For example:

4 3 2 8 0 1 9 5 7 6

~. 'chthonic eleemosynary paronomasiac'

Or (since J defines an item of an n dimensional array as its n-1 dimensional sub arrays):

 

0 0 0

0 1 2

0 0 0

0 1 2

 

=={{header|Java}}==

{{works with|Java|1.5}}

 

class Test {

System.out.printf("%s ", o);

}

<pre>1 a 2 b 3 c d</pre>

 

{{works with|Java|8}}

 

class Test {

Arrays.stream(data).distinct().forEach((o) -> System.out.printf("%s ", o));

}

<pre>1 2 3 a b c d</pre>

 

=={{header|JavaScript}}==

This uses the <code>===</code> "strict equality" operator, which does no type conversions (<code>4 == "4"</code> is true but <code>4 === "4"</code> is false)

// concat() with no args is a way to clone an array

var u = ary.concat().sort();

var uniq = unique(ary);

for (var i = 0; i < uniq.length; i++)

<pre>1 - number

2 - number

 

Or, extend the prototype for Array:

var u = this.concat().sort();

for (var i = 1; i < u.length; ) {

return u;

}

 

With reduce and arrow functions (ES6):

return this.sort().reduce( (a,e) => e === a[a.length-1] ? a : (a.push(e), a), [] )

 

With sets and spread operator (ES6):

return [... new Set(this)]

 

If, however, the array is homogenous, or we wish to interpret it as such by using JavaScript's Abstract Equality comparison (as in '==', see http://www.ecma-international.org/ecma-262/5.1/#sec-11.9.3) then it proves significantly faster to use a hash table.

For example, in ES 5:

 

var u = [],

dct = {},

return u;

 

Or, to allow for customised definitions of equality and duplication, we can follow the Haskell prelude in defining a '''nub :: [a] -> [a] function''' which is a special case of '''nubBy :: (a -> a -> Bool) -> [a] -> [a]'''

{{trans|Haskell}}

 

'use strict';

 

]

 

 

{{Out}}

If it is acceptable to alter the ordering of elements, then

the builtin (fast) filter, '''unique''', can be used. It can be used for arrays with elements of any JSON type and returns the distinct elements in sorted order.

 

If all but the first occurrence of each element should be deleted, then the following function could be used. It retains the advantage of imposing no restrictions on the types of elements in the array and for that reason is slightly more complex than would otherwise be required.

 

# The hash table functions all expect the hash table to be the input.

| .[0];

 

 

=={{header|Julia}}==

{{works with|Julia|0.6}}

 

 

{{out}}

(Inspired by the J version.)

 

(12 7 12 4 3

6 3 7 4 7

(0 0 0

0 1 2

 

=={{header|Klingphix}}==

 

( "Now" "is" "the" "time" "for" "all" "good" "men" "to" "come" "to" "the" "aid" "of" "the" "party." )

swap print drop nl

 

{{out}}

<pre>("Now", "is", "the", "time", "for", "all", "good", "men", "to", "come", "aid", "of", "party.")

=={{header|Kotlin}}==

{{trans|Java}}

val data = listOf(1, 2, 3, "a", "b", "c", 2, 3, 4, "b", "c", "d")

val set = data.distinct()

println(data)

println(set)

{{out}}

<pre>[1, 2, 3, a, b, c, 2, 3, 4, b, c, d]

 

=={{header|Lang5}}==

 

: remove-duplicates

0 extract swap "2dup in if drop else append then" dip ;

 

Built-in function:

 

=={{header|Lasso}}==

x = array(3,4,8,1,8,1,4,5,6,8,9,6),

y = array

)

with n in #x where #y !>> #n do => { #y->insert(#n) }

 

=={{header|Liberty BASIC}}==

LB has arrays, but here the elements are stored in a space-separated string.

a$ =" 1 $23.19 2 elbow 3 2 Bork 4 3 elbow 2 $23.19 "

print "Original set of elements = ["; a$; "]"

removeDuplicates$ =o$

end function

Original set of elements = [ 1 $23.19 2 elbow 3 2 Bork 4 3 elbow 2 $23.19 ]

=={{header|Logo}}==

{{works with|UCB Logo}}

 

=={{header|Lua}}==

flags = {}

io.write('Unique items are:')

end

end

{{out}}

<pre>Unique items are: 1 2 3 4 bird cat dog</pre>

 

Lua doesn't accept Not-a-Number (NaN) and nil as table key, we can handle them like this (Lua 5.3):

 

function rmdup(t)

end

 

{{out}}

<pre>1 2 3 4 nan bird cat dog</pre>

=={{header|Maple}}==

This is simplest with a list, which is an immutable array.

L := [1, 2, 1, 2, 3, 3, 2, 1, "a", "b", "b", "a", "c", "b"]

 

> [op]({op}(L));

That is idiomatic, but perhaps a bit cryptic; here is a more verbose equivalent:

For an Array, which is mutable, the table solution works well in Maple.

> for u in A do T[u] := 1 end: Array( [indices]( T, 'nolist' ) );

Note that the output (due to the Array() constructor) '''is''' in fact an Array.

 

=={{header|Mathematica}}/{{header|Wolfram Language}}==

Built-in function:

gives back:

Delete duplicates and return sorted elements:

{{out|gives back:}}

 

=={{header|MATLAB}}==

MATLAB has a built-in function, "unique(list)", which performs this task.

<br \>Sample Usage:

 

ans =

 

 

NOTE: The unique function only works for vectors and not for true arrays.

=={{header|Maxima}}==

 

 

=={{header|MAXScript}}==

for i in uniques.count to 1 by -1 do

(

id = findItem uniques uniques[i]

if (id != i) do deleteItem uniques i

 

=={{header|Microsoft Small Basic}}==

{{trans|Modula-2}}

' Set the data.

dataArray[1] = 1

TextWindow.WriteLine(resultArray[resultIndex])

EndFor

 

=={{header|MiniScript}}==

d = {}

for i in items

d.push i

end for

{{out}}

<pre>["b", 1, "d", 3, "a", 4, "c", 2]</pre>

==={{header|mLite}}===

A bit like option 3, except copying each element as encountered, and checking to see if it has already been encountered

| (x eql a, a :: as) = true

| (x, _ :: as) = mem (x, as)

println ` implode ` remdup ` explode "the quick brown fox jumped over the lazy dog";

println ` remdup [1,2,3,4,4,3,2,1, "dog","cat","dog", 1.1, 2.2, 3.3, 1.1];

{{out}}

<pre>the quickbrownfxjmpdvlazyg

=={{header|Modula-2}}==

{{works with|ADW Modula-2|any (Compile with the linker option ''Console Application'').}}

MODULE RemoveDuplicates;

 

END;

END RemoveDuplicates.

 

=={{header|MUMPS}}==

<p>We'll take advantage of the fact that an array can only have one index of any specific value. Sorting into canonical order is a side effect.

;L is the input listing

;S is the separator between entries

FOR SET I=$O(Z(I)) QUIT:I="" SET R=$SELECT($L(R)=0:I,1:R_S_I)

KILL Z,I

Example:<pre>USER>W $$REMDUPE^ROSETTA("1,2,3,4,5,2,5,""HELLO"",42,""WORLD""",",")

1,2,3,4,5,42,"HELLO","WORLD"</pre>

=={{header|Nanoquery}}==

After executing, the list 'unique' will contain only the unique items.

unique = {}

 

unique.append(item)

end

 

=={{header|Neko}}==

Remove duplicate elements, in Neko

*/

var show = function(k, v) $print(v, " ")

$hiter(dedup(original), show)

 

{{out}}

 

=={{header|Nemerle}}==

 

module RemDups

WriteLine(unique);

}

 

=={{header|NetRexx}}==

This sample takes advantage of the NetRexx built-in <tt>Rexx</tt> object's indexed string capability (associative arrays). <tt>Rexx</tt> indexed strings act very like hash tables:

options replace format comments java crossref symbols nobinary

 

 

return wordlist.space

{{out}}

<pre>

 

=={{header|NewLISP}}==

 

=={{header|Nial}}==

cull uniques

=+-+-+-+-+-+-+-+-+

=|1|2|3|a|b|c|4|d|

 

Using strand form

 

=={{header|Nim}}==

 

# Go through the list, and for each element, check the rest of the list to see

# Sort the elements and remove consecutive duplicate elements.

sort(items, system.cmp[int]) # O(n log n)

 

=={{header|Objeck}}==

use Structure;

 

}

}

 

=={{header|Objective-C}}==

 

 

=={{header|OCaml}}==

let unique_set = Hashtbl.create (List.length lst) in

List.iter (fun x -> Hashtbl.replace unique_set x ()) lst;

 

let _ =

 

Another solution (preserves order of first occurrence):

let seen = Hashtbl.create (List.length lst) in

List.filter (fun x -> let tmp = not (Hashtbl.mem seen x) in

 

let _ =

 

Solution reversing list order :

let rec tail_uniq a l =

match l with

| [] -> a

| hd::tl -> tail_uniq (hd::a) (List.filter (fun x -> x != hd) tl) in

 

{{works with|OCaml|4.02+}}

 

=={{header|Octave}}==

input=[1 2 6 4 2 32 5 5 4 3 3 5 1 2 32 4 4];

output=unique(input);

 

=={{header|Oforth}}==

 

=={{header|ooRexx}}==

uniqueData = .set~new~union(data)~makearray~sort

 

do item over uniqueData

say item

{{out}}

<pre>Unique elements are

The following solutions only works if the value type is allowed as a key in a dictionary.

 

 

fun {Nub Xs}

in

 

 

=={{header|PARI/GP}}==

Sort and remove duplicates. Other methods should be implemented as well.

vecsort(v,,8)

 

=={{header|Pascal}}==

 

const

for i := low(rArray) to last do

writeln (rArray[i]);

{{out}}

<pre>% ./RemoveDuplicates

{{libheader|List::MoreUtils}}

(this version even preserves the order of first appearance of each element)

 

 

It is implemented like this:

 

Note: the following two solutions convert elements to strings in the result, so if you give it references they will lose the ability to be dereferenced.

 

Alternately:

 

Alternately:

@seen{qw(1 2 3 a b c 2 3 4 b c d)} = ();

 

=={{header|Phix}}==

Standard builtin. The "STABLE" option preserves order of first occurence. Applies to any data type. The default option for unique(), "SORT", obviously produces sorted output, and there is one other recognised option, "PRESORTED", which can be used either to avoid an unnecessary sort, or to only remove adjacent duplicates.

<span style="color: #008080;">with</span> <span style="color: #008080;">javascript_semantics</span>

<span style="color: #0000FF;">?</span><span style="color: #7060A8;">join</span><span style="color: #0000FF;">(</span><span style="color: #7060A8;">unique</span><span style="color: #0000FF;">(</span><span style="color: #7060A8;">split</span><span style="color: #0000FF;">(</span><span style="color: #008000;">"Now is the time for all good men to come to the aid of the party."</span><span style="color: #0000FF;">),</span><span style="color: #008000;">"STABLE"</span><span style="color: #0000FF;">))</span>

<span style="color: #0000FF;">?</span><span style="color: #7060A8;">unique</span><span style="color: #0000FF;">({</span><span style="color: #000000;">1</span><span style="color: #0000FF;">,</span><span style="color: #000000;">3</span><span style="color: #0000FF;">,</span><span style="color: #000000;">2</span><span style="color: #0000FF;">,</span><span style="color: #000000;">9</span><span style="color: #0000FF;">,</span><span style="color: #000000;">1</span><span style="color: #0000FF;">,</span><span style="color: #000000;">2</span><span style="color: #0000FF;">,</span><span style="color: #000000;">3</span><span style="color: #0000FF;">,</span><span style="color: #000000;">8</span><span style="color: #0000FF;">,</span><span style="color: #000000;">8</span><span style="color: #0000FF;">,</span><span style="color: #000000;">1</span><span style="color: #0000FF;">,</span><span style="color: #000000;">0</span><span style="color: #0000FF;">,</span><span style="color: #000000;">2</span><span style="color: #0000FF;">},</span><span style="color: #008000;">"STABLE"</span><span style="color: #0000FF;">)</span>

<span style="color: #0000FF;">?</span><span style="color: #7060A8;">unique</span><span style="color: #0000FF;">(</span><span style="color: #008000;">"chthonic eleemosynary paronomasiac"</span><span style="color: #0000FF;">,</span><span style="color: #008000;">"STABLE"</span><span style="color: #0000FF;">)</span>

{{out}}

<pre>

 

=={{header|Phixmonti}}==

 

0 tolist var newlist

endfor

 

 

=={{header|PHP}}==

 

=={{header|PicoLisp}}==

There is a built-in function

{{out}}

<pre>-> (2 4 6 1 3 5)</pre>

 

=={{header|PL/I}}==

remdup: Proc options(main);

declare t(20) fixed initial (6, 6, 1, 5, 6, 2, 1, 7,

put skip list ('Unique elements are:');

put edit ((t(k) do k = 1 to i)) (skip,20(f(3)));

{{out}}

<pre>Input:

=={{header|Pop11}}==

 

lvars ar = {1 2 3 2 3 4};

;;; Create a hash table

;;; Collect keys into a list

lvars ls = [];

 

=={{header|PostScript}}==

{{libheader|initlib}}

[10 8 8 98 32 2 4 5 10 ] dup length dict begin aload let* currentdict {pop} map end

 

=={{header|PowerShell}}==

The common array for both approaches:

Using a hash table to remove duplicates:

foreach ($x in $data) {

$h[$x] = 1

}

Sorting and removing duplicates along the way can be done with the <code>Sort-Object</code> cmdlet.

Removing duplicates without sorting can be done with the <code>Select-Object</code> cmdlet.

 

=={{header|Prolog}}==

 

Example usage:

 

 

Because sort/2 is GNU prolog and not ISO here is an ISO compliant version:

member1(X,[_|T]) :- member1(X,T).

 

distinct([],[]).

distinct([H|T],C) :- member1(H,T),!, distinct(T,C).

 

Example usage:

 

=={{header|PureBasic}}==

Task solved with the built in Hash Table which are called Maps in PureBasic

 

For i=0 To 9 ;Mark 10 items at random, causing high risk of duplication items.

ForEach MyElements()

Debug MyElements()

Output may look like this, e.g. duplicated items are automatically removed as they have the same hash value.

Number 0 is marked

=={{header|Python}}==

If all the elements are ''hashable'' (this excludes ''list'', ''dict'', ''set'', and other mutable types), we can use a <tt>set</tt>:

 

or if we want to keep the order of the elements

 

unique = []

helperset = set()

if x not in helperset:

unique.append(x)

 

If all the elements are comparable (i.e. <, >=, etc. operators; this works for ''list'', ''dict'', etc. but not for ''complex'' and many other types, including most user-defined types), we can sort and group:

items = [1, 2, 3, 'a', 'b', 'c', 2, 3, 4, 'b', 'c', 'd']

 

If both of the above fails, we have to use the brute-force method, which is inefficient:

unique = []

for x in items:

if x not in unique:

 

 

another way of removing duplicate elements from a list, while preserving the order would be to use OrderedDict module like so

from collections import OrderedDict as od

 

print(list(od.fromkeys([1, 2, 3, 'a', 'b', 'c', 2, 3, 4, 'b', 'c', 'd']).keys()))

 

See also http://www.peterbe.com/plog/uniqifiers-benchmark and http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/52560

 

One way to do this is to require an equality predicate, or perhaps a key function, in addition to a list to be pruned. For example, using itertools.groupby, at the cost of needing a sort and discarding order:

 

 

print (

nubByKey(k, xs)

{{Out}}

<pre>['apple', 'aPPLE', 'Apple', 'orange', 'ORANGE', 'Orange', 'ampersand']

Or alternatively, using an equality predicate with a recursive function which scales less well, but does preserve order:

 

def nubByEq(eq, xs):

def go(yys, xxs):

print (

nubByEq(eq, xs)

 

A briefer definition of which might be in terms of ''filter'':

def nubBy(p, xs):

def go(xs):

else:

return []

 

{{Out}}

=={{header|Qi}}==

 

(define remove-duplicates

[] -> []

 

(remove-duplicates [a b a a b b c d e])

 

=={{header|Quackery}}==

a nest of strings sorted in descending order.

 

' sortwith nested

]'[ nested join do

drop ] is uniquewith ( [ --> [ )

 

{{out}}

=={{header|R}}==

 

 

=={{header|Racket}}==

 

Using the built-in function

-> (remove-duplicates '(2 1 3 2.0 a 4 5 b 4 3 a 7 1 3 x 2))

'(2 1 3 2.0 a 4 5 b 7 x)

 

Using a hash-table:

(define (unique/hash lst)

(hash-keys (for/hash ([x (in-list lst)]) (values x #t))))

 

Using a set:

(define unique/set (compose1 set->list list->set))

 

A definition that works with arbitrary sequences and allows

specification of an equality predicate.

 

(define (unique seq #:same-test [same? equal?])

(for/fold ([res '()])

([x seq] #:unless (memf (curry same? x) res))

(cons x res)))

<pre>

-> (unique '(2 1 3 2.0 a 4 5 b 4 3 a 7 1 3 x 2))

=={{header|Raku}}==

(formerly Perl 6)

Or just make a set of it.

 

=={{header|Raven}}==

 

items copy unique print

 

5 => "c"

6 => 4

 

=={{header|REBOL}}==

 

{{out}}

 

=={{header|Red}}==

>> unique items

 

=={{header|REXX}}==

 

===version 1, using method 1===

$= '2 3 5 7 11 13 17 19 cats 222 -100.2 +11 1.1 +7 7. 7 5 5 3 2 0 4.4 2' /*item list.*/

say 'original list:' $

say

say 'modified list:' space(z) /*stick a fork in it, we're all done. */

{{out|output|text=&nbsp; when using the default input list:}}

<pre>

 

Sorting of the list elements isn't necessary.

$= '2 3 5 7 11 13 17 19 cats 222 -100.2 +11 1.1 +7 7. 7 5 5 3 2 0 4.4 2' /*item list.*/

say 'original list:' $

say

say 'modified list:' space($) /*stick a fork in it, we're all done. */

{{out|output|text=&nbsp; is identical to the 1^st REXX version.}} <br><br>

 

===version 3, using method 3===

old = '2 3 5 7 11 13 17 19 cats 222 -100.2 +11 1.1 +7 7. 7 5 5 3 2 0 4.4 2'

say 'original list:' old

say

say 'modified list:' space(new) /*stick a fork in it, we're all done. */

{{out|output|text=&nbsp; is identical to the 1^st REXX version.}} <br><br>

 

===version 4, using method 1 (hash table) via REXX stems===

* 26.11.2012 Walter Pachl

* added: show multiple occurrences

If count.w>1 Then

Say right(count.w,3) w

{{out}}

<pre>

 

=={{header|Ring}}==

list = ["Now", "is", "the", "time", "for", "all", "good", "men", "to", "come", "to", "the", "aid", "of", "the", "party."]

for i = 1 to len(list)

next

see nl

Output:

<pre>

=={{header|Ruby}}==

Ruby has an <code>Array#uniq</code> built-in method, which returns a new array by removing duplicate values in self.

 

You can also write your own uniq method.

# used Hash

def uniq1

p ary.uniq1 #=> [1, 2, 3, 7, 6, 5, 4]

p ary.uniq2 #=> [1, 2, 3, 4, 5, 6, 7]

 

A version without implementing class declarations:

pure = Array.new

for i in array

 

unique ["hi","hey","hello","hi","hey","heyo"] # => ["hi", "hey", "hello", "heyo"]

 

=={{header|Run BASIC}}==

 

for i = 1 to len(a$)

print "Dups:";a$

<pre>Dups:2 3 5 7 11 13 17 19 cats 222 -100.2 +11 1.1 +7 7. 7 5 5 3 2 0 4.4 2

No Dups:2 3 5 7 11 13 17 19 cats 222 -100.2 +11 1.1 +7 7. 0 4.4 </pre>

 

=={{header|Rust}}==

use std::hash::Hash;

 

remove_duplicate_elements_sorting(&mut sample_elements);

println!("After removal of duplicates : {:?}", sample_elements);

{{out}}

<pre>

 

=={{header|Scala}}==

val l2 = list.distinct

// l2: scala.List[scala.Int] = List(1,2,3,4,99)

val arr2 = arr.distinct

// arr2: Array[Int] = Array(1, 2, 3, 4, 99)

 

=={{header|Scheme}}==

(cond ((null? l)

'())

(cons (car l) (remove-duplicates (cdr l))))))

 

 

 

Alternative approach:

(do ((a '() (if (member (car l) a) a (cons (car l) a)))

(l l (cdr l)))

((null? l) (reverse a))))

 

 

 

The function 'delete-duplicates' is also available in srfi-1.

 

=={{header|Seed7}}==

const proc: main is func

end for;

writeln(dataSet);

 

{{out}}

 

=={{header|SETL}}==

Output in arbitrary order (convert tuple->set then set->tuple):

return [item: item in {item: item in items}];

 

Preserving source order

seen := {};

return [item: item in items, nps in {#seen} | #(seen with:= item) > nps];

 

 

seen := {};

return [item: item in items, nps in {#seen} | #(seen with:= item) > nps];

Output in arbitrary order (convert tuple->set then set->tuple):

return [item: item in {item: item in items}];

 

=={{header|SETL4}}==

 

set = new('set')

* Add all the elements of the array to the set.

add(set,array)

 

=={{header|Sidef}}==

say ary.uniq.dump;

{{out}}

<pre>[1, 2, 'redundant', [1, 2, 3]]

=={{header|Slate}}==

 

 

 

=={{header|Smalltalk}}==

 

|a|

a := #( 1 1 2 'hello' 'world' #symbol #another 2 'hello' #symbol ).

{{out}}

<pre>Set (1 2 #symbol 'world' #another 'hello' )</pre>

on my system. This can be avoided by using an ordered set (which has also O(n) complexity) as below:

{{works with|Smalltalk/X}}

a := #( 1 1 2 'hello' 'world' #symbol #another 2 'hello' #symbol ).

{{out}}

<pre>OrderedSet(1 2 'hello' 'world' #symbol #another)</pre>

 

=={{header|Sparkling}}==

var t = {};

foreach(arr, function(key, val) {

 

return r;

 

=={{header|SQL}}==

This is not a particularly efficient solution, but it gets the job done.

 

/*

This code is an implementation of "Remove duplicate elements" in SQL ORACLE 19c

select remove_duplicate_elements('3 9 1 10 3 7 6 5 2 7 4 7 4 2 2 2 2 8 2 10 4 9 2 4 9 3 4 3 4 7',' ') as res from dual

;

 

{{out}}

Stata can report duplicate lines, or remove them. See '''[http://www.stata.com/help.cgi?duplicates duplicates]''' in Stata help.

 

. input x y

1 1

3. | 2 1 |

4. | 2 2 |

 

===Mata===

The '''[http://www.stata.com/help.cgi?mf_uniqrows uniqrows]''' function removes duplicate rows from a matrix.

 

: a=1,1\1,1\1,2\2,1\2,2\1,1\2,1\2,1\1,2\2,2

 

3 | 2 1 |

4 | 2 2 |

 

=={{header|Swift}}==

Requires elements to be hashable:

{{works with|Swift|1.2+}}

{{out}}

<pre>[2, 3, 1, 4]</pre>

Another solution (preserves order of first occurrence). Also requires elements to be hashable:

{{works with|Swift|1.2+}}

var seen = Set<T>(minimumCapacity: lst.count)

return lst.filter { x in

}

 

{{out}}

<pre>[3, 2, 1, 4]</pre>

 

Only requires elements to be equatable, but runs in O(n^2):

var seen = [T]()

return lst.filter { x in

}

 

{{out}}

<pre>[3, 2, 1, 4]</pre>

What is called "array" in many other languages is probably better represented by the "list" in Tcl (as in LISP).

With the correct option, the <code>lsort</code> command will remove duplicates.

 

 

{{trans|GW-BASIC}}

{{works with|QBasic}}

OPTION BASE 1

LET max = 10

NEXT i

END

 

=={{header|TSE SAL}}==

//

// Go through the list, and for each element, check the rest of the list to see if it appears again, and discard it if it does.

GotoBufferId( bufferI )

END

{{out}} <pre>

Input

 

=={{header|TUSCRIPT}}==

$$ MODE TUSCRIPT

list_old="b'A'A'5'1'2'3'2'3'4"

PRINT list_old

PRINT list_new

{{out}} (sorted)

<pre>

</pre>

or

$$ MODE TUSCRIPT

list_old="b'A'A'5'1'2'3'2'3'4"

PRINT list_old

PRINT list_new

{{out}}

<pre>

=={{header|UnixPipes}}==

Assuming a sequence is represented by lines in a file.

bash$ printf '6\n2\n3\n6\n4\n2\n'

6

4

6

 

or

 

bash$ printf '6\n2\n3\n6\n4\n2\n'|sort -nu

2

4

6

 

=={{header|Ursala}}==

to decide whether that's a relevant criterion for equivalence or else

specify a better one.

 

{{out}}

<pre>'mspi'</pre>

Input list (variant : Long, Double, Boolean and Strings) :

Array(1.23456789101112E+16, True, False, True, "Alpha", 1, 235, 4, 1.25, 1.25, "Beta", 1.23456789101112E+16, "Delta", "Alpha", "Charlie", 1, 2, "Foxtrot", "Foxtrot", "Alpha", 235)

Option Explicit

 

ReDim Preserve Temp(cpt - 1)

Remove_Duplicate = Temp

{{out}}

<pre> 1.23456789101112E+16

=={{header|VBScript}}==

Hash Table Approach

Function remove_duplicates(list)

arr = Split(list,",")

 

WScript.Echo remove_duplicates("a,a,b,b,c,d,e,d,f,f,f,g,h")

 

{{Out}}

 

The input "array" is an edit buffer where each line is one element.

 

=={{header|Vim Script}}==

 

 

=={{header|Visual FoxPro}}==

LOCAL i As Integer, n As Integer, lcOut As String

CLOSE DATABASES ALL

RETURN INT((tnHigh - tnLow + 1)*RAND() + tnLow)

ENDFUNC

{{out}}

<pre>

 

=={{header|Vlang}}==

 

fn main() {

new_list.sort()

return new_list

 

{{out}}

 

=={{header|Wart}}==

let exists (table)

collect+each x l

unless exists.x

yield x

 

{{out}}

 

=={{header|Wortel}}==

 

=={{header|Wren}}==

{{libheader|Wren-sort}}

 

// Using a map - order of distinct items is undefined.

System.print("Method 1: %(removeDuplicates1.call(a.toList))") // copy original each time

System.print("Method 2: %(removeDuplicates2.call(a.toList))")

 

{{out}}

 

=={{header|XBS}}==

set NewArray = [];

foreach(k,v as Array){

log(Arr);

RemoveDuplicates(Arr);

{{out}}

<pre>

 

=={{header|XPL0}}==

string 0; \use zero-terminated strings (not MSb terminated)

 

];

 

 

{{out}}

 

=={{header|Yabasic}}==

 

do

loop

 

 

=={{header|zkl}}==

Using built ins:

L(1,3,2,9,8,"8",0)

zkl: "1,3,2,9,1,2,3,8,8,1,0,2".unique()

Where listUnique is brute force:

 

=={{header|Zoea}}==

program: remove_duplicate_elements

input: [1,2,1,3,2,4,1]

output: [1,2,3,4]

 

=={{header|Zoea Visual}}==