Remove duplicate elements: Difference between revisions

Content added Content deleted

Inline

Revision as of 11:54, 21 August 2022

Given an Array, derive a sequence of elements in which all duplicates are removed.

There are basically three approaches seen here:

Put the elements into a hash table which does not allow duplicates. The complexity is O(n) on average, and O(n²) worst case. This approach requires a hash function for your type (which is compatible with equality), either built-in to your language, or provided by the user.
Sort the elements and remove consecutive duplicate elements. The complexity of the best sorting algorithms is O(n log n). This approach requires that your type be "comparable", i.e., have an ordering. Putting the elements into a self-balancing binary search tree is a special case of sorting.
Go through the list, and for each element, check the rest of the list to see if it appears again, and discard it if it does. The complexity is O(n²). The up-shot is that this always works on any type (provided that you can test for equality).

11l

Translation of: Python

<lang 11l>V items = [‘1’, ‘2’, ‘3’, ‘a’, ‘b’, ‘c’, ‘2’, ‘3’, ‘4’, ‘b’, ‘c’, ‘d’] V unique = Array(Set(items)) print(unique)</lang>

Output:

[1, 2, 3, 4, a, b, c, d]

360 Assembly

Translation of: PL/I

<lang 360asm>* Remove duplicate elements - 18/10/2015 REMDUP CSECT

        USING  REMDUP,R15         set base register
        SR     R6,R6              i=0
        LA     R8,1               k=1

LOOPK C R8,N do k=1 to n

        BH     ELOOPK
        LR     R1,R8              k
        SLA    R1,2
        L      R9,T-4(R1)         e=t(k)
        LR     R7,R8              k
        BCTR   R7,0               j=k-1

LOOPJ C R7,=F'1' do j=k-1 to 1 by -1

        BL     ELOOPJ
        LR     R1,R7              j
        SLA    R1,2
        L      R2,T-4(R1)         t(j)
        CR     R9,R2              if e=t(j) then goto iter
        BE     ITER
        BCTR   R7,0               j=j-1
        B      LOOPJ

ELOOPJ LA R6,1(R6) i=i+1

        LR     R1,R6              i
        SLA    R1,2
        ST     R9,T-4(R1)         t(i)=e

ITER LA R8,1(R8) k=k+1

        B      LOOPK

ELOOPK LA R10,PG pgi=@pg

        LA     R8,1               k=1

LOOP CR R8,R6 do k=1 to i

        BH     ELOOP
        LR     R1,R8              k
        SLA    R1,2
        L      R2,T-4(R1)         t(k)
        XDECO  R2,PG+80           edit t(k)
        MVC    0(3,R10),PG+89     output t(k) on 3 char
        LA     R10,3(R10)         pgi=pgi+3
        LA     R8,1(R8)           k=k+1
        B      LOOP

ELOOP XPRNT PG,80 print buffer

        XR     R15,R15            set return code
        BR     R14                return to caller

T DC F'6',F'6',F'1',F'5',F'6',F'2',F'1',F'7',F'5',F'22'

        DC     F'4',F'19',F'1',F'1',F'6',F'8',F'9',F'10',F'11',F'12'

N DC A((N-T)/4) number of T items PG DC CL92' '

        YREGS
        END    REMDUP</lang>

Output:

  6  1  5  2  7 22  4 19  8  9 10 11 12

8080 Assembly

This routine works on arrays of bytes, and keeps track of which bytes it has seen in a 256-byte array.

<lang 8080asm> org 100h jmp test ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ;; Given an array of bytes starting at HL with length BC, ;; remove all duplicates in the array. The new end of the array ;; is returned in HL. A page of memory (256 bytes) is required ;; to mark which bytes have been seen. uniqpage: equ 3 ; Page to use - a compile-time constant. ; This would need to be set to a page that ; the rest of the program doesn't need. uniq: xra a ; Zero out the page lxi d,uniqpage * 256 uniqzero: stax d ; Zero out a byte inr e ; And do the next byte jnz uniqzero mov d,h ; Keep a second pointer to the array in DE mov e,l uniqpos: ldax d ; Read from high pointer mov m,a ; Write to low pointer inx d ; Increment the high pointer push h ; Keep low pointer around mvi h,uniqpage mov l,a ; Have we seen this byte yet? cmp m mov m,a ; No matter what, we've certainly seen it now pop h ; Bring back the low pointer jz uniqno ; If we already had it, don't increment low ptr inx h ; IF we didn't, do increment it uniqno: dcx b ; One fewer byte left mov a,b ; If there are zero bytes left, ora c rz ; Then return to caller jmp uniqpos ; Otherwise, do the next byte ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ;; Testing code: read a string from the CP/M console, run ;; uniq, then print the output. test: lxi d,bufdef ; Read a string mvi c,10 call 5 lxi d,nl ; Output on new line mvi c,9 call 5 ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; lda bufdef+1 ; Length of input string mov c,a ; Extend to 16-bit (since uniq supports mvi b,0 ; long arrays) lxi h,buf ; Location of input string call uniq ; Only the unique bytes ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; mvi m,'$' ; Mark the (string) end with '$' lxi d,buf ; Print the string, which now has had mvi c,9 ; all duplicates removed. jmp 5 nl: db 13,10,'$' bufdef: db 127,0 buf:</lang>

ACL2

<lang lisp>(remove-duplicates xs)</lang>

Action!

Library: Action! Tool Kit

<lang Action!>INCLUDE "D2:SORT.ACT" ;from the Action! Tool Kit

PROC PrintArray(INT ARRAY a INT size)

 INT i

 Put('[)
 FOR i=0 TO size-1
 DO
   IF i>0 THEN Put(' ) FI
   PrintI(a(i))
 OD
 Put(']) PutE()

RETURN

PROC RemoveDuplicates(INT ARRAY src INT srcLen

 INT ARRAY dst INT POINTER dstLen)
 INT i
 CHAR curr,prev

 IF srcLen=0 THEN
   dstLen^=0
   RETURN
 FI

 SortI(src,srcLen,0)
 dst(0)=src(0)
 dstLen^=1 prev=src(0)
 FOR i=1 TO srcLen-1
 DO
   curr=src(i)
   IF curr#prev THEN
     dst(dstLen^)=curr
     dstLen^==+1
   FI
   prev=curr
 OD

RETURN

PROC Test(INT ARRAY src INT srcLen)

 INT ARRAY dst(100)
 INT dstLen

 PrintE("Input array:")
 PrintArray(src,srcLen)
 RemoveDuplicates(src,srcLen,dst,@dstLen)
 PrintE("Unique items:")
 PrintArray(dst,dstLen)
 PutE()

RETURN

PROC Main()

 INT ARRAY src1(9)=[1 3 65534 0 12 1 65534 52 3]
 INT ARRAY src2(26)=[3 2 1 3 2 5 2 1 6 3 4 2 5 3 1 5 3 5 2 1 3 7 4 5 7 6]
 INT ARRAY src3(1)=[6502]
 
 Put(125) PutE() ;clear screen
 Test(src1,9)
 Test(src2,26)
 Test(src3,1)

RETURN</lang>

Output:

Screenshot from Atari 8-bit computer

Input array:
[1 3 -2 0 12 1 -2 52 3]
Unique items:
[-2 0 1 3 12 52]

Input array:
[3 2 1 3 2 5 2 1 6 3 4 2 5 3 1 5 3 5 2
1 3 7 4 5 7 6]
Unique items:
[1 2 3 4 5 6 7]

Input array:
[6502]
Unique items:
[6502]

Ada

Works with: GNAT version GPL 2018

<lang ada>with Ada.Containers.Ordered_Sets, Ada.Text_IO; use Ada.Text_IO;

procedure Duplicate is package Int_Sets is new Ada.Containers.Ordered_Sets (Integer); Nums : constant array (Natural range <>) of Integer := (1,2,3,4,5,5,6,7,1); Unique : Int_Sets.Set; begin for n of Nums loop Unique.Include (n); end loop; for e of Unique loop Put (e'img); end loop; end Duplicate;</lang>

Aime

Using an index: <lang aime>index x;

list(1, 2, 3, 1, 2, 3, 4, 1).ucall(i_add, 1, x, 0); x.i_vcall(o_, 1, " "); o_newline();</lang>

Output:

 1 2 3 4

Order preserving solution: <lang aime>index x;

for (, integer a in list(8, 2, 1, 8, 2, 1, 4, 8)) {

   if ((x[a] += 1) == 1) {
       o_(" ", a);
   }

} o_newline();</lang>

Output:

 8 2 1 4

ALGOL 68

Using the associative array code from Associative_array/Iteration#ALGOL_68 <lang algol68># use the associative array in the Associate array/iteration task #

this example uses strings - for other types, the associative #
array modes AAELEMENT and AAKEY should be modified as required #

PR read "aArray.a68" PR

returns the unique elements of list #

PROC remove duplicates = ( []STRING list )[]STRING:

    BEGIN
       REF AARRAY elements := INIT LOC AARRAY;
       INT        count    := 0;
       FOR pos FROM LWB list TO UPB list DO
           IF NOT ( elements CONTAINSKEY list[ pos ] ) THEN
               # first occurance of this element                    #
               elements // list[ pos ] := "";
               count +:= 1
           FI
       OD;
       # construct an array of the unique elements from the         #
       # associative array - the new list will not necessarily be   #
       # in the original order                                      #
       [ count ]STRING result;
       REF AAELEMENT e := FIRST elements;
       FOR pos WHILE e ISNT nil element DO
           result[ pos ] := key OF e;
           e := NEXT elements
       OD;
       result
    END; # remove duplicates #

test the duplicate removal #

print( ( remove duplicates( ( "A", "B", "D", "A", "C", "F", "F", "A" ) ), newline ) ) </lang>

Amazing Hopper

include <hopper.h>

main:

  x=-1
  {30} rand array (x), mulby(10), ceil, mov(x)
  {"Original Array:\n",x}, println
  {x}array(SORT), 
  {x}sets(UNIQUE), mov(x)
  {"Final array:\n",x},  println
  
  y={}
  {"C","Go","Go","C","Cobol","java","Ada"} pushall(y)
  {"java","algol-68","C","java","fortran"} pushall(y)
  {"\nOriginal Array:\n",y}, println
  {y}array(SORT), 
  {y}sets(UNIQUE), mov(y)
  {"Final array:\n",y},  println

exit(0) </lang>

Output:

Original Array:
3 9 1 10 3 7 6 5 2 7 4 7 4 2 2 2 2 8 2 10 4 9 2 4 9 3 4 3 4 7
Final array:
1 2 3 4 5 6 7 8 9 10

Original Array:
C Go Go C Cobol java Ada java algol-68 C java fortran
Final array:
Ada C Cobol Go algol-68 fortran java

APL

Works with: Dyalog APL

Works with: GNU APL

The primitive monad ∪ means "unique", so: <lang apl>∪ 1 2 3 1 2 3 4 1 1 2 3 4</lang>

Works with: APL2

Works with: GNU APL

<lang apl>w←1 2 3 1 2 3 4 1

    ((⍳⍨w)=⍳⍴w)/w

1 2 3 4</lang>

AppleScript

Idiomatic

<lang applescript>unique({1, 2, 3, "a", "b", "c", 2, 3, 4, "b", "c", "d"})

on unique(x)

   set R to {}
   repeat with i in x
       if i is not in R then set end of R to i's contents
   end repeat
   return R

end unique</lang>

Idiomatic 2 (more robust)

While it's quite common to see 'is in', 'is not in', 'contains', and 'does not contain' used in the above way for convenience when the scripter knows the code will only be used to check for simple objects, the commands actually compare sections of list rather than individual elements.

<lang applescript>{1, 2, 3, {4, 5}} contains {2, 3} --> true {1, 2, 3, {4, 5}} contains {4, 5} --> false {1, 2, 3, {4, 5}} contains Template:4, 5 --> true</lang>

If the search term presented is anything other than a list, it's silently coerced to one for the search.

So for robustness in case the elements sought may already be lists, or records, it's best to wrap them explicitly in list braces.

<lang applescript>unique({1, 2, 3, "a", "b", "c", 2, 3, 4, "c", {b:"c"}, {"c"}, "c", "d"})

on unique(x)

   set R to {}
   repeat with i in x
       set i to i's contents
       if {i} is not in R then set end of R to i
   end repeat
   return R

end unique</lang>

Output:

Functional

Or, more generally, we can allow for customised definitions of equality and duplication, by following the Haskell prelude in defining a nub :: [a] -> [a] function which is a special case of nubBy :: (a -> a -> Bool) -> [a] -> [a]

In the following example, equality is defined as case-insensitive for strings. We would obtain a different list of unique strings by adjusting the Eq :: a -> a -> Bool function to make it consider case.

Translation of: JavaScript

Translation of: Haskell

<lang AppleScript>-- CASE-INSENSITIVE UNIQUE ELEMENTS ------------------------------------------

-- nub :: [a] -> [a] on nub(xs)

   -- Eq :: a -> a -> Bool
   script Eq
       on |λ|(x, y)
           ignoring case
               x = y
           end ignoring
       end |λ|
   end script
   
   nubBy(Eq, xs)

end nub

-- TEST ---------------------------------------------------------------------- on run

   {intercalate(space, ¬
       nub(splitOn(space, "4 3 2 8 0 1 9 5 1 7 6 3 9 9 4 2 1 5 3 2"))), ¬
       intercalate("", ¬
           nub(characters of "abcabc ABCABC"))}
   
   --> {"4 3 2 8 0 1 9 5 7 6", "abc "}

end run

-- GENERIC FUNCTIONS ---------------------------------------------------------

-- filter :: (a -> Bool) -> [a] -> [a] on filter(f, xs)

   tell mReturn(f)
       set lst to {}
       set lng to length of xs
       repeat with i from 1 to lng
           set v to item i of xs
           if |λ|(v, i, xs) then set end of lst to v
       end repeat
       return lst
   end tell

end filter

-- intercalate :: Text -> [Text] -> Text on intercalate(strText, lstText)

   set {dlm, my text item delimiters} to {my text item delimiters, strText}
   set strJoined to lstText as text
   set my text item delimiters to dlm
   return strJoined

end intercalate

-- Lift 2nd class handler function into 1st class script wrapper -- mReturn :: Handler -> Script on mReturn(f)

   if class of f is script then
       f
   else
       script
           property |λ| : f
       end script
   end if

end mReturn

-- nubBy :: (a -> a -> Bool) -> [a] -> [a] on nubBy(fnEq, xxs)

   set lng to length of xxs
   if lng > 1 then
       set x to item 1 of xxs
       set xs to items 2 thru -1 of xxs
       set p to mReturn(fnEq)
       
       -- notEq :: a -> Bool
       script notEq
           on |λ|(a)
               not (p's |λ|(a, x))
           end |λ|
       end script
       
       {x} & nubBy(fnEq, filter(notEq, xs))
   else
       xxs
   end if

end nubBy

-- splitOn :: Text -> Text -> [Text] on splitOn(strDelim, strMain)

   set {dlm, my text item delimiters} to {my text item delimiters, strDelim}
   set lstParts to text items of strMain
   set my text item delimiters to dlm
   return lstParts

end splitOn</lang>

Output:

AppleScriptObjC

<lang applescript>use AppleScript version "2.4" -- OS X 10.10 (Yosemite) or later use framework "Foundation"

set aList to {1, 2, 3, "a", "b", "c", 2, 3, 4, "c", {b:"c"}, {"c"}, "c", "d"} set orderedSet to current application's class "NSOrderedSet"'s orderedSetWithArray:(aList) return orderedSet's array() as list</lang>

Output:

Applesoft BASIC

<lang basic>100 DIM L$(15) 110 L$(0) = "NOW" 120 L$(1) = "IS" 130 L$(2) = "THE" 140 L$(3) = "TIME" 150 L$(4) = "FOR" 160 L$(5) = "ALL" 170 L$(6) = "GOOD" 180 L$(7) = "MEN" 190 L$(8) = "TO" 200 L$(9) = "COME" 210 L$(10) = "TO" 220 L$(11) = "THE" 230 L$(12) = "AID" 240 L$(13) = "OF" 250 L$(14) = "THE" 260 L$(15) = "PARTY."

300 N = 15 310 GOSUB 400 320 FOR I = 0 TO N 330 PRINT L$(I) " " ; 340 NEXT 350 PRINT 360 END

400 REMREMOVE DUPLICATES 410 FOR I = N TO 1 STEP -1 420 I$ = L$(I) 430 FOR J = 0 TO I - 1 440 EQ = I$ = L$(J) 450 IF NOT EQ THEN NEXT J 460 IF EQ THEN GOSUB 500 470 NEXT I 480 RETURN

500 REMREMOVE ELEMENT 510 L$(I) = L$(N) 520 L$(N) = "" 530 N = N - 1 540 RETURN</lang>

Arturo

<lang rebol>arr: [1 2 3 2 1 2 3 4 5 3 2 1]

print unique arr</lang>

Output:

1 2 3 4 5

AutoHotkey

Built in Sort has an option to remove duplicates <lang AutoHotkey>a = 1,2,1,4,5,2,15,1,3,4 Sort, a, a, NUD`, MsgBox % a ; 1,2,3,4,5,15</lang>

AWK

We produce an array a with duplicates from a string; then index a second array b with the contents of a, so that duplicates make only one entry; then produce a string with the keys of b, which is finally output. <lang awk>$ awk 'BEGIN{split("a b c d c b a",a);for(i in a)b[a[i]]=1;r="";for(i in b)r=r" "i;print r}' a b c d</lang>

BASIC256

Translation of: True BASIC

<lang BASIC256> arraybase 1 max = 10 dim res(max) dim dat(max) dat[1] = 1: dat[2] = 2: dat[3] = 1: dat[4] = 4: dat[5] = 5 dat[6] = 2: dat[7] = 15: dat[8] = 1: dat[9] = 3: dat[10] = 4 res[1] = dat[1]

cont = 1 posic = 1 while posic < max posic += 1 esnuevo = 1 indice = 1 while indice <= cont and esnuevo = 1 if dat[posic] = res[indice] then esnuevo = 0 indice += 1 end while if esnuevo = 1 then cont += 1 res[cont] = dat[posic] end if end while

for i = 1 to cont print res[i]; " "; next i end </lang>

BBC BASIC

Works with: BBC BASIC for Windows

<lang bbcbasic> DIM list$(15)

     list$() = "Now", "is", "the", "time", "for", "all", "good", "men", \
     \         "to", "come", "to", "the", "aid", "of", "the", "party."
     num% = FNremoveduplicates(list$())
     FOR i% = 0 TO num%-1
       PRINT list$(i%) " " ;
     NEXT
     PRINT
     END
     
     DEF FNremoveduplicates(l$())
     LOCAL i%, j%, n%, i$
     n% = 1
     FOR i% = 1 TO DIM(l$(), 1)
       i$ = l$(i%)
       FOR j% = 0 TO i%-1
         IF i$ = l$(j%) EXIT FOR
       NEXT
       IF j%>=i% l$(n%) = i$ : n% += 1
     NEXT
     = n%</lang>

Output:

Now is the time for all good men to come aid of party.

Bracmat

Here are three solutions. The first one (A) uses a hash table, the second (B) uses a pattern for spotting the elements that have a copy further on in the list and only adds those elements to the answer that don't have copies further on. The third solution (C) utilises an mechanism that is very typical of Bracmat, namely that sums (and also products) always are transformed to a normalised form upon evaluation. Normalisation means that terms are ordered in a unique way and that terms that are equal, apart from a numerical factor, are replaced by a single term with a numerical factor that is the sum of the numerical factors of each term. The answer is obtained by replacing all numerical factors by 1 as the last step.

The list contains atoms and also a few non-atomic expressions. The hash table needs atomic keys, so we apply the str function when searching and inserting elements. <lang bracmat>2 3 5 7 11 13 17 19 cats 222 (-100.2) "+11" (1.1) "+7" (7.) 7 5 5 3 2 0 (4.4) 2:?LIST

(A=

 ( Hashing
 =   h elm list
   .   new$hash:?h
     &   whl
       ' ( !arg:%?elm ?arg
         & ( (h..find)$str$!elm
           | (h..insert)$(str$!elm.!elm)
           )
         )
     & :?list
     &   (h..forall)
       $ (
         = .!arg:(?.?arg)&!arg !list:?list
         )
     & !list
 )

& put$("Solution A:" Hashing$!LIST \n,LIN) );

(B=

 ( backtracking
 =   answr elm
   .     :?answr
       &   !arg
         :   ?
             (   %?`elm
                 ?
                 ( !elm ?
                 | &!answr !elm:?answr
                 )
             & ~
             )
     | !answr
 )

& put$("Solution B:" backtracking$!LIST \n,LIN) );

(C=

 ( summing
 =   sum car LIST
   .   !arg:?LIST
     & 0:?sum
     &   whl
       ' ( !LIST:%?car ?LIST
         & (.!car)+!sum:?sum
         )
     &   whl
       ' ( !sum:#*(.?el)+?sum
         & !el !LIST:?LIST
         )
     & !LIST
 )

& put$("Solution C:" summing$!LIST \n,LIN) );

( !A & !B & !C & )</lang> Only solution B produces a list with the same order of elements as in the input.

Solution A: 19 (4.4) 17 11 13 (1.1) (7.) 222 +11 7 5 3 2 0 cats (-100.2) +7
Solution B: 11 13 17 19 cats 222 (-100.2) +11 (1.1) +7 (7.) 7 5 3 0 (4.4) 2
Solution C: (7.) (4.4) (1.1) (-100.2) cats 222 19 17 13 11 7 5 3 2 0 +7 +11

Brat

<lang brat>some_array = [1 1 2 1 'redundant' [1 2 3] [1 2 3] 'redundant']

unique_array = some_array.unique</lang>

C

O(n^2) version, using linked lists

Since there's no way to know ahead of time how large the new data structure will need to be, we'll return a linked list instead of an array.

<lang c>#include <stdio.h>

include <stdlib.h>

struct list_node {int x; struct list_node *next;}; typedef struct list_node node;

node * uniq(int *a, unsigned alen)

{if (alen == 0) return NULL;
 node *start = malloc(sizeof(node));
 if (start == NULL) exit(EXIT_FAILURE);
 start->x = a[0];
 start->next = NULL;

 for (int i = 1 ; i < alen ; ++i)
    {node *n = start;
     for (;; n = n->next)
        {if (a[i] == n->x) break;
         if (n->next == NULL)
            {n->next = malloc(sizeof(node));
             n = n->next;
             if (n == NULL) exit(EXIT_FAILURE);
             n->x = a[i];
             n->next = NULL;
             break;}}}

 return start;}

int main(void)

  {int a[] = {1, 2, 1, 4, 5, 2, 15, 1, 3, 4};
   for (node *n = uniq(a, 10) ; n != NULL ; n = n->next)
       printf("%d ", n->x);
   puts("");
   return 0;}</lang>

Output:

1 2 4 5 15 3

O(n^2) version, pure arrays

<lang c>#include <stdio.h>

include <stdlib.h>
include <stdbool.h>
include <string.h>

/* Returns `true' if element `e' is in array `a'. Otherwise, returns `false'.

* Checks only the first `n' elements. Pure, O(n).
*/

bool elem(int *a, size_t n, int e) {

   for (size_t i = 0; i < n; ++i)
       if (a[i] == e)
           return true;

   return false;

}

/* Removes the duplicates in array `a' of given length `n'. Returns the number

* of unique elements. In-place, order preserving, O(n ^ 2).
*/

size_t nub(int *a, size_t n) {

   size_t m = 0;

   for (size_t i = 0; i < n; ++i)
       if (!elem(a, m, a[i]))
           a[m++] = a[i];

   return m;

}

/* Out-place version of `nub'. Pure, order preserving, alloc < n * sizeof(int)

* bytes, O(n ^ 2).
*/

size_t nub_new(int **b, int *a, size_t n) {

   int *c = malloc(n * sizeof(int));
   memcpy(c, a, n * sizeof(int));
   int m = nub(c, n);
   *b = malloc(m * sizeof(int));
   memcpy(*b, c, m * sizeof(int));
   free(c);
   return m;

}

int main(void) {

   int a[] = {1, 2, 1, 4, 5, 2, 15, 1, 3, 4};
   int *b;

   size_t n = nub_new(&b, a, sizeof(a) / sizeof(a[0]));

   for (size_t i = 0; i < n; ++i)
       printf("%d ", b[i]);
   puts("");

   free(b);
   return 0;

}</lang>

Output:

1 2 4 5 15 3

Sorting method

Using qsort and return uniques in-place:<lang c>#include <stdio.h>

include <stdlib.h>

int icmp(const void *a, const void *b) {

define _I(x) *(const int*)x

return _I(a) < _I(b) ? -1 : _I(a) > _I(b);

undef _I

}

/* filter items in place and return number of uniques. if a separate

  list is desired, duplicate it before calling this function */

int uniq(int *a, int len) { int i, j; qsort(a, len, sizeof(int), icmp); for (i = j = 0; i < len; i++) if (a[i] != a[j]) a[++j] = a[i]; return j + 1; }

int main() { int x[] = {1, 2, 1, 4, 5, 2, 15, 1, 3, 4}; int i, len = uniq(x, sizeof(x) / sizeof(x[0])); for (i = 0; i < len; i++) printf("%d\n", x[i]);

return 0; }</lang>

Output:

C#

Works with: C# version 2+

<lang csharp>int[] nums = { 1, 1, 2, 3, 4, 4 }; List<int> unique = new List<int>(); foreach (int n in nums)

   if (!unique.Contains(n))
       unique.Add(n);</lang>

Works with: C# version 3+

<lang csharp>int[] nums = {1, 1, 2, 3, 4, 4}; int[] unique = nums.Distinct().ToArray();</lang>

C++

This version uses std::set, which requires its element type be comparable using the < operator. <lang cpp>#include <set>

include <iostream>

using namespace std;

int main() {

   typedef set<int> TySet;
   int data[] = {1, 2, 3, 2, 3, 4};

   TySet unique_set(data, data + 6);

   cout << "Set items:" << endl;
   for (TySet::iterator iter = unique_set.begin(); iter != unique_set.end(); iter++)
         cout << *iter << " ";
   cout << endl;

}</lang>

This version uses hash_set, which is part of the SGI extension to the Standard Template Library. It is not part of the C++ standard library. It requires that its element type have a hash function.

Works with: GCC

<lang cpp>#include <ext/hash_set>

include <iostream>

using namespace std;

int main() {

   typedef __gnu_cxx::hash_set<int> TyHash;
   int data[] = {1, 2, 3, 2, 3, 4};

   TyHash unique_set(data, data + 6);

   cout << "Set items:" << endl;
   for (TyHash::iterator iter = unique_set.begin(); iter != unique_set.end(); iter++)
         cout << *iter << " ";
   cout << endl;

}</lang>

This version uses unordered_set, which is part of the TR1, which is likely to be included in the next version of C++. It is not part of the C++ standard library. It requires that its element type have a hash function.

Works with: GCC

<lang cpp>#include <tr1/unordered_set>

include <iostream>

using namespace std;

int main() {

   typedef tr1::unordered_set<int> TyHash;
   int data[] = {1, 2, 3, 2, 3, 4};

   TyHash unique_set(data, data + 6);

   cout << "Set items:" << endl;
   for (TyHash::iterator iter = unique_set.begin(); iter != unique_set.end(); iter++)
         cout << *iter << " ";
   cout << endl;

}</lang>

Alternative method working directly on the array:

<lang cpp>#include <iostream>

include <iterator>
include <algorithm>

// helper template template<typename T> T* end(T (&array)[size]) { return array+size; }

int main() {

 int data[] = { 1, 2, 3, 2, 3, 4 };
 std::sort(data, end(data));
 int* new_end = std::unique(data, end(data));
 std::copy(data, new_end, std::ostream_iterator<int>(std::cout, " ");
 std::cout << std::endl;

}</lang>

Using sort, unique, and erase on a vector.

Works with: C++11

<lang cpp>#include <algorithm>

include <iostream>
include <vector>

int main() {

 std::vector<int> data = {1, 2, 3, 2, 3, 4};

 std::sort(data.begin(), data.end());
 data.erase(std::unique(data.begin(), data.end()), data.end());

 for(int& i: data) std::cout << i << " ";
 std::cout << std::endl;
 return 0;

}</lang>

CafeOBJ

The parametrized module NO-DUP-LIST(ELEMENTS :: TRIV) defines the signature of a space separated list structure. The removal of duplicates is handled by the equational properties listed after the signature in brackets {}. There is no user written code in module NO-DUP-LIST. The binary operation (_ _) is associative, commutative, and idempotent, meaning (x , x) = x. This list structure does not permit duplicates, they are removed during evaluation (called reduction in CafeOBJ). Using this module we can perform set union, just by evaluating two lists e.g: <lang> red (1 2 3 4) (3 2 1 5) .

 --> (4 5 1 2 3):Int

</lang>

<lang CafeOBJ> mod! NO-DUP-LIST(ELEMENTS :: TRIV) {

   op __ : Elt Elt -> Elt { comm assoc idem assoc }

}

-- Runs on Version 1.5.1(PigNose0.99) of CafeOBJ -- The tests are performed after opening instantiated NO-DUP-LIST with various concrete types. -- Test on lists of INT open NO-DUP-LIST(INT) . red 2 1 2 1 2 1 3 . -- Gives (2 1 3):Int open NO-DUP-LIST(INT) . reduce 1 1 2 1 1 . close open NO-DUP-LIST(CHARACTER) . reduce 'a' 'b' 'a' 'a' . close open NO-DUP-LIST(STRING) . reduce "abc" "def" "abc" "abc" "abc" . close </lang>

How does work?

The evaluation automatically uses right associativity. So starting with:

<lang> (1  1  2  1  1)</lang>

The system places appropriate brackets on the entire expression:

<lang> (1  ((1  2)  (1  1))) </lang>

Idempotency is applied at rightmost inner brackets: We get rewrite (1 1) -> 1. The term is now:

Any further occurrence of 1 will be removed.

Idempotency is applied at inner brackets (1 2) . We get rewrite (1 2) -> 2 . The term is now:

By already established idempotency we finally get

Ceylon

<lang ceylon><String|Integer>[] data = [1, 2, 3, "a", "b", "c", 2, 3, 4, "b", "c", "d"]; <String|Integer>[] unique = HashSet { *data }.sequence();</lang>

Clojure

<lang lisp>user=> (distinct [1 3 2 9 1 2 3 8 8 1 0 2]) (1 3 2 9 8 0) user=></lang>

CoffeeScript

Translation of: Kotlin

<lang coffeescript>data = [ 1, 2, 3, "a", "b", "c", 2, 3, 4, "b", "c", "d" ] set = [] set.push i for i in data when not (i in set)

console.log data console.log set</lang>

Output:

[ 1, 2, 3, 'a', 'b', 'c', 2, 3, 4, 'b', 'c', 'd' ]
[ 1, 2, 3, 'a', 'b', 'c', 4, 'd' ]

Common Lisp

To remove duplicates non-destructively:

<lang lisp>(remove-duplicates '(1 3 2 9 1 2 3 8 8 1 0 2)) > (9 3 8 1 0 2)</lang>

Or, to remove duplicates in-place:

<lang lisp>(delete-duplicates '(1 3 2 9 1 2 3 8 8 1 0 2)) > (9 3 8 1 0 2)</lang>

Crystal

Copied and modified from the Ruby version.

[1, 2, "a", [1, 2, 3]]

D

<lang d>void main() {

   import std.stdio, std.algorithm;

   [1, 3, 2, 9, 1, 2, 3, 8, 8, 1, 0, 2]
   .sort()
   .uniq
   .writeln;

}</lang>

Output:

[0, 1, 2, 3, 8, 9]

Using an associative array: <lang d>void main() {

   import std.stdio;

   immutable data = [1, 3, 2, 9, 1, 2, 3, 8, 8, 1, 0, 2];

   bool[int] hash;
   foreach (el; data)
       hash[el] = true;
   hash.byKey.writeln;

}</lang>

Output:

[8, 0, 1, 9, 2, 3]

Like code D#1, but with an array returned: <lang d>void main() {

   import std.stdio, std.algorithm, std.array;

   auto a = [5,4,32,7,6,4,2,6,0,8,6,9].sort.uniq.array;
   a.writeln;

}</lang>

Output:

[0, 2, 4, 5, 6, 7, 8, 9, 32]

Delphi

Generics were added in Delphi2009.

<lang Delphi>program RemoveDuplicateElements;

{$APPTYPE CONSOLE}

uses Generics.Collections;

var

 i: Integer;
 lIntegerList: TList<Integer>;

const

 INT_ARRAY: array[1..7] of Integer = (1, 2, 2, 3, 4, 5, 5);

begin

 lIntegerList := TList<Integer>.Create;
 try
 for i in INT_ARRAY do
   if not lIntegerList.Contains(i) then
     lIntegerList.Add(i);

 for i in lIntegerList do
   Writeln(i);
 finally
   lIntegerList.Free;
 end;

end.</lang>

Output:

Déjà Vu

<lang dejavu>} for item in [ 1 10 1 :hi :hello :hi :hi ]: @item !. keys set{</lang>

Output:

[ 1 :hello 10 :hi ]

E

<lang e>[1,2,3,2,3,4].asSet().getElements()</lang>

ECL

<lang ecl> inNumbers := DATASET([{1},{2},{3},{4},{1},{1},{7},{8},{9},{9},{0},{0},{3},{3},{3},{3},{3}], {INTEGER Field1}); DEDUP(SORT(inNumbers,Field1)); </lang>

Output:

Elena

ELENA 5.0 : <lang elena>import extensions; import system'collections; import system'routines;

public program() {

   var nums := new int[]{1,1,2,3,4,4};
   auto unique := new Map<int, int>();

   nums.forEach:(n){ unique[n] := n };

   console.printLine(unique.MapValues.asEnumerable())

}</lang>

Output:

1,2,3,4

Elixir

Elixir has an Enum.uniq built-in function.

Works with: Elixir version 1.2

<lang elixir>defmodule RC do

 # Set approach
 def uniq1(list), do: MapSet.new(list) |> MapSet.to_list
 
 # Sort approach
 def uniq2(list), do: Enum.sort(list) |> Enum.dedup
 
 # Go through the list approach
 def uniq3(list), do: uniq3(list, [])
 
 defp uniq3([], res), do: Enum.reverse(res)
 defp uniq3([h|t], res) do
   if h in res, do: uniq3(t, res), else: uniq3(t, [h | res])
 end

end

num = 10000 max = div(num, 10) list = for _ <- 1..num, do: :rand.uniform(max) funs = [&Enum.uniq/1, &RC.uniq1/1, &RC.uniq2/1, &RC.uniq3/1] Enum.each(funs, fn fun ->

 result = fun.([1,1,2,1,'redundant',1.0,[1,2,3],[1,2,3],'redundant',1.0])
 :timer.tc(fn ->
   Enum.each(1..100, fn _ -> fun.(list) end)
 end)
 |> fn{t,_} -> IO.puts "#{inspect fun}:\t#{t/1000000}\t#{inspect result}" end.()

end)</lang>

Output:

&Enum.uniq/1:   0.296   [1, 2, 'redundant', 1.0, [1, 2, 3]]
&RC.uniq1/1:    0.686   [1, 2, 1.0, [1, 2, 3], 'redundant']
&RC.uniq2/1:    0.921   [1, 1.0, 2, [1, 2, 3], 'redundant']
&RC.uniq3/1:    1.497   [1, 2, 'redundant', 1.0, [1, 2, 3]]

Erlang

<lang erlang>List = [1, 2, 3, 2, 2, 4, 5, 5, 4, 6, 6, 5]. UniqueList = gb_sets:to_list(gb_sets:from_list(List)). % Alternatively the builtin: Unique_list = lists:usort( List ). </lang>

Euphoria

<lang euphoria>include sort.e

function uniq(sequence s)

   sequence out
   s = sort(s)
   out = s[1..1]
   for i = 2 to length(s) do
       if not equal(s[i],out[$]) then
           out = append(out, s[i])
       end if
   end for
   return out

end function

constant s = {1, 2, 1, 4, 5, 2, 15, 1, 3, 4} ? s ? uniq(s)</lang>

Output:

{1,2,1,4,5,2,15,1,3,4}
{1,2,3,4,5,15}

F#

The simplest way is to build a set from the given array (this actually works for any enumerable input sequence type, not just arrays): <lang fsharp> set [|1;2;3;2;3;4|] </lang> gives: <lang fsharp> val it : Set<int> = seq [1; 2; 3; 4] </lang>

Factor

<lang factor>USING: sets ; V{ 1 2 1 3 2 4 5 } members .

V{ 1 2 3 4 5 }</lang>

Forth

Forth has no built-in hashtable facility, so the easiest way to achieve this goal is to take the "uniq" program as an example.

The word uniq, if given a sorted array of cells, will remove the duplicate entries and return the new length of the array. For simplicity, uniq has been written to process cells (which are to Forth what "int" is to C), but could easily be modified to handle a variety of data types through deferred procedures, etc.

The input data is assumed to be sorted.

<lang forth>\ Increments a2 until it no longer points to the same value as a1 \ a3 is the address beyond the data a2 is traversing.

skip-dups ( a1 a2 a3 -- a1 a2+n )

   dup rot ?do
     over @ i @ <> if drop i leave then
   cell +loop ;

\ Compress an array of cells by removing adjacent duplicates \ Returns the new count

uniq ( a n -- n2 )

  over >r             \ Original addr to return stack
  cells over + >r     \ "to" addr now on return stack, available as r@
  dup begin           ( write read )
     dup r@ <
  while
     2dup @ swap !    \ copy one cell
     cell+ r@ skip-dups
     cell 0 d+        \ increment write ptr only
  repeat  r> 2drop  r> - cell / ;</lang>

Here is another implementation of "uniq" that uses a popular parameters and local variables extension words. It is structurally the same as the above implementation, but uses less overt stack manipulation.

<lang forth>: uniqv { a n \ r e -- n }

   a n cells+ to e
   a dup to r
   \ the write address lives on the stack
   begin
     r e <
   while
     r @ over !
     r cell+ e skip-dups to r
     cell+
   repeat
   a - cell / ;</lang>

To test this code, you can execute:

<lang forth>create test 1 , 2 , 3 , 2 , 6 , 4 , 5 , 3 , 6 , here test - cell / constant ntest

.test ( n -- ) 0 ?do test i cells + ? loop ;

test ntest 2dup cell-sort uniq .test</lang>

Output:

1 2 3 4 5 6 ok

Fortran

Fortran has no built-in hash functions or sorting functions but the code below implements the compare all elements algorithm.

program remove_dups

 implicit none
 integer :: example(12)         ! The input
 integer :: res(size(example))  ! The output
 integer :: k                   ! The number of unique elements
 integer :: i, j

 example = [1, 2, 3, 2, 2, 4, 5, 5, 4, 6, 6, 5]
 k = 1
 res(1) = example(1)
 outer: do i=2,size(example)
    do j=1,k
       if (res(j) == example(i)) then
          ! Found a match so start looking again
          cycle outer
       end if
    end do
    ! No match found so add it to the output
    k = k + 1
    res(k) = example(i)
 end do outer
 write(*,advance='no',fmt='(a,i0,a)') 'Unique list has ',k,' elements: '
 write(*,*) res(1:k)

end program remove_dups

</lang>

Same as above but using 'ANY' to check if the input number already exists in the array of unique elements:

<lang fortran> program remove_dups

   implicit none
   integer :: example(12)         ! The input
   integer :: res(size(example))  ! The output
   integer :: k                   ! The number of unique elements
   integer :: i

   example = [1, 2, 3, 2, 2, 4, 5, 5, 4, 6, 6, 5]
   k = 1
   res(1) = example(1)
   do i=2,size(example)
       ! if the number already exist in res check next
       if (any( res == example(i) )) cycle
       ! No match found so add it to the output
       k = k + 1
       res(k) = example(i)
   end do

   write(*,advance='no',fmt='(a,i0,a)') 'Unique list has ',k,' elements: '
   write(*,*) res(1:k)

end program remove_dups

</lang>

Output:

Unique list has 6 elements:            1           2           3           4           5           6

FreeBASIC

<lang freebasic>' FB 1.05.0 Win64

Sub removeDuplicates(a() As Integer, b() As Integer)

 Dim lb As Integer = LBound(a)
 Dim ub As Integer = UBound(a)
 If ub = -1 Then Return  empty array
 Redim b(lb To ub)
 b(lb) = a(lb)
 Dim count As Integer = 1
 Dim unique As Boolean
 
 For i As Integer = lb + 1 To ub
   unique = True 
   For j As Integer = lb to i - 1
     If a(i) = a(j) Then
       unique = False
       Exit For
     End If
   Next j
   If unique Then
     b(lb + count) = a(i)
     count += 1
   End If
 Next i

 If count > 0 Then Redim Preserve b(lb To lb + count - 1)

End Sub

Dim a(1 To 10) As Integer = {1, 2, 1, 4, 5, 2, 15, 1, 3, 4} Dim b() As Integer removeDuplicates a(), b()

For i As Integer = LBound(b) To UBound(b)

 Print b(i); " ";

Output:

 1  2  4  5  15  3

Frink

The following demonstrates two of the simplest ways of removing duplicates. <lang frink> b = [1, 5, 2, 6, 6, 2, 2, 1, 9, 8, 6, 5]

// One way, using OrderedList. An OrderedList is a type of array that keeps // its elements in order. The items must be comparable. a = new OrderedList println[a.insertAllUnique[b]]

// Another way, using the "set" datatype and back to an array. println[toArray[toSet[b]] </lang>

Output:

Note that sets are not guaranteed to be printed in any specific order.

[1, 2, 5, 6, 8, 9]
[9, 8, 6, 5, 2, 1]

Futurebasic

<lang futurebasic> include "NSLog.incl"

CFArrayRef array, unique OrderedSetRef ordered

array = @[@"A", @"B", @"C", @"B", @"A", @"C", @"A", @"C", @"A", @"B", @"C"] ordered = fn OrderedSetWithArray( array ) NSLog( @"%@", fn OrderedSetArray( ordered ) )

HandleEvents </lang>

Output:

(
    A,
    B,
    C
)

Gambas

Click this link to run this code <lang gambas>Public Sub Main() Dim sString As String[] = Split("Now is the time for all the good men to come to the aid of the good party 1 2 1 3 3 3 2 1 1 2 3 4 33 2 5 4 333 5", " ") Dim sFix As New String[] Dim sTemp As String

For Each sTemp In sString

 sTemp &= " "
 If InStr(sFix.Join(" ") & " ", sTemp) Then Continue
 sFix.Add(Trim(sTemp))

End</lang> Output:

Now is the time for all good men to come aid of party 1 2 3 4 33 5 333

GAP

<lang gap># Built-in, using sets (which are also lists) a := [ 1, 2, 3, 1, [ 4 ], 5, 5, [4], 6 ];

[ 1, 2, 3, 1, [ 4 ], 5, 5, [ 4 ], 6 ]

b := Set(a);

[ 1, 2, 3, 5, 6, [ 4 ] ]

IsSet(b);

true

IsList(b);

true</lang>

Go

Map solution

<lang go>package main

import "fmt"

func uniq(list []int) []int { unique_set := make(map[int]bool, len(list)) for _, x := range list { unique_set[x] = true } result := make([]int, 0, len(unique_set)) for x := range unique_set { result = append(result, x) } return result }

func main() { fmt.Println(uniq([]int{1, 2, 3, 2, 3, 4})) // prints: [3 4 1 2] (but in a semi-random order) }</lang>

Map preserving order

It takes only small changes to the above code to preserver order. Just store the sequence in the map: <lang go>package main

import "fmt"

func uniq(list []int) []int { unique_set := make(map[int]int, len(list)) i := 0 for _, x := range list { if _, there := unique_set[x]; !there { unique_set[x] = i i++ } } result := make([]int, len(unique_set)) for x, i := range unique_set { result[i] = x } return result }

func main() { fmt.Println(uniq([]int{1, 2, 3, 2, 3, 4})) // prints: [1 2 3 4] }</lang>

Float64, removing duplicate NaNs

In solutions above, you just replace int with another type to use for a list of another type. (See Associative_arrays/Creation#Go for acceptable types.) Except a weird thing happens with NaNs. They (correctly) don't compare equal, so you have to special case them if you want to remove duplicate NaNs: <lang go>package main

import ( "fmt" "math" )

func uniq(list []float64) []float64 { unique_set := map[float64]int{} i := 0 nan := false for _, x := range list { if _, exists := unique_set[x]; exists { continue } if math.IsNaN(x) { if nan { continue } else { nan = true } } unique_set[x] = i i++ } result := make([]float64, len(unique_set)) for x, i := range unique_set { result[i] = x } return result }

func main() { fmt.Println(uniq([]float64{1, 2, math.NaN(), 2, math.NaN(), 4})) // Prints [1 2 NaN 4] }</lang>

Any type using reflection

Go doesn't have templates or generics, but it does have reflection. Using this it's possible to build a version that will work on almost any array or slice type. Using the reflect package for this does make the code less readable.

Normally in Go this type of solution is somewhat rare. Instead, for very short code (such as min, max, abs) it's common to cast or make a type specific function by hand as needed. For longer code, often an interface can be used instead (see the sort package for an example).

Note: due to details with how Go handles map keys that contain a NaN somewhere (including within a complex or even within a sub struct field) this version simply omits any NaN containing values it comes across and returns a bool to indicate if that happened. This version is otherwise a translation of the above order preserving map implementation, it does not for example call reflect.DeepEqual so elements with pointers to distinct but equal values will be treated as non-equal. <lang go>package main

import ( "fmt" "math" "reflect" )

func uniq(x interface{}) (interface{}, bool) { v := reflect.ValueOf(x) if !v.IsValid() { panic("uniq: invalid argument") } if k := v.Kind(); k != reflect.Array && k != reflect.Slice { panic("uniq: argument must be an array or a slice") } elemType := v.Type().Elem() intType := reflect.TypeOf(int(0)) mapType := reflect.MapOf(elemType, intType) m := reflect.MakeMap(mapType) i := 0 for j := 0; j < v.Len(); j++ { x := v.Index(j) if m.MapIndex(x).IsValid() { continue } m.SetMapIndex(x, reflect.ValueOf(i)) if m.MapIndex(x).IsValid() { i++ } } sliceType := reflect.SliceOf(elemType) result := reflect.MakeSlice(sliceType, i, i) hadNaN := false for _, key := range m.MapKeys() { ival := m.MapIndex(key) if !ival.IsValid() { hadNaN = true } else { result.Index(int(ival.Int())).Set(key) } }

return result.Interface(), hadNaN }

type MyType struct { name string value float32 }

func main() { intArray := [...]int{5, 1, 2, 3, 2, 3, 4} intSlice := []int{5, 1, 2, 3, 2, 3, 4} stringSlice := []string{"five", "one", "two", "three", "two", "three", "four"} floats := []float64{1, 2, 2, 4, math.NaN(), 2, math.NaN(), math.Inf(1), math.Inf(1), math.Inf(-1), math.Inf(-1)} complexes := []complex128{1, 1i, 1 + 1i, 1 + 1i, complex(math.NaN(), 1), complex(1, math.NaN()), complex(math.Inf(+1), 1), complex(1, math.Inf(1)), complex(math.Inf(-1), 1), complex(1, math.Inf(1)), } structs := []MyType{ {"foo", 42}, {"foo", 2}, {"foo", 42}, {"bar", 42}, {"bar", 2}, {"fail", float32(math.NaN())}, }

fmt.Print("intArray: ", intArray, " → ") fmt.Println(uniq(intArray)) fmt.Print("intSlice: ", intSlice, " → ") fmt.Println(uniq(intSlice)) fmt.Print("stringSlice: ", stringSlice, " → ") fmt.Println(uniq(stringSlice)) fmt.Print("floats: ", floats, " → ") fmt.Println(uniq(floats)) fmt.Print("complexes: ", complexes, "\n → ") fmt.Println(uniq(complexes)) fmt.Print("structs: ", structs, " → ") fmt.Println(uniq(structs)) // Passing a non slice or array will compile put // then produce a run time panic: //a := 42 //uniq(a) //uniq(nil) }</lang>

Output:

intArray: [5 1 2 3 2 3 4] → [5 1 2 3 4] false
intSlice: [5 1 2 3 2 3 4] → [5 1 2 3 4] false
stringSlice: [five one two three two three four] → [five one two three four] false
floats: [1 2 2 4 NaN 2 NaN +Inf +Inf -Inf -Inf] → [1 2 4 +Inf -Inf] true
complexes: [(1+0i) (0+1i) (1+1i) (1+1i) (NaN+1i) (1+NaNi) (+Inf+1i) (1+Infi) (-Inf+1i) (1+Infi)]
 → [(1+0i) (0+1i) (1+1i) (+Inf+1i) (1+Infi) (-Inf+1i)] true
structs: [{foo 42} {foo 2} {foo 42} {bar 42} {bar 2} {fail NaN}] → [{foo 42} {foo 2} {bar 42} {bar 2}] true

Groovy

<lang groovy>def list = [1, 2, 3, 'a', 'b', 'c', 2, 3, 4, 'b', 'c', 'd'] assert list.size() == 12 println " Original List: ${list}"

// Filtering the List (non-mutating) def list2 = list.unique(false) assert list2.size() == 8 assert list.size() == 12 println " Filtered List: ${list2}"

// Filtering the List (in place) list.unique() assert list.size() == 8 println " Original List, filtered: ${list}"

def list3 = [1, 2, 3, 'a', 'b', 'c', 2, 3, 4, 'b', 'c', 'd'] assert list3.size() == 12

// Converting to Set def set = list as Set assert set.size() == 8 println " Set: ${set}"</lang>

Output:

             Original List: [1, 2, 3, a, b, c, 2, 3, 4, b, c, d]
             Filtered List: [1, 2, 3, a, b, c, 4, d]
   Original List, filtered: [1, 2, 3, a, b, c, 4, d]
                       Set: [1, 2, 3, a, b, c, 4, d]

GW-BASIC

Translation of: Modula-2

Works with: PC-BASIC version any

<lang qbasic> 10 ' Remove Duplicates 20 OPTION BASE 1 30 LET MAXI% = 7 40 DIM D(7), R(7): ' data, result 50 ' Set the data. 60 FOR I% = 1 TO 7 70 READ D(I%) 80 NEXT I% 90 ' Remove duplicates. 100 LET R(1) = D(1) 110 LET LRI% = 1: ' last index of result 120 LET P% = 1: ' position 130 WHILE P% < MAXI% 140 LET P% = P% + 1 150 LET ISNEW = 1: ' is a new number? 160 LET RI% = 1: ' current index of result 170 WHILE (RI% <= LRI%) AND ISNEW 180 IF D(P%) = R(RI%) THEN LET ISNEW = 0 190 LET RI% = RI% + 1 200 WEND 210 IF ISNEW THEN LET LRI% = LRI% + 1: LET R(LRI%) = D(P%) 220 WEND 230 FOR RI% = 1 TO LRI% 240 PRINT R(RI%) 250 NEXT RI% 260 END 1000 DATA 1, 2, 2, 3, 4, 5, 5 </lang>

Output:

Haskell

Usage

<lang haskell> print $ unique [4, 5, 4, 2, 3, 3, 4]

[4,5,2,3]</lang>

Sorted result using Set

O(n ln(n)). Requires there is a partial ordering of elements.

<lang haskell>import qualified Data.Set as Set

unique :: Ord a => [a] -> [a] unique = Set.toList . Set.fromList</lang>

Unsorted result using Set

O(n ln(n)). Retains original order. Requires there is a partial ordering of elements.

<lang haskell>import Data.Set

unique :: Ord a => [a] -> [a] unique = loop empty

 where
   loop s []                    = []
   loop s (x : xs) | member x s = loop s xs
                   | otherwise  = x : loop (insert x s) xs</lang>

Using filter

O(n^2). Retains original order. Only requires that elements can be compared for equality.

<lang haskell>import Data.List

unique :: Eq a => [a] -> [a] unique [] = [] unique (x : xs) = x : unique (filter (x /=) xs)</lang>

Standard Library

<lang haskell>import Data.List Data.List.nub :: Eq a => [a] -> [a] Data.List.Unique.unique :: Ord a => [a] -> [a]</lang>

HicEst

<lang hicest>REAL :: nums(12) CHARACTER :: workspace*100

nums = (1, 3, 2, 9, 1, 2, 3, 8, 8, 1, 0, 2) WRITE(Text=workspace) nums ! convert to string EDIT(Text=workspace, SortDelDbls=workspace) ! do the job for a string READ(Text=workspace, ItemS=individuals) nums ! convert to numeric

WRITE(ClipBoard) individuals, "individuals: ", nums ! 6 individuals: 0 1 2 3 8 9 0 0 0 0 0 0 </lang>

Icon and Unicon

This solution preserves the original order of the elements. <lang Icon>procedure main(args)

   every write(!noDups(args))

end

procedure noDups(L)

   every put(newL := [], notDup(set(),!L))
   return newL

end

procedure notDup(cache, a)

   if not member(cache, a) then {
        insert(cache, a)
        return a
        }

end</lang> A sample run is:

->noDups a b c d c a b e
a
b
c
d
e
->

IDL

<lang idl>non_repeated_values = array[uniq(array, sort( array))]</lang>

Inform 7

<lang inform7>To decide which list of Ks is (L - list of values of kind K) without duplicates: let result be a list of Ks; repeat with X running through L: add X to result, if absent; decide on result.</lang>

IS-BASIC

<lang IS-BASIC>100 PROGRAM "RemoveDu.bas" 110 RANDOMIZE 120 NUMERIC ARR(1 TO 20),TOP 130 LET TOP=FILL(ARR) 140 CALL WRITE(ARR,TOP) 150 LET TOP=REMOVE(ARR) 160 CALL WRITE(ARR,TOP) 170 DEF WRITE(REF A,N) 180 FOR I=1 TO N 190 PRINT A(I); 200 NEXT 210 PRINT 220 END DEF 230 DEF FILL(REF A) 240 LET FILL=UBOUND(A):LET A(LBOUND(A))=1 250 FOR I=LBOUND(A)+1 TO UBOUND(A) 260 LET A(I)=A(I-1)+RND(3) 270 NEXT 280 END DEF 290 DEF REMOVE(REF A) 300 LET ST=0 310 FOR I=LBOUND(A)+1 TO UBOUND(A) 320 IF A(I-1)=A(I) THEN LET ST=ST+1 330 IF ST>0 THEN LET A(I-ST)=A(I) 340 NEXT 350 LET REMOVE=UBOUND(A)-ST 360 END DEF</lang>

Output:

START
 1  1  2  4  5  7  9  10  12  14  16  16  16  17  18  20  20  22  23  23 
 1  2  4  5  7  9  10  12  14  16  17  18  20  22  23 
ok
START
 1  2  4  5  5  5  7  8  9  9  10  10  10  12  14  15  17  17  18  20 
 1  2  4  5  7  8  9  10  12  14  15  17  18  20 
ok
START
 1  3  3  4  5  6  8  10  11  12  14  16  16  16  16  18  18  19  21  21 
 1  3  4  5  6  8  10  11  12  14  16  18  19  21 
ok
START
 1  3  3  4  5  5  7  9  11  13  13  14  16  17  17  18  19  19  20  21 
 1  3  4  5  7  9  11  13  14  16  17  18  19  20  21 
ok
START
 1  2  3  5  5  6  6  7  8  10  12  14  15  17  17  19  21  23  25  25 
 1  2  3  5  6  7  8  10  12  14  15  17  19  21  23  25
ok

J

The verb ~. removes duplicate items from any array (numeric, character, or other; vector, matrix, rank-n array). For example: <lang j> ~. 4 3 2 8 0 1 9 5 1 7 6 3 9 9 4 2 1 5 3 2 4 3 2 8 0 1 9 5 7 6

  ~. 'chthonic eleemosynary paronomasiac'

chtoni elmsyarp</lang> Or (since J defines an item of an n dimensional array as its n-1 dimensional sub arrays):

<lang j> 0 1 1 2 0 */0 1 2 0 0 0 0 1 2 0 1 2 0 2 4 0 0 0

  ~. 0 1 1 2 0 */0 1 2

0 0 0 0 1 2 0 2 4</lang>

Java

Works with: Java version 1.5

<lang java5>import java.util.*;

class Test {

   public static void main(String[] args) {

       Object[] data = {1, 1, 2, 2, 3, 3, 3, "a", "a", "b", "b", "c", "d"};
       Set<Object> uniqueSet = new HashSet<Object>(Arrays.asList(data));
       for (Object o : uniqueSet)
           System.out.printf("%s ", o);
   }

}</lang>

1 a 2 b 3 c d

Works with: Java version 8

<lang java>import java.util.*;

class Test {

   public static void main(String[] args) {

       Object[] data = {1, 1, 2, 2, 3, 3, 3, "a", "a", "b", "b", "c", "d"};
       Arrays.stream(data).distinct().forEach((o) -> System.out.printf("%s ", o));
   }

}</lang>

1 2 3 a b c d

JavaScript

This uses the === "strict equality" operator, which does no type conversions (4 == "4" is true but 4 === "4" is false) <lang javascript>function unique(ary) {

   // concat() with no args is a way to clone an array
   var u = ary.concat().sort();
   for (var i = 1; i < u.length; ) {
       if (u[i-1] === u[i])
           u.splice(i,1);
       else
           i++;
   }
   return u;

}

var ary = [1, 2, 3, "a", "b", "c", 2, 3, 4, "b", "c", "d", "4"]; var uniq = unique(ary); for (var i = 0; i < uniq.length; i++)

   print(uniq[i] + "\t" + typeof(uniq[i]));</lang>

1 - number
2 - number
3 - number
4 - number
4 - string
a - string
b - string
c - string
d - string

Or, extend the prototype for Array: <lang javascript>Array.prototype.unique = function() {

   var u = this.concat().sort();
   for (var i = 1; i < u.length; ) {
       if (u[i-1] === u[i])
           u.splice(i,1);
       else
           i++;
   }
   return u;

} var uniq = [1, 2, 3, "a", "b", "c", 2, 3, 4, "b", "c", "d"].unique();</lang>

With reduce and arrow functions (ES6): <lang javascript>Array.prototype.unique = function() {

  return this.sort().reduce( (a,e) => e === a[a.length-1] ? a : (a.push(e), a), [] )

}</lang>

With sets and spread operator (ES6): <lang javascript>Array.prototype.unique = function() {

   return [... new Set(this)]

}</lang>

If, however, the array is homogenous, or we wish to interpret it as such by using JavaScript's Abstract Equality comparison (as in '==', see http://www.ecma-international.org/ecma-262/5.1/#sec-11.9.3) then it proves significantly faster to use a hash table.

For example, in ES 5:

<lang JavaScript>function uniq(lst) {

 var u = [],
   dct = {},
   i = lst.length,
   v;

 while (i--) {
   v = lst[i], dct[v] || (
     dct[v] = u.push(v)
   );
 }
 u.sort(); // optional
 
 return u;

}</lang>

Or, to allow for customised definitions of equality and duplication, we can follow the Haskell prelude in defining a nub :: [a] -> [a] function which is a special case of nubBy :: (a -> a -> Bool) -> [a] -> [a]

Translation of: Haskell

<lang JavaScript>(function () {

   'use strict';

   // nub :: [a] -> [a]
   function nub(xs) {

       // Eq :: a -> a -> Bool
       function Eq(a, b) {
           return a === b;
       }

       // nubBy :: (a -> a -> Bool) -> [a] -> [a]
       function nubBy(fnEq, xs) {
           var x = xs.length ? xs[0] : undefined;

           return x !== undefined ? [x].concat(
               nubBy(fnEq, xs.slice(1)
                   .filter(function (y) {
                       return !fnEq(x, y);
                   }))
           ) : [];
       }

       return nubBy(Eq, xs);
   }

   // TEST
   
   return [
       nub('4 3 2 8 0 1 9 5 1 7 6 3 9 9 4 2 1 5 3 2'.split(' '))
       .map(function (x) {
           return Number(x);
       }),
       nub('chthonic eleemosynary paronomasiac'.split())
       .join()
   ]

})();</lang>

Output:

[[4, 3, 2, 8, 0, 1, 9, 5, 7, 6], "chtoni elmsyarp"]

jq

If it is acceptable to alter the ordering of elements, then the builtin (fast) filter, unique, can be used. It can be used for arrays with elements of any JSON type and returns the distinct elements in sorted order. <lang jq>[4,3,2,1,1,2,3,4] | unique</lang>

If all but the first occurrence of each element should be deleted, then the following function could be used. It retains the advantage of imposing no restrictions on the types of elements in the array and for that reason is slightly more complex than would otherwise be required. <lang jq>def removeAllButFirst:

 # The hash table functions all expect the hash table to be the input.
 
 # Is x in the hash table?
 def hashed(x):
   (x|tostring) as $value
   | .[$value] as $bucket
   | $bucket and (.[$value] | index([x]));

 # Add x to the hash table:
 def add_hash(x):
   (x|tostring) as $value
   | .[$value] as $bucket
   | if $bucket and ($bucket | index([x])) then .
     else .[$value] += [x]
     end;

 reduce .[] as $item
   ( [[], {}]; # [array, hash]
     if .[1] | hashed($item) then .
     else [ (.[0] + [$item]), (.[1] | add_hash($item)) ]
     end)
 | .[0];

</lang>

Julia

Works with: Julia version 0.6

<lang julia>a = [1, 2, 3, 4, 1, 2, 3, 4] @show unique(a) Set(a)</lang>

Output:

unique(a) = [1, 2, 3, 4]
Set(a) = Set([4, 2, 3, 1])

K

(Inspired by the J version.)

<lang K> a:4 5#20?13 / create a random 4 x 5 matrix (12 7 12 4 3

6 3 7 4 7
3 8 3 1 2
2 12 6 4 1)

  ,/a           / flatten to array

12 7 12 4 3 6 3 7 4 7 3 8 3 1 2 2 12 6 4 1

  ?,/a          / distinct elements

12 7 4 3 6 8 1 2

  ?"chthonic eleemosynary paronomasiac"

"chtoni elmsyarp"

  ?("this";"that";"was";"that";"was";"this")

("this"

"that"
"was")

  0 1 1 2 0 *\: 0 1 2

(0 0 0

  ?0 1 1 2 0 *\: 0 1 2

(0 0 0

0 1 2
0 2 4)</lang>

Klingphix

<lang Klingphix>( )

( "Now" "is" "the" "time" "for" "all" "good" "men" "to" "come" "to" "the" "aid" "of" "the" "party." )

len [

   get rot over find
   not (
       [swap 0 put]
       [nip]
   ) if
   swap

] for

swap print drop nl

"End " input</lang>

Output:

("Now", "is", "the", "time", "for", "all", "good", "men", "to", "come", "aid", "of", "party.")
End

Kotlin

Translation of: Java

<lang scala>fun main(args: Array<String>) {

   val data = listOf(1, 2, 3, "a", "b", "c", 2, 3, 4, "b", "c", "d")
   val set = data.distinct()

   println(data)
   println(set)

}</lang>

Output:

[1, 2, 3, a, b, c, 2, 3, 4, b, c, d]
[1, 2, 3, a, b, c, 4, d]

Lang5

<lang lang5>: dip swap '_ set execute _ ;

remove-duplicates

   [] swap do unique? length 0 == if break then loop drop ;

unique?

   0 extract swap "2dup in if drop else append then" dip ;

[1 2 6 3 6 4 5 6] remove-duplicates .</lang> Built-in function: <lang lang5>[1 2 6 3 6 4 5 6] 's distinct [1 2 6 3 6 4 5 6] 's dress dup union .</lang>

Lasso

<lang Lasso >local( x = array(3,4,8,1,8,1,4,5,6,8,9,6), y = array ) with n in #x where #y !>> #n do => { #y->insert(#n) } // result: array(3, 4, 8, 1, 5, 6, 9)</lang>

Liberty BASIC

LB has arrays, but here the elements are stored in a space-separated string. <lang lb> a$ =" 1 $23.19 2 elbow 3 2 Bork 4 3 elbow 2 $23.19 " print "Original set of elements = ["; a$; "]"

b$ =removeDuplicates$( a$) print "With duplicates removed = ["; b$; "]"

end

function removeDuplicates$( in$)

   o$ =" "
   i  =1
   do
       term$    =word$( in$, i, " ")
       if instr( o$, " "; term$; " ") =0 and term$ <>" " then o$ =o$ +term$ +" "
       i        =i +1
   loop until term$ =""
   removeDuplicates$ =o$

end function </lang>

Original set of elements = [ 1 $23.19 2 elbow 3 2 Bork 4 3 elbow 2 $23.19 ]
With duplicates removed  = [ 1 $23.19 2 elbow 3 Bork 4  ]

Logo

Works with: UCB Logo

<lang logo>show remdup [1 2 3 a b c 2 3 4 b c d] ; [1 a 2 3 4 b c d]</lang>

Lua

<lang Lua>items = {1,2,3,4,1,2,3,4,"bird","cat","dog","dog","bird"} flags = {} io.write('Unique items are:') for i=1,#items do

  if not flags[items[i]] then
     io.write(' ' .. items[i])
     flags[items[i]] = true
  end

end io.write('\n')</lang>

Output:

Unique items are: 1 2 3 4 bird cat dog

Lua doesn't accept Not-a-Number (NaN) and nil as table key, we can handle them like this (Lua 5.3): <lang Lua>local items = {1,2,3,4,1,2,3,4,0/0,nil,"bird","cat","dog","dog","bird",0/0}

function rmdup(t)

 local r,dup,c,NaN = {},{},1,{}  
 for i=1,#t do 
   local e = t[i]
   local k = e~=e and NaN or e
   if k~=nil and not dup[k] then  
     c, r[c], dup[k]= c+1, e, true 
   end
 end
 return r

end

print(table.concat(rmdup(items),' '))</lang>

Output:

1 2 3 4 nan bird cat dog

Maple

This is simplest with a list, which is an immutable array. <lang Maple>> L := [ 1, 2, 1, 2, 3, 3, 2, 1, "a", "b", "b", "a", "c", "b" ];

     L := [1, 2, 1, 2, 3, 3, 2, 1, "a", "b", "b", "a", "c", "b"]

> [op]({op}(L));

                       [1, 2, 3, "a", "b", "c"]</lang>

That is idiomatic, but perhaps a bit cryptic; here is a more verbose equivalent: <lang Maple>> convert( convert( L, 'set' ), 'list' );

                       [1, 2, 3, "a", "b", "c"]</lang>

For an Array, which is mutable, the table solution works well in Maple. <lang Maple>> A := Array( L ): > for u in A do T[u] := 1 end: Array( [indices]( T, 'nolist' ) );

                       [1, 2, 3, "c", "a", "b"]</lang>

Note that the output (due to the Array() constructor) is in fact an Array.

Mathematica/Wolfram Language

Built-in function: <lang Mathematica>DeleteDuplicates[{0, 2, 1, 4, 2, 0, 3, 1, 1, 1, 0, 3}]</lang> gives back: <lang Mathematica>{0, 2, 1, 4, 3}</lang> Delete duplicates and return sorted elements: <lang Mathematica>Union[{0, 2, 1, 4, 2, 0, 3, 1, 1, 1, 0, 3}]</lang>

gives back: :

MATLAB

MATLAB has a built-in function, "unique(list)", which performs this task.
Sample Usage: <lang MATLAB>>> unique([1 2 6 3 6 4 5 6])

ans =

    1     2     3     4     5     6</lang>

NOTE: The unique function only works for vectors and not for true arrays.

Maxima

<lang maxima>unique([8, 9, 5, 2, 0, 7, 0, 0, 4, 2, 7, 3, 9, 6, 6, 2, 4, 7, 9, 8, 3, 8, 0, 3, 7, 0, 2, 7, 6, 0]); [0, 2, 3, 4, 5, 6, 7, 8, 9]</lang>

MAXScript

<lang maxscript>uniques = #(1, 2, 3, "a", "b", "c", 2, 3, 4, "b", "c", "d") for i in uniques.count to 1 by -1 do (

   id = findItem uniques uniques[i]
   if (id != i) do deleteItem uniques i

)</lang>

Microsoft Small Basic

Translation of: Modula-2

<lang microsoftsmallbasic> ' Set the data. dataArray[1] = 1 dataArray[2] = 2 dataArray[3] = 2 dataArray[4] = 3 dataArray[5] = 4 dataArray[6] = 5 dataArray[7] = 5

resultArray[1] = dataArray[1] lastResultIndex = 1 position = 1 While position < Array.GetItemCount(dataArray)

 position = position + 1
 isNewNumber = 1 ' logical 1
 resultIndex = 1
 While (resultIndex <= lastResultIndex) And isNewNumber = 1
   If dataArray[position] = resultArray[resultIndex] Then
     isNewNumber = 0
   EndIf
   resultIndex = resultIndex + 1
 EndWhile
 If isNewNumber = 1 Then
   lastResultIndex = lastResultIndex + 1
   resultArray[lastResultIndex] = dataArray[position]
 EndIf

EndWhile For resultIndex = 1 To lastResultIndex

 TextWindow.WriteLine(resultArray[resultIndex])

EndFor </lang>

MiniScript

<lang MiniScript>items = [1, 2, 3, "a", "b", "c", 2, 3, 4, "b", "c", "d"] d = {} for i in items

   d.push i

end for print d.indexes</lang>

Output:

["b", 1, "d", 3, "a", 4, "c", 2]

ML

mLite

A bit like option 3, except copying each element as encountered, and checking to see if it has already been encountered <lang ocaml>fun mem (x, []) = false

     | (x eql a, a :: as) = true
     | (x, _ :: as) = mem (x, as)

fun remdup ([], uniq) = rev uniq | (h :: t, uniq) = if mem(h, uniq) then remdup (t, uniq) else remdup (t, h :: uniq) | L = remdup (L, [])

println ` implode ` remdup ` explode "the quick brown fox jumped over the lazy dog"; println ` remdup [1,2,3,4,4,3,2,1, "dog","cat","dog", 1.1, 2.2, 3.3, 1.1]; </lang>

Output:

the quickbrownfxjmpdvlazyg
[1, 2, 3, 4, dog, cat, 1.1, 2.2, 3.3]

Modula-2

Works with: ADW Modula-2 version any (Compile with the linker option Console Application).

<lang modula2> MODULE RemoveDuplicates;

FROM STextIO IMPORT

 WriteLn;

FROM SWholeIO IMPORT

 WriteInt;

TYPE

 TArrayRange = [1 .. 7];
 TArray = ARRAY TArrayRange OF INTEGER;

VAR

 DataArray, ResultArray: TArray;
 ResultIndex, LastResultIndex, Position: CARDINAL;
 IsNewNumber: BOOLEAN;

BEGIN

 (* Set the data. *);
 DataArray[1] := 1;
 DataArray[2] := 2;
 DataArray[3] := 2;
 DataArray[4] := 3;
 DataArray[5] := 4;
 DataArray[6] := 5;
 DataArray[7] := 5;

 ResultArray[1] := DataArray[1];
 LastResultIndex := 1;
 Position := 1;
 WHILE Position < HIGH(DataArray) DO
   INC(Position);
   IsNewNumber := TRUE;
   ResultIndex := 1;
   WHILE (ResultIndex <= LastResultIndex) AND IsNewNumber DO
     IF DataArray[Position] = ResultArray[ResultIndex] THEN
       IsNewNumber := FALSE;
     END;
     INC(ResultIndex);
   END;
   IF IsNewNumber THEN
     INC(LastResultIndex);
     ResultArray[LastResultIndex] := DataArray[Position];
   END
 END;
 FOR ResultIndex := 1 TO LastResultIndex DO
   WriteInt(ResultArray[ResultIndex], 1);
   WriteLn;
 END;

END RemoveDuplicates. </lang>

MUMPS

We'll take advantage of the fact that an array can only have one index of any specific value. Sorting into canonical order is a side effect. If the indices are strings containing the separator string, they'll be split apart.

<lang MUMPS>REMDUPE(L,S)

;L is the input listing
;S is the separator between entries
;R is the list to be returned
NEW Z,I,R
FOR I=1:1:$LENGTH(L,S) SET Z($PIECE(L,S,I))=""
;Repack for return
SET I="",R=""
FOR  SET I=$O(Z(I)) QUIT:I=""  SET R=$SELECT($L(R)=0:I,1:R_S_I)
KILL Z,I
QUIT R</lang>

Example:

USER>W $$REMDUPE^ROSETTA("1,2,3,4,5,2,5,""HELLO"",42,""WORLD""",",")
1,2,3,4,5,42,"HELLO","WORLD"

Nanoquery

After executing, the list 'unique' will contain only the unique items. <lang nanoquery>items = {1, 2, 3, "a", "b", "c", 2, 3, 4, "b", "c", "d"} unique = {}

for item in items

   if not item in unique
       unique.append(item)
   end

end</lang>

Neko

<lang ActionScript>/**

Remove duplicate elements, in Neko

/

var original = $array(1, 2, 1, 4, 5, 2, 15, 1, 3, 4)

/* Create a table with only unique elements from the array */ var dedup = function(a) {

   var size = $asize(a)
   var hash = $hnew(size)
   while size > 0 {
       var v = a[size - 1]
       var k = $hkey(v)
       $hset(hash, k, v, null)
       size -= 1
   }
   return hash

}

/* Show the original list and the unique values */ $print(original, "\n") var show = function(k, v) $print(v, " ") $hiter(dedup(original), show) $print("\n")</lang>

Output:

prompt$ nekoc remove-duplicates.neko
prompt$ neko remove-duplicates.n
[1,2,1,4,5,2,15,1,3,4]
1 2 3 4 5 15

Nemerle

<lang Nemerle>using System.Console;

module RemDups {

   Main() : void
   {
       def nums = array[1, 4, 6, 3, 6, 2, 7, 2, 5, 2, 6, 8];
       def unique = $[n | n in nums].RemoveDuplicates();
       WriteLine(unique);
   }

}</lang>

NetRexx

This sample takes advantage of the NetRexx built-in Rexx object's indexed string capability (associative arrays). Rexx indexed strings act very like hash tables: <lang NetRexx>/* NetRexx */ options replace format comments java crossref symbols nobinary

-- Note: Task requirement is to process "arrays". The following converts arrays into simple lists of words: -- Putting the resulting list back into an array is left as an exercise for the reader. a1 = [2, 3, 5, 7, 11, 13, 17, 19, 'cats', 222, -100.2, +11, 1.1, +7, '7.', 7, 5, 5, 3, 2, 0, 4.4, 2] a2 = [1, 2, 3, 'a', 'b', 'c', 2, 3, 4, 'b', 'c', 'd'] a3 = ['Now', 'is', 'the', 'time', 'for', 'all', 'good', 'men', 'to', 'come', 'to', 'the', 'aid', 'of', 'the', 'party.'] x = 0 lists = x = x + 1; lists[0] = x; lists[x] = array2wordlist(a1) x = x + 1; lists[0] = x; lists[x] = array2wordlist(a2) x = x + 1; lists[0] = x; lists[x] = array2wordlist(a3)

loop ix = 1 to lists[0]

 nodups_list = remove_dups(lists[ix])
 say ix.right(4)':' lists[ix]
 say .right(4)':' nodups_list
 say
 end ix

return

-- ============================================================================= method remove_dups(list) public static

 newlist = 
 nodups = '0'
 loop w_ = 1 to list.words()
   ix = list.word(w_)
   nodups[ix] = nodups[ix] + 1 -- we can even collect a count of dups if we want
   end w_
 loop k_ over nodups
   newlist = newlist k_
   end k_

 return newlist.space

-- ============================================================================= method array2wordlist(ra = Rexx[]) public static

 wordlist = 
 loop r_ over ra
   wordlist = wordlist r_
   end r_

 return wordlist.space

</lang>

Output:

   1: 2 3 5 7 11 13 17 19 cats 222 -100.2 11 1.1 7 7. 7 5 5 3 2 0 4.4 2
    : 13 2 3 17 19 7. 4.4 5 222 7 -100.2 1.1 cats 0 11

   2: 1 2 3 a b c 2 3 4 b c d
    : c 2 d 3 4 a b 1

   3: Now is the time for all good men to come to the aid of the party.
    : Now aid for men to the party. come time of is all good

NewLISP

<lang NewLISP>(unique '(1 2 3 a b c 2 3 4 b c d))</lang>

Nial

<lang nial>uniques := [1, 2, 3, 'a', 'b', 'c', 2, 3, 4, 'b', 'c', 'd'] cull uniques =+-+-+-+-+-+-+-+-+ =|1|2|3|a|b|c|4|d| =+-+-+-+-+-+-+-+-+</lang>

Using strand form <lang nial>cull 1 1 2 2 3 3 =1 2 3</lang>

Nim

<lang nim>import sequtils, algorithm, intsets

Go through the list, and for each element, check the rest of the list to see
if it appears again,

var items = @[1, 2, 3, 2, 3, 4, 5, 6, 7] echo deduplicate(items) # O(n^2)

proc filterDup(xs: openArray[int]): seq[int] =

 result = @[xs[0]]
 var last = xs[0]
 for x in xs[1..xs.high]:
   if x != last:
     result.add(x)
     last = x

Put the elements into a hash table which does not allow duplicates.

var s = initIntSet() for x in items:

 s.incl(x)

echo s

Sort the elements and remove consecutive duplicate elements.

sort(items, system.cmp[int]) # O(n log n) echo filterDup(items) # O(n)</lang>

Objeck

<lang objeck> use Structure;

bundle Default {

 class Unique {
   function : Main(args : String[]) ~ Nil {
     nums := [1, 1, 2, 3, 4, 4];
     unique := IntVector->New();

     each(i : nums) {
       n := nums[i];
       if(unique->Has(n) = false) {
         unique->AddBack(n);
       };
     };

     each(i : unique) {
       unique->Get(i)->PrintLine();
     };
   }
 }

} </lang>

Objective-C

<lang objc>NSArray *items = [NSArray arrayWithObjects:@"A", @"B", @"C", @"B", @"A", nil];

NSSet *unique = [NSSet setWithArray:items];</lang>

OCaml

<lang ocaml>let uniq lst =

 let unique_set = Hashtbl.create (List.length lst) in
 List.iter (fun x -> Hashtbl.replace unique_set x ()) lst;
 Hashtbl.fold (fun x () xs -> x :: xs) unique_set []

let _ =

 uniq [1;2;3;2;3;4]</lang>

Another solution (preserves order of first occurrence): <lang ocaml>let uniq lst =

 let seen = Hashtbl.create (List.length lst) in
 List.filter (fun x -> let tmp = not (Hashtbl.mem seen x) in
                       Hashtbl.replace seen x ();
                       tmp) lst

let _ =

 uniq [1;2;3;2;3;4]</lang>

Solution reversing list order : <lang ocaml>let uniq l =

 let rec tail_uniq a l =
   match l with
     | [] -> a
     | hd::tl -> tail_uniq (hd::a) (List.filter (fun x -> x  != hd) tl) in
 tail_uniq [] l</lang>

Works with: OCaml version 4.02+

<lang ocaml>List.sort_uniq compare [1;2;3;2;3;4]</lang>

Octave

<lang octave> input=[1 2 6 4 2 32 5 5 4 3 3 5 1 2 32 4 4]; output=unique(input); </lang>

Oforth

The list is converted to a set to remove duplicate elements

Output:

import: set

[ 1, 2, 3, 1, 2, 4, 1, 3, 4, 5 ] asSet println
[1, 2, 3, 4, 5]

ooRexx

<lang ooRexx>data = .array~of(1, 2, 3, "a", "b", "c", 2, 3, 4, "b", "c", "d") uniqueData = .set~new~union(data)~makearray~sort

say "Unique elements are" say do item over uniqueData

  say item

end</lang>

Output:

Unique elements are

1
2
3
4
a
b
c
d

Oz

The following solutions only works if the value type is allowed as a key in a dictionary.

<lang oz>declare

 fun {Nub Xs}
    D = {Dictionary.new}
 in
    for X in Xs do D.X := unit end
    {Dictionary.keys D}
 end

in

 {Show {Nub [1 2 1 3 5 4 3 4 4]}}</lang>

PARI/GP

Sort and remove duplicates. Other methods should be implemented as well. <lang parigp>rd(v)={

 vecsort(v,,8)

};</lang>

Pascal

<lang pascal>Program RemoveDuplicates;

const

 iArray: array[1..7] of integer = (1, 2, 2, 3, 4, 5, 5);

var

 rArray: array[1..7] of integer;
 i, pos, last: integer;
 newNumber: boolean;

begin

 rArray[1] := iArray[1];
 last := 1;
 pos := 1;
 while pos < high(iArray) do
 begin
   inc(pos);
   newNumber := true;
   for i := low(rArray) to last do
     if iArray[pos] = rArray[i] then
     begin
       newNumber := false;

break;

     end;
   if newNumber then
   begin
     inc(last);
     rArray[last] := iArray[pos];
   end;
 end;
 for i := low(rArray) to last do
   writeln (rArray[i]);

end.</lang>

Output:

% ./RemoveDuplicates
1
2
3
4
5

Perl

Library: List::MoreUtils MoreUtils

(this version even preserves the order of first appearance of each element) <lang perl>use List::MoreUtils qw(uniq);

my @uniq = uniq qw(1 2 3 a b c 2 3 4 b c d);</lang>

It is implemented like this: <lang perl>my %seen; my @uniq = grep {!$seen{$_}++} qw(1 2 3 a b c 2 3 4 b c d);</lang>

Note: the following two solutions convert elements to strings in the result, so if you give it references they will lose the ability to be dereferenced.

Alternately: <lang perl>my %hash = map { $_ => 1 } qw(1 2 3 a b c 2 3 4 b c d); my @uniq = keys %hash;</lang>

Alternately: <lang perl>my %seen; @seen{qw(1 2 3 a b c 2 3 4 b c d)} = (); my @uniq = keys %seen;</lang>

Phix

Standard builtin. The "STABLE" option preserves order of first occurence. Applies to any data type. The default option for unique(), "SORT", obviously produces sorted output, and there is one other recognised option, "PRESORTED", which can be used either to avoid an unnecessary sort, or to only remove adjacent duplicates.

with javascript_semantics
?join(unique(split("Now is the time for all good men to come to the aid of the party."),"STABLE"))
?unique({1, 2, 1, 4, 5, 2, 15, 1, 3, 4},"STABLE")
?unique({1, 2, 3, "a", "b", "c", 2, 3, 4, "b", "c", "d"},"STABLE")
?unique({1,3,2,9,1,2,3,8,8,1,0,2},"STABLE")
?unique("chthonic eleemosynary paronomasiac","STABLE")

Output:

"Now is the time for all good men to come aid of party."
{1,2,4,5,15,3}
{1,2,3,"a","b","c",4,"d"}
{1,3,2,9,8,0}
"chtoni elmsyarp"

Phixmonti

<lang Phixmonti>"Now" "is" "the" "time" "for" "all" "good" "men" "to" "come" "to" "the" "aid" "of" "the" "party." stklen tolist

0 tolist var newlist

len for

   get newlist over find
   not if
       swap 0 put var newlist
   else
       drop drop
   endif

endfor

newlist print drop</lang>

PHP

<lang php>$list = array(1, 2, 3, 'a', 'b', 'c', 2, 3, 4, 'b', 'c', 'd'); $unique_list = array_unique($list);</lang>

PicoLisp

There is a built-in function <lang PicoLisp>(uniq (2 4 6 1 2 3 4 5 6 1 3 5))</lang>

Output:

-> (2 4 6 1 3 5)

PL/I

<lang PL/I>*process mar(1,72); remdup: Proc options(main);

  declare t(20) fixed initial (6, 6, 1, 5, 6, 2, 1, 7,
     5, 22, 4, 19, 1, 1, 6, 8, 9, 10, 11, 12);
  declare (i, j, k, n, e) fixed;

  put skip list ('Input:');
  put edit ((t(k) do k = 1 to hbound(t))) (skip,20(f(3)));
  n = hbound(t,1);
  i = 0;

outer:

  do k = 1 to n;
     e = t(k);
     do j = k-1 to 1 by -1;
         if e = t(j) then iterate outer;
     end;
     i = i + 1;
     t(i) = e;
  end;

  put skip list ('Unique elements are:');
  put edit ((t(k) do k = 1 to i)) (skip,20(f(3)));

end;</lang>

Output:

Input:
  6  6  1  5  6  2  1  7  5 22  4 19  1  1  6  8  9 10 11 12
Unique elements are:
  6  1  5  2  7 22  4 19  8  9 10 11 12

Pop11

<lang pop11>;;; Initial array lvars ar = {1 2 3 2 3 4};

Create a hash table

lvars ht= newmapping([], 50, 0, true);

Put all array as keys into the hash table

lvars i; for i from 1 to length(ar) do

  1 -> ht(ar(i))

endfor;

Collect keys into a list

lvars ls = []; appdata(ht, procedure(x); cons(front(x), ls) -> ls; endprocedure);</lang>

PostScript

Library: initlib

[10 8 8 98 32 2 4 5 10 ] dup length dict begin  aload let* currentdict {pop} map end

</lang>

PowerShell

The common array for both approaches: <lang powershell>$data = 1,2,3,1,2,3,4,1</lang> Using a hash table to remove duplicates: <lang powershell>$h = @{} foreach ($x in $data) {

   $h[$x] = 1

} $h.Keys</lang> Sorting and removing duplicates along the way can be done with the Sort-Object cmdlet. <lang powershell>$data | Sort-Object -Unique</lang> Removing duplicates without sorting can be done with the Select-Object cmdlet. <lang powershell>$data | Select-Object -Unique</lang>

Prolog

<lang prolog>uniq(Data,Uniques) :- sort(Data,Uniques).</lang>

Example usage: <lang prolog>?- uniq([1, 2, 3, 2, 3, 4],Xs). Xs = [1, 2, 3, 4]</lang>

Because sort/2 is GNU prolog and not ISO here is an ISO compliant version: <lang prolog>member1(X,[H|_]) :- X==H,!. member1(X,[_|T]) :- member1(X,T).

distinct([],[]). distinct([H|T],C) :- member1(H,T),!, distinct(T,C). distinct([H|T],[H|C]) :- distinct(T,C).</lang>

Example usage: <lang prolog>?- distinct([A, A, 1, 2, 3, 2, 3, 4],Xs). Xs = [A, 1, 2, 3, 4]</lang>

PureBasic

Task solved with the built in Hash Table which are called Maps in PureBasic <lang PureBasic>NewMap MyElements.s()

For i=0 To 9 ;Mark 10 items at random, causing high risk of duplication items.

 x=Random(9)
 t$="Number "+str(x)+" is marked"
 MyElements(str(x))=t$   ; Add element 'X' to the hash list or overwrite if already included.

 Debug MyElements()

Next</lang> Output may look like this, e.g. duplicated items are automatically removed as they have the same hash value.

Number 0 is marked
Number 2 is marked
Number 5 is marked
Number 6 is marked

Python

If all the elements are hashable (this excludes list, dict, set, and other mutable types), we can use a set: <lang python>items = [1, 2, 3, 'a', 'b', 'c', 2, 3, 4, 'b', 'c', 'd'] unique = list(set(items))</lang>

or if we want to keep the order of the elements

<lang python>items = [1, 2, 3, 'a', 'b', 'c', 2, 3, 4, 'b', 'c', 'd'] unique = [] helperset = set() for x in items:

   if x not in helperset:
       unique.append(x)
       helperset.add(x)</lang>

If all the elements are comparable (i.e. <, >=, etc. operators; this works for list, dict, etc. but not for complex and many other types, including most user-defined types), we can sort and group: <lang python>import itertools items = [1, 2, 3, 'a', 'b', 'c', 2, 3, 4, 'b', 'c', 'd'] unique = [k for k,g in itertools.groupby(sorted(items))]</lang>

If both of the above fails, we have to use the brute-force method, which is inefficient: <lang python>items = [1, 2, 3, 'a', 'b', 'c', 2, 3, 4, 'b', 'c', 'd'] unique = [] for x in items:

   if x not in unique:
       unique.append(x)</lang>

another way of removing duplicate elements from a list, while preserving the order would be to use OrderedDict module like so <lang python> from collections import OrderedDict as od

print(list(od.fromkeys([1, 2, 3, 'a', 'b', 'c', 2, 3, 4, 'b', 'c', 'd']).keys())) </lang>

We may also need to specify the particular type (or degree) of uniqueness and duplication that is at issue. Case-insensitive, with strings ? Unique with respect to a particular key in the case of dictionaries ?

One way to do this is to require an equality predicate, or perhaps a key function, in addition to a list to be pruned. For example, using itertools.groupby, at the cost of needing a sort and discarding order: <lang python>from itertools import (groupby)

nubByKey :: (a -> b) -> [a] -> [a]

def nubByKey(k, xs):

   return list(list(v)[0] for _, v in groupby(sorted(xs, key=k), key=k))

xs = [

   'apple', 'apple',
   'ampersand', 'aPPLE', 'Apple',
   'orange', 'ORANGE', 'Orange', 'orange', 'apple'

] for k in [

   id,                      # default case sensitive uniqueness
   lambda x: x.lower(),     # case-insensitive uniqueness
   lambda x: x[0],          # unique first character (case-sensitive)
   lambda x: x[0].lower(),  # unique first character (case-insensitive)

]:

   print (
       nubByKey(k, xs)
   )</lang>

Output:

['apple', 'aPPLE', 'Apple', 'orange', 'ORANGE', 'Orange', 'ampersand']
['ampersand', 'apple', 'orange']
['Apple', 'ORANGE', 'apple', 'orange']
['apple', 'orange']

Or alternatively, using an equality predicate with a recursive function which scales less well, but does preserve order:

<lang python># nubByEq :: (a -> a -> Bool) -> [a] -> [a] def nubByEq(eq, xs):

   def go(yys, xxs):
       if yys:
           y = yys[0]
           ys = yys[1:]
           return go(ys, xxs) if (
               elemBy(eq, y, xxs)
           ) else (
               [y] + go(ys, [y] + xxs)
           )
       else:
           return []
   return go(xs, [])

elemBy :: (a -> a -> Bool) -> a -> [a] -> Bool

def elemBy(eq, x, xs):

   if xs:
       return eq(x, xs[0]) or elemBy(eq, x, xs[1:])
   else:
       return False

xs = [

   'apple', 'apple',
   'ampersand', 'aPPLE', 'Apple',
   'orange', 'ORANGE', 'Orange', 'orange', 'apple'

] for eq in [

   lambda a, b: a == b,                   # default case sensitive uniqueness
   lambda a, b: a.lower() == b.lower(),   # case-insensitive uniqueness
   lambda a, b: a[0] == b[0],             # unique first char (case-sensitive)
   lambda a, b: a[0].lower() == b[0].lower(),   # unique first char (any case)

]:

   print (
       nubByEq(eq, xs)
   )</lang>

A briefer definition of which might be in terms of filter: <lang python># nubBy :: (a -> a -> Bool) -> [a] -> [a] def nubBy(p, xs):

   def go(xs):
       if xs:
           x = xs[0]
           return [x] + go(
               list(filter(
                   lambda y: not p(x, y),
                   xs[1:]
               ))
           )
       else:
           return []
   return go(xs)</lang>

Output:

['apple', 'ampersand', 'aPPLE', 'Apple', 'orange', 'ORANGE', 'Orange']
['apple', 'ampersand', 'orange']
['apple', 'Apple', 'orange', 'ORANGE']
['apple', 'orange']

Qi

<lang qi> (define remove-duplicates

 []    -> []
 [A|R] -> (remove-duplicates R) where (element? A R)
 [A|R] -> [A|(remove-duplicates R)])

(remove-duplicates [a b a a b b c d e]) </lang>

Quackery

uniquewithsorts a nest and removes duplicate items from it.

The appropriate test for ordering two items is specified after uniquewith, so uniquewith > would be appropriate for a nest of numbers, and return the nest sorted in ascending numerical order, and uniquewith $< would be appropriate for a nest of strings sorted in descending order.

<lang Quackery> [ dup [] = iff ]else[ done

   ' sortwith nested 
   ]'[ nested join do
   behead dup dip nested rot
   witheach 
     [ tuck != if
         [ dup dip
           [ nested join ] ] ] 
   drop ]                      is uniquewith ( [ --> [ )
   
 ' [ 1 2 3 5 6 7 8 1 2 3 4 5 6 7 ] uniquewith > echo</lang>

Output:

[ 1 2 3 4 5 6 7 8 ]

R

<lang r>items <- c(1,2,3,2,4,3,2) unique (items)</lang>

Racket

Using the built-in function <lang Racket> -> (remove-duplicates '(2 1 3 2.0 a 4 5 b 4 3 a 7 1 3 x 2)) '(2 1 3 2.0 a 4 5 b 7 x) </lang>

Using a hash-table: <lang Racket> (define (unique/hash lst)

 (hash-keys (for/hash ([x (in-list lst)]) (values x #t))))

</lang>

Using a set: <lang Racket> (define unique/set (compose1 set->list list->set)) </lang>

A definition that works with arbitrary sequences and allows specification of an equality predicate.

<lang Racket> (define (unique seq #:same-test [same? equal?])

 (for/fold ([res '()])
           ([x seq] #:unless (memf (curry same? x) res))
   (cons x res)))

</lang>

-> (unique '(2 1 3 2.0 a 4 5 b 4 3 a 7 1 3 x 2))
'(1 2 3 a b x 4 5 7 2.0)
-> (unique '(2 1 3 2.0 4 5 4.0 3 7 1 3 2) #:same-test =)
'(7 5 4 3 1 2)
-> (unique #(2 1 3 2.0 4 5 4.0 3 7 1 3 2))
'(7 5 4 3 1 2)
-> (apply string (unique "absbabsbdbfbd"))
"fdsba"

Raku

(formerly Perl 6) <lang perl6>my @unique = [1, 2, 3, 5, 2, 4, 3, -3, 7, 5, 6].unique;</lang> Or just make a set of it. <lang perl6>set(1,2,3,5,2,4,3,-3,7,5,6).list</lang>

Raven

<lang raven>[ 1 2 3 'a' 'b' 'c' 2 3 4 'b' 'c' 'd' ] as items items copy unique print

list (8 items)

0 => 1
1 => 2
2 => 3
3 => "a"
4 => "b"
5 => "c"
6 => 4
7 => "d"</lang>

REBOL

<lang REBOL>print mold unique [1 $23.19 2 elbow 3 2 Bork 4 3 elbow 2 $23.19]</lang>

Output:

[1 $23.19 2 elbow 3 Bork 4]

Red

<lang Red>>> items: [1 "a" "c" 1 3 4 5 "c" 3 4 5] >> unique items == [1 "a" "c" 3 4 5]</lang>

REXX

Note that in REXX, strings are quite literal.

+7 is different from 7 (but compares numerically equal).
00 is different from 0 (but compares numerically equal).
─0 is different from 0 (but compares numerically equal).
7. is different from 7 (but compares numerically equal).
Ab is different from AB (but can compare equal if made case insensitive).

Note that all the REXX examples below don't care what type of element is used, integer, floating point, character, binary, ···

version 1, using method 1

<lang rexx>/*REXX program removes any duplicate elements (items) that are in a list (using a hash).*/ $= '2 3 5 7 11 13 17 19 cats 222 -100.2 +11 1.1 +7 7. 7 5 5 3 2 0 4.4 2' /*item list.*/ say 'original list:' $ say right( words($), 17, '─') 'words in the original list.' z=; @.= /*initialize the NEW list & index list.*/

    do j=1  for words($);       y= word($, j)   /*process the words (items) in the list*/
    if @.y==  then z= z y;    @.y= .          /*Not duplicated? Add to Z list,@ array*/
    end   /*j*/

say say 'modified list:' space(z) /*stick a fork in it, we're all done. */ say right( words(z), 17, '─') 'words in the modified list.'</lang>

output when using the default input list:

original list: 2 3 5 7 11 13 17 19 cats 222 -100.2 +11 1.1 +7 7. 7 5 5 3 2 0 4.4
──────────────23  words in the original list.

modified list: 2 3 5 7 11 13 17 19 cats 222 -100.2 +11 1.1 +7 7. 0 4.4
──────────────17  words in the modified list.

version 2, using a modified method 3

Instead of discard an element if it's a duplicated, it just doesn't add it to the new list.

Sorting of the list elements isn't necessary. <lang rexx>/*REXX program removes any duplicate elements (items) that are in a list (using a list).*/ $= '2 3 5 7 11 13 17 19 cats 222 -100.2 +11 1.1 +7 7. 7 5 5 3 2 0 4.4 2' /*item list.*/ say 'original list:' $ say right( words($), 17, '─') 'words in the original list.'

= words($) /*process all the words in the list. */

    do j=#  by -1  for #;     y= word($, j)                       /*get right-to-Left. */
    _= wordpos(y, $, j + 1);  if _\==0  then $= delword($, _, 1)  /*Dup? Then delete it*/
    end   /*j*/

say say 'modified list:' space($) /*stick a fork in it, we're all done. */ say right( words(z), 17, '─') 'words in the modified list.'</lang>

output is identical to the 1^st REXX version.

version 3, using method 3

<lang rexx>/*REXX program removes any duplicate elements (items) that are in a list (using 2 lists)*/ old = '2 3 5 7 11 13 17 19 cats 222 -100.2 +11 1.1 +7 7. 7 5 5 3 2 0 4.4 2' say 'original list:' old say right( words(old), 17, '─') 'words in the original list.' new= /*start with a clean (list) slate. */

    do j=1  for words(old);     _= word(old, j) /*process the words in the  OLD  list. */
    if wordpos(_, new)==0  then new= new _      /*Doesn't exist?  Then add word to NEW.*/
    end   /*j*/

say say 'modified list:' space(new) /*stick a fork in it, we're all done. */ say right( words(new), 17, '─') 'words in the modified list.'</lang>

output is identical to the 1^st REXX version.

version 4, using method 1 (hash table) via REXX stems

<lang rexx>/* REXX ************************************************************

26.11.2012 Walter Pachl
added: show multiple occurrences
- - - - /

old='2 3 5 7 11 13 17 19 cats 222 -100.2 +11 1.1 +7 7. 7 5 5',

   '3 2 0 4.4 2'

say 'old list='old say 'words in the old list=' words(old) new= found.=0 count.=0 Do While old<>

 Parse Var old w old                                                   
 If found.w=0 Then Do                                                  
   new=new w                                                           
   found.w=1                                                           
   End                                                                 
 count.w=count.w+1                                                     
 End

say 'new list='strip(new) say 'words in the new list=' words(new) Say 'Multiple occurrences:' Say 'occ word' Do While new<>

 Parse Var new w new                                                   
 If count.w>1 Then                                                     
   Say right(count.w,3) w                                              
 End</lang>

Output:

old list=2 3 5 7 11 13 17 19 cats 222 -100.2 +11 1.1 +7 7. 7 5 5 3 2 0 4.4 2 
words in the old list= 23                                                    
new list=2 3 5 7 11 13 17 19 cats 222 -100.2 +11 1.1 +7 7. 0 4.4             
words in the new list= 17                                                    
Multiple occurrences:                                                        
occ word                                                                     
  3 2                                                                        
  2 3                                                                        
  3 5                                                                        
  2 7

Ring

<lang ring> list = ["Now", "is", "the", "time", "for", "all", "good", "men", "to", "come", "to", "the", "aid", "of", "the", "party."] for i = 1 to len(list)

   for j = i + 1 to len(list) 
       if list[i] = list[j] del(list, j) j-- ok
   next

   see list[n] + " "

next see nl </lang> Output:

Now is the time for all good men to come aid of party.

Ruby

Ruby has an Array#uniq built-in method, which returns a new array by removing duplicate values in self. <lang ruby>ary = [1,1,2,1,'redundant',[1,2,3],[1,2,3],'redundant'] p ary.uniq # => [1, 2, "redundant", [1, 2, 3]]</lang>

You can also write your own uniq method. <lang ruby>class Array

 # used Hash
 def uniq1
   each_with_object({}) {|elem, h| h[elem] = true}.keys
 end
 # sort (requires comparable)
 def uniq2
   sorted = sort
   pre = sorted.first
   sorted.each_with_object([pre]){|elem, uniq| uniq << (pre = elem) if elem != pre}
 end
 # go through the list
 def uniq3
   each_with_object([]) {|elem, uniq| uniq << elem unless uniq.include?(elem)}
 end

end

ary = [1,1,2,1,'redundant',[1,2,3],[1,2,3],'redundant'] p ary.uniq1 #=> [1, 2, "redundant", [1, 2, 3]] p ary.uniq2 rescue nil # Error (not comparable) p ary.uniq3 #=> [1, 2, "redundant", [1, 2, 3]]

ary = [1,2,3,7,6,5,2,3,4,5,6,1,1,1] p ary.uniq1 #=> [1, 2, 3, 7, 6, 5, 4] p ary.uniq2 #=> [1, 2, 3, 4, 5, 6, 7] p ary.uniq3 #=> [1, 2, 3, 7, 6, 5, 4]</lang>

A version without implementing class declarations: <lang ruby>def unique(array)

   pure = Array.new
   for i in array
       flag = false
       for j in pure
           flag = true if j==i
       end
       pure << i unless flag
   end
   return pure

end

unique ["hi","hey","hello","hi","hey","heyo"] # => ["hi", "hey", "hello", "heyo"] unique [1,2,3,4,1,2,3,5,1,2,3,4,5] # => [1,2,3,4,5]</lang>

Run BASIC

<lang runbasic>a$ = "2 3 5 7 11 13 17 19 cats 222 -100.2 +11 1.1 +7 7. 7 5 5 3 2 0 4.4 2"

for i = 1 to len(a$)

 a1$ = word$(a$,i)
 if a1$ = "" then exit for
 for i1 = 1 to len(b$)
   if a1$ = word$(b$,i1) then [nextWord]
 next i1
 b$ = b$ + a1$ + " "

[nextWord] next i

print "Dups:";a$
print "No Dups:";b$</lang>

Dups:2 3 5 7 11 13 17 19 cats 222 -100.2 +11 1.1 +7 7. 7 5 5 3 2 0 4.4 2
No Dups:2 3 5 7 11 13 17 19 cats 222 -100.2 +11 1.1 +7 7. 0 4.4

Rust

<lang rust>use std::collections::HashSet; use std::hash::Hash;

fn remove_duplicate_elements_hashing<T: Hash + Eq>(elements: &mut Vec<T>) {

   let set: HashSet<_> = elements.drain(..).collect();
   elements.extend(set.into_iter());

}

fn remove_duplicate_elements_sorting<T: Ord>(elements: &mut Vec<T>) {

   elements.sort_unstable(); // order does not matter
   elements.dedup();

}

fn main() {

   let mut sample_elements = vec![0, 0, 1, 1, 2, 3, 2];
   println!("Before removal of duplicates : {:?}", sample_elements);
   remove_duplicate_elements_sorting(&mut sample_elements);
   println!("After removal of duplicates : {:?}", sample_elements);

}</lang>

Output:

Before removal of duplicates : [0, 0, 1, 1, 2, 3, 2]
After removal of duplicates : [1, 0, 3, 2]

Scala

<lang scala>val list = List(1,2,3,4,2,3,4,99) val l2 = list.distinct // l2: scala.List[scala.Int] = List(1,2,3,4,99)

val arr = Array(1,2,3,4,2,3,4,99) val arr2 = arr.distinct // arr2: Array[Int] = Array(1, 2, 3, 4, 99) </lang>

Scheme

<lang scheme>(define (remove-duplicates l)

 (cond ((null? l)
        '())
       ((member (car l) (cdr l))
        (remove-duplicates (cdr l)))
       (else
        (cons (car l) (remove-duplicates (cdr l))))))

(remove-duplicates '(1 2 1 3 2 4 5))</lang>

Alternative approach: <lang scheme>(define (remove-duplicates l)

 (do ((a '() (if (member (car l) a) a (cons (car l) a)))
      (l l (cdr l)))
   ((null? l) (reverse a))))

(remove-duplicates '(1 2 1 3 2 4 5))</lang>

The function 'delete-duplicates' is also available in srfi-1.

Seed7

<lang seed7>$ include "seed7_05.s7i";

const proc: main is func

 local
   const array integer: data is [] (1, 3, 2, 9, 1, 2, 3, 8, 8, 1, 0, 2);
   var set of integer: dataSet is (set of integer).value;
   var integer: number is 0;
 begin
   for number range data do
     incl(dataSet, number);
   end for;
   writeln(dataSet);
 end func;</lang>

Output:

{0, 1, 2, 3, 8, 9}

SETL

<lang SETL>items := [0,7,6,6,4,9,7,1,2,3,2]; print(unique(items));</lang> Output in arbitrary order (convert tuple->set then set->tuple): <lang SETL>proc unique(items);

 return [item: item in {item: item in items}];

end proc;</lang>

Preserving source order <lang SETL>proc unique(items);

 seen := {};
 return [item: item in items, nps in {#seen} | #(seen with:= item) > nps];

end proc;</lang>

<lang SETL>proc unique(items);

 seen := {};
 return [item: item in items, nps in {#seen} | #(seen with:= item) > nps];

end proc;</lang> <lang SETL>items := [0,7,6,6,4,9,7,1,2,3,2]; print(unique(items));</lang> Output in arbitrary order (convert tuple->set then set->tuple): <lang SETL>proc unique(items);

 return [item: item in {item: item in items}];

end proc;</lang>

SETL4

<lang SETL4> set = new('set')

Add all the elements of the array to the set.

add(set,array) </lang>

Sidef

<lang ruby>var ary = [1,1,2,1,'redundant',[1,2,3],[1,2,3],'redundant']; say ary.uniq.dump; say ary.last_uniq.dump;</lang>

Output:

[1, 2, 'redundant', [1, 2, 3]]
[2, 1, [1, 2, 3], 'redundant']

Slate

<lang slate>[|:s| #(1 2 3 4 1 2 3 4) >> s] writingAs: Set.

"==> {"Set traitsWindow" 1. 2. 3. 4}"</lang>

Smalltalk

<lang smalltalk>"Example of creating a collection" |a| a := #( 1 1 2 'hello' 'world' #symbol #another 2 'hello' #symbol ). a asSet.</lang>

Output:

Set (1 2 #symbol 'world' #another 'hello' )

the above has the disadvantage of loosing the original order (because Sets are unordered, and the hashing shuffles elements into an arbitrary order). When tried, I got:

Set('world' 1 #another 'hello' #symbol 2)

on my system. This can be avoided by using an ordered set (which has also O(n) complexity) as below:

Works with: Smalltalk/X

<lang smalltalk>|a| a := #( 1 1 2 'hello' 'world' #symbol #another 2 'hello' #symbol ). a asOrderedSet.</lang>

Output:

OrderedSet(1 2 'hello' 'world' #symbol #another)

Sparkling

<lang sparkling>function undupe(arr) { var t = {}; foreach(arr, function(key, val) { t[val] = true; });

var r = {}; foreach(t, function(key) { r[sizeof r] = key; });

return r; }</lang>

SQL

Works with: ORACLE 19c

This is not a particularly efficient solution, but it gets the job done.

<lang SQL> /* This code is an implementation of "Remove duplicate elements" in SQL ORACLE 19c p_in_str -- input string p_delimiter -- delimeter

/

with

 function remove_duplicate_elements(p_in_str in varchar2, p_delimiter in varchar2 default ',') return varchar2 is
 v_in_str varchar2(32767) := replace(p_in_str,p_delimiter,',');
 v_res    varchar2(32767);

begin

 --
 execute immediate 'select listagg(distinct cv,:p_delimiter) from (select (column_value).getstringval() cv from xmltable(:v_in_str))' 
    into v_res using p_delimiter, v_in_str;
 --
 return v_res;
 --

end;

--Test select remove_duplicate_elements('1, 2, 3, "a", "b", "c", 2, 3, 4, "b", "c", "d"') as res from dual union all select remove_duplicate_elements('3 9 1 10 3 7 6 5 2 7 4 7 4 2 2 2 2 8 2 10 4 9 2 4 9 3 4 3 4 7',' ') as res from dual

</lang>

Output:

1,2,3,4,a,b,c,d
1 10 2 3 4 5 6 7 8 9

Stata

Duplicates in a dataset

Stata can report duplicate lines, or remove them. See duplicates in Stata help.

<lang stata>. clear all . input x y 1 1 1 1 1 2 2 1 2 2 1 1 2 1 2 1 1 2 2 2 end

. duplicates drop x y, force . list

    +-------+
    | x   y |
    |-------|
 1. | 1   1 |
 2. | 1   2 |
 3. | 2   1 |
 4. | 2   2 |
    +-------+</lang>

Mata

The uniqrows function removes duplicate rows from a matrix.

<lang stata>. mata

a=1,1\1,1\1,2\2,1\2,2\1,1\2,1\2,1\1,2\2,2

a

       1   2
    +---------+
  1 |  1   1  |
  2 |  1   1  |
  3 |  1   2  |
  4 |  2   1  |
  5 |  2   2  |
  6 |  1   1  |
  7 |  2   1  |
  8 |  2   1  |
  9 |  1   2  |
 10 |  2   2  |
    +---------+

uniqrows(a)

      1   2
   +---------+
 1 |  1   1  |
 2 |  1   2  |
 3 |  2   1  |
 4 |  2   2  |
   +---------+</lang>

Swift

Requires elements to be hashable:

Works with: Swift version 1.2+

<lang swift>println(Array(Set([3,2,1,2,3,4])))</lang>

Output:

[2, 3, 1, 4]

Another solution (preserves order of first occurrence). Also requires elements to be hashable:

Works with: Swift version 1.2+

<lang swift>func uniq<T: Hashable>(lst: [T]) -> [T] {

 var seen = Set<T>(minimumCapacity: lst.count)
 return lst.filter { x in
   let unseen = !seen.contains(x)
   seen.insert(x)
   return unseen
 }

}

println(uniq([3,2,1,2,3,4]))</lang>

Output:

[3, 2, 1, 4]

Only requires elements to be equatable, but runs in O(n^2): <lang swift>func uniq<T: Equatable>(lst: [T]) -> [T] {

 var seen = [T]()
 return lst.filter { x in
   let unseen = find(seen, x) == nil
   if (unseen) {
     seen.append(x)
   }
   return unseen
 }

}

println(uniq([3,2,1,2,3,4]))</lang>

Output:

[3, 2, 1, 4]

Tcl

The concept of an "array" in Tcl is strictly associative - and since there cannot be duplicate keys, there cannot be a redundant element in an array. What is called "array" in many other languages is probably better represented by the "list" in Tcl (as in LISP). With the correct option, the lsort command will remove duplicates. <lang tcl>set result [lsort -unique $listname]</lang>

True BASIC

Translation of: GW-BASIC

Works with: QBasic

<lang basic> OPTION BASE 1 LET max = 10 DIM dat(10), res(10) FOR i = 1 TO max

   READ dat(i)

NEXT i

DATA 1, 2, 1, 4, 5, 2, 15, 1, 3, 4

LET res(1) = dat(1) LET count = 1 LET posic = 1 DO WHILE posic < max

  LET posic = posic + 1
  LET esnuevo = 1
  LET indice = 1
  DO WHILE (indice <= count) AND esnuevo = 1
     IF dat(posic) = res(indice) THEN LET esnuevo = 0
     LET indice = indice + 1
  LOOP
  IF esnuevo = 1 THEN
     LET count = count + 1
     LET res(count) = dat(posic)
  END IF

LOOP

FOR i = 1 TO count

   PRINT res(i);

NEXT i END </lang>

TSE SAL

<lang TSESAL> // // Go through the list, and for each element, check the rest of the list to see if it appears again, and discard it if it does. // INTEGER PROC FNBlockGetUniqueAllToBufferB( INTEGER bufferI )

INTEGER B = FALSE
INTEGER downB = TRUE
STRING s[255] = ""
IF ( NOT ( IsBlockInCurrFile() ) ) Warn( "Please mark a block" ) B = FALSE RETURN( B ) ENDIF // return from the current procedure if no block is marked
PushPosition()
PushBlock()
GotoBlockBegin()
WHILE ( ( IsCursorInBlock() ) AND ( downB ) )
 s = GetText( 1, MAXSTRINGLEN )
 PushPosition()
 PushBlock()
 GotoBufferId( bufferI )
 IF NOT LFind( s, "" )
  AddLine( s )
 ENDIF
 PopBlock()
 PopPosition()
 downB = Down()
ENDWHILE
PopPosition()
PopBlock()
B = TRUE
RETURN( B )

END // PROC Main()

INTEGER bufferI = 0
PushPosition()
bufferI = CreateTempBuffer()
PopPosition()
Message( FNBlockGetUniqueAllToBufferB( bufferI ) ) // gives e.g. TRUE
GotoBufferId( bufferI )

END </lang>

Output:

Input This is a test1 This is a test2 This is a test2 This is a test2 This is a test2 This is a test2 This is a test3 This is a test3 This is a test3 This is a test3 This is a test3 This is a test4

Output This is a test1 This is a test2 This is a test3 This is a test4

TUSCRIPT

<lang tuscript> $$ MODE TUSCRIPT list_old="b'A'A'5'1'2'3'2'3'4" list_sort=MIXED_SORT (list_old) list_new=REDUCE (list_sort) PRINT list_old PRINT list_new </lang>

Output:

(sorted)

b'A'A'5'1'2'3'2'3'4
1'2'3'4'5'A'b

or <lang tuscript> $$ MODE TUSCRIPT list_old="b'A'A'5'1'2'3'2'3'4" DICT list CREATE LOOP l=list_old DICT list LOOKUP l,num IF (num==0) DICT list ADD l ENDLOOP DICT list unload list list_new=JOIN (list) PRINT list_old PRINT list_new </lang>

Output:

b'A'A'5'1'2'3'2'3'4
b'A'5'1'2'3'4

UnixPipes

Assuming a sequence is represented by lines in a file. <lang bash>bash$ # original list bash$ printf '6\n2\n3\n6\n4\n2\n' 6 2 3 6 4 2 bash$ # made uniq bash$ printf '6\n2\n3\n6\n4\n2\n'|sort -n|uniq 2 3 4 6 bash$</lang>

or

<lang bash>bash$ # made uniq bash$ printf '6\n2\n3\n6\n4\n2\n'|sort -nu 2 3 4 6 bash$</lang>

Ursala

The algorithm is to partition the list by equality and take one representative from each class, which can be done by letting the built in partition operator, |=, use its default comparison relation. This works on lists of any type including character strings but the comparison is based only on structural equivalence. It's up to the programmer to decide whether that's a relevant criterion for equivalence or else specify a better one. <lang Ursala>#cast %s

example = |=hS& 'mississippi'</lang>

Output:

'mspi'

VBA

Hash Table Approach Input list (variant : Long, Double, Boolean and Strings) : Array(1.23456789101112E+16, True, False, True, "Alpha", 1, 235, 4, 1.25, 1.25, "Beta", 1.23456789101112E+16, "Delta", "Alpha", "Charlie", 1, 2, "Foxtrot", "Foxtrot", "Alpha", 235) <lang vb> Option Explicit

Sub Main() Dim myArr() As Variant, i As Long

   myArr = Remove_Duplicate(Array(1.23456789101112E+16, True, False, True, "Alpha", 1, 235, 4, 1.25, 1.25, "Beta", 1.23456789101112E+16, "Delta", "Alpha", "Charlie", 1, 2, "Foxtrot", "Foxtrot", "Alpha", 235))

'return :

   For i = LBound(myArr) To UBound(myArr)
       Debug.Print myArr(i)
   Next

End Sub

Private Function Remove_Duplicate(Arr As Variant) As Variant() Dim myColl As New Collection, Temp() As Variant, i As Long, cpt As Long

   ReDim Temp(UBound(Arr))
   For i = LBound(Arr) To UBound(Arr)
       On Error Resume Next
       myColl.Add CStr(Arr(i)), CStr(Arr(i))
       If Err.Number > 0 Then
           On Error GoTo 0
       Else
           Temp(cpt) = Arr(i)
           cpt = cpt + 1
       End If
   Next i
   ReDim Preserve Temp(cpt - 1)
   Remove_Duplicate = Temp

End Function</lang>

Output:

 1.23456789101112E+16 
True
False
Alpha
 1 
 235 
 4 
 1.25 
Beta
Delta
Charlie
 2 
Foxtrot

VBScript

Hash Table Approach <lang vb> Function remove_duplicates(list) arr = Split(list,",") Set dict = CreateObject("Scripting.Dictionary") For i = 0 To UBound(arr) If dict.Exists(arr(i)) = False Then dict.Add arr(i),"" End If Next For Each key In dict.Keys tmp = tmp & key & "," Next remove_duplicates = Left(tmp,Len(tmp)-1) End Function

WScript.Echo remove_duplicates("a,a,b,b,c,d,e,d,f,f,f,g,h") </lang>

Output:

a,b,c,d,e,f,g,h

Vedit macro language

The input "array" is an edit buffer where each line is one element. <lang vedit>Sort(0, File_Size) // sort the data While(Replace("^(.*)\N\1$", "\1", REGEXP+BEGIN+NOERR)){} // remove duplicates</lang>

Vim Script

<lang vim>call filter(list, 'count(list, v:val) == 1')</lang>

Visual FoxPro

<lang vfp> LOCAL i As Integer, n As Integer, lcOut As String CLOSE DATABASES ALL CLEAR CREATE CURSOR nums (num I) INDEX ON num TAG num COLLATE "Machine" SET ORDER TO 0 n = 50 RAND(-1) FOR i = 1 TO n

   INSERT INTO nums VALUES (RanInt(1, 10))

ENDFOR SELECT num, COUNT(num) As cnt FROM nums ; GROUP BY num INTO CURSOR grouped LIST OFF TO FILE grouped.txt NOCONSOLE lcOut = "" SCAN

   lcOut = lcOut + TRANSFORM(num) + ","

ENDSCAN lcOut = LEFT(lcOut, LEN(lcOut)-1) ? lcOut

FUNCTION RanInt(tnLow As Integer, tnHigh As Integer) As Integer RETURN INT((tnHigh - tnLow + 1)*RAND() + tnLow) ENDFUNC </lang>

Output:

NUM          COUNT
  1              6
  2              5
  3              6
  4              8
  5              4
  6              3
  7              8
  8              7
  9              3

Unique Values: 1,2,3,4,5,6,7,8,9

Vlang

<lang vlang>const list = 'a,1,a,b,b,c,d,e,d,0,f,f,5,f,g,h,1,3,3'

fn main() { println(unique(list)) }

fn unique(list string) []string {

   mut new_list := []string{}
   for word in list.split(',') {
       if new_list.any(it == word.str()) == false {
           new_list << word
       }
   }
   new_list.sort()
   return new_list

}</lang>

Output:

['0', '1', '3', '5', 'a', 'b', 'c', 'd', 'e', 'f', 'g', 'h']

Wart

<lang python>def (dedup l)

 let exists (table)
   collect+each x l
     unless exists.x
       yield x
     exists.x <- 1</lang>

Output:

dedup '(1 3 2 9 1 2 3 8 8 1 0 2)
=> (1 3 2 9 8 0)

Wortel

<lang wortel>@uniq [1 2 3 2 1 2 3] ; returns [1 2 3]</lang>

Wren

Library: Wren-sort

<lang ecmascript>import "/sort" for Sort

// Using a map - order of distinct items is undefined. var removeDuplicates1 = Fn.new { |a|

   if (a.count <= 1) return a
   var m = {}
   for (e in a) m[e] = true
   return m.keys.toList

}

// Sort elements and remove consecutive duplicates. var removeDuplicates2 = Fn.new { |a|

   if (a.count <= 1) return a
   Sort.insertion(a)
   for (i in a.count-1..1) {
       if (a[i] == a[i-1]) a.removeAt(i)
   }
   return a

}

// Iterate through list and for each element check if it occurs later in the list. // If it does, discard it. var removeDuplicates3 = Fn.new { |a|

   if (a.count <= 1) return a
   var i = 0
   while (i < a.count-1) {
       var j = i + 1
       while(j < a.count) {
           if (a[i] == a[j]) {
               a.removeAt(j)
           } else { 
               j = j + 1
           }
       }
       i = i + 1
   }
   return a

}

var a = [1,2,3,4,1,2,3,5,1,2,3,4,5] System.print("Original: %(a)") System.print("Method 1: %(removeDuplicates1.call(a.toList))") // copy original each time System.print("Method 2: %(removeDuplicates2.call(a.toList))") System.print("Method 3: %(removeDuplicates3.call(a.toList))")</lang>

Output:

Original: [1, 2, 3, 4, 1, 2, 3, 5, 1, 2, 3, 4, 5]
Method 1: [2, 1, 3, 5, 4]
Method 2: [1, 2, 3, 4, 5]
Method 3: [1, 2, 3, 4, 5]

XBS

<lang XBS>func RemoveDuplicates(Array){ set NewArray = []; foreach(k,v as Array){ if (NewArray->includes(v)){ Array->splice(toint(k),1); } else { NewArray->push(v); } } }

set Arr = ["Hello",1,2,"Hello",3,1]; log(Arr); RemoveDuplicates(Arr); log(Arr);</lang>

Output:

Hello,1,2,Hello,3,1
Hello,1,2,3

XPL0

<lang XPL0>code Text=12; \built-in routine to display a string of characters string 0; \use zero-terminated strings (not MSb terminated)

func StrLen(S); \Return number of characters in an ASCIIZ string char S; int I; for I:= 0, -1>>1-1 do \(limit = 2,147,483,646 if 32 bit, or 32766 if 16 bit)

       if S(I) = 0 then return I;

func Unique(S); \Remove duplicate bytes from string char S; int I, J, K, L; [L:= StrLen(S); \string length for I:= 0 to L-1 do \for all characters in string...

   for J:= I+1 to L-1 do               \scan rest of string for duplicates
       if S(I) = S(J) then             \if duplicate then
           [for K:= J+1 to L do        \ shift rest of string down (including
               S(K-1):= S(K);          \ terminating zero)
           L:= L-1                     \ string is now one character shorter
           ];

return S; \return pointer to string ];

Text(0, Unique("Pack my box with five dozen liquor jugs."))</lang>

Output:

Pack myboxwithfvedznlqurjgs.

Yabasic

<lang Yabasic>data "Now", "is", "the", "time", "for", "all", "good", "men", "to", "come", "to", "the", "aid", "of", "the", "party.", ""

do

   read p$
   if p$ = "" break
   if not instr(r$, p$) r$ = r$ + p$ + " "

loop

print r$</lang>

zkl

Using built ins: <lang zkl>zkl: Utils.Helpers.listUnique(T(1,3,2,9,1,2,3,8,8,"8",1,0,2,"8")) L(1,3,2,9,8,"8",0) zkl: "1,3,2,9,1,2,3,8,8,1,0,2".unique() ,012389</lang> Where listUnique is brute force: <lang zkl>fcn listUnique(xs){

  xs.reduce(fcn(us,s){us.holds(s) and us or us.append(s)},L()) }</lang>

Zoea

<lang Zoea> program: remove_duplicate_elements

 input: [1,2,1,3,2,4,1]
 output: [1,2,3,4]

</lang>

Zoea Visual

Remove duplicate elements

@@ Line 999: / Line 999: @@
 Idempotency is applied at rightmost inner brackets:
-We get rewrite  ''(1  1) -> 1''. The term is now:
+We get rewrite ''(1  1) -> 1''. The term is now:
 <lang>  (1  ((1  2)  1)) </lang>
@@ Line 1,006: / Line 1,006: @@
 Idempotency is applied at inner brackets  ''(1  2) ''.
-We get rewrite {{Imp}} ''(1  2) -> 2 ''. The term is now:
+We get rewrite  ''(1  2) -> 2 ''. The term is now:
 <lang>(1  (2  1))</lang>