Remove duplicate elements
From Rosetta Code
You are encouraged to solve this task according to the task description, using any language you may know.
There are basically three approaches seen here:
- Put the elements into a hash table which does not allow duplicates. The complexity is O(n) on average, and O(n2) worst case. This approach requires a hash function for your type (which is compatible with equality), either built-in to your language, or provided by the user.
- Sort the elements and remove consecutive duplicate elements. The complexity of the best sorting algorithms is O(n log n). This approach requires that your type be "comparable", i.e., have an ordering. Putting the elements into a self-balancing binary search tree is a special case of sorting.
- Go through the list, and for each element, check the rest of the list to see if it appears again, and discard it if it does. The complexity is O(n2). The up-shot is that this always works on any type (provided that you can test for equality).
[edit] Ada
Works with: GNAT version GPL 2007
with Ada.Containers.Ordered_Sets;
with Ada.Text_IO; use Ada.Text_IO;
procedure Unique_Set is
package Int_Sets is new Ada.Containers.Ordered_Sets(Integer);
use Int_Sets;
Nums : array (Natural range <>) of Integer := (1,2,3,4,5,5,6,7,1);
Unique : Set;
Set_Cur : Cursor;
Success : Boolean;
begin
for I in Nums'range loop
Unique.Insert(Nums(I), Set_Cur, Success);
end loop;
Set_Cur := Unique.First;
loop
Put_Line(Item => Integer'Image(Element(Set_Cur)));
exit when Set_Cur = Unique.Last;
Set_Cur := Next(Set_Cur);
end loop;
end Unique_Set;
[edit] APL
Works with: Dyalog APL The primitive monad ∪ means "unique", so:
∪ 1 2 3 1 2 3 4 1
1 2 3 4
Works with: APL2
w←1 2 3 1 2 3 4 1
((⍳⍨w)=⍳⍴w)/w
1 2 3 4
[edit] AppleScript
unique({1, 2, 3, "a", "b", "c", 2, 3, 4, "b", "c", "d"})
on unique(x)
set R to {}
repeat with i in x
if i is not in R then set end of R to i's contents
end repeat
return R
end unique
[edit] AutoHotkey
Built in Sort has an option to remove duplicates
a = 1,2,1,4,5,2,15,1,3,4
Sort, a, a, NUD`,
MsgBox % a ; 1,2,3,4,5,15
[edit] AWK
We produce an array a with duplicates from a string; then index a second array b with the contents of a, so that duplicates make only one entry; then produce a string with the keys of b, which is finally output.
$ awk 'BEGIN{split("a b c d c b a",a);for(i in a)b[a[i]]=1;r="";for(i in b)r=r" "i;print r}'
a b c d
[edit] C
Since there's no way to know ahead of time how large the new data structure will need to be, we'll return a linked list instead of an array.
#include <stdio.h>
#include <stdlib.h>
struct list_node {int x; struct list_node *next;};
typedef struct list_node node;
node * uniq(int *a, unsigned alen)
{if (alen == 0) return NULL;
node *start = malloc(sizeof(node));
if (start == NULL) exit(EXIT_FAILURE);
start->x = a[0];
start->next = NULL;
for (int i = 1 ; i < alen ; ++i)
{node *n = start;
for (;; n = n->next)
{if (a[i] == n->x) break;
if (n->next == NULL)
{n->next = malloc(sizeof(node));
n = n->next;
if (n == NULL) exit(EXIT_FAILURE);
n->x = a[i];
n->next = NULL;
break;}}}
return start;}
int main(void)
{int a[] = {1, 2, 1, 4, 5, 2, 15, 1, 3, 4};
for (node *n = uniq(a, 10) ; n != NULL ; n = n->next)
printf("%d ", n->x);
puts("");
return 0;}
[edit] C++
This version uses std::set, which requires its element type be comparable using the < operator.
#include <set>
#include <iostream>
using namespace std;
int main() {
typedef set<int> TySet;
int data[] = {1, 2, 3, 2, 3, 4};
TySet unique_set(data, data + 6);
cout << "Set items:" << endl;
for (TySet::iterator iter = unique_set.begin(); iter != unique_set.end(); iter++)
cout << *iter << " ";
cout << endl;
}
This version uses hash_set, which is part of the SGI extension to the Standard Template Library. It is not part of the C++ standard library. It requires that its element type have a hash function.
Works with: GCC
#include <ext/hash_set>
#include <iostream>
using namespace std;
int main() {
typedef __gnu_cxx::hash_set<int> TyHash;
int data[] = {1, 2, 3, 2, 3, 4};
TyHash unique_set(data, data + 6);
cout << "Set items:" << endl;
for (TyHash::iterator iter = unique_set.begin(); iter != unique_set.end(); iter++)
cout << *iter << " ";
cout << endl;
}
This version uses unordered_set, which is part of the TR1, which is likely to be included in the next version of C++. It is not part of the C++ standard library. It requires that its element type have a hash function.
Works with: GCC
#include <tr1/unordered_set>
#include <iostream>
using namespace std;
int main() {
typedef tr1::unordered_set<int> TyHash;
int data[] = {1, 2, 3, 2, 3, 4};
TyHash unique_set(data, data + 6);
cout << "Set items:" << endl;
for (TyHash::iterator iter = unique_set.begin(); iter != unique_set.end(); iter++)
cout << *iter << " ";
cout << endl;
}
Alternative method working directly on the array:
#include <iostream>
#include <iterator>
#include <algorithm>
// helper template
template<typename T> T* end(T (&array)[size]) { return array+size; }
int main()
{
int data[] = { 1, 2, 3, 2, 3, 4 };
std::sort(data, end(data));
int* new_end = std::unique(data, end(data));
std::copy(data, new_end, std::ostream_iterator<int>(std::cout, " ");
std::cout << std::endl;
}
[edit] C#
Works with: C# version 2+
int[] nums = { 1, 1, 2, 3, 4, 4 };
List<int> unique = new List<int>();
foreach (int n in nums)
if (!unique.Contains(n))
unique.Add(n);
Works with: C# version 3+
int[] nums = {1, 1, 2, 3, 4, 4};
int[] unique = nums.Distinct().ToArray();
[edit] Clojure
user=> (distinct [1 3 2 9 1 2 3 8 8 1 0 2])
(1 3 2 9 8 0)
user=>
[edit] Common Lisp
To remove duplicates non-destructively:
(remove-duplicates '(1 3 2 9 1 2 3 8 8 1 0 2))
> (9 3 8 1 0 2)
Or, to remove duplicates in-place:
(delete-duplicates '(1 3 2 9 1 2 3 8 8 1 0 2))
> (9 3 8 1 0 2)
[edit] D
void main() {
int[] data = [1, 2, 3, 2, 3, 4];
int[int] hash;
foreach(el; data)
hash[el] = 0;
}
[edit] E
[1,2,3,2,3,4].asSet().getElements()
[edit] Erlang
List = [1, 2, 3, 2, 2, 4, 5, 5, 4, 6, 6, 5].
Set = sets:from_list(List).
[edit] F#
The simplest way is to build a set from the given array (this actually works for any enumerable input sequence type, not just arrays):
set [|1;2;3;2;3;4|]
gives:
val it : Set<int> = seq [1; 2; 3; 4]
[edit] Factor
USING: sets ;
V{ 1 2 1 3 2 4 5 } prune .
V{ 1 2 3 4 5 }
[edit] Forth
Forth has no built-in hashtable facility, so the easiest way to achieve this goal is to take the "uniq" program as an example.
The word uniq, if given a sorted array of cells, will remove the duplicate entries and return the new length of the array. For simplicity, uniq has been written to process cells (which are to Forth what "int" is to C), but could easily be modified to handle a variety of data types through deferred procedures, etc.
The input data is assumed to be sorted.
\ Increments a2 until it no longer points to the same value as a1
\ a3 is the address beyond the data a2 is traversing.
: skip-dups ( a1 a2 a3 -- a1 a2+n )
dup rot ?do
over @ i @ <> if drop i leave then
cell +loop ;
\ Compress an array of cells by removing adjacent duplicates
\ Returns the new count
: uniq ( a n -- n2 )
over >r \ Original addr to return stack
cells over + >r \ "to" addr now on return stack, available as r@
dup begin ( write read )
dup r@ <
while
2dup @ swap ! \ copy one cell
cell+ r@ skip-dups
cell 0 d+ \ increment write ptr only
repeat r> 2drop r> - cell / ;
Here is another implementation of "uniq" that uses a popular parameters and local variables extension words. It is structurally the same as the above implementation, but uses less overt stack manipulation.
: uniqv { a n \ r e -- n }
a n cells+ to e
a dup to r
\ the write address lives on the stack
begin
r e <
while
r @ over !
r cell+ e skip-dups to r
cell+
repeat
a - cell / ;
To test this code, you can execute:
create test 1 , 2 , 3 , 2 , 6 , 4 , 5 , 3 , 6 ,
here test - cell / constant ntest
: .test ( n -- ) 0 ?do test i cells + ? loop ;
test ntest 2dup cell-sort uniq .test
output
1 2 3 4 5 6 ok
[edit] Fortran
Fortran has no built-in hash functions or sorting functions but the code below implements the compare all elements algorithm.
program remove_dups
implicit none
integer :: example(12) ! The input
integer :: res(size(example)) ! The output
integer :: k ! The number of unique elements
integer :: i, j
example = [1, 2, 3, 2, 2, 4, 5, 5, 4, 6, 6, 5]
k = 1
res(1) = example(1)
outer: do i=2,size(example)
do j=1,k
if (res(j) == example(i)) then
! Found a match so start looking again
cycle outer
end if
end do
! No match found so add it to the output
k = k + 1
res(k) = example(i)
end do outer
write(*,advance='no',fmt='(a,i0,a)') 'Unique list has ',k,' elements: '
write(*,*) res(1:k)
end program remove_dups
[edit] Go
package main
import "fmt"
func uniq(list []int) []int {
unique_set := make(map[int] bool, len(list))
for _, x := range list {
unique_set[x] = true
}
result := make([]int, len(unique_set))
i := 0
for x := range unique_set {
result[i] = x
i++
}
return result
}
func main() {
fmt.Println(uniq([]int {1,2,3,2,3,4})) // prints: [3 1 4 2]
}
replace "int" with another type (must be boolean, numeric, string, pointer, function, interface, map, or channel type) to use for a list of another type
[edit] Groovy
def list = [1, 2, 3, 'a', 'b', 'c', 2, 3, 4, 'b', 'c', 'd']
assert list.size() == 12
println " Original List: ${list}"
// Filtering the List
list.unique()
assert list.size() == 8
println " Filtered List: ${list}"
list = [1, 2, 3, 'a', 'b', 'c', 2, 3, 4, 'b', 'c', 'd']
assert list.size() == 12
// Converting to Set
def set = new HashSet(list)
assert set.size() == 8
println " Set: ${set}"
// Converting to Order-preserving Set
set = new LinkedHashSet(list)
assert set.size() == 8
println "List-ordered Set: ${set}"
Output:
Original List: [1, 2, 3, a, b, c, 2, 3, 4, b, c, d]
Filtered List: [1, 2, 3, a, b, c, 4, d]
Set: [1, d, 2, 3, 4, b, c, a]
List-ordered Set: [1, 2, 3, a, b, c, 4, d]
[edit] Haskell
values = [1,2,3,2,3,4]
unique = List.nub values
[edit] HicEst
REAL :: nums(12)
CHARACTER :: workspace*100
nums = (1, 3, 2, 9, 1, 2, 3, 8, 8, 1, 0, 2)
WRITE(Text=workspace) nums ! convert to string
EDIT(Text=workspace, SortDelDbls=workspace) ! do the job for a string
READ(Text=workspace, ItemS=individuals) nums ! convert to numeric
WRITE(ClipBoard) individuals, "individuals: ", nums ! 6 individuals: 0 1 2 3 8 9 0 0 0 0 0 0
[edit] Icon and Unicon
[edit] Icon
This solution preserves the original order of the elements.
procedure main(args)
every write(!noDups(args))
end
procedure noDups(L)
every put(newL := [], notDup(set(),!L))
return newL
end
procedure notDup(cache, a)
if not member(cache, a) then {
insert(cache, a)
return a
}
end
A sample run is:
->noDups a b c d c a b e a b c d e ->
[edit] Unicon
This Icon solution works in Unicon.
[edit] IDL
non_repeated_values = array[uniq(array, sort( array))]
[edit] J
] a=: 4 5 ?@$ 13 NB. 4 by 5 matrix of numbers chosen from 0 to 12
4 3 2 8 0
1 9 5 1 7
6 3 9 9 4
2 1 5 3 2
, a NB. sequence of the same elements
4 3 2 8 0 1 9 5 1 7 6 3 9 9 4 2 1 5 3 2
~. , a NB. unique elements
4 3 2 8 0 1 9 5 7 6
The verb ~. removes duplicate items from any array (numeric, character, or other; vector, matrix, rank-n array). For example:
~. 'chthonic eleemosynary paronomasiac'
chtoni elmsyarp
Or
0 1 1 2 0 */0 1 2
0 0 0
0 1 2
0 1 2
0 2 4
0 0 0
~. 0 1 1 2 0 */0 1 2
0 0 0
0 1 2
0 2 4
[edit] Java
Works with: Java version 1.5
import java.util.Set;
import java.util.HashSet;
import java.util.Arrays;
Object[] data = {1, 2, 3, "a", "b", "c", 2, 3, 4, "b", "c", "d"};
Set<Object> uniqueSet = new HashSet<Object>(Arrays.asList(data));
Object[] unique = uniqueSet.toArray();
[edit] JavaScript
This uses the === "strict equality" operator, which does no type conversions (4 == "4" is true but 4 === "4" is false)
function unique(ary) {
// concat() with no args is a way to clone an array
var u = ary.concat().sort();
for (var i = 1; i < u.length; ) {
if (u[i-1] === u[i])
u.splice(i,1);
else
i++;
}
return u;
}
var ary = [1, 2, 3, "a", "b", "c", 2, 3, 4, "b", "c", "d", "4"];
var uniq = unique(ary);
for (var i = 0; i < uniq.length; i++)
print(uniq[i] + "\t" + typeof(uniq[i]));
1 - number 2 - number 3 - number 4 - number 4 - string a - string b - string c - string d - string
Or, extend the prototype for Array:
Array.prototype.unique = function() {
var u = this.concat().sort();
for (var i = 1; i < u.length; ) {
if (u[i-1] === u[i])
u.splice(i,1);
else
i++;
}
return u;
}
var uniq = [1, 2, 3, "a", "b", "c", 2, 3, 4, "b", "c", "d"].unique();
[edit] Logo
Works with: UCB Logo
show remdup [1 2 3 a b c 2 3 4 b c d] ; [1 a 2 3 4 b c d]
[edit] Lua
items = {1,2,3,4,1,2,3,4,"bird","cat","dog","dog","bird"}
flags = {}
io.write('Unique items are:')
for i=1,#items do
if not flags[items[i]] then
io.write(' ' .. items[i])
flags[items[i]] = true
end
end
io.write('\n')
Output:
Unique items are: 1 2 3 4 bird cat dog
[edit] MAXScript
uniques = #(1, 2, 3, "a", "b", "c", 2, 3, 4, "b", "c", "d")
for i in uniques.count to 1 by -1 do
(
id = findItem uniques uniques[i]
if (id != i) do deleteItem uniques i
)
[edit] Mathematica
Built-in function:
DeleteDuplicates[{0, 2, 1, 4, 2, 0, 3, 1, 1, 1, 0, 3}]
gives back:
{0, 2, 1, 4, 3}
Custom function (reordering of elements is possible):
NoDupes[input_List] := Split[Sort[input]][[All, 1]]
NoDupes[{0, 2, 1, 4, 2, 0, 3, 1, 1, 1, 0, 3}]
gives back:
{0, 1, 2, 3, 4}
[edit] MATLAB
MATLAB has a built-in function, "unique(list)", which performs this task.
Sample Usage:
>> unique([1 2 6 3 6 4 5 6])
ans =
1 2 3 4 5 6
[edit] MUMPS
We'll take advantage of the fact that an array can only have one index of any specific value. Sorting into canonical order is a side effect. If the indices are strings containing the separator string, they'll be split apart.
REMDUPE(L,S)Example:
;L is the input listing
;S is the separator between entries
;R is the list to be returned
NEW Z,I,R
FOR I=1:1:$LENGTH(L,S) SET Z($PIECE(L,S,I))=""
;Repack for return
SET I="",R=""
FOR SET I=$O(Z(I)) QUIT:I="" SET R=$SELECT($L(R)=0:I,1:R_S_I)
KILL Z,I
QUIT R
USER>W $$REMDUPE^ROSETTA("1,2,3,4,5,2,5,""HELLO"",42,""WORLD""",",")
1,2,3,4,5,42,"HELLO","WORLD"
[edit] Nial
uniques := [1, 2, 3, 'a', 'b', 'c', 2, 3, 4, 'b', 'c', 'd']
cull uniques
=+-+-+-+-+-+-+-+-+
=|1|2|3|a|b|c|4|d|
=+-+-+-+-+-+-+-+-+
Using strand form
cull 1 1 2 2 3 3
=1 2 3
[edit] Objective-C
NSArray *items = [NSArray arrayWithObjects:@"A", @"B", @"C", @"B", @"A", nil];
NSSet *unique = [NSSet setWithArray:items];
[edit] Objeck
use Structure;
bundle Default {
class Unique {
function : Main(args : String[]) ~ Nil {
nums := [1, 1, 2, 3, 4, 4];
unique := IntVector->New();
each(i : nums) {
n := nums[i];
if(unique->Has(n) = false) {
unique->AddBack(n);
};
};
each(i : unique) {
unique->Get(i)->PrintLine();
};
}
}
}
[edit] OCaml
let uniq lst =
let unique_set = Hashtbl.create (List.length lst) in
List.iter (fun x -> Hashtbl.replace unique_set x ()) lst;
Hashtbl.fold (fun x () xs -> x :: xs) unique_set []
let _ =
uniq [1;2;3;2;3;4]
[edit] Oz
The following solutions only works if the value type is allowed as a key in a dictionary.
declare
fun {Nub Xs}
D = {Dictionary.new}
in
for X in Xs do D.X := unit end
{Dictionary.keys D}
end
in
{Show {Nub [1 2 1 3 5 4 3 4 4]}}
[edit] Perl
Library: List::MoreUtilsMoreUtils (this version even preserves the order of first appearance of each element)
use List::MoreUtils qw(uniq);
my @uniq = uniq qw(1 2 3 a b c 2 3 4 b c d);
It is implemented like this:
my %seen;
my @uniq = grep {!$seen{$_}++} qw(1 2 3 a b c 2 3 4 b c d);
Note: the following two solutions convert elements to strings in the result, so if you give it references they will lose the ability to be dereferenced.
Alternately:
my %hash = map { $_ => 1 } qw(1 2 3 a b c 2 3 4 b c d);
my @uniq = keys %hash;
Alternately:
my %seen;
@seen{qw(1 2 3 a b c 2 3 4 b c d)} = ();
my @uniq = keys %seen;
[edit] Perl 6
Works with: Rakudo version #23 "Lisbon"
sub nub (@a) {
my @b;
none(@b) eqv $_ and push @b, $_ for @a;
return @b;
}
my @unique = nub [1, 2, 3, 5, 2, 4, 3, -3, 7, 5, 6];
Or just use the .uniq builtin.
[edit] PHP
$list = array(1, 2, 3, 'a', 'b', 'c', 2, 3, 4, 'b', 'c', 'd');
$unique_list = array_unique($list);
[edit] PicoLisp
There is a built-in function
(uniq (2 4 6 1 2 3 4 5 6 1 3 5))
Output:
-> (2 4 6 1 3 5)
[edit] PL/I
declare t(20) fixed initial (1, 5, 6, 2, 1, 7,
5, 22, 4, 19, 1, 1, 6, 6, 6, 8, 9, 10, 11, 12);
declare (i, j, k, n, e) fixed;
n = hbound(t,1);
i = 0;
outer:
do k = 1 to n;
e = t(k);
do j = k-1 to 1 by -1;
if e = t(j) then iterate outer;
end;
i = i + 1;
t(i) = e;
end;
put skip list ('Unique elements are:');
put edit ((t(k) do k = 1 to i)) (skip, f(11));
[edit] Pop11
;;; Initial array
lvars ar = {1 2 3 2 3 4};
;;; Create a hash table
lvars ht= newmapping([], 50, 0, true);
;;; Put all array as keys into the hash table
lvars i;
for i from 1 to length(ar) do
1 -> ht(ar(i))
endfor;
;;; Collect keys into a list
lvars ls = [];
appdata(ht, procedure(x); cons(front(x), ls) -> ls; endprocedure);
[edit] PowerShell
The common array for both approaches:
$data = 1,2,3,1,2,3,4,1
Using a hash table to remove duplicates:
$h = @{}
foreach ($x in $data) {
$h[$x] = 1
}
$h.Keys
Sorting and removing duplicates along the way can be done with the Sort-Object cmdlet.
$data | Sort-Object -Unique
[edit] Prolog
uniq(Data,Uniques) :- sort(Data,Uniques).
Example usage:
?- uniq([1, 2, 3, 2, 3, 4],Xs).
Xs = [1, 2, 3, 4]
[edit] PureBasic
Task solved with the built in Hash Table which are called Maps in PureBasic
NewMap MyElements.s()
For i=0 To 9 ;Mark 10 items at random, causing high risk of duplication items.
x=Random(9)
t$="Number "+str(x)+" is marked"
MyElements(str(x))=t$ ; Add element 'X' to the hash list or overwrite if already included.
Next
ForEach MyElements()
Debug MyElements()
Next
Output may look like this, e.g. duplicated items are automatically removed as they have the same hash value.
Number 0 is marked Number 2 is marked Number 5 is marked Number 6 is marked
[edit] Python
If all the elements are hashable (this excludes list, dict, set, and other mutable types), we can use a set:
items = [1, 2, 3, 'a', 'b', 'c', 2, 3, 4, 'b', 'c', 'd']
unique = list(set(items))
If all the elements are comparable (i.e. <, >=, etc. operators; this works for list, dict, etc. but not for complex and many other types, including most user-defined types), we can sort and group:
import itertools
items = [1, 2, 3, 'a', 'b', 'c', 2, 3, 4, 'b', 'c', 'd']
unique = [k for k,g in itertools.groupby(sorted(items))]
If both of the above fails, we have to use the brute-force method, which is inefficient:
items = [1, 2, 3, 'a', 'b', 'c', 2, 3, 4, 'b', 'c', 'd']
unique = []
for x in items:
if x not in unique:
unique.append(x)
See also http://www.peterbe.com/plog/uniqifiers-benchmark and http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/52560
[edit] R
items <- c(1,2,3,2,4,3,2)
unique (items)
[edit] REBOL
print mold unique [1 $23.19 2 elbow 3 2 Bork 4 3 elbow 2 $23.19]
Output:
[1 $23.19 2 elbow 3 Bork 4]
[edit] Raven
[ 1 2 3 'a' 'b' 'c' 2 3 4 'b' 'c' 'd' ] as items
items copy unique print
list (8 items)
0 => 1
1 => 2
2 => 3
3 => "a"
4 => "b"
5 => "c"
6 => 4
7 => "d"
[edit] Ruby
ary = [1,1,2,1,'redundant',[1,2,3],[1,2,3],'redundant']
uniq_ary = ary.uniq
# => [1, 2, "redundant", [1, 2, 3]]
[edit] Scala
val list = List(1,2,3,4,2,3,4,99)
val l2 = list.distinct
// l2: scala.List[scala.Int] = List(1,2,3,4,99)
[edit] Scheme
(define (remove-duplicates l)
(do ((a '() (if (member (car l) a) a (cons (car l) a)))
(l l (cdr l)))
((null? l) (reverse a))))
(remove-duplicates (list 1 2 1 3 2 4 5))
(1 2 3 4 5)
Some implementations provide remove-duplicates in their standard library.
[edit] SETL
items := [0,7,6,6,4,9,7,1,2,3,2];
print(unique(items));
Output in arbitrary order (convert tuple->set then set->tuple):
proc unique(items);
return [item: item in {item: item in items}];
end proc;
Preserving source order
proc unique(items);
seen := {};
return [item: item in items, nps in {#seen} | #(seen with:= item) > nps];
end proc;
[edit] Slate
[|:s| #(1 2 3 4 1 2 3 4) >> s] writingAs: Set.
"==> {"Set traitsWindow" 1. 2. 3. 4}"
[edit] Smalltalk
|aCollection|
"Example of creating a collection"
a := #( 1 1 2 'hello' 'world' #symbol #another 2 'hello' #symbol ).
a asSet printNl.
Output:
Set (1 2 #symbol 'world' #another 'hello' )
[edit] Tcl
The concept of an "array" in Tcl is strictly associative - and since there cannot be duplicate keys, there cannot be a redundant element in an array. What is called "array" in many other languages is probably better represented by the "list" in Tcl (as in LISP). With the correct option, the lsort command will remove duplicates.
set result [lsort -unique $listname]
[edit] UnixPipes
Assuming a sequence is represented by lines in a file.
bash$ # original list
bash$ printf '6\n2\n3\n6\n4\n2\n'
6
2
3
6
4
2
bash$ # made uniq
bash$ printf '6\n2\n3\n6\n4\n2\n'|sort -n|uniq
2
3
4
6
bash$
or
bash$ # original list
bash$ printf '6\n2\n3\n6\n4\n2\n'
6
2
3
6
4
2
bash$ # made uniq
bash$ printf '6\n2\n3\n6\n4\n2\n'|sort -u
2
3
4
6
bash$
[edit] Ursala
The algorithm is to partition the list by equality and take one representative from each class, which can be done by letting the built in partition operator, |=, use its default comparison relation. This works on lists of any type including character strings but the comparison is based only on structural equivalence. It's up to the programmer to decide whether that's a relevant criterion for equivalence or else specify a better one.
#cast %s
example = |=hS& 'mississippi'
output:
'mspi'
[edit] Vedit macro language
The input "array" is an edit buffer where each line is one element.
Sort(0, File_Size) // sort the data
While(Replace("^(.*)\N\1$", "\1", REGEXP+BEGIN+NOERR)){} // remove duplicates
