Rosetta Code/Rank languages by number of users

Rosetta Code/Rank languages by number of users is a draft programming task. It is not yet considered ready to be promoted as a complete task, for reasons that should be found in its talk page.

Sort most popular programming languages based on the number of users on Rosetta Code. Show the languages with at least 100 users.

A way to solve the task:

Users of a language X are those referenced in the page https://rosettacode.org/wiki/Category:X_User, or preferably https://rosettacode.org/mw/index.php?title=Category:X_User&redirect=no to avoid redirections. In order to find the list of such categories, it's possible to first parse the entries of http://rosettacode.org/mw/index.php?title=Special:Categories&limit=5000. Then download and parse each language users category to count the users.

Sample output on 17 december 2017:

Language      Users   Rank
--------------------------
C               373      1
C++             261      2
Java            257      3
Python          243      4
JavaScript      228      5
PHP             163      6
Perl            162      7
SQL             131      8
UNIX Shell      120      9
BASIC           118     10
C sharp         113     11
Pascal          109     12

A Rosetta Code user usually declares using a language with the mylang template. This template is expected to appear on the User page. However, in some cases it appears in a user Talk page. It's not necessary to take this into account. For instance, among the 373 C users in the table above, 3 are actually declared in a Talk page.

Perl 6

Works with: Rakudo version 2017.11

Use the mediawiki API rather than web scraping since it is much faster and less resource intensive. Show languages with more than 25 users since that is still a pretty short list and to demonstrate how tied rankings are handled. Change the $minimum parameter to adjust what the cut-off point will be.

This is all done in a single pass; ties are not detected until a language has the same count as a previous one, so ties are marked by a T next to the count indicating that this language has the same count as the previous.

<lang perl6>use HTTP::UserAgent; use URI::Escape; use JSON::Fast;

my $client = HTTP::UserAgent.new;

my $url = 'http://rosettacode.org/mw';

my $start-time = now;

say "========= Generated: { DateTime.new(time) } =========";

my $lang = 1; my $rank = 0; my $last = 0; my $tie = ' '; my $minimum = 25;

.say for

   mediawiki-query(
       $url, 'pages',
       :generator<categorymembers>,
       :gcmtitle<Category:Language users>,
       :gcmlimit<350>,
       :rawcontinue(),
       :prop<categoryinfo>
   )

   .map({ %( count => .<categoryinfo><pages> || 0,
             lang  => .<title>.subst(/^'Category:' (.+) ' User'/, ->$/ {$0}) ) })

   .sort( { -.<count>, .<lang> } )

   .map( { last if .<count> < $minimum; display(.<count>, .<lang>) } );

say "========= elapsed: {(now - $start-time).round(.01)} seconds =========";

sub display ($count, $which) {

   if $last != $count { $last = $count; $rank = $lang; $tie = ' ' } else { $tie = 'T' };
   sprintf "#%3d  Rank: %2d %s  with %-4s users:  %s", $lang++, $rank, $tie, $count, $which;

}

sub mediawiki-query ($site, $type, *%query) {

   my $url = "$site/api.php?" ~ uri-query-string(
       :action<query>, :format<json>, :formatversion<2>, |%query);
   my $continue = ;

   gather loop {
       my $response = $client.get("$url&$continue");
       my $data = from-json($response.content);
       take $_ for $data.<query>.{$type}.values;
       $continue = uri-query-string |($data.<query-continue>{*}».hash.hash or last);
   }

}

sub uri-query-string (*%fields) {

   join '&', %fields.map: { "{.key}={uri-escape .value}" }

}</lang>

Output:

========= Generated: 2018-05-19T12:08:51Z =========
#  1  Rank:  1    with 378  users:  C
#  2  Rank:  2    with 267  users:  Java
#  3  Rank:  3    with 265  users:  C++
#  4  Rank:  4    with 251  users:  Python
#  5  Rank:  5    with 233  users:  JavaScript
#  6  Rank:  6    with 165  users:  PHP
#  7  Rank:  6 T  with 165  users:  Perl
#  8  Rank:  8    with 133  users:  SQL
#  9  Rank:  9    with 124  users:  UNIX Shell
# 10  Rank: 10    with 118  users:  BASIC
# 11  Rank: 11    with 116  users:  C sharp
# 12  Rank: 12    with 112  users:  Pascal
# 13  Rank: 13    with 99   users:  Haskell
# 14  Rank: 14    with 92   users:  Ruby
# 15  Rank: 15    with 73   users:  Fortran
# 16  Rank: 16    with 67   users:  Visual Basic
# 17  Rank: 17    with 62   users:  Prolog
# 18  Rank: 18    with 61   users:  Scheme
# 19  Rank: 19    with 58   users:  Common Lisp
# 20  Rank: 20    with 55   users:  Lua
# 21  Rank: 21    with 53   users:  AWK
# 22  Rank: 22    with 52   users:  HTML
# 23  Rank: 23    with 46   users:  Assembly
# 24  Rank: 24    with 44   users:  Batch File
# 25  Rank: 25    with 42   users:  Bash
# 26  Rank: 25 T  with 42   users:  X86 Assembly
# 27  Rank: 27    with 40   users:  Erlang
# 28  Rank: 28    with 38   users:  Forth
# 29  Rank: 29    with 37   users:  MATLAB
# 30  Rank: 30    with 36   users:  Lisp
# 31  Rank: 31    with 35   users:  J
# 32  Rank: 31 T  with 35   users:  Visual Basic .NET
# 33  Rank: 33    with 34   users:  Delphi
# 34  Rank: 34    with 33   users:  Ada
# 35  Rank: 34 T  with 33   users:  Brainf***
# 36  Rank: 34 T  with 33   users:  Objective-C
# 37  Rank: 34 T  with 33   users:  Tcl
# 38  Rank: 38    with 32   users:  APL
# 39  Rank: 38 T  with 32   users:  R
# 40  Rank: 40    with 31   users:  COBOL
# 41  Rank: 41    with 30   users:  Go
# 42  Rank: 42    with 28   users:  Perl 6
# 43  Rank: 43    with 27   users:  Clojure
# 44  Rank: 43 T  with 27   users:  Mathematica
# 45  Rank: 45    with 25   users:  AutoHotkey
========= elapsed: 1.41 seconds =========

Stata

<lang stata>copy "http://rosettacode.org/mw/index.php?title=Special:Categories&limit=5000" categ.html, replace import delimited categ.html, delim("@") enc("utf-8") clear keep if ustrpos(v1,"/wiki/Category:") & ustrpos(v1,"_User") gen i = ustrpos(v1,"href=") gen j = ustrpos(v1,char(34),i+1) gen k = ustrpos(v1,char(34),j+1) gen s = usubstr(v1,j+7,k-j-7) replace i = ustrpos(v1,"title=") replace j = ustrpos(v1,">",i+1) replace k = ustrpos(v1," User",j+1) gen lang = usubstr(v1,j+1,k-j) keep s lang gen users=.

forval i=1/`c(N)' { local s preserve copy `"https://rosettacode.org/mw/index.php?title=`=s[`i']'&redirect=no"' `i'.html, replace import delimited `i'.html, delim("@") enc("utf-8") clear count if ustrpos(v1,"/wiki/User") local m `r(N)' restore replace users=`m' in `i' erase `i'.html }

drop s gsort -users lang list if users>=100 save rc_users, replace</lang>

Output

     +---------------------+
     |        lang   users |
     |---------------------|
  1. |          C      373 |
  2. |        C++      261 |
  3. |       Java      257 |
  4. |     Python      243 |
  5. | JavaScript      228 |
     |---------------------|
  6. |        PHP      163 |
  7. |       Perl      162 |
  8. |        SQL      131 |
  9. | UNIX Shell      120 |
 10. |      BASIC      118 |
     |---------------------|
 11. |    C sharp      113 |
 12. |     Pascal      109 |
     +---------------------+

zkl

Uses libraries cURL and YAJL (yet another json library) <lang zkl>const MIN_USERS=60; var [const] CURL=Import("zklCurl"), YAJL=Import("zklYAJL")[0];

fcn rsGet{

  continueValue,r,curl := "",List, CURL();
  do{	// eg 5 times
     page:=("http://rosettacode.org/mw/api.php?action=query"
       "&generator=categorymembers&prop=categoryinfo"

"&gcmtitle=Category%%3ALanguage%%20users" "&rawcontinue=&format=json&gcmlimit=350" "%s").fmt(continueValue);

     page=curl.get(page);
     page=page[0].del(0,page[1]);  // get rid of HTML header
     json:=YAJL().write(page).close();
     json["query"]["pages"].pump(r.append,'wrap(x){ x=x[1];
        //("2708",Dictionary(title:Category:C User,...,categoryinfo:D(pages:373,size:373,...)))

// or title:SmartBASIC if((pgs:=x.find("categoryinfo")) and (pgs=pgs.find("pages")) and pgs>=MIN_USERS) return(pgs,x["title"].replace("Category:","").replace(" User","")); return(Void.Skip);

     });
     if(continueValue=json.find("query-continue",""))
       continueValue=String("&gcmcontinue=",

continueValue["categorymembers"]["gcmcontinue"]);

  }while(continueValue);
  r

}

allLangs:=rsGet(); allLangs=allLangs.sort(fcn(a,b){ a[0]>b[0] }); println("========== ",Time.Date.prettyDay()," =========="); foreach n,pgnm in ([1..].zip(allLangs))

  { println("#%3d with %4s users: %s".fmt(n,pgnm.xplode())) }</lang>

Output:

========== Wednesday, the 20th of December 2017 ==========
#  1 with  373 users: C
#  2 with  261 users: C++
#  3 with  257 users: Java
#  4 with  243 users: Python
#  5 with  228 users: JavaScript
#  6 with  163 users: PHP
#  7 with  162 users: Perl
#  8 with  131 users: SQL
#  9 with  120 users: UNIX Shell
# 10 with  118 users: BASIC
# 11 with  113 users: C sharp
# 12 with  109 users: Pascal
# 13 with   98 users: Haskell
# 14 with   91 users: Ruby
# 15 with   71 users: Fortran
# 16 with   65 users: Visual Basic
# 17 with   60 users: Scheme