User talk:Paddy3118/Vanity Search Updated

Vanity Search Updated

After the 4th of July RC updates, My RC Vanity Searcher needed to be updated due to HTML changes. I decided to add a modification, that shows the movement between the order a page was created in, vs the max views order called +/-.

<lang python> Rosetta Code Vanity search:

   How many new pages has someone created?

import urllib, re, pdb

user = 'Paddy3118'

site = 'http://www.rosettacode.org' nextpage = site + '/wiki/Special:Contributions/' + user nextpage_re = re.compile(

   #r'<a href="([^"]+)" title="[^"]+" rel="next">older '
   r'<a href="([^"]+)" title="[^"]+" rel="next"[^>]*>older '
   )

newpages = [] pagecount = 0 while nextpage:

   page = urllib.urlopen(nextpage)
   pagecount +=1
   nextpage = 
   for line in page:
       if not nextpage:
           # Search for URL to next page of results for download
           nextpage_match = re.search(nextpage_re, line)
           if nextpage_match:
               nextpage = (site + nextpage_match.groups()[0]).replace('&', '&')
               #print nextpage
               npline=line
       if '' in line:
           # extract N page name from title
           newpages.append(line.partition(' title="')[2].partition('"')[0])
   page.close()

nontalk = [p for p in newpages if not ':' in p] nontalk.reverse()

print "User: %s has created %i new pages of which %i were not Talk: pages, from approx %i edits" % (

   user, len(newpages), len(nontalk), pagecount*50 )

print "New pages created, in order, are:\n ", print "\n ".join(nontalk)

nextpage = site + '/w/index.php?title=Special:PopularPages' nextpage_re = re.compile(

   #r'<a href="([^"]+)" class="mw-nextlink">next '
   r'<a href="([^"]+)"[^>]* class="mw-nextlink"[^>]*>next'
   )

data_re = re.compile(

r'^

<a href="[^"]+" title="([^"]+)".*</a>.*$([0-9,]+) views$' ) title2rankviews = {} rank = 1 pagecount = 0 while nextpage: page = urllib.urlopen(nextpage) pagecount +=1 nextpage = for line in page: if not nextpage: # Search for URL to next page of results for download nextpage_match = re.search(nextpage_re, line) if nextpage_match: nextpage = (site + nextpage_match.groups()[0]).replace('&', '&') # print nextpage npline=line datamatch = re.search(data_re, line) if datamatch: title, views = datamatch.groups() views = int(views.replace(',', )) title2rankviews[title] = [rank, views] rank += 1 page.close() print "\n\n Highest page Ranks for user pages:\n" fmt = " %-4s %-6s %-3s %s" # rank, views, +/- title print fmt % ('RANK', 'VIEWS', '+/-', 'TITLE') fmt = " %4s %6s %+3i %s" # rank, views, +/- title highrank = [title2rankviews.get(t,[99999, 0]) + [t] for t in nontalk] highrank.sort() for i,x in enumerate(highrank): rank, view, title = x movement = nontalk.index(title) - i print fmt % (rank, view, movement, title) </lang>

Sample output on 12 July 2009

Since the 25th of June I've added two tasks: 'Send an email' and 'Topological (dependency) sort'.

With the new +/- column showing views w.r.t. creation order, you can see that the Knapsack Problem has captured the readers imagination. The Y combinator has a strong showing too.

Is it possible to get views in the last three months rather than total views? That would allow me to discount some of the bias due to date created.

User: Paddy3118 has created 63 new pages of which 31 were not Talk: pages, from approx 600 edits
New pages created, in order, are:
  Spiral
  Monty Hall simulation
  Web Scraping
  Sequence of Non-squares
  Anagrams
  Max Licenses In Use
  One dimensional cellular automata
  Conway's Game of Life
  Data Munging
  Data Munging 2
  Column Aligner
  Probabilistic Choice
  Knapsack Problem
  Yuletide Holiday
  Common number base formatting
  Octal
  Integer literals
  Command Line Interpreter
  First-class functions
  Y combinator
  Functional Composition
  Exceptions Through Nested Calls
  Look-and-say sequence
  Mutual Recursion
  Bulls and Cows
  Testing a Function
  Select
  Sort stability
  Moving Average
  Send an email
  Topological (dependency) sort


 Highest page Ranks for user pages:

  RANK VIEWS  +/- TITLE
    84   2792 +12 Knapsack Problem
   100   2629  +0 Monty Hall simulation
   107   2431  +5 Conway's Game of Life
   132   2013  +1 Anagrams
   199   1368  +2 One dimensional cellular automata
   211   1233  -3 Web Scraping
   218   1189  -1 Max Licenses In Use
   221   1158  -7 Spiral
   256    919  -5 Sequence of Non-squares
   261    900  +4 Yuletide Holiday
   267    866  +0 Column Aligner
   281    809  +8 Y combinator
   290    780  +6 First-class functions
   303    732  -4 Data Munging 2
   309    718  -6 Data Munging
   315    706  -4 Probabilistic Choice
   332    642  +1 Command Line Interpreter
   351    598  +6 Mutual Recursion
   365    568  +6 Bulls and Cows
   366    566  +1 Functional Composition
   388    501  -4 Integer literals
   407    463  -7 Common number base formatting
   418    436  +0 Look-and-say sequence
   446    375  -8 Octal
   455    347  -3 Exceptions Through Nested Calls
   493    248  +2 Sort stability
   525    207  -1 Testing a Function
   548    186  -1 Select
   559    174  +0 Moving Average
   703     69  +0 Send an email
   725     43  +0 Topological (dependency) sort

Sample output on 24 July 2009

What has happened in the last twelve days?

Ethiopian Multiplication is newly added and is off to a great start. Editors seem attracted to the challenge. I decided to add this as I was drawn by the simplicity of the algorithm. To make it a better RC task I decided on the restriction that you must create the function using a functions to half, double, and test for even-ness.
I guess viewers are tired of yet another sort: Topological (dependency) sort still languishes at the bottom.
I created my first #REDIRECT page: Russian Peasant Multiplication
Knapsack Problem still continues its rise in ranking (thank you).
At the top of my table, everything above Sequence of Non-squares has improved its rank. I tried to think of some other thing thats links them, but nothing special comes to mind.
I am not sure if spiral is unpopular, as it has a lot of views, was my first page ever, but appears seven places down from its original top spot - I remember writing it after creating the Python solution to Zig Zag.

Here's the table:

 Highest page Ranks for user pages:

  RANK VIEWS  +/- TITLE
    79   2898 +12 Knapsack Problem
    97   2707  +0 Monty Hall simulation
   106   2490  +5 Conway's Game of Life
   131   2059  +1 Anagrams
   196   1391  +2 One dimensional cellular automata
   208   1257  -3 Web Scraping
   216   1207  -1 Max Licenses In Use
   220   1182  -7 Spiral
   256    936  -5 Sequence of Non-squares
   259    915  +4 Yuletide Holiday
   269    878  +0 Column Aligner
   270    871  +8 Y combinator
   285    803  +6 First-class functions
   300    744  -4 Data Munging 2
   305    733  -6 Data Munging
   312    717  -4 Probabilistic Choice
   321    678  +1 Command Line Interpreter
   331    648  +6 Mutual Recursion
   348    614  +6 Bulls and Cows
   360    582  +1 Functional Composition
   384    525  -4 Integer literals
   396    486  -7 Common number base formatting
   407    466  +0 Look-and-say sequence
   432    410  -8 Octal
   450    367  -3 Exceptions Through Nested Calls
   479    275  +6 Ethiopian Multiplication
   491    255  +1 Sort stability
   516    218  -2 Testing a Function
   537    200  -2 Select
   540    192  -1 Moving Average
   669     90  -1 Send an email
   676     79  -1 Topological (dependency) sort
  99999      0  +0 Russian Peasant Multiplication

Sample output on 6 August 2009

So, what's been moving!
Well I am glad to see that I have to go right down to Sort stability before I see a task that is lower in the rankings than last time. Y combinator has leap-frogged even more of my tasks, from +8 to +11; and Bulls and Cows, and Ethiopian Multiplication are also doing well.

Newly created is First-class Numbers. When reviewing wp:First-class function, There were comments in the talk page about what languages really have FcF. I decided that rather than bring the discussion to RC, I would create a sister task - similar but using numbers and asking examples to compare the numbers solution with the functions solution. I guess people can still get around it, but I hope they follow the spirit of the task.


Highest page Ranks for user pages:


 RANK VIEWS  +/- TITLE
   76   3064 +12 Knapsack Problem
   93   2826  +0 Monty Hall simulation
  103   2611  +5 Conway's Game of Life
  127   2156  +1 Anagrams
  195   1442  +2 One dimensional cellular automata
  208   1285  -3 Web Scraping
  213   1259  -1 Max Licenses In Use
  219   1218  -7 Spiral
  249    999 +11 Y combinator
  259    966  -6 Sequence of Non-squares
  261    956  +3 Yuletide Holiday
  268    919  -1 Column Aligner
  274    861  +6 First-class functions
  298    785  -5 Data Munging
  301    776  -3 Probabilistic Choice
  302    770  -6 Data Munging 2
  315    729  +1 Command Line Interpreter
  317    721  +7 Bulls and Cows
  321    711  +5 Mutual Recursion
  350    631  +1 Functional Composition
  373    562  -4 Integer literals
  393    508  -7 Common number base formatting
  398    501  +0 Look-and-say sequence
  424    446  -8 Octal
  437    427  +7 Ethiopian Multiplication
  451    392  -4 Exceptions Through Nested Calls
  496    276  +1 Sort stability
  499    267  -2 Testing a Function
  502    265  +0 Moving Average
  529    226  -3 Select
  576    160  -1 Send an email
  590    146  -1 Topological (dependency) sort
  720     25  +1 First-class Numbers
 99999      0  -1 Russian Peasant Multiplication

Sample output on 29 August 2009

The big change that warrants its own section below, is that Ethiopian Multiplication is at the head off the list of pages I started, and ranked number three on RC as a whole.

Now if only I can discover the recipe and start some more, equally attractive pages...


Highest page Ranks for user pages:
 RANK VIEWS  +/- TITLE
    3  17327 +31 Ethiopian Multiplication
   71   3351 +11 Knapsack Problem
   89   2943  -1 Monty Hall simulation
  104   2726  +4 Conway's Game of Life
  120   2392  +0 Anagrams
  191   1547  +1 One dimensional cellular automata
  206   1409  -4 Web Scraping
  210   1324  -2 Max Licenses In Use
  219   1273  -8 Spiral
  228   1171 +10 Y combinator
  256   1007  +3 Yuletide Holiday
  261    999  -8 Sequence of Non-squares
  262    990  -2 Column Aligner
  272    947  +5 First-class functions
  288    852  -6 Data Munging
  293    826  +9 Bulls and Cows
  294    823  -7 Data Munging 2
  301    812  -6 Probabilistic Choice
  302    808  -1 Command Line Interpreter
  306    789  +4 Mutual Recursion
  337    694  +0 Functional Composition
  369    623  -5 Integer literals
  385    562  -8 Common number base formatting
  388    553  -1 Look-and-say sequence
  421    480  -9 Octal
  440    438  -4 Exceptions Through Nested Calls
  465    365  +2 Average/Simple moving average
  485    323  +0 Sort stability
  487    313  -3 Testing a Function
  517    261  -3 Select
  518    261  -1 Send an email
  559    211  -1 Topological (dependency) sort
  620    147  +3 Simple Random Distribution Checker
  661    128  +0 First-class Numbers
  667    121  +0 Seven-dice from Five-dice
  681    105  +1 List Flattening
 99999      0  -4 Russian Peasant Multiplication

Sample output on 17 September 2009

Well, Ethiopian Multiplication is now the most popular page after the front page. Scary! (As I will need to live up to it).

I still like the explanation of the knapsack problem after all these months, so its nice to see it doing well.


Highest page Ranks for user pages:
 RANK VIEWS  +/- TITLE
    2  29238 +31 Ethiopian Multiplication
   68   3599 +11 Knapsack Problem
   81   3186  -1 Monty Hall simulation
   93   3008  +4 Conway's Game of Life
  117   2534  +0 Anagrams
  190   1605  +1 One dimensional cellular automata
  201   1507  -4 Web Scraping
  214   1367  -2 Max Licenses In Use
  220   1319  -8 Spiral
  226   1285 +10 Y combinator
  254   1085  +3 Yuletide Holiday
  256   1058  -1 Column Aligner
  258   1046  +6 First-class functions
  260   1032 -10 Sequence of Non-squares
  278    929 +10 Bulls and Cows
  281    915  -7 Data Munging
  286    887  +1 Command Line Interpreter
  289    864  -8 Data Munging 2
  291    859  +5 Mutual Recursion
  297    836  -8 Probabilistic Choice
  333    747  +0 Functional Composition
  360    673  -5 Integer literals
  374    625  +0 Look-and-say sequence
  375    623  -9 Common number base formatting
  406    523  -9 Octal
  417    503  -4 Exceptions Through Nested Calls
  434    451  +2 Average/Simple moving average
  460    358  +0 Sort stability
  463    351  -3 Testing a Function
  465    346  +0 Send an email
  479    310  -4 Select
  480    310  -1 Topological (dependency) sort
  515    228  +4 List Flattening
  518    218  +2 Simple Random Distribution Checker
  525    191  +0 Seven-dice from Five-dice
  526    191  -2 First-class Numbers
 99999      0  -4 Russian Peasant Multiplication