Showing posts with label SPARQL. Show all posts
Showing posts with label SPARQL. Show all posts

Monday, July 14, 2014

Harald's Original Miscellany - More Truth about Football - Part 4

Finally, Germany has won the Soccer Worldcup 2014. Therefore, also our little statistics on soccer will come to an end with the post today. You might ask yourself what kind of data is left for soccer players in Wikipedia and DBpedia. Well, unfortunately only a little. But, we will try to make something out of it. Last time, we've ask for the number of team changes and the correlation to popularity vs. scored goals for soccer players. What is left, if we look at the available data?





We have the data about the years in which the football players were active or have played in their national soccer team. Let's start with the national team years [1]:

nationalyears NumPlayers
8 3
7 20
6 85
5 371
4 1070
3 2479
2 6116
1 28211

Well, it was obvious that the most players have only 1 or two years in the national team. But, there are exceptional players who achieved even 8 years. But, who are these long term players? [2]:

nationalyears Player Team
8 Wojciech Łobodziński "Poland"@en
8 Wojciech Łobodziński "Poland Under 16"@en
8 Wojciech Łobodziński "Poland Under 17"@en
8 Wojciech Łobodziński "Poland Under 21"@en
8 Wojciech Łobodziński "Poland Under 18"@en
8 Santiago Cañizares "Spain Under-17"@en
8 Santiago Cañizares "Spain Under-16"@en
8 Santiago Cañizares "Spain Under-21"@en
8 Santiago Cañizares "Spain Under-18"@en
8 Santiago Cañizares "Spain Under-23"@en
8 Santiago Cañizares "Spain Under-19"@en
8 Santiago Cañizares "Spain Under-20"@en
7 Aydın Yılmaz "Turkey Under-21"@en
7 Aydın Yılmaz "Turkey"@en
7 Aydın Yılmaz "Turkey"@en
7 Aydın Yılmaz "Turkey Under-19s"@en
7 Aydın Yılmaz "Turkey Under-17"@en
7 Aydın Yılmaz "Turkey A2"@en
7 Ismael Urzaiz "Spain Under-17"@en
7 Ismael Urzaiz "Spain Under-16"@en

Possibly you will never have heard of Poland's Wojciech Łobodziński or Spain's Santiago Cañizares. Here another flaw in the data becomes visible. There is no such thing as the unique national team. We have "under 16", "under 17", "under 18", and so on... So you start your career already with 15 and after 8 years you would be 23 and possibly be in the "real" national team.

Let's have a look at the active years of players. Unfortunately, here the data is rather messy [3]: 
person From To
http://dbpedia.org/resource/Marta_(footballer) 2000 9223372036854775807
http://dbpedia.org/resource/Carlos_Alberto_Gomes_de_Lima 2006 200720082008
http://dbpedia.org/resource/Blake_Camp 2008 200420052006
http://dbpedia.org/resource/Birgit_Prinz 1998 200320042005
http://dbpedia.org/resource/Dejan_Damjanovi%C4%87 1998 20112012
http://dbpedia.org/resource/Inka_Grings 1995 20092010
http://dbpedia.org/resource/Breno_Silva 2003 20092010
http://dbpedia.org/resource/Vin%C3%ADcius_Calamari 2007 20092010
http://dbpedia.org/resource/Samuel_Jos%C3%A9_da_Silva_Vieira 1994 20082009
http://dbpedia.org/resource/Chad_Marshall 2004 20082009
http://dbpedia.org/resource/Michael_Parkhurst 2003 20072008
http://dbpedia.org/resource/Geison_Rodrigues_Marrote 2004 20072008
http://dbpedia.org/resource/Dejan_Stankovi%C4%87 1994 20062010
http://dbpedia.org/resource/Glauber_Da_Silva 2001 20062007
http://dbpedia.org/resource/Hugo_Sarmiento 1999 20032007
http://dbpedia.org/resource/Obafemi_Martins 2000 20032004
http://dbpedia.org/resource/Heslley_Couto 2005 20032006
http://dbpedia.org/resource/Dalibor_Filipovi%C4%87 1992 20022003
http://dbpedia.org/resource/Dmytro_Zayko 2005 20022004
http://dbpedia.org/resource/Aviv_Volnerman 1998 20012004

Possibly this is because of people not only writing single year's into the Wikipedia infoboxes, but time spans and other things. In the list above are only the Top 20. Just remove the LIMIT from the SPARQL query and further down you will find more valid data.

OK, let's come to our last problem related to football. Is there a correlation between the height of a player (simply because we have that data) and the number of achieved goals [4]?
Height sumgoals
243.84 1
23.0 59
215.9 1
206.0 0
204.0 0
203.2 24
203.2 47
203.0 0
203.0 0
203.0 51
202.0 19
202.0 0
202.0 15
201.0 20
201.0 0
201.0 0
200.66 11
200.66 48
200.66 0
200.66 0
200.66 0
200.66 124
200.66 146
200.66 0
200.66 1
200.66 16
200.66 0
200.66 102
200.66 6
200.66 0
200.0 0
200.0 20
199.0 0
199.0 0
199.0 78
199.0 47
199.0 0
199.0 62
199.0 32
199.0 84

It is rather difficult to recognize something in this data. You see heights and the number of goals that player of that height have scored. Fascinating that there seem to be a significant number of players that are taller than 2 meters. I guess that the list leader with 2.43 meters is just incorrect data. Now. this is so many data that we have to visualize it to recognize something....

Correlation between height of soccer players and scored goals
In the diagram to the left you see the soccer players height (x-axis) vs. the number of scored goals (y-axis). Interesting thing to notice is that the heights approximately show a Gaussian distribution, i.e most players have a "middle" height, on the extremes there are only a few. Well, there seems to be an exception. on the outer left you will notice a large fraction of players with a height of 1.52m with goal scores ranging among 0 to 300. This is extraordinary, because I have no idea what kind of group this is or if there is simply an error in the data again. What I noticed is that among this group there seems to be a larger fraction of female Asian soccer players. Maybe they are responsible for that large number of outliers, but this requires further investigation.

Alas, I want to add one last table. For each player there is the information on which position she or he is playing. Of course Wikipedia authors are far from any sort of agreement how to name the player's position. Thus, there is a rather huge variety. Nevertheless, I will leave you with the table to make any sense from it. Enjoy [5]:
position number avheight avgoals
midfielder 4450 176.822828644929299 18.077078651685393
defender 3429 181.37523034450506 8.058326042578011
striker 2643 180.169761559433703 65.213015512674991
goalkeeper 2086 186.186064557569709 0.174496644295302
forward 1787 178.865346269495465 46.388919977616116
centre back 786 185.7391350821381 10.651399491094148
winger 648 174.69354896725695 27.583333333333333
attacking midfielder 490 175.897877222177931 38.006122448979592
left back 461 177.661735253323698 6.995661605206074
defensive midfielder 427 179.741452037869347 11.978922716627635
right back 405 177.829456545982828 7.301234567901235
central defender 196 185.005408462213008 8.897959183673469
central midfielder 161 178.223105294363834 20.099378881987578
full back 158 174.659366173080242 6.759493670886076
centre forward 140 179.09685701642717 80.792857142857143
left winger 113 174.790176753449224 28.398230088495575
defender, midfielder 102 179.789412255380659 10.607843137254902
inside forward 96 172.094789346059145 64.083333333333333
defender/midfielder 82 179.178292995545911 11.463414634146341
defender / midfielder 79 180.188100887250282 15.050632911392405
left-back 65 175.642922504131603 7.692307692307692
right winger 62 175.247418803553424 30.096774193548387
centre-back 56 521.327500479561941 9.375
right-back 54 176.101110952871815 6.111111111111111
midfielder/forward 52 172.013075924836657 22.557692307692308
midfield 52 176.982691251314588 30.673076923076923
defender (retired) 50 181.37344055175781 14.48
second striker 48 174.673332850138346 63.854166666666667
full-back 44 175.908408771861666 2.681818181818182
striker / winger 41 178.235609845417295 49.219512195121951

Of course I limited the table to 30 rows. Interestingly, the average height of goalkeepers is larger than for midfielders or strikers. As expected, strikers on the average score more goals than defenders or goalkeepers.


SPARQL Queries and original data:

Thursday, July 03, 2014

Harald's Original Miscellany - More Truth about Football - Part 3

To change the team means to earn more
money...what about the football millionaires?
How often do they change the team?
Are you ready for more statistics on your favorite kind of sports? Well, data is fun, and obviously Big Data means Big Fun. There are lot's of interesting things to discover while exploring data, and wikipedia (i.e. dbpedia for the insiders) provides all the necessary means.

Have you ever wondered about this kind of slave trade in professional football? Well, I wouldn't exactly call the transfer of a millionaire to a higher paying job a 'slave trade'. But, have you ever thought about the following question: Do the real good (and well paid) players more often change the team - or is it vice versa, that teams try to get rid of players that have a bad season or are on the decline? Who knows? Let's have a look on the data:
TeamChanges NumPlayers
16 2
15 26
14 84
13 287
12 792
11 2247
10 3848
9 5109
8 6464
7 8110
6 9790
5 11264
4 11837
3 11448
2 10961
1 6515
Here, we have a table providing an overview about how many players (in wikipedia) have changed their team for how many times [1]. Obviously, it seems to be some kind of Gaussian distribution with a peak between 2 and 6 team switches. OK, what about the players? Where are the top players listed in this table? Well, David Beckham switched team 11 times according to wikipedia, Cristiano Ronaldo 8 times, Thierry Henry 9 times. At least these numbers are above average which we had identified to be between 2 and 6. This seems to give proof to our original assumption.

person TeamChanges popularity
http://dbpedia.org/resource/Cristiano_Ronaldo 8 1794
http://dbpedia.org/resource/David_Beckham 11 1572
http://dbpedia.org/resource/Thierry_Henry 9 1414
http://dbpedia.org/resource/Lionel_Messi 7 1404
http://dbpedia.org/resource/Wayne_Rooney 5 1343
http://dbpedia.org/resource/Frank_Lampard 5 1188
http://dbpedia.org/resource/Pel%C3%A9 4 1111
http://dbpedia.org/resource/Didier_Drogba 8 1047
http://dbpedia.org/resource/Ronaldo 10 1037
http://dbpedia.org/resource/Michael_Owen 6 1011

But, we get a better overview, if we look at the average popularity of each switching group in the table [3]:
TeamChanges NumPlayers avgindegree
16 2 107.5
15 26 76.769230769230769
14 84 47.297619047619048
13 287 60.885017421602787
12 788 45.073604060913706
11 2235 37.206263982102908
10 3800 32.577631578947368
9 5023 30.939080230937687
8 6332 29.685881238155401
7 7935 25.499054820415879
6 9525 23.188346456692913
5 10886 18.682895462061363
4 11423 14.937844699290904
3 10842 11.131802250507286
2 10089 7.704628803647537
1 5534 5.19588001445609
As we had originally thought, on the average, the popularity of the players rises with the number of team changes. Although the top group with 16 changes is far from the highest possible popularity scores (as e.g. 1572 for David Beckham). Hmm, maybe there is a correlation among the number of achieved goals with the number of team changes? Is it more likely that a top goal hunter switches team more often? Let's have a look [4]:
TeamChanges NumPlayers AvgGoals
16 2 22.5
15 26 28.807692307692308
14 83 37.855421686746988
13 287 38.062717770034843
12 781 39.939820742637644
11 2205 38.625850340136054
10 3784 35.646141649048626
9 4980 32.826907630522088
8 6296 29.489517153748412
7 7826 24.455788397648863
6 9314 21.937835516426884
5 10471 18.655142775284118
4 10627 16.317869577491296
3 9612 12.756450270495214
2 7193 9.911858751564021
1 4624 4.444204152249135

Looks interesting. Top goal scorer have 9 to 14 team switches. This is way above the average. Thus, the more goals you score, the more often you will have the chance of being transferred (and thus earn more money). Players that don't score goals will obviously not be transferred (that often).

References:

Wednesday, June 25, 2014

Harald's Original Miscellany - The Truth about Football - Part 2

John Terry Celebration Meme, read on and you will understand...
Of course you always wanted to know, who is the best football player of all times. Sure this might be a question about which real football afficionados might argue forever. Also Wikipedia will not be able to give you the definite answer. But, we can play around with the available data and maybe we find out something interesting about football players again ...

But, first at all, I want to say thank you to Kingsley Idehen, who gave me the hint for my SPARQL query links to use the parameter "qtxt=" instead of "query=", which enables others to see the original query and to use it for further data explorations. Thus, all SPARQL query links will be given in this form.

So let's start with the most simple query: Select all football players and their popularity (indegree) in descending order starting with the most popular player. We must be a little bit careful, because the class SoccerPlayer does not only contain "real persons" but also popular roles of football players such as e.g. "Captain". Therefore, we filter the results for entities that have a name (via foaf:name). Here are the Top50 football players according to wikipedia. For the entire list, please refer to the references [1].
Name Popularity
Cristiano Ronaldo 1794
David Beckham 1572
Thierry Henry 1414
Lionel Messi 1404
Wayne Rooney 1343
Frank Lampard 1188
Pelé 1111
Didier Drogba 1047
Ronaldo 1037
Michael Owen 1011
Steven Gerrard 1002
Zlatan Ibrahimović 964
Alessandro Del Piero 926
Ronaldinho 914
Raúl (footballer) 903
Ryan Giggs 894
Fernando Torres 889
Zinedine Zidane 867
Ruud van Nistelrooy 861
Robbie Keane 861
Samuel Eto'o 859
Landon Donovan 835
Andriy Shevchenko 823
Kaká 804
Francesco Totti 730
Robin van Persie 720
Paul Scholes 692
Hernán Crespo 680
David Villa 669
John Terry 669
Cesc Fàbregas 669
George Best 667
Carlos Tévez 666
Robinho 643
Gary Lineker 641
Teddy Sheringham 633
Andrew Cole 620
Dwayne De Rosario 617
Xavi 616
Jermain Defoe 613
Craig Bellamy 609
Dimitar Berbatov 587
David Trezeguet 587
Luis Suárez 581
Peter Crouch 577
Michael Ballack 572
Miroslav Klose 568
Luís Figo 567
Lee Dong-Gook 558
Filippo Inzaghi 557
Yes, it was obvious for everybody that names such as Ronaldo, Beckham, Thierry, Pelé occur among the top popular players. Unfortunately, I'm not a football expert to comment further on that. Let's have a look, whether popularity corresponds with the number of achieved goals. However, this information is not easy to extract. For some of the football players, there's a property dbprop:totalGoals, while most of them has dbprop:goals. But the later sometimes exists multiple times for single years or periods. Thus, we have to sum up all dbprop:goals, while keeping in mind not to count any number more often than once (because an entry might be reproduced in our result list for several reasons).
Name Goals Popularity
David Schofield (footballer) 76543210 9
Alcindo Sartori 5019110 160
Oh Seung-Bum 1842256 32
Marei Al Ramly 6037 11
Darío Espínola 1715 6
Kim Andersson 1537 23
Stefan Lövgren 1328 18
Nikola Karabatić 1318 84
Elias Ribeiro de Oliveira 1187 26
Mohd Amar Rohidan 1020 38
Slaviša Žungul 856 113
John Bartley (footballer) 762 1
Zoran Karić 759 11
Jimmy Greaves 748 342
Ernest Spiteri Gonzi 704 11
Pierre van Hooijdonk 670 238
Reg Date 664 3
Trevor Phillips (footballer) 655 4
Joan Linares 645 12
Domenic Mobilio 625 58
Pelé 620 1111
Harry Johnson (footballer born 1899) 610 25
Ernst Stojaspal 602 23
Max Morlock 588 66
Ernie Hine 574 72
Branko Šegota 561 38
Serhiy Koridze 557 4
Salvinu Schembri 538 6
Konstantin Yeryomenko 537 13
Tony Brown (English footballer) 498 47
Waldo Machado 497 36
Nguyen Minh Phuong 496 50
Tony Cascarino 496 141
Leônidas da Silva 484 98
Zeki Rıza Sporel 470 75
Ángeles Parejo 469 9
Tommy Dickson 457 13
Elisabetta Vignotto 454 22
Alberto Spencer 445 99
Stefan Schwoch 435 7
Peter Kitchen 429 16
Edgar Kail 427 7
Eusébio 423 433
Giorgos Sideris 415 51
Tommy Browell 414 88
Patricio Margetic 412 14
Arsénio Trindade Duarte 409 19
Uwe Seeler 406 153
Hughie Gallacher 406 131
Dragan Džajić 401 150
Again we see, that DBpedia data (resp. Wikipedia data) is somehow 'noisy'. The first 3 ranks are obviously wrong concerning the number of goals. Simply because if David Schofield really would have achieved 76,543,210 goals, it would mean that he had won about 5 goals per minute of all the 32 years of his entire life so far. This must be kind of an extraction error. If we look at the players with more than 1000 goals, then a closer inspection reveals some handballers that either are also footballers or are wrongly declared to be footballers. In handball it is easier to achieve a higher number of goals compared to football. Trevor Phillips and John Bartley really achieved more than 600 goals, but their popularity score signals that they did achieve this not necessarely in the major league. The first top ranked prominent football player in this list definitely is Pelé with 620 goals. The only other two in this Top50 list I have already heard of are Eusébio and Uwe Seeler, but don't take me as a reference :)

Lets order the list again the other way around according to the most popular players to investigate their goal score:
Name Goals Popularity
Cristiano Ronaldo 227 1794
David Beckham 95 1572
Thierry Henry 265 1414
Lionel Messi 223 1404
Wayne Rooney 156 1343
Frank Lampard 163 1188
Pelé 620 1111
Didier Drogba 160 1047
Ronaldo 217 1037
Michael Owen 163 1011
Steven Gerrard 98 1002
Zlatan Ibrahimović 198 964
Alessandro Del Piero 223 926
Ronaldinho 157 914
Raúl (footballer) 280 903
Ryan Giggs 114 894
Fernando Torres 161 889
Zinedine Zidane 95 867
Ruud van Nistelrooy 249 861
Robbie Keane 179 861
Samuel Eto'o 219 859
Landon Donovan 135 835
Andriy Shevchenko 219 823
Kaká 114 804
Francesco Totti 226 730
Robin van Persie 130 720
Paul Scholes 107 692
Hernán Crespo 198 680
John Terry 30 669
David Villa 234 669
Cesc Fàbregas 50 669
George Best 238 667
Carlos Tévez 135 666
Robinho 122 643
Gary Lineker 243 641
Teddy Sheringham 289 633
Andrew Cole 226 620
Dwayne De Rosario 94 617
Xavi 57 616
Jermain Defoe 151 613
Craig Bellamy 113 609
Dimitar Berbatov 189 587
David Trezeguet 218 587
Luis Suárez 128 581
Peter Crouch 102 577
Michael Ballack 117 572
Miroslav Klose 181 568
Luís Figo 91 567
Filippo Inzaghi 184 557
Patrick Vieira 45 551
As we would expect, most of the popular football players are also good goal scorers. Well, there are a few exceptions. Take John Terry with a popularity score of 669 and only 30 goals. Why might he be so popular then? Taking a closer look at Wikipedia reveals that Terry plays at centre back position and is the captain of Chelsea in the Premier League. Well, that's already something for popularity. But, if you look even closer, you will find more: under the topic 'Controversies' you will find charges for assault and affray, a £60 fine for parking his Bentley in a disabled bay, extramarital affair allegations as well as racial abuse allegations. But, neither of these is directly responsible for Terry's popularity. In fact it's an internet meme (cf. introductory picture of this article). John Terry was suspended for the UEFA Final and had to watch his team in a suit and tie on the sidelines. He did look quite miserable as he sat there, watching his team defend for their lives and then miraculously pull out the victory. However, as soon as Chelsea made the victory, it was party time for Terry! He immediately threw off his suit like Superman and revealed his full Chelsea kit underneath his suit. The internet community enjoyed his dedication to his club and soccer so much that immediately a popular internet meme lampooning his behaviour appeared on the web, becoming one of the most popular online jokes in 2012. Terry has been pictured taking part in great moments in history and fiction. These included the fall of the Berlin Wall, the freeing of Nelson Mandela, the triumph of Rocky Balboa, as well as the first landing on the Moon [3]. Well, this should be reason for some popularity :)

Monday, June 23, 2014

Harald's Original Miscellany - The Truth about Football

The England National football team, 1893, photo: wikipedia
Well, it's the time of the Worldcup 2014. Why should I bother you with the peculiarities of authors and writers, when we can also have a look on Football! As you might remember, we had about 15,328 individuals in Wikipedia classified as authors [1]. What do you think, how many footballers are there compared to authors? Well there's a huge difference: 162,597 referenced footballer players, i.e. about 10 times as much as authors [2]. Maybe you think now there might be an overlap. How many football players are also listed as authors? I am sorry to disappoint you, but there is no overlap. No football player is also listed as being an author.

Fact No. 1: Footballers are no authors, and vice versa.

So you might wonder, what other categories these football players are in to get a better overview about what we are talking about. Interestingly, when looking at the most popular categories, you will soon find the large number of expatriates among those players. The Top5 expatriate nationalities among football players are: Brazil, Argentina, Russia, France, Serbia [3]. If you look at the bottom of the list, you will find the more or less exotic combinations, such as e.g. Hungarian expatriates in Uzbekistan, or Cameroonian expatriates in Venezuela, or even German expatriates in the Netherlands ;-)

Fact No. 2: Only a few Hungarian football players emigrate to Uzbekistan.

And in which countries they prefer to emigrate? The Top5 countries for football players to emigrate are: England, Germany, Spain, Italy, France [4]. At least for France the statistics seems to be balanced somehow, while England and Germany are the leading nations to attract foreign football players all around the world. Very interesting also the bottom of the list, where as the "least attractive" countries Gambia, Guam, Nepal, or Antigua and Barbuda are listed.

Fact No.3: French football players seem to be undecided whether to stay or leave the country.

But what about the categories that do not have a direct relationship to football in the first place? Let's filter out these categories and let's have a look on what football players are up to.

alternative professions of football players #players
http://dbpedia.org/class/yago/Director110014939 279
http://dbpedia.org/class/yago/Migrant110314952 271
http://dbpedia.org/class/yago/National109625401 250
http://dbpedia.org/class/yago/Intellectual109621545 237
http://dbpedia.org/class/yago/Alumnus109786338 234
http://dbpedia.org/class/yago/Scholar110557854 234
http://dbpedia.org/class/yago/Striker110663996 134
http://dbpedia.org/class/yago/Worker109632518 73
http://dbpedia.org/class/yago/SkilledWorker110605985 64
http://dbpedia.org/class/yago/Unfortunate109630641 56
http://dbpedia.org/class/yago/Trainer110722575 55
http://dbpedia.org/class/yago/Coach109931640 54
http://dbpedia.org/class/yago/Communicator109610660 51
http://dbpedia.org/class/yago/Serviceman110582746 51
http://dbpedia.org/class/yago/Expert109617867 43
http://dbpedia.org/class/yago/Observer110369528 33
http://dbpedia.org/class/yago/Cricketer109977326 32
http://dbpedia.org/class/yago/Adult109605289 28
http://dbpedia.org/class/yago/Victim110752093 28
http://dbpedia.org/class/yago/Rival110533013 27
http://dbpedia.org/class/yago/Official110372076 26
http://dbpedia.org/class/yago/Adjudicator109769636 26
http://dbpedia.org/class/yago/MilitaryOfficer110317007 25
http://dbpedia.org/class/yago/EnlistedPerson110058777 25
http://dbpedia.org/class/yago/Soldier110622053 25
http://dbpedia.org/class/yago/Referee110514429 25
http://dbpedia.org/class/yago/BadPerson109831962 23
http://dbpedia.org/class/yago/Wrongdoer109633969 23
http://dbpedia.org/class/yago/Writer110794014 21
http://dbpedia.org/class/yago/Professional110480253 21
http://dbpedia.org/class/yago/EnglishCricketers 20
http://dbpedia.org/class/yago/OlympicBronzeMedalistsForGermany 18
http://dbpedia.org/class/yago/Creator109614315 18
http://dbpedia.org/class/yago/AsianGamesGoldMedalistsForIran 16
http://dbpedia.org/class/yago/OlympicGoldMedalistsForTheUnitedStates 16
http://dbpedia.org/class/yago/Presenter110466387 16
http://dbpedia.org/class/yago/Survivor110681194 16
http://dbpedia.org/class/yago/Entertainer109616922 16
http://dbpedia.org/class/yago/Performer110415638 16
http://dbpedia.org/class/yago/Captain109893191 15
http://dbpedia.org/class/yago/CommissionedMilitaryOfficer109943239 15
http://dbpedia.org/class/yago/CommissionedOfficer109942970 15
http://dbpedia.org/class/yago/Artist109812338 15
http://dbpedia.org/class/yago/OlympicSilverMedalistsForParaguay 14
http://dbpedia.org/class/yago/OlympicSilverMedalistsForPoland 14
http://dbpedia.org/class/yago/Journalist110224578 14
http://dbpedia.org/class/yago/Principal110474950 13
http://dbpedia.org/class/yago/Criminal109977660 13
http://dbpedia.org/class/yago/OlympicGoldMedalistsForArgentina 12
http://dbpedia.org/class/yago/OlympicBronzeMedalistsForItaly 12
Please find the entire list in the references [5].

Fact No. 4: There are more intellectuals among football players than bad persons.

Wow, 237 football players are also categorized as being intellectuals, while 23 football players are listed as "bad persons". But, in this statistics, we will also find 21 writers(!) among the football players, as well as 15 artists, 14 journalists, and 13 criminals.
Further down the list, you will also find
  • 12 politicians, 
  • 8 identical twins, 
  • 8(!) head of state, 
  • 7 musicians, 
  • 4 comedians (I'll bet there are more...), 
  • 4 scientists, 
  • 3 singers, 
  • 2 mammals,
  • 2 Gentleman Cricketeers,
  • 2 gambling addicts,
  • 2 aviators,
  • 2 painters,
  • 1 UFO conspiracy theorists,
  • 1 bank robber,
  • 1 rapper,
  • 1 plumber,

etc.

So what does this tell us about football players?

Fact No. 5: There are more politicians among football players than comedians.

Well, in general, and according to Wikipedia, football players most times stick to their original profession. While emigrating here and there sometimes, there are only a few among them who actually have a second career outside of their original profession. Please note that we did not follow categories like football manager, football trainer, football coach, etc. Anyway, we finally did find also some authors among them....

Fact No. 6: There are writers among the football players...although they are not listed as authors.

to be continued....


Please find the full tables with all the results listed here in the References:

[1] total number of authors (?author rdf:type dbpedia-owl:Writer .)
[2] total number of football players (?player rdf:type dbpedia-owl:SoccerPlayer .)
[3] expatriate football players by home country
[4] expatriate football players by emigration country
[5] occupations of football players other than football

Saturday, June 21, 2014

Harald's Original Miscellany - Prolificacy vs. Popularity in Literature, Part 2

It might have been a surprise for you that according to Wikipedia, editor in chief of the Oxford English Dictionary John Simpson is the most popular author [1]. Thus, we have to take a look on the notion of "popularity". In scientific publishing, an "important" author is an author whose works are cited (referenced) by many other authors. In this way, a ranking among the most important authors has been established. In Wikipedia, we can follow this approach and simply count, how many articles are referencing the article of an author or the articles of the author`s books. In our approach, this was simply the number of incoming pageLinks of the referenced articles. Currently, we are adopting the PageRank algorithm for achieving a better measure of "importance" of an article. You all know the PageRank algorithm named after Larry Page, one of the founders of Google. PageRank is a way of measuring the importance of website pages [2]. We will get back on this, when we have finished PageRank integration into DBpedia.

But, let's discuss the results we had achieved in our last blog post. We stated that there are 15,328 authors [3] referenced in DBpedia. Well, of course there might be more authors, but this is the number of individuals that belong to the DBpedia class authors. The question is, when we made the popularity vs prolificacy statistics, did we include the works of 15,328 authors? Surely not. Let's first check, for how many authors, there are books written by these authors referenced in DBpedia: Its only 3,763 authors with referenced books [4]. Thus, we don't have any idea about the remaining 11,000 authors who don't have one of their books referenced in DBpedia. They must have written some book, otherwise they would not be an "author"....

OK, so we are dealing with insufficient information. If we look at the available data, we could also use another property, called dbpedia-owl:notableWork denoting only the "important" works of an author (if we relate it with authors and not with other artists). Now it would be interesting, to repeat our statistics based on this property and look, if there is a significant difference. First let's look at the most prolific authors with respect to referenced "notable works":

name numOfNotableWorks popularityOfNotableWorks
"Roald Dahl"@en 16 53.8
"Joseph Conrad"@en 9 68.2
"Charles Dickens"@en 8 398.7
"Roger Zelazny"@en 7 15.7
"Henryk Sienkiewicz"@en 6 53.8
"Tom Wolfe"@en 6 44.3
"Margaret Atwood"@en 6 41.8
"Henry James"@en 6 74.0
"Ismail Kadare"@en 6 7.8
"Alfred Döblin"@en 6 13.3
"Stephen King"@en 6 133.1
"Robert Girardi"@en 6 3.0
"Hunter S. Thompson"@en 5 56.4
"Joanne Harris"@en 5 16.4
"David Mitchell (author)"@en 5 25.6
"Chris Kuzneski"@en 5 7.0
"Jackie French"@en 5 8.2
"Sita Ram Goel"@en 5 7.0
"Peter Hitchens"@en 5 9.2
"Chinua Achebe"@en 5 31.6
"Colm Tóibín"@en 5 17.4
"E. L. Doctorow"@en 5 26.4
"Don DeLillo"@en 5 27.4
"Johann Wolfgang von Goethe"@en 5 72.6
"Samuel Beckett"@en 5 25.4
"Cormac McCarthy"@en 5 55.6
"Brian O'Nolan"@en 5 29.8
"Poppy Z. Brite"@en 5 11.2
"William Trevor"@en 5 20.2
"Neil Gaiman"@en 5 84.8
"Dean Koontz"@en 5 10.4
"George MacDonald"@en 5 21.6
"Ann Bannon"@en 5 9.0
"William Faulkner"@en 4 70.7
"Will Self"@en 4 10.0
"Alan Hollinghurst"@en 4 23.5
"George Eliot"@en 4 81.0
"Malcolm Gladwell"@en 4 39.0
"C. S. Lewis"@en 4 43.2
"Hermann Hesse"@en 4 53.2

There is a huge difference. Less science fiction authors, more international authors, and the average "popularity" of the top 40 author's works has also significantly increased. We also realize that considering Charles Dickens, the popularity of his 8 notable works (398) doesn't differ so much from the popularity of his overall 30 works (159.5) as for Stephen King, where the popularity of his 5 notable works (133) is much higher compared to the overall average from last time (44) for his 75 books. Interesting also that Road Dahl is mentioned with 16 notable works. Either the wikipedia authors could not decide which of the works of Road Dahl really was notable or the article must have been written by a huge fan. OK, so we might learn that is in the eye of the beholder, which works of an author are referenced as being "notable".

Let's order the list now again according to the popularity measure and see what happens. This is the Top 40 list of authors with the most popular works (on average) with respect to "notable works" only:

name numOfNotableWorks popularityOfNotableWorks
"Bram Stoker"@en 1 1145.0
"Lewis Carroll"@en 2 939.0
"Lucifer Chu"@en 1 612.0
"Miguel de Cervantes"@en 2 611.0
"J. R. R. Tolkien"@en 2 566.0
"Robert Louis Stevenson"@en 3 448.0
"Alexandre Dumas"@en 2 427.5
"Oscar Wilde"@en 1 427.0
"Kenneth Grahame"@en 1 423.0
"Walter Prescott Webb"@en 1 423.0
"James Vincent Murphy"@en 1 421.0
"George Orwell"@en 3 412.3
"Charles Dickens"@en 8 398.7
"Emily Brontë"@en 1 387.0
"Leo Tolstoy"@en 2 374.5
"John Bunyan"@en 1 341.0
"Louisa May Alcott"@en 1 314.0
"Mark Twain"@en 2 307.5
"Jonathan Swift"@en 2 303.5
"Emma Orczy"@en 1 282.0
"Charlotte Brontë"@en 2 258.0
"Ian McFarlane"@en 1 255.0
"William Golding"@en 1 251.0
"Anne Frank"@en 1 246.0
"James Fenimore Cooper"@en 1 246.0
"Margaret Mitchell"@en 2 242.0
"Frans Sammut"@en 2 233.5
"Rudyard Kipling"@en 2 230.5
"Dan Brown"@en 3 224.3
"John Steinbeck"@en 3 218.0
"William Gibson"@en 1 216.0
"Ayn Rand"@en 2 210.5
"Harriet McDougal"@en 1 207.0
"Jim Butcher"@en 1 200.0
"Richard Adams"@en 1 192.0
"William Makepeace Thackeray"@en 1 191.0
"T. S. Eliot"@en 2 186.0
"Aldous Huxley"@en 3 175.3
"Hitoshi Igarashi"@en 1 175.0
"Alice Walker"@en 1 174.0

This looks a lot more like we would have expect it to be in the first place. Bram Stoker on the first place with only 1 notable work, we all know which book is meant by that :) impressive score. Lewis Carroll on the second place with his Alice in Wonderland stories is also no wonder. But, who is Lucifer Chu? This is some kind of surprise for me. You won't find Lucifer Chu in the German version of DBpedia, so why does our result suggest that he is a popular author? Well, if you look at the very short Wikipedia article in the English version, you will find out that he is the author of the Chinese translation of J.R.R.Tolkien's The Hobbit and The Lord of the Rings. Tolkien's books follow on rank number 5 with a slightly less popularity. How can that be? Well we must examine the available data a little bit closer.

Chu is referenced in our list with 1 book only although we know that he has translated The Hobbit as well as The Lord of the Rings. And J.R.R. Tolkien is referenced with 2 books. Obviously, for Chu only the Hobbit as being the most popular of these books has been counted. But, if we look at the DBpedia page of Lucifer Chu, we realize that for Chu there are 2 books referenced as notable Work, as there are 3 books for Tolkien (including The Silmarillion). The reason for this is that The Lord of the Rings is not a member of the class dbpedia-owl:Book. Why is that so? Maybe because The Lord of the Rings consists out of 3 different books. Thus, to get a more complete result, we should think of how we get all notable works into that list. But, we should simultaneously take care of only including books or series of books and to exclude other pieces or work such as e.g. paintings, sculptures, movies, photographs, etc.

We will take care of this next time... :)

References:
[1] Harald's Original Miscellany - Prolificacy vs. Popularity in Literature, Part 1, 2014/06/20
[2] Brin, S.; Page, L. (1998). "The anatomy of a large-scale hypertextual Web search engine". Computer Networks and ISDN Systems 30: 107–117.
[3] live SPARQL query for the number or authors
[4] live SPARQL query for the number of distinct authors who have written a book, referenced in DBpedia
[5] live SPARQL query for the Top 40 most prolific authors (based on notable works)
[6] live SPARQL query for the Top 40 most popular authors (based on notable works)

Friday, June 20, 2014

Harald's Original Miscellany - Prolificacy vs. Popularity in Literature

Oh wow, it's quite a while that I wrote my last post here in the blog... But, while preparing exercises for the OpenHPI MOOC course 'Knowledge Engineering with Semantic Technologies', I was about to play around a little with SPARQL to come up with new exercises for the students of the course. To make it short, our current lecture examples all deal with writers and books. Thus, to learn how to query RDF knowledge bases with the SPARQL query language, I chose the DBpedia. While trying to think of some interesting toy examples, I started to play around and the facts that I discovered by chance were so interesting that I totally forgot about my lunch break :)

So here are some interesting facts about books and authors that will be continued in later posts. All presented statistics is based on the (English) Wikipedia (of course for the SPARQL queries we use DBpedia)...but nevertheless, it is wikipedia knowledge.

There are currently 15,328 authors listed (i.e. they are member of the class dbpedia-owl:Writer). First thing I wanted to find out was, who are the most prolific authors according to Wikipedia (at least this means, whose works also exist as Wikipedia Pages and who are connected via dbpedia-owl:author).

Well, here are the Top 40 Most Prolific Writers:

name numOfWorks popularityOfWorks
"L. Sprague de Camp"@en 128 10.9
"Agatha Christie"@en 103 32.7
"Isaac Asimov"@en 75 28.3
"Stephen King"@en 75 44.0
"Philip K. Dick"@en 74 14.4
"Edgar Rice Burroughs"@en 73 18.7
"Ruth Rendell"@en 70 5.3
"Dean Koontz"@en 67 6.2
"Lin Carter"@en 64 9.0
"Terry Pratchett"@en 63 41.7
"Jules Verne"@en 63 31.2
"P. G. Wodehouse"@en 63 20.9
"Robert E. Howard"@en 62 10.5
"Gary Paulsen"@en 61 4.1
"K. A. Applegate"@en 61 18.5
"August Derleth"@en 60 5.2
"John Dickson Carr"@en 59 5.9
"H. G. Wells"@en 57 30.0
"James Patterson"@en 56 12.0
"Leslie Charteris"@en 55 8.8
"Robert A. Heinlein"@en 52 36.5
"Rex Stout"@en 52 21.4
"Arthur C. Clarke"@en 50 20.8
"Harry Turtledove"@en 49 11.3
"Ray Bradbury"@en 49 15.3
"David Weber"@en 49 30.9
"Danielle Steel"@en 48 3.3
"J. M. G. Le Clézio"@en 48 3.7
"Henry James"@en 48 18.5
"Piers Anthony"@en 48 14.0
"Clive Cussler"@en 47 10.1
"Roger Zelazny"@en 45 10.5
"Alan Dean Foster"@en 44 6.7
"Joe R. Lansdale"@en 43 4.7
"Gordon R. Dickson"@en 41 5.0
"Marion Zimmer Bradley"@en 41 7.6
"Samuel R. Delany"@en 41 9.0
"Bernard Cornwell"@en 40 16.6
"Enid Blyton"@en 40 6.8
"Dr. Seuss"@en 40 26.5
The average Popularity Score that you see in the third column corresponds to the number of references (links) from other wikipedia articles to these books. Interestingly, Agatha Christie as well as Isaac Asimov are rather prolific authors whose books also have an above the average popularity. On the other hand, Ruth Rendell or Dean Koontz are rather prolific, but not very popular (at least according to wikipedia). Most popular in this list are Stephen King and Terry Prachett.

Well, let's turn it the other way around. Let's sort this list by the average popularity of the books of these authors....

Here is the Top 40 list of the authors with the most popular books (on average):
name numOfWorks popularityOfWorks
"John Simpson (lexicographer)"@en 1 1627.0
"John Milton"@en 1 669.0
"Kenneth Grahame"@en 1 423.0
"Emily Brontë"@en 1 387.0
"Wilhelm Grimm"@en 1 343.0
"Jacob Grimm"@en 1 343.0
"Harper Lee"@en 1 334.0
"Lewis Carroll"@en 6 319.8
"Miguel de Cervantes"@en 4 310.5
"Jonathan Swift"@en 2 303.5
"Wilbert Awdry"@en 1 296.0
"Cao Xueqin"@en 1 291.0
"Giovanni Boccaccio"@en 1 287.0
"William Shakespeare"@en 3 264.6
"Ian McFarlane"@en 1 255.0
"Antoine de Saint-Exupéry"@en 1 253.0
"Margaret Mitchell"@en 2 242.0
"Suetonius"@en 1 229.0
"Roger Hargreaves"@en 2 214.0
"Joseph O'Neill (writer)"@en 1 211.0
"T. S. Eliot"@en 2 186.0
"George Bernard Shaw"@en 2 184.5
"Harriet Beecher Stowe"@en 3 173.6
"Petronius"@en 1 162.0
"Charles Dickens"@en 30 159.5
"Johanna Spyri"@en 1 156.0
"Herman Melville"@en 7 154.7
"Jaroslav Hašek"@en 1 152.0
"Pierre Choderlos de Laclos"@en 1 152.0
"Oscar Wilde"@en 3 148.0
"Dave Arneson"@en 3 146.0
"Ngô Sĩ Liên"@en 1 145.0
"John Eric Holmes"@en 2 143.5
"Carlo Collodi"@en 1 142.0
"George Orwell"@en 11 135.0
"Erik Mona"@en 3 134.6
"Daniel Defoe"@en 5 132.8
"Monte Cook"@en 3 132.3
"Apuleius"@en 1 132.0
"Dan Brown"@en 6 126.6
Possibly you have never heard of John Simpson? But you will have heard about the Oxford English Dictionary. Well John Simpson was its Chief Editor...that makes sense, doesn't it? What about Kenneth Graham? Maybe you know his 1908 published novel The Wind and the Willows...

In this list it seems that it is more about literary excellency. Only one author with a rather prolific output is found, which is Charles Dickens with 30 listed Books.  But, to find also Dan Brown on this list tells me, that popularity doesn't hold for literary excellency or quality. At least he is last among the Top 40 after Herman Melville, George Orwell, Daniel Defoe, Lewis Caroll or Jonathan Swift. On the other hand, John Milton did not become rich with his one shot Paradise Lost although it is rather popular.

Here are the links to the online queries to get the most recent and complete results:
Enjoy....I'll be back, when I will find again something interesting ;-)