New DBpedia Graph Statistics

Recently, we have been working on the DBpedia / Wikipedia Page Link dataset. We have considered the English and the German language versions for this project. In the current DBpedia 3.9 page links English and German datasets 18 million and 6 million entities are represented respectively. But the original DBpedia only contains about 4 million and 1 million distinct entities for English and German versions. This significant difference is mainly due to the current DBpedia pagelinks dataset include redirect pages and pagelinks with resources that are not considered as entites (as e.g. thumbnails and other images). So we considered cleaning up DBpedia pagelinks dataset for the computation of statistical parameters (a.g. pagerank or HITS). For the Cleanup we have removed all unnecessary and redundant RDF-Triples from the pagelinks dataset, i.e all removing the redirect pages (Redirection pages are just URIs that automatically forward a user to another Wikipedia page, but do not represent entities) as well as RDF-Triples representing resources that do not have an own rdfs:label (as per DBpedia documentation every entity has an rdfs:label reference).

One of the benefits of the cleaned up pagelink dataset is the faster computation of statistical graph measures (while not influencing the overall statistics, i.e. redirect pages usually don't have incoming links and theother removed resources (as e.g. images) don't have outgoing links). Based on this dataset we have computed PageRank, Hub and Authorities (HITS), PageInlink Counts and PageOutLink Counts. Please find the details of the datasets here on our research group's webpage [1].

For Computation of the DBpedia graph statistics we have used JUNG — the Java Universal Network/Graph Framework. Please find the source code for PageRank and HITS computation here via GitHub [2].

References and further Reading:
[1] New PageRank Computations for DBpedia 3.9 (English/German) at SemanticMultimedia
[2] Source code for DBpedia Graph Statistics

Harald's Original Miscellany - More Truth about Football - Part 4

Finally, Germany has won the Soccer Worldcup 2014. Therefore, also our little statistics on soccer will come to an end with the post today. You might ask yourself what kind of data is left for soccer players in Wikipedia and DBpedia. Well, unfortunately only a little. But, we will try to make something out of it. Last time, we've ask for the number of team changes and the correlation to popularity vs. scored goals for soccer players. What is left, if we look at the available data?





We have the data about the years in which the football players were active or have played in their national soccer team. Let's start with the national team years [1]:

nationalyears	NumPlayers
8	3
7	20
6	85
5	371
4	1070
3	2479
2	6116
1	28211

Well, it was obvious that the most players have only 1 or two years in the national team. But, there are exceptional players who achieved even 8 years. But, who are these long term players? [2]:

nationalyears	Player	Team
8	Wojciech Łobodziński	"Poland"@en
8	Wojciech Łobodziński	"Poland Under 16"@en
8	Wojciech Łobodziński	"Poland Under 17"@en
8	Wojciech Łobodziński	"Poland Under 21"@en
8	Wojciech Łobodziński	"Poland Under 18"@en
8	Santiago Cañizares	"Spain Under-17"@en
8	Santiago Cañizares	"Spain Under-16"@en
8	Santiago Cañizares	"Spain Under-21"@en
8	Santiago Cañizares	"Spain Under-18"@en
8	Santiago Cañizares	"Spain Under-23"@en
8	Santiago Cañizares	"Spain Under-19"@en
8	Santiago Cañizares	"Spain Under-20"@en
7	Aydın Yılmaz	"Turkey Under-21"@en
7	Aydın Yılmaz	"Turkey"@en
7	Aydın Yılmaz	"Turkey"@en
7	Aydın Yılmaz	"Turkey Under-19s"@en
7	Aydın Yılmaz	"Turkey Under-17"@en
7	Aydın Yılmaz	"Turkey A2"@en
7	Ismael Urzaiz	"Spain Under-17"@en
7	Ismael Urzaiz	"Spain Under-16"@en

Possibly you will never have heard of Poland's Wojciech Łobodziński or Spain's Santiago Cañizares. Here another flaw in the data becomes visible. There is no such thing as the unique national team. We have "under 16", "under 17", "under 18", and so on... So you start your career already with 15 and after 8 years you would be 23 and possibly be in the "real" national team.

Let's have a look at the active years of players. Unfortunately, here the data is rather messy [3]: 

person	From	To
http://dbpedia.org/resource/Marta_(footballer)	2000	9223372036854775807
http://dbpedia.org/resource/Carlos_Alberto_Gomes_de_Lima	2006	200720082008
http://dbpedia.org/resource/Blake_Camp	2008	200420052006
http://dbpedia.org/resource/Birgit_Prinz	1998	200320042005
http://dbpedia.org/resource/Dejan_Damjanovi%C4%87	1998	20112012
http://dbpedia.org/resource/Inka_Grings	1995	20092010
http://dbpedia.org/resource/Breno_Silva	2003	20092010
http://dbpedia.org/resource/Vin%C3%ADcius_Calamari	2007	20092010
http://dbpedia.org/resource/Samuel_Jos%C3%A9_da_Silva_Vieira	1994	20082009
http://dbpedia.org/resource/Chad_Marshall	2004	20082009
http://dbpedia.org/resource/Michael_Parkhurst	2003	20072008
http://dbpedia.org/resource/Geison_Rodrigues_Marrote	2004	20072008
http://dbpedia.org/resource/Dejan_Stankovi%C4%87	1994	20062010
http://dbpedia.org/resource/Glauber_Da_Silva	2001	20062007
http://dbpedia.org/resource/Hugo_Sarmiento	1999	20032007
http://dbpedia.org/resource/Obafemi_Martins	2000	20032004
http://dbpedia.org/resource/Heslley_Couto	2005	20032006
http://dbpedia.org/resource/Dalibor_Filipovi%C4%87	1992	20022003
http://dbpedia.org/resource/Dmytro_Zayko	2005	20022004
http://dbpedia.org/resource/Aviv_Volnerman	1998	20012004

Possibly this is because of people not only writing single year's into the Wikipedia infoboxes, but time spans and other things. In the list above are only the Top 20. Just remove the LIMIT from the SPARQL query and further down you will find more valid data.

OK, let's come to our last problem related to football. Is there a correlation between the height of a player (simply because we have that data) and the number of achieved goals [4]?

Height	sumgoals
243.84	1
23.0	59
215.9	1
206.0	0
204.0	0
203.2	24
203.2	47
203.0	0
203.0	0
203.0	51
202.0	19
202.0	0
202.0	15
201.0	20
201.0	0
201.0	0
200.66	11
200.66	48
200.66	0
200.66	0
200.66	0
200.66	124
200.66	146
200.66	0
200.66	1
200.66	16
200.66	0
200.66	102
200.66	6
200.66	0
200.0	0
200.0	20
199.0	0
199.0	0
199.0	78
199.0	47
199.0	0
199.0	62
199.0	32
199.0	84

It is rather difficult to recognize something in this data. You see heights and the number of goals that player of that height have scored. Fascinating that there seem to be a significant number of players that are taller than 2 meters. I guess that the list leader with 2.43 meters is just incorrect data. Now. this is so many data that we have to visualize it to recognize something....

Correlation between height of soccer players and scored goals

In the diagram to the left you see the soccer players height (x-axis) vs. the number of scored goals (y-axis). Interesting thing to notice is that the heights approximately show a Gaussian distribution, i.e most players have a "middle" height, on the extremes there are only a few. Well, there seems to be an exception. on the outer left you will notice a large fraction of players with a height of 1.52m with goal scores ranging among 0 to 300. This is extraordinary, because I have no idea what kind of group this is or if there is simply an error in the data again. What I noticed is that among this group there seems to be a larger fraction of female Asian soccer players. Maybe they are responsible for that large number of outliers, but this requires further investigation.

Alas, I want to add one last table. For each player there is the information on which position she or he is playing. Of course Wikipedia authors are far from any sort of agreement how to name the player's position. Thus, there is a rather huge variety. Nevertheless, I will leave you with the table to make any sense from it. Enjoy [5]:

position	number	avheight	avgoals
midfielder	4450	176.822828644929299	18.077078651685393
defender	3429	181.37523034450506	8.058326042578011
striker	2643	180.169761559433703	65.213015512674991
goalkeeper	2086	186.186064557569709	0.174496644295302
forward	1787	178.865346269495465	46.388919977616116
centre back	786	185.7391350821381	10.651399491094148
winger	648	174.69354896725695	27.583333333333333
attacking midfielder	490	175.897877222177931	38.006122448979592
left back	461	177.661735253323698	6.995661605206074
defensive midfielder	427	179.741452037869347	11.978922716627635
right back	405	177.829456545982828	7.301234567901235
central defender	196	185.005408462213008	8.897959183673469
central midfielder	161	178.223105294363834	20.099378881987578
full back	158	174.659366173080242	6.759493670886076
centre forward	140	179.09685701642717	80.792857142857143
left winger	113	174.790176753449224	28.398230088495575
defender, midfielder	102	179.789412255380659	10.607843137254902
inside forward	96	172.094789346059145	64.083333333333333
defender/midfielder	82	179.178292995545911	11.463414634146341
defender / midfielder	79	180.188100887250282	15.050632911392405
left-back	65	175.642922504131603	7.692307692307692
right winger	62	175.247418803553424	30.096774193548387
centre-back	56	521.327500479561941	9.375
right-back	54	176.101110952871815	6.111111111111111
midfielder/forward	52	172.013075924836657	22.557692307692308
midfield	52	176.982691251314588	30.673076923076923
defender (retired)	50	181.37344055175781	14.48
second striker	48	174.673332850138346	63.854166666666667
full-back	44	175.908408771861666	2.681818181818182
striker / winger	41	178.235609845417295	49.219512195121951

Of course I limited the table to 30 rows. Interestingly, the average height of goalkeepers is larger than for midfielders or strikers. As expected, strikers on the average score more goals than defenders or goalkeepers.

SPARQL Queries and original data:

[1] number of players wrt. to years in the national team

[2] player name and team name of national team players with the longest playing terms
[3] players ordered by number of active years
[4] is there a correlation between soccer player height and scored goals?
[5] is there any correlation between player position, height, and scored goals?

Harald's Original Miscellany - More Truth about Football - Part 3

To change the team means to earn more
money...what about the football millionaires?
How often do they change the team?

Are you ready for more statistics on your favorite kind of sports? Well, data is fun, and obviously Big Data means Big Fun. There are lot's of interesting things to discover while exploring data, and wikipedia (i.e. dbpedia for the insiders) provides all the necessary means.

Have you ever wondered about this kind of slave trade in professional football? Well, I wouldn't exactly call the transfer of a millionaire to a higher paying job a 'slave trade'. But, have you ever thought about the following question: Do the real good (and well paid) players more often change the team - or is it vice versa, that teams try to get rid of players that have a bad season or are on the decline? Who knows? Let's have a look on the data:

TeamChanges	NumPlayers
16	2
15	26
14	84
13	287
12	792
11	2247
10	3848
9	5109
8	6464
7	8110
6	9790
5	11264
4	11837
3	11448
2	10961
1	6515

Here, we have a table providing an overview about how many players (in wikipedia) have changed their team for how many times [1]. Obviously, it seems to be some kind of Gaussian distribution with a peak between 2 and 6 team switches. OK, what about the players? Where are the top players listed in this table? Well, David Beckham switched team 11 times according to wikipedia, Cristiano Ronaldo 8 times, Thierry Henry 9 times. At least these numbers are above average which we had identified to be between 2 and 6. This seems to give proof to our original assumption.

person	TeamChanges	popularity
http://dbpedia.org/resource/Cristiano_Ronaldo	8	1794
http://dbpedia.org/resource/David_Beckham	11	1572
http://dbpedia.org/resource/Thierry_Henry	9	1414
http://dbpedia.org/resource/Lionel_Messi	7	1404
http://dbpedia.org/resource/Wayne_Rooney	5	1343
http://dbpedia.org/resource/Frank_Lampard	5	1188
http://dbpedia.org/resource/Pel%C3%A9	4	1111
http://dbpedia.org/resource/Didier_Drogba	8	1047
http://dbpedia.org/resource/Ronaldo	10	1037
http://dbpedia.org/resource/Michael_Owen	6	1011

But, we get a better overview, if we look at the average popularity of each switching group in the table [3]:

TeamChanges	NumPlayers	avgindegree
16	2	107.5
15	26	76.769230769230769
14	84	47.297619047619048
13	287	60.885017421602787
12	788	45.073604060913706
11	2235	37.206263982102908
10	3800	32.577631578947368
9	5023	30.939080230937687
8	6332	29.685881238155401
7	7935	25.499054820415879
6	9525	23.188346456692913
5	10886	18.682895462061363
4	11423	14.937844699290904
3	10842	11.131802250507286
2	10089	7.704628803647537
1	5534	5.19588001445609

As we had originally thought, on the average, the popularity of the players rises with the number of team changes. Although the top group with 16 changes is far from the highest possible popularity scores (as e.g. 1572 for David Beckham). Hmm, maybe there is a correlation among the number of achieved goals with the number of team changes? Is it more likely that a top goal hunter switches team more often? Let's have a look [4]:

TeamChanges	NumPlayers	AvgGoals
16	2	22.5
15	26	28.807692307692308
14	83	37.855421686746988
13	287	38.062717770034843
12	781	39.939820742637644
11	2205	38.625850340136054
10	3784	35.646141649048626
9	4980	32.826907630522088
8	6296	29.489517153748412
7	7826	24.455788397648863
6	9314	21.937835516426884
5	10471	18.655142775284118
4	10627	16.317869577491296
3	9612	12.756450270495214
2	7193	9.911858751564021
1	4624	4.444204152249135

Looks interesting. Top goal scorer have 9 to 14 team switches. This is way above the average. Thus, the more goals you score, the more often you will have the chance of being transferred (and thus earn more money). Players that don't score goals will obviously not be transferred (that often).

References:

[1] How many players have how many team changes overall?
[2] The Top10 popular soccer players and their number of team changes
[3] The average popularity of football players regarding the number of team switches
[4] Number of average goals per soccer player with respect to the number of team changes

Harald's Original Miscellany - The Truth about Football - Part 2

John Terry Celebration Meme, read on and you will understand...

Of course you always wanted to know, who is the best football player of all times. Sure this might be a question about which real football afficionados might argue forever. Also Wikipedia will not be able to give you the definite answer. But, we can play around with the available data and maybe we find out something interesting about football players again ...

But, first at all, I want to say thank you to Kingsley Idehen, who gave me the hint for my SPARQL query links to use the parameter "qtxt=" instead of "query=", which enables others to see the original query and to use it for further data explorations. Thus, all SPARQL query links will be given in this form.

So let's start with the most simple query: Select all football players and their popularity (indegree) in descending order starting with the most popular player. We must be a little bit careful, because the class SoccerPlayer does not only contain "real persons" but also popular roles of football players such as e.g. "Captain". Therefore, we filter the results for entities that have a name (via foaf:name). Here are the Top50 football players according to wikipedia. For the entire list, please refer to the references [1].

Name	Popularity
Cristiano Ronaldo	1794
David Beckham	1572
Thierry Henry	1414
Lionel Messi	1404
Wayne Rooney	1343
Frank Lampard	1188
Pelé	1111
Didier Drogba	1047
Ronaldo	1037
Michael Owen	1011
Steven Gerrard	1002
Zlatan Ibrahimović	964
Alessandro Del Piero	926
Ronaldinho	914
Raúl (footballer)	903
Ryan Giggs	894
Fernando Torres	889
Zinedine Zidane	867
Ruud van Nistelrooy	861
Robbie Keane	861
Samuel Eto'o	859
Landon Donovan	835
Andriy Shevchenko	823
Kaká	804
Francesco Totti	730
Robin van Persie	720
Paul Scholes	692
Hernán Crespo	680
David Villa	669
John Terry	669
Cesc Fàbregas	669
George Best	667
Carlos Tévez	666
Robinho	643
Gary Lineker	641
Teddy Sheringham	633
Andrew Cole	620
Dwayne De Rosario	617
Xavi	616
Jermain Defoe	613
Craig Bellamy	609
Dimitar Berbatov	587
David Trezeguet	587
Luis Suárez	581
Peter Crouch	577
Michael Ballack	572
Miroslav Klose	568
Luís Figo	567
Lee Dong-Gook	558
Filippo Inzaghi	557

Yes, it was obvious for everybody that names such as Ronaldo, Beckham, Thierry, Pelé occur among the top popular players. Unfortunately, I'm not a football expert to comment further on that. Let's have a look, whether popularity corresponds with the number of achieved goals. However, this information is not easy to extract. For some of the football players, there's a property dbprop:totalGoals, while most of them has dbprop:goals. But the later sometimes exists multiple times for single years or periods. Thus, we have to sum up all dbprop:goals, while keeping in mind not to count any number more often than once (because an entry might be reproduced in our result list for several reasons).

Name	Goals	Popularity
David Schofield (footballer)	76543210	9
Alcindo Sartori	5019110	160
Oh Seung-Bum	1842256	32
Marei Al Ramly	6037	11
Darío Espínola	1715	6
Kim Andersson	1537	23
Stefan Lövgren	1328	18
Nikola Karabatić	1318	84
Elias Ribeiro de Oliveira	1187	26
Mohd Amar Rohidan	1020	38
Slaviša Žungul	856	113
John Bartley (footballer)	762	1
Zoran Karić	759	11
Jimmy Greaves	748	342
Ernest Spiteri Gonzi	704	11
Pierre van Hooijdonk	670	238
Reg Date	664	3
Trevor Phillips (footballer)	655	4
Joan Linares	645	12
Domenic Mobilio	625	58
Pelé	620	1111
Harry Johnson (footballer born 1899)	610	25
Ernst Stojaspal	602	23
Max Morlock	588	66
Ernie Hine	574	72
Branko Šegota	561	38
Serhiy Koridze	557	4
Salvinu Schembri	538	6
Konstantin Yeryomenko	537	13
Tony Brown (English footballer)	498	47
Waldo Machado	497	36
Nguyen Minh Phuong	496	50
Tony Cascarino	496	141
Leônidas da Silva	484	98
Zeki Rıza Sporel	470	75
Ángeles Parejo	469	9
Tommy Dickson	457	13
Elisabetta Vignotto	454	22
Alberto Spencer	445	99
Stefan Schwoch	435	7
Peter Kitchen	429	16
Edgar Kail	427	7
Eusébio	423	433
Giorgos Sideris	415	51
Tommy Browell	414	88
Patricio Margetic	412	14
Arsénio Trindade Duarte	409	19
Uwe Seeler	406	153
Hughie Gallacher	406	131
Dragan Džajić	401	150

Again we see, that DBpedia data (resp. Wikipedia data) is somehow 'noisy'. The first 3 ranks are obviously wrong concerning the number of goals. Simply because if David Schofield really would have achieved 76,543,210 goals, it would mean that he had won about 5 goals per minute of all the 32 years of his entire life so far. This must be kind of an extraction error. If we look at the players with more than 1000 goals, then a closer inspection reveals some handballers that either are also footballers or are wrongly declared to be footballers. In handball it is easier to achieve a higher number of goals compared to football. Trevor Phillips and John Bartley really achieved more than 600 goals, but their popularity score signals that they did achieve this not necessarely in the major league. The first top ranked prominent football player in this list definitely is Pelé with 620 goals. The only other two in this Top50 list I have already heard of are Eusébio and Uwe Seeler, but don't take me as a reference :)

Lets order the list again the other way around according to the most popular players to investigate their goal score:

Name	Goals	Popularity
Cristiano Ronaldo	227	1794
David Beckham	95	1572
Thierry Henry	265	1414
Lionel Messi	223	1404
Wayne Rooney	156	1343
Frank Lampard	163	1188
Pelé	620	1111
Didier Drogba	160	1047
Ronaldo	217	1037
Michael Owen	163	1011
Steven Gerrard	98	1002
Zlatan Ibrahimović	198	964
Alessandro Del Piero	223	926
Ronaldinho	157	914
Raúl (footballer)	280	903
Ryan Giggs	114	894
Fernando Torres	161	889
Zinedine Zidane	95	867
Ruud van Nistelrooy	249	861
Robbie Keane	179	861
Samuel Eto'o	219	859
Landon Donovan	135	835
Andriy Shevchenko	219	823
Kaká	114	804
Francesco Totti	226	730
Robin van Persie	130	720
Paul Scholes	107	692
Hernán Crespo	198	680
John Terry	30	669
David Villa	234	669
Cesc Fàbregas	50	669
George Best	238	667
Carlos Tévez	135	666
Robinho	122	643
Gary Lineker	243	641
Teddy Sheringham	289	633
Andrew Cole	226	620
Dwayne De Rosario	94	617
Xavi	57	616
Jermain Defoe	151	613
Craig Bellamy	113	609
Dimitar Berbatov	189	587
David Trezeguet	218	587
Luis Suárez	128	581
Peter Crouch	102	577
Michael Ballack	117	572
Miroslav Klose	181	568
Luís Figo	91	567
Filippo Inzaghi	184	557
Patrick Vieira	45	551

As we would expect, most of the popular football players are also good goal scorers. Well, there are a few exceptions. Take John Terry with a popularity score of 669 and only 30 goals. Why might he be so popular then? Taking a closer look at Wikipedia reveals that Terry plays at centre back position and is the captain of Chelsea in the Premier League. Well, that's already something for popularity. But, if you look even closer, you will find more: under the topic 'Controversies' you will find charges for assault and affray, a £60 fine for parking his Bentley in a disabled bay, extramarital affair allegations as well as racial abuse allegations. But, neither of these is directly responsible for Terry's popularity. In fact it's an internet meme (cf. introductory picture of this article). John Terry was suspended for the UEFA Final and had to watch his team in a suit and tie on the sidelines. He did look quite miserable as he sat there, watching his team defend for their lives and then miraculously pull out the victory. However, as soon as Chelsea made the victory, it was party time for Terry! He immediately threw off his suit like Superman and revealed his full Chelsea kit underneath his suit. The internet community enjoyed his dedication to his club and soccer so much that immediately a popular internet meme lampooning his behaviour appeared on the web, becoming one of the most popular online jokes in 2012. Terry has been pictured taking part in great moments in history and fiction. These included the fall of the Berlin Wall, the freeing of Nelson Mandela, the triumph of Rocky Balboa, as well as the first landing on the Moon [3]. Well, this should be reason for some popularity :)

[1] List of most popular Football Players

[2] List of top goal scorer among popular Football players

[3] John Terry's Celebration Memes