Tuesday, September 02, 2014

More intermediate Results from our Fact Ranking Challenge

Fact Ranking Challenge
Again, here is an update about our currently gathered data about our fact ranking experiment.

We started the original challenge about 5 weeks ago and now are able to present you some more intermediate results [1]. Nevertheless, the challenge is still running. Therefore, please distribute, participate, advertise, and help us to generate a fully fledged ground truth for fact ranking [2]. All data will be made publicly available for further research.

To determine the importance of a fact is of utmost importance, if you want to properly understand the content of information. Usually, you have a rich variety of possible interpretations of information. To determine the proper interpretation, you are going to use the context, i.e. further available information. So, the question develops from "what is important?" to "what is important with regard of this specific context?".

Current Intermediate Statistics (from Aug 28):

Number of users who participated: 388

Sum of concepts done: 1736

446 unique concepts are covered (out of 541). 

Average concepts done per user: 4.47

CONCEPTS DONE:
0 concepts were done by 79 users. 
1 concepts were done by 79 users. 
2 concepts were done by 68 users. 
3 concepts were done by 41 users. 
4 concepts were done by 25 users. 
5 concepts were done by 22 users. 
6 concepts were done by 8 users. 
7 concepts were done by 7 users. 
8 concepts were done by 9 users. 
9 concepts were done by 6 users. 
10 concepts were done by 8 users. 
11 concepts were done by 4 users. 
12 concepts were done by 2 users. 
13 concepts were done by 1 users. 
14 concepts were done by 4 users. 
15 concepts were done by 3 users. 
16 concepts were done by 2 users. 
19 concepts were done by 1 users. 
20 concepts were done by 3 users. 
21 concepts were done by 2 users. 
22 concepts were done by 1 users. 
23 concepts were done by 1 users. 
24 concepts were done by 2 users. 
25 concepts were done by 1 users. 
31 concepts were done by 1 users. 
38 concepts were done by 2 users. 
40 concepts were done by 1 users. 
42 concepts were done by 1 users. 
58 concepts were done by 1 users. 
59 concepts were done by 1 users. 
62 concepts were done by 1 users. 
64 concepts were done by 1 users. 

EDUCATION:
highschool : 41 users.
phd : 57 users.
other : 22 users.
bachelors : 93 users.
masters : 174 users.

AGE:
33+ :  216 users.
19-25 :  68 users.
26-32 :  98 users.
0-18 :  5 users.

GENDER:
female :  90 users.
male :  297 users.

COUNTRY OF ORIGIN:
Angola :  1 users.
Belarus :  1 users.
Portugal :  4 users.
Philippines :  2 users.
Morocco :  3 users.
Greece :  5 users.
Ukraine :  3 users.
Indonesia :  3 users.
Sri Lanka :  1 users.
Italy :  13 users.
Iraq :  1 users.
India :  40 users.
France :  11 users.
Latvia :  1 users.
Pakistan :  3 users.
Syrian Arab Republic :  1 users.
Montenegro :  1 users.
Armenia :  1 users.
Mexico :  2 users.
Brazil :  10 users.
Venezuela :  1 users.
Croatia :  2 users.
Macedonia, The Former Yugoslav Republic of :  1 users.
Romania :  2 users.
Western Sahara :  1 users.
Algeria :  4 users.
Sweden :  2 users.
United States :  21 users.
Serbia :  8 users.
Nigeria :  2 users.
Estonia :  1 users.
Spain :  8 users.
Taiwan, Republic of China :  2 users.
Ireland :  1 users.
Israel :  1 users.
Russian Federation :  9 users.
Colombia :  3 users.
Switzerland :  1 users.
Azerbaijan :  1 users.
Kenya :  2 users.
Yemen :  1 users.
Malaysia :  2 users.
Viet Nam :  1 users.
Australia :  4 users.
Peru :  1 users.
Albania :  1 users.
South Africa :  2 users.
Netherlands :  8 users.
China :  2 users.
Somalia :  1 users.
Slovenia :  1 users.
Finland :  3 users.
Lithuania :  1 users.
Austria :  6 users.
Sudan :  1 users.
United Kingdom :  15 users.
Egypt :  2 users.
Bahamas :  1 users.
Hungary :  1 users.
Poland :  4 users.
Iran, Islamic Republic of :  2 users.
Bulgaria :  3 users.
Norway :  1 users.
Germany :  140 users.
New Zealand :  3 users.

COUNTRY OF RESIDENCE:
United Arab Emirates :  1 users.
Belarus :  1 users.
Portugal :  2 users.
Philippines :  1 users.
Morocco :  2 users.
Greece :  3 users.
Ukraine :  1 users.
Indonesia :  2 users.
Luxembourg :  1 users.
Sri Lanka :  1 users.
Italy :  11 users.
India :  31 users.
France :  13 users.
Jordan :  1 users.
Denmark :  1 users.
Latvia :  1 users.
Pakistan :  3 users.
Oman :  1 users.
Turkey :  1 users.
Czech Republic :  1 users.
Armenia :  1 users.
Canada :  3 users.
Brazil :  9 users.
Croatia :  1 users.
Romania :  3 users.
Algeria :  4 users.
Sweden :  3 users.
United States :  23 users.
Serbia :  4 users.
Nigeria :  1 users.
Saudi Arabia :  1 users.
Estonia :  2 users.
Spain :  7 users.
Taiwan, Republic of China :  1 users.
Ireland :  3 users.
Israel :  2 users.
Russian Federation :  2 users.
Colombia :  2 users.
Switzerland :  7 users.
Azerbaijan :  2 users.
Kenya :  2 users.
Norfolk Island :  1 users.
Yemen :  1 users.
Malaysia :  2 users.
Australia :  6 users.
Peru :  1 users.
Albania :  1 users.
South Africa :  1 users.
Netherlands :  12 users.
Somalia :  1 users.
Slovenia :  1 users.
Gambia :  1 users.
Finland :  3 users.
Lithuania :  1 users.
Austria :  5 users.
United Kingdom :  17 users.
Egypt :  1 users.
Bahamas :  1 users.
Belgium :  2 users.
Poland :  5 users.
Singapore :  1 users.
Iran, Islamic Republic of :  1 users.
Bulgaria :  3 users.
Norway :  1 users.
Germany :  154 users.
New Zealand :  2 users.

Overall confidence of users about the seen concepts: 2.687

NO. OF USERS PER CONCEPT:
On AVERAGE there are 2.18 users per concept.

NO. OF ANSWERS PER CONCEPT (STEP 1):
On AVERAGE there are 4.71 answers per concept.

NONSENSE STATEMENTS:

TOTAL number of nonsense sentences = 1371


Hint: You might wonder about the impressive high scores on the top of the list? Well, actually points are given exponentially, i.e. the longer you play, the more points you will score per processed concept.

References:
[1] Help Us with a Research Problem, July 30, 2914
[2] Fact Ranking Web-Application, http://s16a.org/fr/