The Importance of Relevance - Intermediate Results

Current Highscore List of our fact ranking challenge

In my last post, we invited you to take part in our research challenge, which was about creating a ground truth for fact ranking algorithms. To determine the importance of a fact is of utmost importance, if you want to properly understand the content of information. Usually, you have a rich variety of possible interpretations of information. To determine the proper interpretation, you are going to use the context, i.e. further available information. So, the question develops from "what is important?" to "what is important with regard of this specific context?".

We started the original challenge about 3 weeks ago and now are able to present you first intermediate results [1]. Nevertheless, the challenge is still running. Therefore, please distribute, participate, advertise, and help us to generate a fully fledged ground truth for fact ranking [2]. All data will be made publicly available for further research.

Current Intermediate Statistics:
Number of users who participated: 110 (Thanks to all you you!!!)
Number of overall processed concepts: 509
Overall 200 unique concepts are covered (out of 541).

Average concepts processed per user: 4.63

Detailed number of processed concepts per user:
0 concepts were done by 15 users.
1 concepts were done by 27 users.
2 concepts were done by 21 users.
3 concepts were done by 12 users.
4 concepts were done by 7 users.
5 concepts were done by 8 users.
6 concepts were done by 2 users.
7 concepts were done by 1 users.
8 concepts were done by 4 users.
9 concepts were done by 1 users.
10 concepts were done by 2 users.
11 concepts were done by 2 users.
14 concepts were done by 1 users.
16 concepts were done by 1 users.
20 concepts were done by 1 users.
22 concepts were done by 1 users.
25 concepts were done by 1 users.
31 concepts were done by 1 users.
53 concepts were done by 2 users.

Participant statistics:

EDUCATION:
highschool : 7 users.
bachelors : 28 users.
masters : 47 users.
phd : 23 users.

other : 5 users. 

AGE:
33+ : 43 users.
26-32 : 34 users.
19-25 : 33 users.

GENDER:
female : 25 users.
male : 85 users.

COUNTRY OF ORIGIN:
United States : 9 users.
Serbia : 6 users.
Spain : 2 users.
Ukraine : 1 users.
Russian Federation : 3 users.
Colombia : 1 users.
Italy : 6 users. India : 2 users.
France : 3 users.
Malaysia : 1 users.
Australia : 1 users.
Albania : 1 users.
China : 1 users.
Pakistan : 1 users.
Finland : 1 users.
Austria : 2 users.
Montenegro : 1 users.
United Kingdom : 8 users.
Brazil : 3 users.
Poland : 2 users.
Iran, Islamic Republic of : 1 users.
Macedonia, The Former Yugoslav Republic of : 1 users.
Croatia : 1 users.
Germany : 49 users.
Algeria : 1 users.
New Zealand : 1 users.
Sweden : 1 users.

Overall confidence of users about the seen concepts: 2.585

NO. OF USERS PER CONCEPT:
On AVERAGE there are 1.295 users per concept.

NO. OF ANSWERS PER CONCEPT (STEP 1):
On AVERAGE there are 4.825 answers per concept.

We will keep you posted about the results.
Please distribute, participate, advertise, and help us to generate a fully fledged ground truth for fact ranking.

Hint: You might wonder about the impressive high scores on the top of the list? Well, actually points are given exponentially, i.e. the longer you play, the more points you will score per processed concept.

References:
[1] Help Us with a Research Problem, July 30, 2914
[2] Fact Ranking Web-Application, http://s16a.org/fr/

Do Students really appreciate online lecture recordings...??

For another paper I was doing some research on how students are using online lecture recordings. I started with asking Google about studies and evaluations on the topic. But, as I had to find out, there are only some primitive evaluations (with a small number of participating students often limited to a spectial course or a single university) as to the satisfaction of the students -- often with 'overwhelming' results (as for advertising the used lecture recording technology...).

Here you may find a few examples (recent evaluations with more than 100 students seem to be rare...):

• Maree Gosper, Margot McNeill, Karen Woo et. al.: Web-based Lecture Recording Technologies: Do Students Learn From Them?, in In. Adelaide: Higher Education Research and Development Society of Australasia, 2007.
  (interesting summary of several surveys down under in Australia with Lectopia/iLectures, more than 10.000 students addressed but only about 800 answered...)


• Marc Krüger: Pädagogische Betrachtungen zu Vortragsaufzeichnungen (eLectures), in i-com, Zeitschrift für interaktive und kooperative Medien, Oldenbourg Wissenschaftsverlag 3 56--60 (2005).
  (first try of a more general study...)

I was not able to find a more general, summarizing report dealing with a variety of scenarios and clustering the results (inluding methodological soundness and decent empirical basis).

So...first thing -- if you know about any study or evaluation (with significant empirical basis), please write a comment with a link to the source (!)
Second -- I know that this will also only be a rather limited and primitive poll -- please fill out the small poll below. Maybe we gain a little bit more insight on student's satisfaction concerning online lecture recordings -- not only restricted to a single course or a single university....