Visualizing video archive content -- arte.tv

Okay, first at all, it's been a while that I have written a blog post here. I guess that's some tribute to the ever faster spinning world of digital media as I had concentrated more on shorter and therefore, faster means of communication such as twitter and (shame on me...) facebook. Nevertheless, while skipping through the pages of moresemantic I decided to revive the blog and to keep on posting about current research work....

This morning, I had to look up a documentary 'The Digital Bomb', which had been broadcasted yesterday evening on the German/French arte television channel. Arte is one of the public service television channels focussing on culture and arts. As many other television broadcasters, arte of course maintains a website and being as a television broadcaster there is also some sort of media archive. As being a public service television broadcaster, some strange regulations keep the archive from maintaining more than 7 days of tv-program -- but this is something completely different (as to speak with Monty Python). This morning, I made some discoveries in arte's tv archive, esp. about their way of visualizing content.

Besides being a little bit difficult to find the right mode of access -- esp. if you are looking for a specific date of broadcast, I succeeded in finding this nice portal page showing the most featured videos of yesterday's tv program including a timeline (at the top) for growing back (and forth) in time. I really like the 2-D tile pattern relating the size of the videos (represented by some significant key frame) to their popularity (or any other ranking). When you place the mouse pointer over a frame you will get more detailed information about the video and by clicking on it the video opens for reviewing.

All in all it seems to be inspired by the TED video archive and there's still room for improvement. I would like to see also timelines for shown content (not only broadcast or production date) as well as geographical information about production/content shown in interactive maps.

Now I have become curious about what else is out there? Any new innovative, interactive visualizations for displaying video archive content aside from the youtube mainstream??

Open PhD Positions in Semantic Multimedia Retrieval Project

OPEN Ph.D. POSITIONS at Hasso-Plattner-Institute (HPI), Potsdam (Germany) starting on the fourth quarter of 2009

Hasso-Plattner-Institute (HPI) is a privately financed institute affiliated with the University of Potsdam, Germany. The Institute's founder and benefactor Professor Hasso Plattner, who is also co-founder and chairman of the supervisory board of SAP AG, has created an opportunity for students to experience a unique education in IT systems engineering in a professional research environment with a strong practice orientation.

(for more information on HPI, c.f. http://www.hpi.uni-potsdam.de/ )

Project Description:
MEDIAGLOBE is part of the THESEUS research program initiated by the German Federal Ministry of Economy and Technology (BMWi), with the goal of developing a new Internet-based infrastructure in order to better use and utilize the knowledge available on the Internet. The focus of the research program is on semantic technologies, which determine contents (words, images, sounds, and videos) not through conventional methods (e.g., combinations of letters) but which are able to recognize and place the meaning of a content in its proper context. MEDIAGLOBE deals with digitalization, analysis, and semantic retrieval of historical, documentary audiovisual content. (for more information on MEDIAGLOBE, c.f. http://theseus-programm.de/theseus-mittelstand-2009/ )

The ideal candidate holds a MS degree in Computer Science or related field and is able to consider both theoretical and practical/implementation aspects in her/his work. Fluent english communication and programming skills are fundamental requirements. Since we are working on a multimedia repository with resources in German language, German language skills are welcome! Preferably the candidate has a background in one of the following
fields:
• semantic web technologies
• knowledge representations and ontology engineering
• audiovisual retrieval and analysis
• semantic search
• innovative web development
• user interface design for audiovisual content

The position starts as soon as possible and is full-time (40h/week) for the duration of the project until Oct 2011. Review of applications will begin immediately and will continue until the position is filled. The successful candidate will tightly work with international partners and has the possibility to pursue PhD work within the scope of the project.

How to apply:
Excellent candidates are invited to apply with:
• Curriculum vitae and copies of degree certificates/transcripts,
• Writing samples/copies of relevant scientific papers (e.g. thesis, etc.),
• Letters of recommendation.

Please send your application in PDF format indicating in the subject 'Application for PhD position‘ via email or via traditional mail to the following contact.

Contact and application:
Harald Sack
Hasso-Plattner-Institut für Softwaresystemtechnik GmbH
Universität Potsdam
Prof.-Dr.-Helmert-Str. 2-3
D-14482 Potsdam, Germany
phone: +49 (0)331-5509-527
fax: +49 (0)331-5509-325
email: harald.sack@hpi.uni-potsdam.de
web: http://www.hpi.uni-potsdam.de/meinel/persons/sack.html

Digitale Kommunikation

Am 21. Mai 2009 ist unser Neues Buch 'Ch. Meinel, H. Sack: Digitale Kommunikation' bei Springer erschienen, das ich hier heute vorstellen möchte. Hervorgegangen ist das Buch aus dem Absicht, unserem 2003 erschienenen Buch 'WWW - Kommunikation, Internetworking, Web-Technologien' eine zweite Auflage folgen zu lassen. Dies allerdings erwies sich als schwierig. 1200 Seiten in einer Disziplin, die sich so rasant weiterentwickelt, dass sich in den mehr als 5 Jahren, die seither vergangen sind eine Stofffülle angesammelt hat, die in einem Band einfach nicht mehr ausreichend behandelt werden kann.

Daher unternahmen wir Absprache mit dem Verlag das Wagnis, den Band in seine drei Grundbestandteile zu zerlegen und diese separat als einzelne Bände einer Trilogie zu veröffentlichen. Deren erster Band, die 'Digitale Kommunikation' liegt nunmehr vor. Die beiden Folgebände 'Internetworking' und 'Web-Technologien' sind in Vorbereitung und werden bald erscheinen.

Worum geht es im ersten Band der WWW-Trilogie? Wie schon im ersten Teil des 2003 erschienenen WWW-Buches dreht es sich in diesem Band um die Grundlagen der Rechnerkommunikation, die durch eine ausführliche historische Betrachtung eingeleitet werden und insbesondere die Gebiete der Kodierungstheorie und der Multimedia-Kodierung und -Komprimierung, sowie die Grundlagen der Kryptografie abdecken.

Hier das Inhaltsverzeichnis:

DIGITALE KOMMUNIKATION

(1) Prolog
(2) Geschichtlicher Rückblick
(3) Grundlagen der Kommunikation in Rechnernetzen
(4) Multimediale Daten und ihre Kodierung
(5) Digitale Sicherheit
(6) Epilog

In den Anhängen befindet sich ein ausführliches Personenregister, das vom ägyptischen Pharao Ramses II. und seiner 'ersten' Bibliothek bis hin zum 1970 geborenen Vincent Rijmen, dem Miterfinder des AES-Verschlüsselungsverfahren reicht. Das Buch stellt auf gut 430 Seiten mit zahlreichen Abbildungen die fundamentalen Grundlagen der digitalen Kommunikation dar. 17 einzelne Exkurse vertiefen dabei wichtige Themengebiete, die vielleicht nicht für jeden Leser gleichermaßen von Interesse sind. Jedes Kapitel ist mit einem ausführlichen Glossar abgeschlossen und über 250 Literaturverweise und Referenzen regen zum Weiterlesen an.

Weitere Informationen:

• Hier das ausführliche Inhaltsverzeichnis als pdf


• 'Digitale Kommunikation' bei amazon.de


• 'Digitale Kommunikation' bei springer.de

3. tele-TASK Symposium am HPI in Potsdam

Heute und morgen (13./14. November 2008) findet am Hasso-Plattner-Institut in Potsdam das 3. tele-Task Symposium statt. Ich freue mich auf ein spannendes Programm als auch auf interessante Gäste (unter anderem von der ETH Zürich mit dem Projekt REPLAY, Andreas Nürnberger von der Uni Magdeburg, das Fraunhoher IDM aus Illmenau und viele mehr...).

Natürlich werde ich selbst auch im Programm vertreten sein zum Thema "Semantisch unterstützteu Suche und Navigation in audiovisuellen Datenbeständen" (Slides gibt es später hier via slideshare).

Adaptive Multimedia Retrieval 2008 in Berlin, June 26-27, 2008 - Day 02

After the dinner cruise along the river Spree, the second day of Adaptive Multimedia Retrieval 2008 again starts with an interesting invited talk on the European answer to Google search engine technology - THESEUS.

Karsten Müller from Fraunhofer Heinrich-Hertz-Institute is presenting on "THESEUS Project - Applications and Core Technologies for the Semantic Web". First, Karsten makes clear, that THESEUS doesn't want to be Google ;-) THESEUS is a research program for a new internetbased knowledge infrastructure....which from my point of view means nothing else but "the semantic web"....
One part of the THESEUS project is ALEXANDRIA, the virtual library, being lead by Yahoo! with the objective of semantic processing of different forms of content to enable faster access to relevant content, which again means an increase in information quality. Concepts such as an automated tagging framework (including language error correction, synonym & tag merging, and topic focussing, identification of semantic relations), innovative navigation (by presenting thematically related contents) and interaction concepts are involved.
Another part is ORDO, which deals with "Organizing your digital life" with the goal to unify various data formats, multilingual information, structured and unstructured data on the web to enable homogeneous information sources.Problems such as separating important from unimportant, ordering information instead of searching, priorization, identification and visualization of interrelations are addressed.
TEXO is another part with the objective of "Realizing the internet of services" (being lead by SAP Research), offering personalized customized services, community involvement to improve services, as well as a smooth & seamless (userfriendly) adaption and integration of services.
PROCESSUS deals with the "Optimization of business processes" aiming for the objective of anytime providing the user with theright information at any stage of the business process.
MEDICO is another subproject dealing with "Towards Scalable Semantic Image Search in Medicine" and being lead by Siemens.
CONTENTUS, as being the last Use case "Content access and generation from cultural institutions is lead by the Deutsche Nationalbibliothek. Being part of CONTENTUS are tasks such as Digitizing books as well as audiovisual material (including the German Music Archive in Berlin) protecting the cultural heritage. The goal is the semantically interlinked collection of content to achieve a next generation multimedia library.
.....impressive and ambitious project!

The upcoming section this morning is on "Image Tagging" and Marius Renn (at least I hope so) from TU Kaiserslautern is givig a presentation on "Automatic Image Tagging using Community-Driven Online Image Database". Automatic image tagging requires a lot of training data....and flickr is delivering tons of tags per day...but are these flickr data really good candidates for learning? So, in the end, unfiltered community image sets directly do not provide satisfying results. Alas, these databases at least allow large scale image aggregation...
The next talk in this session is given by Christian Hentschel from Fraunhofer HHI Berlin about "Automatic Image Annotation Refinement using Object Co-Occurences". Again, flickr is the target image set with its huge collection of more than 2 billion images, growing by 3 million photos every day. Objects always appear and are perceived in a semantic context.

The following session is on "Symbolic Music Retrieval" and starts with Rainer Typke from Austrian Research Institute for Artificial Intelligence (ÖFAI), but I had to skip this talk. Anyway, the samples of the reduced MIDI files were quite interesting (although I'm not a fan of the Scorpions!). OK, I had to ask afterwards about the usefulness and application of his approach. In music retrieval it can be used to reduce the index size down to 30% of the original index. Also QBE-processing will become much easier while on the other hand you might connect this MIDI-collection to real music files.
The last talk of the morning session is given by Giancarlo Vercellesi from University of Milan on "Automatic synchronization between audio and partial music score presentation". He presents the ParSi architecture, which perfoms an alignment of PCM signal and partial MIDI scores.

The afternoon session is simply entitled with "Systems". Fernando Lopéz from Madrid is giving a presentation on "Towards a fully MPEG-21 compliant adaption engine: complementary description tools and architectural models". Within the MPEG-21 framework several aspects of metadata-driven adaption is not clearly covered. He introduces CAIN, a tool for adapting Digital Items e.g. to different output devices.
The session continues with a presentation on "Mobile museum guide based on fast SIFT recognition" with the objective to identify paintings in galleries simply with the help of mobile pattern recognition without any extra installation on site. The SIFT (Scale Invariant Feature Transform) method is a rather cool algorithm for detecting local features within images that are used to map photographs taken with your PDA or mobile phone in the image gallery with reference pictures from a given database. And actually the live demo did work :)
I guess, we will also use the SIFT-algorithm in yovisto for synchronization of ppt/pdf-slides with the lecture video.

For the last session - "Structuring of Image Collections" - only one speaker showed up. Marc Gelgon is presenting on "Geo-temporal structuring of a personal image database with two-level variational Bayes mixture estimation".

[to be continued...]

Adaptive Multimedia Retrieval 2008 in Berlin, June 26-27, 2008

The next two days, we are attending the Berlin Adaptve Multimedia Retrieval 2008 Workshop at the Heinrich Hertz Institute being located in downtown Berlin. So, it's pretty close to home and the only travelling involved was by S-Bahn :)

The first speaker is Francois Pachett from Sony CSL giving a keynote entitled "What are our audio features worth?"
The fundamental questions are "What makes objects what they are?", ""What are the features of subjectivity?", "How do we perceive objects and how can we transfer this to a machine?" Pachet's research is concerned with the classification of musical objects based on the so called polyphonic timbre that describes the sum of all features of a music object. Interesting thing is the identification of hubs, i.e. songs that are pretty close to every other song. Hubs in general seem to be mere artefacts of static models.
Interesting fact ist that there are companies now, predicting if your song is going to be a hit. Their judgement also relies on feature analysis and they even give recommendations how your song can be improvent to become a hit. Of course you have to pay for that service...but does it really work??

After the coffee break, there's a session on User-Adaptive Music Retrieval. The first talak is presented by Kay Wolter from Fraunhofer IDMT Ilmenau on "Adaptive User-Modelling for Content-Based Music Retrieval". They are adapting a content-based music retrieval system (CBMR) according to user preferences that are determined by acceptances and rejections of recommended songs by the user, which is furthermore used to improve the quality of music recommendations....Reminds me somehow to Pandora or last.fm...
The second talk is presented by Sebastian Stober from Otto-von-Guericke-Universität Magdeburg on "Towards User-Adaptive Structuring and Organization of Music Collections". So, wouldn't it be nice to structure your music collection automatically...but not in the way the software tells you, but the way you like it? The presented system is based on an general adaption approach using self-organizing maps that can be adapted by user interaction.

The first afternoon session is on "User-adaptive Web Retrieval" and starts with a presentation of Florian König from Johannes-Kepler-Universität Linz on "Using thematic ontologies for user- and group-based adaptive personalization in web searching". He introduces Prospector, which is a generic meta-search layer for Google, not constrained only to web search, based on re-ranking of search results and deploying user modells based on Open Directory Project (ODP) taxonomies. As far as I have understood, the applcation is based on the carrot2 framework for open source search engine result clustering.
Next, David Zellhöfer from BTU Cottbus presents on "A Poset Based Approach for Condition Weighting". Similarity search can be determined according to different conditions w.r.t. the search query. Esp. different people have different expectations if it comes to similarity. So, condition weights have to be determined by psychological experiments.

The second afternoon session is about "Music Tracking and Tumbnailing" and starts with a presentation of Tim Pohle from Johannes-Kepler-Universität Linz on "An Approach to Automatically Tracking Music Preference on Mobile Players". Ok, so the basic problem is, someday you will get bored by the music selection on your ipod. Therefore, the goal is to remove songs that you don't like anymore and replace them with new songs that you probably will like. How do you achieve this? Well, with according user feedback, i.e. by tracking the user's decision on choosing or skipping tracks. Tracks that have recently been skipped often will be dropped and replaced by tracks that are similar (according to some feature analyses) to the remaning tracks.
Next, Björn Schuller from Technische Universität Münschen is presenting on "One Day in Half an Hour: Music Thumbnailing Incorporating Harmony- and Rythm Structure". Music thumbnailing is some really cool feature, Just imagine, your sitting in your car and you are looking for another track to hear, but your player always starts songs at the beginning and they have long and boring intros. Therefore, getting to the most interesting (or significant) part of the song immediately would really be something...

The sessions close with an invited talk given by Stefan Weinzierl and Sascha Spors on "The Future of Audio Reproduction. Technology - Formats - Applications". Promissing title, let's see.... We start with a brief history of audio recording and reproduction technology starting from the very first phonograph to modern multichannel spatial surround sound systems. So, the future seems to be real sound field synthesis (wavefield synthesis, WFS) instead of relying on psycho-acustic effects as in today's stereo. Here, an array of loudspeakers reproduces exactly the wave front of the original sound source. For transmitting signals like this, no single channels are recorded anymore, but the original sound signal (without spatial characteristics of the room where it has been recorded, because this would interfere with the characteristics of the room, where it is reproduced) including movement and position of the sound source. Besides existing VRML and MPEG-4 Audio BIFS that focus more on visual scene description than on audio scene descriptions, there is the proposal of a new modeling language for high resolution spatial sound events called ASDF (Audio Scene Description Format).

[...to be continued in Adaptive Multimedia Retrieval 2008 in Berlin, June 26-27, 2008 - Day 02]