Semantic Video Search Engine

September 2, 2018

I saw a link to a Semantic Video Search Engine” with the logo of MediaMill attached. Curious I did a bit of exploring and noted a video at this link. I learned that MediaMill is the name of the multimedia search engine. The system “watches” or “processes” a video and then assigns an index term or category to the subject of the video scene; for example a scene with a boat is tagged “boat.”

The function is to identify specific video fragments. The system provides automatic content detection. The goal is to make huge amounts of video data accessible. The video i watched was dated 2009. I located the MediaMill Web site and learned:

MediaMill has its roots in the ICES-KIS Multimedia Information Analysis project (in conjunction with TNO) and the Innovative Research Program for Image processing (IOP). It blossomed in the BSIK program MultimediaN the EU FP-6 program VIDI-Video, the Dutch/Flemish IM-Pact BeeldCanon project, and the Dutch VENISEARCHER project. The MediaMill team is currently funded by the Dutch VIDI STORYproject, the Dutch FES COMMIT program, and the US IARPA SESAME project.

The project’s news ended in 2015. Bing and Google searches turn up a significant amount of academic-oriented information. TREC data, technical papers, and links to the MediaMill Web site abound.

The question becomes:

Why has video search remained a non starter?

Since we started our DarkCyber video series, available on YouTube and Vimeo, we have had an opportunity to monitor how these two services index videos. YouTube, for example, makes the video available in the YouTube index in about a day, sometimes more. Vimeo does not index DarkCyber on a regular schedule. We provide an explicit link to the Vimeo video in our Beyond Search announcement of each week’s video.

It is possible to get a listing of DarkCyber videos on the not-well-known Google Video search service. You can find this index at Run the query “arnold darkcyber” and you will see a list of DarkCyber videos. Note that these are not in chronological order. In fact, running the “arnold darkcyber” query at different times generates results lists with different items and a similarly jumbled  or non chronological order. Why? Google search does not handle time in its public facing services. For high accuracy time based queries, you will have to use the commercial Google technology. Check out Recorded Future for some additional details.

Searching for video is a difficult task. YouTube search is quirky. For example, search for “hawaii volcano live shipley” and one does not get a link to the current live stream. YouTube provides links to old videos. To find the live stream, one has to click on the picture of Mr. Shipley and then select the live stream. Vimeo has its oddities as well. When I post a DarkCyber to Vimeo, I cannot search for it. The new video just sort of shows up on my Vimeo dashboard but I cannot locate the most recent video with a query. So much for real time.

Exalead tried its hand at video search, enlisting a partner for the effort. The test was interesting, but I heard chatter that the computational demand (think expense) made the project less than attractive.

My hunch is that video search is lousy because of the costs associated with processing video. Even basic rendering is a slog. Imagine the expense of grinding through a day’s worth of YouTube or Vimeo output?

To sum up, nifty video search ideas abound. Academics have a treasure trove of opportunity. But despite the talk about the cloud and the magic of modern technology. Video search remains difficult and mostly unsatisfying.

Maybe that’s why social media sites rely on those posting the video to tell friends where the content resides? Searching for a snippet of video is almost as difficult as wrestling with a Modern Talking Pictures catalog.

Stephen E Arnold, September 2, 2018



One Response to “Semantic Video Search Engine”

  1. aumentare mi piace video youtube sito on November 6th, 2018 5:03 am

    Sono più che felice per trovare questa pagina. Ho bisogno di grazie davvero per questo fantastico leggi !! Ho sicuramente goduto ogni piccolo it e anche io ho segnato nel libro vedere nuove cose nel tuo sito.

  • Archives

  • Recent Posts

  • Meta