Customization and Delivery of Multimedia Information

J Kay and R.J.Kummerfeld
Department of Computer Science
University of Sydney

(PLEASE NOTE: this paper appeared in Proceedings of MULTICOMM '94, Vancouver, Nov 1994)

Abstract

Current research into architectures for delivery of multimedia (video, audio and data) to the home have concentrated on real-time video-on-demand. This paper describes a project that treats multimedia objects as messages and uses a store-and-multicast delivery system for multimedia objects. The system uses a directory service to store descriptions of objects. These descriptions are then processed by filters that use a model of the user of the system to discover objects of interest. An Individualised News Service then takes the filtered object descriptions and constructs a composite news program for presentation to the user. The actual objects (movies, TV shows, news programmes, radio shows) are then transferred to the user system using a multicast message delivery protocol.

Introduction

The mass media of the future will involve delivery of multimedia (video, audio, text) information to the home. In its simplest form this will be a simple replacement for the video store: online selection and immediate delivery of television programmes or video-on-demand.

Current proposals (eg Ramanathan 1994, Federighi 1994) for delivering this type of service utilise high performance servers with large amounts of storage and high speed networks capable of delivering the programmes in real time to a number of simultaneous users. Users of this service are expected to browse an online TV-Guide and select programmes that are delivered for immediate viewing. The video-on-demand services would also allow arbitrary stopping, starting, backtracking of the program at any time, much like a video recorder. Much more interactive services are envisaged with instant surveys after a program, interactive advertising etc. Two-way, interpersonal communication services are also predicted: the long awaited video-phone.

This vision of the future presents a number of problems: the servers must be capable of delivering high data rates from the storage in real time; the network must be able to give quality of service guarantees to allow real time viewing; there is little opportunity for the user to customise the material they view and little attention has been paid to the problem of getting the programmes to the servers.

Video-on-demand and interpersonal communications require a reasonably fixed bandwidth and Quality of Service during the life of the session. In the case of video-on-demand this will be at least 8Mbps for a compressed broadcast quality video/audio stream. This rate will be required between the subscriber and the provider's head-end based server for each subscriber concurrently viewing a program. This is because each subscriber may view a different program, start a program at a different time or pause it at a different time. For a typical cable system covering 5000 subscribers at the head-end a bandwidth exceeding 40Gbps (5000 x 8Mbps) would be required. This is well beyond the current state of the art. Clearly, a future increase in performance of servers and an increase in bandwidth between the subscriber and the head-end is required to solve the problem.

An Alternative Approach

The services commonly thought of when the network of the future is discussed are the broadcast services of today (or video-on-tap), video-on-demand and interactive-video. We propose another service and accompanying architecture: video-on-order: the asynchronous selection and delivery of programmes.

Current and proposed home systems are relatively simple terminal devices that simply display the programmes. They may have key pads or even full keyboards and mice for interactive sessions but the programme material is not stored in the terminal. Instead, storage is relegated to an external device, the VCR. With the fall in price and increase in capacity of mass storage devices, random access digital storage of video material in the home will be available in the future.

Two hours of broadcast quality, compressed video/audio require a storage device of approximately 8Gigabytes. Single magnetic disk drives with that capacity are available today and capacities will increase and prices fall dramatically over the next few years.

While such storage devices will be used in the same way as the current VCR, as offline storage, it will also be connected to the display device (the TV set of tomorrow) via a high speed network and have substantial local intelligence. Such a system would allow programmes to be delivered to the subscriber from the head-end server at any time and stored in the subscribers own storage device.

The architecture of the future home system would then involve bringing the network cable into a closet box instead of the set top box. From there a home network would connect terminal devices of varying capability. The current TV in the living room would be replaced with an almost identical device (but perhaps with a higher resolution screen and better sound system) and remote control keypad. In the study the terminal may have a smaller screen and lower quality sound but would include a full keyboard and mouse. The closet box would contain substantial mass storage and a powerful CPU as well as network interfaces.

Delivery of Programs

Our proposal is that programmes (movies, newscasts, radio programs etc) should be delivered to several end-points simultaneously in the form of multimedia messages using a multicast message delivery protocol. This will be built above a multicast packet delivery service at the network layer.

A subscriber or a subscriber's agent may initiate delivery of the programme from a server. The programme would then be sent to the subscriber end-point as a message using the multicast message delivery protocol. Other end-points may join the multicast group at any time and receive a partial message but subsequently request retransmission of the missing parts.

This process may operate between end-points and a local (neighbourhood) server for popular material, between end-points and a remote server for more specialised material or between local and remote servers for the distribution of material from production point to servers.

We describe this method of delivery as a store-and-multicast message delivery system because it is similar to traditional store-and-forward email delivery systems.

As well as delivery of multimedia objects such as movies and other programs between homes and servers it could also be used for distribution, or publication, of such objects. Production houses would use MMD to distribute programmes to servers as well as distribute directory entries to a global distributed directory service.

These directory servers are then available for browsing by the home user to see "what's on" and possible ordering of programmes for delivery. They are also the place where intelligent agents can scan available material to build personalised news programs or notify a subscriber of a new movie that may interest them.

The architecture consists of three main entities: information providers such as television and radio production houses as well as data providers; neighbourhood and regional (head-end) servers that store substantial amounts of information and end-points in homes and offices. Messages are passed between these entities using the store- and-multicast system.

Customisation

The video-on-order service opens up vast possibilities of customisation and selection of programme material. Users are already swamped by the quantity of material available on conventional mass media such as cable television as well as the Internet. In the future an even larger range of multimedia material: television programmes, movies, radio programmes as well as data will be available. This can only serve to provide users with so much choice that they are overwhelmed by it.

Our approach is to provide a filtering system that can take descriptions of the objects available to the user and pass these through a filter. Objects that seem likely to appeal are allowed to pass through; others, that are unlikely to interest the user are discarded.

The keystone of the filtering process is a user model, a collection of useful information about the user. The particular aspects that interest each user and the degree of their interest defines which items can pass the filter.

To see how this operates in the filtering of news items, consider how a form of filtering is applied to selection of material for current newspapers. Editors have models of their readers and they use this to decide how much space to allot to each aspect. This process involves a set of sub- models, one for each of the various parts of the paper: world news, local, sports and so on. Different papers have different reader-models, based on factors like locality and different target populations for the paper.

The next stage in this filtering process is that the user decides which paper to buy. From there, the reader decides which parts of the paper to read: they may ignore the sports section entirely; they may skim foreign news involving countries and issues they are not deeply concerned about; they may pour over the local news that affects them directly.

Our filtering mechanisms allow the user to define the filtering from the very first. They do have to be satisfied with the priorities defined by editors of a newspaper. If they are deeply interested in a subject, they can have access to the very substantial set of all news that is produced about it, including source materials.

Of course, for all this to work, we need to build the user model. There are many possible strategies for this. The simplest is to interview the user, either with a series of simple questions or with a more sophisticated graphical interface.(Kay 1990) Another simple approach is to require the user to rate unfiltered news items and use this to refine the user model: over a period of time, this should give a detailed and effective user model.(Orwant 1993)

Perhaps the most promising approach is to build stereotypes(Kay1994) user models for a set of opinion leaders and allow the user to select from these. If these are helpful for many users, they will have considerable commercial value. This reduces the burden on the users: they need only examine these stereotypes and select the ones that they like best. They can also make personal amendments to these.

Although this example has been in terms of newspapers, we will see a merging of the roles of newspapers and the various electronic media. The principles are the same. Personalised filters have an extremely important role to play in making it feasible for users to benefit from the explosion in available material.

Implementation

A prototype of the proposed architecture is being constructed using currently available workstations and networks. The programme material is restricted to text and audio since the storage and bandwidth requirements of video make it unwieldy for experimental purposes.

We describe the system in terms of an Individualised News Service (INS) (see diagram below). The INS is given information about objects available from the remote object store that have been deemed `interesting' to an individual, family or group by one or more filter programs. It then constructs a news program from these individual objects according to the user's preferences stored in the user model. The news program is then available for viewing in its entirety by the user or individual parts may be viewed.

There are at least two filtering steps in the process (the diagram shows two): one is carried out remotely using a partial user model and examines descriptions of objects as they are published in the directory service, and the other occurs locally with a more complete user model that may contain more personal aspects.

For simplicity, only the main data paths, programs and storage areas are shown in the figure.

Components on the `local' side of the diagram are located at the user site (home, office). Those on the `remote' side are elsewhere in the network, eg at a neighbourhood or regional server..

The `UI' is Mosaic, a World Wide Browser produced by NCSA (Mosaic 1994) Other World Wide Web Browsers could be used and we plan to experiment with browsers that offer a simplified user interface.

The `Server' component is based on the NCSA HTTPD server (McCool 1994) but enhanced with a number of programs that are invoked in response to the selection of objects by the Mosaic client.

The `Local Object Store' is simply the file system of the local system. However, the `Remote Object Store' is potentially the entire World Wide Web (Berners-Lee 1994)

The `Directory' components are servers for the Whois++ protocol(Deutsch 1994) modified to allow new entries to be sent to them as messages for incorporation into their respective databases (publication) and to invoke a `Filter' program on the new entries. The filtering process uses components of a partial user model (`PUM') to select or reject directory entries for possible use by the local system.

The `Filter' may in turn send an entry to a local directory server and the process repeats with the final filter passing selected entries to the `INS' or Individualised News Service.

The INS then uses the full User Model (`UM') to make a final selection of programmes to be incorporated into the customised view of the news. The programme objects are then requested from the remote object store and delivered, using the multicast message delivery protocol, to the local object store.

When the user wishes to view the current newscast the INS presents several possible views of the selected items. One view is as individual items under topic headings (`local news', `world news', `Middle-East', `sports') another view is a composite of these and the simplest view is a sequential presentation of all the items.

Conclusion

The important contributions of our project are: the treatment of multimedia objects as messages instead of as streams of data; the individualised selection and customisation of programmes using descriptions of objects; the store-and-multicast message delivery architecture; and the formalisation of the `publication' process.

Nicholas Negroponte, writing in Wired Magazine, May 1994 has said: `I see a huge market in the agent business, modeled more after the added value of an English butler or the Librarian of Congress...... So far, in the theater of Wall Street, the personal information filter business has only played a bit part. I assure you that it will be tomorrow's lead role on the stage of success.'

References

Berners-Lee 1994
T Berners-Lee, The World Wide Web Initiative: The Project

Deutsch 1994
P Deutsch, R Schoultz, P Faltstrom, and C Weider, Architecture of the WHOIS++ service IETF draft document: draft-ietf-wnils-whois-arch-00.txt 1994.

Federighi 1994
C Federighi and L A Rowe, A Distributed Hierarchical Storage Manager for a Video-on-Demand System Symposium on Elec. Imagin Sci. & Tech., San Jose, CA, 1994.

Kay 1990
J Kay, um: a user modelling toolkit Second Intl User Modelling Workshop 1990.

Kay 1994
J Kay, Invited keynote address: Lies, damned lies and stereotypes: pragmatic approximations of users UM94 - 1994 User modeling Conference 1994.

McCool 1994
R McCool, NCSA httpd Overview

Mosaic 1994
Mosaic, Description of Mosaic

Orwant 1993
K Orwant, Doppelganger goes to school: machine learning for user modeling MIT MS Thesis, MIT Media Laboratory 1993.

Ramanathan 1994
S Ramanathan and P V Rangan, Architectures for Personalized Multimedia Services IEEE MultiMedia, Vol. 1, No. 1, Spring 1994 1994.