Implementation of the Experimental System

We have implemented an experimental distributed multimedia system, called the Active Multimedia System (AMS), based on the concept of transformation among multimedia schemas and the object exchange manager, as shown in Figure . The system is implemented in the X-Window environment on the SUN SPARCstation 5. The object exchange manager is written in C++ programming language, while the other modules are written in C programming language.

On the top level of the system is an application window which provides an integrated user interface for performing various tasks. Illustrated in Figure  is a screen dump of the application window. The pull-down menus in the top portion of the window provide functionalities for editing, attributes defining, playback, transformation, packing/unpacking to/from object exchange format, transmission, etc. When composing a multimedia object, the user can choose buttons in the left panel of the window to create links and nodes for constructing the hypergraph structure of the MSS.

  


Figure: User interface window of the AMS system.

  


Figure: Example of creating a composite object and invoking transformation in the AMS system.

  


Figure: A screen dump of multimedia objects playback in AMS.

  


Figure: Another screen dump of multimedia objects playback in AMS.

Referring to the system structure shown in Figure , the steps of a typical scenario to compose, view, send, and receive a composite multimedia object in our system are as follows:

1. Editing the hypergraph structure of the MSS:
   The user can click on Add in Edit pull-down menu and buttons in the left panel to create nodes and links for the MSS. For example, in Figure , a composite object C0 composed of basic objects M0, M1, and M2 is created. A temporal link defining the temporal relation between M0, M1, and M2 is created. A location link and an annotation link defining the spatial relation and the annotation between M0 and another composite object C1 are also created, respectively. This hypergraph structure corresponds to the one shown in Figure (a).
2. Defining attribute:
   In the pop-up dialogue box brought up by clicking on the options in the Attr pull-down menu, the user can define the attributes of nodes and links in the MSS, such as media type of an object, the corresponding file name, temporal relation defined by a temporal link, and so on. For example, we can define in Figure  the type of M0 as image and M1 as audio as well as their corresponding file names. For the temporal link linking M0, M1, and M2, the temporal attribute is defined as co-begin.
3. Performing transformation from MSS to MDS:
   After the MSS is constructed, MDS in the Gnet pull-down menu, as shown in Figure , is chosen to perform the transformation from MSS to MDS.
4. Playback:
   Once the MDS is created, the user can view the presentation of the composite multimedia objects according to the MDS by clicking on Playback in Viewer pull-down menu. The playback by the Presentation module is based on the token flow in the MDS. When a token flows to a place in the MDS, the corresponding object is displayed. For example, in Figure , a text, an image, and an audio are displayed as there are tokens in their corresponding places. The corresponding MDS is also shown at the lower right corner of the screen. Later on, when tokens flow to , , and , two more texts (``Tumor???" and ``OK here.") and one animation (the freehand circle) are displayed as the annotation of the image as shown in Figure . Note that a reference window with text ``Please see reference [C2]." is shown on the screen despite the referenced object C2 is not a part of the MDS.
5. Performing transformation from MDS to MCS:
   By clicking on MCS in the Gnet pull-down menu (see Figure ), the MDS is transformed into MCS.
6. Packing into OEF:
   The user then packs objects into an OEF file by choosing Export in the Ex_Format pull-down menu.
7. Sending and/or depositing multimedia objects:
   The multimedia objects will be sent after the user clicks on Send in the Transmit pull-down menu. A Deposit option is also available in this pull-down menu for the user to store the objects in a specified directory.

At the receiving end, the receiver can choose Receive in the Transmit pull-down menu to retrieve the multimedia object and the corresponding G-Net specification of the MDS from the mailbox. The user can then playback the object according to the MDS.