A smart image is an image plus the associated knowledge structure. It is a unique technology developed jointly by researchers from the University of Pittsburgh and Siemens Corporate Research.
Why we need smart images? Because images require knowledge to be interpreted correctly. Image operations are often time-consuming and must be explicitly requested in existing systems. In the smart image system, the smart image is capable of performing actions by itself. It can also provide more information to the user. The concept is similar to a Smart House.
In what follows, we present an example, using medical image information systems as the potential application. Other applications include the intelligent factory, the intelligent office, the service engineer's assistant, and active image information management.
Frame #1 Initial frame for the smart image system, showing the medical information system application example. For this smart image application, the system will allow the doctor to examine the image data for the patients.
Frame #2 First, the user can select the name of the physician.
Frame #3 Then, the user enters the password.
Frame #4 Once the physician is selected, his name appears in the upperleft area of the smart image frame.
Frame #5 The patients are displayed, for the user to select.
Frame #6 The selected patient is highlighted.
Frame #7 The patient information is displayed in the upperleft area of the smart image frame.
Frame #8 One of the main purposes of the smart image is to predict the user's intention. When the smart image is capable of predicting some action, such action will be performed as a background process. The background window is the light green window on the right hand side. We show this window to illustrate the smart image capability. Normally, this window need not be shown.
Frame #9 The examination window is shown. Now the physician can select a particular type of examination that is available.
Frame #10 For example, the physician selected the most recent MUGA study.
Frame #11 Since the physician will most likely also examine the previous MUGA studies, in the background process the system prefetches these images from the PACS, as shown by the two iconic images in the background window.
Frame #12 The current image is retrieved first and displayed.
Frame #13 Once a background image prefetching task is complete, that icon disappears.
Frame #14 The background prefetching task is complete for both studies, therefore both icons disappear.
Frame #15 We close the background window. Now the physician is examining the current MUGA image. He can define hotspots to be associated with a smart image. For example, the current MUGA image already has some associated hotspots.
Frame #16 Again, to illustrate the smart image concept, we will bring up the POSTQUEL window, and show a database query to retrieve the information associated with the smart image from the database.
Frame #17 In the database window, three previously created hotspots are shown. For each hotspot instance, we can show the image it is associated with, the hotspot type (in this case it is type 10), the region of interest ROI (255,255,0,0), and the agent(s) that created these hotspots (in this case, the hotspots were created by the protocol automatically).
Frame #18 The protocol is the user model for a physician to examine the patient. The different states reflect the different tasks the physician want to perform. For example, we are currently at the state "identify abnormalities", shown in red.
Frame #19 Now if the physician introduces a new hotspot, it will activate a hotspot rule, leading to the performing of some actions and the change of protocol state.
Frame #20 The protocol state now changes to "time domain study". It is shown in red.
Frame #21 Now if we examine the contents of the database, three new hotspots have been created. The first one is created for the current image under examination. Therefore, its creator is the user. The other two hotspots are created by the smart image system, by propagating the hotspots to images of the same type in previous studies. Therefore, the creator is "propagate". This illustrates the smart image can automatically add knowledge to itself, i.e. the hotspots can be created and propagated to other images. The hotspots enable the smart image to automatically react to certain events, such as the presence of a tumor in a medical image.
Frame #22 The physician now finishes the study. He can record his findings in the diagnosis report, and suggests a treatment plan. The smart image system has the capability to allow the user to add text information (as shown in the next frame), or voice recording, or links to other studies. So it is also a powerful multimedia or hypermedia system.
Frame #23 The text information on the diagnosis report and recommendation is illustrated. For example, the report may say the following: |NMR images examined. Ischemia (heart wall muscle damage)| |due to a stenosis (full or partial block in a vein). | |Recommend a cardiac angiogram be performed. |
Frame #24 We return to the main smart image frame.
Frame #25 Now the physician can send the smart images to other physicians. The purple "Imail Utility" window allows the user to select the receiver's name, and the name of the smart image to be sent.
Frame #26 The smart image is transmitted progressively, so that the important area (such as the region of interest in the hotspot) will be transmitted and displayed in full resolution, and the background area will become clearer and clearer as time goes by.
The above demo has illustrated some of the important characteristics of the smart image. First, it is capable of performing operations by itself, such as the prefetching of images. Second, it is capable of adding knowledge by itself, such as the propagation of hot spots. Third, it is capable of reacting to an event, such as the triggering of a hotspot rule. Fourth, all kinds of information, including text, audio, video, graphics, can be appended to a smart image. Finally, the smart image knows how to transmit itself in the most efficient and economical manner. The University of Pittsburgh and Siemens team is now exploring new ways of applying the smart images.
