The Challenges of Networking Multimedia applications

The Challenges of Networking Multimedia applications

Multimedia implies video, and because of the high information content, video implies high network bitrates. Much of what applies to video is also true of audio, although the network speed requirements are lower, timing constraints are just as important. In fact, one of the most noticable problems with some mulitmedia applications occurs when the audio and vido get out of ‘synch’, that is, the timing relationships between them are not preserved during capture, transmission or presentation. This results in problems like loss of ‘lip synch’ for example. Networking multimedia poses a different set of challenges to networking non real-time data, that is, data for which the time of presentation is not critical at the milli-second level. Although the systems may be superficially similar, multimedia data has quite different requirements.

Multimedia data has very tight bounds on delay and delay variation (jitter). Unlike non real-time data, elements of the video sequence must be delivered across the network in strict order. Frame n must arrive before frame n+1. Similarly, the variability of the time between frames must be within strict bounds. To avoid jerkiness and flicker, the time between the display of frames, must be as near constant as possible.
Video streams require a large amount of bandwidth to be available for a sustained length of time, whereas non real-time data is able to fit into whatever amount of bandwidth is available. If a non real-time data file is transferred over a low bitrate connection, the transfer time is increased. The file is still useable, whereas if a real-time multimedia stream is to be delivered over a connection at a bitrate lower than required, it will be unviewable, or at least degraded in quality.
Users of real-time audio/visual data can tolerate some loss, as long as it isn’t too noticable, whereas users of data transfers such as financial transactions, database records and so on cannot. Clicks or gaps in the audio can be ignored, and some video frames, or parts of video frames, can be lost without the viewer really noticing, but for normal data, errors or loss can be catastrophic, particularly if the errors are initially undetected. At first sight this might seem to be an advantage of multimedia data over error-sensitive data. Unfortunately, most protocols (such as TCP/IP) are designed for error-sensitive data, and if a part of a message is lost, they will invoke mechanisms for recovering the lost data, usually through re-transmissions. This is exactly what is NOT wanted for real-time mulltimedia, since the re-transmissions introduce additional delay and jitter into the video stream so that not only does the re-transmitted data arrive too late to be useable, it also occupies vital bandwidth unnecessarily.
In summary, the characteristics of multimedia transmissions over a network are quite different to those of non real-time, error-sensitive data, and introduce a whole new set of challenges for server, network and client design.