PERFORMANCE EVALUATION OF MPEG-4 VIDEO OVER REALISTIC EDGE WIRELESS NETWORKS Andrea Basso

Byoung-Jo “J” Kim

University of Victoria BC, CA [email protected]

AT&T Labs – Research, USA [email protected]

Abstract: This paper examines the performance of the delivery of MPEG-4 video over a realistically emulated EDGE (Enhanced Data for GSM Evolution) wireless network. The video delivery system is based on MPEG-4 video simple profile and on the IETF protocols RTSP, RTP, and RTCP for the media delivery and synchronization in compliance with the 3GPP specifications for Packet Streaming services (PSS). The EDGE network emulator is based on a well-verified Markov chain model. The effect of the different EDGE network parameters is evaluated in terms of video quality. Keywords: wireless packet video, MPEG-4, EDGE, 3G cellular, streaming video, emulation INTRODUCTION Recently, the research community and wireless service providers have put a lot of efforts in evaluating the performances of EGDE (Enhanced Data for GSM Evolution) networks and their capability to offer multimedia services that include video and audio delivery. This paper examines the performance of the delivery of MPEG-4 video over a realistically emulated EDGE wireless network [4] that includes the effects of background traffic, adaptive modulation, time-varying propagation, and system-wide interference and draws some conclusions in terms of optimal parameter set for the wireless infrastructure and the video delivery system. Our EDGE network emulator is based on a well-verified model described in [2]. EDGE is the evolutionary upgrade of GSM (Global System for Mobile) [4]. It adds always-on packet data capabilities with the same carrier spacing, and provides higher data rates than GPRS by using a larger number and more sophisticated adaptive modulations and coding schemes. EDGE is designed to allow GSM and GPRS network upgrades with no or little additional spectrum, possibly offering an interim solution before 3G cellular services which have designated spectrum bands, or an alternative to 3G networks for operators with no additional spectrum. EDGE is in the process of being deployed by several operators in the US and Europe. In this paper we will present various scenarios of MPEG-4 video streaming over EDGE. The video delivery system is based on MPEG-4 video simple profile and on the standard IETF protocols RTSP, RTP, and RTCP for the media delivery and synchronization. In our simulation scenario, the radio link layer is generic and does not provide video-specific support. Several research proposals can be found in the literature claiming

Zhimei Jiang AT&T Labs – Research, USA [email protected]

substantial performance gains thanks to the tight integration of radio link layer and video transmission, e.g., retransmission and/or FEC are adapted for the relative importance of a video packet such as a packet containing Iframes. However, such approaches are difficult to deploy in practice, since they require all base stations to implement such an integration to support video streaming. This paper is organized as follows. In Section 2, we review the basics of the EDGE wireless infrastructure. In section 3, we review the MPEG-4 video and its error resilience capabilities. In section 4 we discuss our EDGE wireless network simulator. In section 5, we present experimental results showing the performances of MPEG-4 streaming over EDGE. Finally we summarize our major findings. BRIEF OVERVIEW OF EDGE GPRS is a wireless packet-based network architecture using GSM radio systems. The original design of GPRS has been driven by non real-time requirements. Nevertheless the adaptive multislot capability of GPRS which allows for dynamic allocations of timeslots to a given terminal, providing enough bandwidth for the support of a limited set of multimedia enabled services. Furthermore the native support of the IP protocol allows a simple interfacing of current IP/RTP based multimedia applications such as video streaming to a GPRS network. Beyond GPRS, EDGE (Enhanced Data Rates for GSM Evolution) is a generation 2.5 air interface which represents a step forward to UMTS [4]. It provides higher data rates than GPRS and introduces a new modulation scheme called eight-phase-shift keying (8 PSK) that allows a much higher bit rate and automatically adapts to radio conditions. EDGE shares its available bandwidth among users on one carrier in a sector, which ranges from several 10’s of kbps to 384 kbps, depending on various conditions such as propagation, interference, and traffic load. The network can choose a maximum number of retransmissions that may be attempted for each link layer segment. Link adaptation is used in EDGE so that the system can select the most efficient modulation and coding scheme for each mobile based on its current channel conditions. Different modulation and coding schemes are associated with different data rates over the airlink, thus resulting in temporal variations of bandwidth and error probability. EDGE uses 8 different channel coding schemes. Some of them are based on convolutional codes with different error correction capabilities. Others instead provide only error detection and no error correction.

The RLC/MAC layers implement procedures to allow the sharing of multiple physical channels and handle the multiplexing of timeslots. In EDGE, every timeslot can be multiplexed up to eight users. A user may make use of all the eight timeslots reaching the rate of 384Kb/s. The RLC blocks are delivered as GSM bursts across the radio link. Detailed descriptions of EDGE cellular systems are found in [2, 5]. For a typical deployment scenario, 5 MHz total spectrum for uplink and downlink used with (4, 12) reuse (3 sectors per cell, 4 cell spectrum clusters), and one carrier is available per sector. With more spectrum, more carriers can be used per sector for higher capacity. It is expected that most high-end terminals will support 4 time slots, instead of 8. BRIEF OVERVIEW OF MPEG-4 VIDEO The MPEG-4 Standard MPEG-4 [10] is an ISO/IEC standard that provides a broad framework for creating, representing, distributing, and accessing digital audiovisual content. The standard defines tools with which to create, represent and distribute individual audiovisual objects, both natural and synthetic, ranging from arbitrarily shaped natural video objects to sprites and face and body animations. These objects are encoded separately into their own elementary streams (ES). For the synchronization and the transmission of audio and video elementary streams over IP, MPEG-4 relies on the latest IETF RFCs [11, 12] for media encapsulation and synchronization by means of RTP/RTCP/RTSP. In the scenarios considered in this paper, MPEG-4 audio and video are delivered according to RFC 3016 [11]. MPEG-4 error resilience MPEG-4 provides several tools for error mitigation: they include error detection and localization, data recovery methods as well as visual concealment. Error detection consists of detection of invalid data, data out of range, or data in excess. Start codes 4 or 5 bytes in length are used across all the MPEG standards for coarse error localization and synchronization. Furthermore the MPEG-4 syntax includes special codes called resynchronization points which act as markers in the stream for error localization. The MPEG-4 ‘video packet mode’ [10] introduces resynchronization codes at fixed intervals in time. Each video packet is coded independently so that temporal and spatial error propagation is minimized. The MPEG-4 ‘video packet mode’ [10] allows resynchronization codes at fixed intervals in time and at precise spatial locations (i.e. the beginning of a slice). Their position is related to the number of macroblocks required to form a ‘packet’ of video data of given bit length. Each video packet is coded independently

Streaming Video Server

100BT Ethernet

Router

Figure 2 Experiment setup

so that temporal and spatial error propagation is minimized. An MPEG-4 video bitstream can be structured in a way to allow data partitioning and protection of header information. This provides redundancy information for the most critical parts of the stream. In MPEG-4, motion header information can be separated from the texture information by means of marker codes. The use of reversible VLC codes, i.e. codes that can be decoded in forward and backward direction is allowed in MPEG-4. Finally, mechanisms for forced intra frame refresh are provided to reduce the effect of temporal propagation of errors. Finally, mechanisms for forced intra frame refresh are provided to reduce the effect of temporal propagation of errors. MPEG4 provides two methods of intra frame refresh. Intra frame refresh can be cyclic i.e., the refresh rate of all the macroblocks in the picture is fixed, or otherwise the refresh rate can be adaptive depending on the video content. In general, the use of all such techniques introduces an overhead up to 10% of the total video bitstream. Finally, RTP packet encapsulation plays a key role in the overall performances. The RTP payload should match the VOP size and VOP fragmentation should be avoided whenever possible [11].

EDGE WIRELESS NETWORK EMULATOR The system setup for our experiment is shown in Figure 2 and includes a RTSP/RTP/RTCP video server and client, a router, and our EDGE simulator. The video server and the client are connected through a router and an EDGE emulator. The EDGE emulator is a high-end PC running FreeBSD. The IP stack of the kernel has been modified to delay or drop IP packets according to the Markov chain models derived from EDGE system level simulations [2]. The emulator also takes into account the packet fragmentation in the MAC layer of EDGE and emulates link layer retransmissions. The emulator is based on the Network Trace Replay tool for NetBSD, developed by groups at Carnegie Mellon University [1]. We made several modifications to and ported it to FreeBSD. The EDGE Emulator delays and drops packets passing through to provide the channel conditions that client would experience in a real EDGE system. The traffic model was generated based on a large amount of data obtained from comprehensive EDGE system simulations [2], in which detailed interactions between many mobiles and multiple base stations at physical level are painstakingly captured. SIMULATION MODEL

10BT Ethernet

EDGE Emulator

10BT Ethernet

Client

is streaming video. The rest of the users are modeled as web traffic generators. “Mode 0” [6] is a radio link layer control method where a packet transmission over a very poor channel to a particular user is deferred until the channel condition improves. This improves the system-wide performance of EDGE under heavy load by reducing the interference from packet transmissions that are likely to be unsuccessful even with high transmission power.

Our simulation model assumes a single video stream per sector per carrier, with the number of web users per sector per carrier variable. Wireless users belong to one of the three groups. Within each group, users may be at one of the five states corresponding roughly to different EDGE modulation and coding levels. Note that while EDGE supports eight modulation and coding schemes only five of them have distinctively different performances in most practical situations. As we change parameters such as number of users in the system and maximum number of retransmissions, the distribution of users among these groups changes accordingly as well as the transition probability among channel states, resulting different channel characteristics of the emulated channel. For details, refer to [2]. Each mobile web user generates traffic files based on a Poisson arrival process with average arrival rate of 1.5 seconds. Each traffic file consists of geometrically distributed number of pages with the mean of 10. The size of the pages is lognormal distributed with the mean of 4.1 Kbytes and the standard deviation of 30 Kbytes, truncated at 100 Kbytes, assuming very large page sizes are not likely. The inter-arrival time between pages is Pareto distributed with the mean of 3 seconds and the shape parameter of 1.4. More details can be found in [5]. Along with the variable number of web users per sector per

Available bandwidth For video streams with error concealment and other ways to mitigate the effect of IP packet losses, the “available bandwidth” must be defined differently from the conventional definitions, especially when the underlying layer 2 also employs retransmissions. So-called “goodput” where only successfully delivered bits are counted is not suitable for a video stream that can mitigate the effect of packet losses to some degree. Likewise, “raw bandwidth” where all bits attempted for delivery is counted regardless of success or failure is not suitable with link layer retransmission invisible at the IP layer, since the retransmission attempts at the link layer consumes a part of the raw bandwidth. This portion of the bandwidth is effectively invisible to the video stream operating at the IP layer when a packet is eventually delivered successfully. However, when a packet is dropped after exhausting all retransmission attempts, all the bandwidth used for this failed delivery is visible to the IP layer, since it causes at least one IP packet to be lost. (The link layer retransmission can be persistent, in which case the video stream never experiences a packet loss. In this case, the available bandwidth is the same as the goodput.) In the cases that we considered, the link layer retransmission is not persistent, i.e., link level packets are dropped after a certain number of failed retransmissions. Thus, the available bandwidth must be defined as the raw bandwidth divided by the average

carrier, the maximum number of EDGE radio link layer retransmissions and the presence or absence of “Mode 0” are the set of parameters considered for the simulations along with the MPEG-4 error concealment overhead of 0 or 10%. Channel loading is a complex issue determined by several elements including number of users in the system, spectrum reuse factor, and maximum slot number allowed to one user. In the simulations we fixed the number of maximum slots allocated to a given user to 4 and fixed also the spectrum reuse factor. We considered as parameter the number of simultaneous users where only one among them

Table 1: Available bandwidth for video and packet error rates with or without Mode 0 # of Users per Sector per Carrier

Max. # of Link Layer Retransmission

2 2 2 2 4 4 4 4 8 8 8 8 12 12 12 12

0 1 2 3 0 1 2 3 0 1 2 3 0 1 2 3

Available Bandwidth(kb/s) (Mode0)

208.9 188.2 186.2 185.9 195.2 168.3 164.6 164.1 171.7 140.7 135.4 134.2 144.3 110.2 102.6 100.5

208.9 188.2 186.2 185.9 196.5 169.4 165.7 165.2 175.3 146.1 141.4 140.5 151.2 121.9 116.5 115.3

Average Link Layer IP Packet Error Rate(%) for IP Packet Error Rate(%) for Packet Error Rate(%) 340 byte IP packets 1050 byte IP packets with No (Mode 0) (Mode 0) retransmission

11

16

22

31

11 1.21 0.133 0.0146 16 2.56 0.41 0.0655 22 4.84 1.06 0.234 31 9.61 2.98 0.924

11 1.21 0.133 0.0146 16 2.56 0.41 0.0655 20 4 0.8 0.16 24 5.76 1.38 0.332

29.5 3.59 0.399 0.0439 40.7 7.49 1.22 0.196 52.5 13.8 3.16 0.701 67.1 26.1 8.67 2.75

29.5 3.59 0.399 0.0439 40.7 7.49 1.22 0.196 48.8 11.5 2.38 0.479 56.1 16.3 4.09 0.992

delivery and N f the number of attempts for a failed delivery. Let’s call Ba the Available bandwidth, Br the raw bandwidth, elink the average link layer packet error probability,

and

Nm

the

maximum

number

of

retransmissions. We can express the available bandwidth as:

Ba = Br

{E (N s ) + E (N f )}

 N m +1  i −1 = B r  ∑ i (1 − e link )e link + ( N m + 1)e link   i =1  1 − e link = Br N m +1 1 − e link N m +1

(1)

This definition for the available bandwidth is equal to the goodput when N m → ∞ , and is equal to the raw bandwidth when no link level retransmission is employed, i.e., N m = 0 . Table 1 summarizes the average available bandwidth for the video stream per sector per carrier at the IP layer with and without mode 0 respectively. They also contain the packet error rates for the link layer packets (for reference) and for IP packets of 340 and 1050 bytes, most common average IP packet sizes observed during MPEG4 streaming. The 340 bytes packets correspond to the average size of MPEG-4 PVOPs while the 1050 bytes packets correspond to the average size of segmented MPEG-4 IVOPs. The model takes into account the effects of link layer fragmentation and the link layer retransmissions. The range of web users and the maximum number of retransmissions have been selected as good compromise between packet error rate reduction and bandwidth consumption. The maximum number of web users was chosen on the basis of practical experiences. We note that even for a scenario with only 2 users the IP packet error rate will still affect the quality of UDP/RTP-based video delivery, thus the MPEG-4 error resilience cannot be turned off completely in any of the scenarios. Furthermore we note that the cost in terms of bandwidth for 2 retransmissions is around 10% for 2 users the sector per carrier going up to around 15% when the number of users increases to 12. This value is comparable with the cost, in terms of bandwidth of the MPEG-4 error resilience. According to Table 1 and estimating the MPEG-4 error resilience capable of compensating up to 2% of packet losses for both IVOPs and PVOPs, we can identify the expected performances of the system. In the case in which Mode 0 is considered the model predicts that 12 web users can be active at the same time while the video stream can be delivered with acceptable quality. If Mode 0 is not considered the number of web users is reduced to 8. This is an observation based on average performance, and we show how instantaneous

performances differ later, since the emulator varies the bandwidth and packet error rates according to the Markov chain models at 100 msec intervals (5 EDGE frames) to emulate time-varying wireless channel conditions, a common parameter in real networks. Experimental Setup The EDGE parameters chosen for our test are of a typical EDGE deployment scenario [5]: (4, 12) reuse, 4 time slot capable EDGE devices. An error resilient scheme was used in the experiments. The overhead of such redundant information is around 10% of the video bitstream. The intraframe distance has been set to 7 seconds. Adaptive MultiRate GSM Voice coding is included in the delivered streams with the average bit rate of 4.75 Kb/s. For the experiments we used 2 video clips. The first one, ‘video music’, is a 900 frame long QCIF of size 176x144 coded at 10 fps at 100 kbps. The second one is a movie trailer encoded at 100kbps up to 10 fps. The encoder we used is capable of variable playback frame rate and privileges picture quality versus frame rate. Video is delivered over UDP/RTP. PERFORMANCE ANALYSIS MPEG-4 error resilience performance The results obtained are in agreement with the literature.MPEG-4 error resilience produced improvements up to 10dB. The average PSNR for the 2 video clips in an error free delivery is 38dB and 37.8dB respectively. Effect of the link layer ARQ The case of 0 retransmissions, i.e. LLC ARQ disabled, caused the decoder to freeze very often and led to unacceptable video quality at the client side. Thus, the corresponding data are not considered in the analysis. Only the cases with 1, 2, or 3 retransmissions are taken into account with MPEG-4 error resilience enabled. The results are presented in figure 3. EDGE LLC ARQ Performance 40

35

30 PSNR in dB

number of link layer transmission attempt up to the maximum number of attempts. Let’s call N s the number of attempts for a successful packet

25

20

15

10

1

1.5

2 2.5 Max N. of Retransmissions

Figure 3: Experiment results

3

The dotted line with “o” shows the evolution of the PSNR when the maximum number of retransmissions goes from 1 to 3 for the case of 2 users. The segmented line with “*” and the continuous line instead are for the case of 4 and 8 simultaneous users per carrier per sector respectively. Note that we used the average PSNR of the sequence to measure the video quality. This is a quite rudimental video quality estimation method. Nevertheless the differences in video quality are so substantial that PSNR was a sufficient estimator. As can be noted from figure 3, ARQ plays a quite relevant role and affects substantially the quality of the user experience. Experiments show that in the hypothesis of a MPEG-4 video delivery schemes that uses only the error concealment and resilience capabilities described in Sect. 2.2 a single retransmission is not sufficient for assuring an acceptable video quality. The choice of 2 or 3 retransmissions depends on the service model that will be implemented. Effect of Mode 0 Experimental results show that the improvement introduced by Mode 0 is not significant in terms of image quality. This result should be expected due to the nature of the Mode0 scheme, which starts to yield substantial benefits only when the system is heavily loaded. The channel conditions are already too poor to make substantial difference in PSNR. CONCLUSIONS We evaluated the MPEG4 streaming performance over an emulated EDGE wireless cellular network. The EDGE network emulator is based on a well-verified model using Markov chains, including background traffic and systemswide effects of propagation and interference. An IP-based MPEG4 streaming system is tested using a few typical clips at 100kbps encoded rate. The maximum number of link layer retransmissions, the use of Mode 0, and the amount of error concealment coding are varied to examine how such an MPEG4 streaming performs over a time-varying EDGE system under various loads. The experiments showed that the error resilience provided by MPEG-4 is needed to assure an acceptable video quality on a wireless network. Simpler schemes as H.263 baseline that do not provide the richness that MPEG-4 has may fail as coding schemes for wireless content. Furthermore, we identified link layer ARQ as a key element for the quality of video delivery over EDGE. Experiment showed that at least 2 attempts to retransmit lost frames should be allowed. The actual value may vary depending on the service model that a given wireless service provider plans to implement.

REFERENCES [1] B. D. Noble, M. Satyanarayanan, G. T. Nguyen, and R. H. Katz, “Trace-based mobile network emulation,” Proceedings of the ACM SIGCOMM Conference, France, Sep. 1997, in Computer-Communication-Review, vol.27, no.4, p.51-61, Oct. 1997 [2] Man Cal, Li Fung Chang, Kapil Chawla, K., and Xiaoxin Qiu, “Providing differentiated services in EGPRS through packet scheduling,” Proceeding of GLOBECOM '00, Nov. 2000, Page(s): 1515 –1521. [3] H. Holma and A. Toskala, WCDMA for UMTS: Radio Access for Third Generation Mobile Communications, John Wiley & Sons, 2000. [4] Molkdar, D., Featherstone, W. Larnbotharan, S., “An overview of EGPRS: the packet data component of EDGE,” Electronics & Communication Engineering Journal, Volume: 14 Issue: 1, Feb. 2002, Page(s): 21 –38 [5] Furuska, A., Frodigh, M, Olofson, H., Skold, J., “System Performance of EDGE, a Proposal for Enhanced Data Rates in Existing Digital Cellular Systems,” Proceedings of VTC’98 [6] Balachandran, K.; Kirk Chang; Wei Luo; Nanda, S., “System level interference mitigation schemes in EGPRS: Mode-0 and scheduling, ” Vehicular Technology Conference, 2001. VTC 2001 Spring, IEEE VTS 53rd, Volume: 4, 2001, Page(s): 2489 -2493 vol.4 [7] Talluri, R., “Error-Resilient Video Coding in the ISO MPEG-4 Video Standard” 1998 IEEE Communications Magazine pp 112-119. [8] Cai, J.; Qian Zhang; Wenwu Zhu; Chen, C.W. “An FEC-based error control scheme for wireless MPEG-4 video transmission”, Wireless Communications and Networking Conference, 2000. WCNC, 2000 IEEE, 2000 Page(s): 1243 -1247 vol.3 [9] Budagavi, M.; Heinzelman, W.R.; Webb, J.; Talluri, R. Wireless MPEG-4 video communication on DSP chips IEEE Signal Processing Magazine , Volume: 17 Issue: 1 , Jan. 2000 Page(s): 36 –53 [10] ISO/IEC 14496-1, 2, 3:1999, "Information technology - Coding of audio-visual objects – Part1 Systems, Part2: Visual, Part3: Audio [11] Y. Kikuchi, T. Nomura, S. Fukunaga, Y. Matsui, H. Kimata, “ RTP playload format for video and audio MPEG4 streams” RFC3016 Nov 2000 [12] Schulzrinne, H., Casner, S., Frederick, R. and V. Jacobson "RTP: A Transport Protocol for Real Time Applications", RFC 1889, January 1996.