Digital video technology has evolved from VGA to UHD and 4K in last few decades. With recent advances in technology, HD and Ultra HD digital video content has become ubiquitous Many of the latest digital multimedia devices like LED TV’s, smartphones, tablets and portable media players are capable of streaming HD and UHD content over wireless mediums. The main challenges in transmitting HD and UHD video content over wireless mediums is that it requires high compression efficiency and low video distortion. By using High Efficiency Video Coding (HEVC) standard we can achieve high compression ratios but unfortunately the distortion caused by packet losses while transmitting HEVC video bit stream over wireless channels cannot be avoided. This thesis is aimed at designing an Unequal Error Protection (UEP) scheme for efficient transmission of HEVC video bit streams over wireless channels. First, we created a database of video parameters extracted from several test videos and developed a statistical regression model for finding the most important video parameters that play role in causing transmission distortion. Next, we assigned priorities to each individual slice of test video based on regression analysis. Finally, we developed a model to assign a Forward Error Correction (FEC) code rate based on the priority ranking of each video slice. By predicting vital factors in transmission distortion and assigning unequal error protection code rates our scheme provides low transmission distortion and high Peak Signal to Noise Ratio (PSNR) for received videos
Digital video technology has evolved from a low resolution (SD) to very high resolution (ultra HD or 4K) in this decade. Many digital multimedia devices, such as smartphones, smart television and tablets, can now support high resolution video content over wireless mediums. The main challenges in supporting the high resolution mobile video applications over wireless medium are the limited and variable channel bandwidth, and channel induced packet losses. Due to limited available channel bandwidth, the video streams are highly compressed by using the video compression technology, such as the recently developed High Efficiency Video Coding (HEVC) standard. However, the compressed video bitstream is very sensitive to the packet losses during transmission, which results in distorted video at the user end. In this thesis, we have designed a new scheme for transmitting HEVC video bitstreams over wireless channels by using unequal error protection (UEP). In UEP scheme, the HEVC video packets are divided in four priority classes, based on the distortion their loss would introduce to the video quality. In the next step, a suitable low density parity check (LDPC) based forward error correction (FEC) code is applied to each packet based on its priority class, video bit rate, available channel bandwidth and channel SNR, by using a genetic algorithm based optimization which minimizes the video distortion. The LDPC FEC codes are used as they give error-free performance near Shannon's channel capacity limit. Effectively, the higher (lower) priority packets are transmitted with stronger (weaker) FEC protection. Finally, the error concealment schemes (including the temporal error concealment (TEC) and motion compensation error concealment (MCEC) algorithms are used to conceal the effect of lost video packets at the receiver. The use of UEP and error concealment schemes provide high video quality in the presence of channel losses. It validates our primary objective which was to better utilize the available channel bandwidth to improve the video quality.
With the advent of technology, video resolution has increased up to 8K. But the wireless channel bandwidth available to transmit such high resolution videos is limited. Therefore, a better video coding standard, known as the high efficiency video coding (HEVC), has been developed that can compress the video data more efficiently than the currently deployed H.264 AVC standard, in order to fit high resolution videos on available wireless channels. For compressing videos using HEVC, first we need to set the encoding parameters appropriately so that the application requirements can be met, while increasing the compression ratio and video quality at the end user. In this thesis, we have studied the impact of various encoding parameters (such as group of pictures (GOP) size, GOP structure, and bitrate) on HEVC bitstream. Not every packet in the compressed video bitstream has the same contribution to video quality. If the most important video packets are lost, the error propagation will be more and the decoded video will be highly distorted. Since measuring video distortion contributed by the loss of a slice is computationally intensive and introduces delay, a prediction model is used to predict the distortion value and assign priority to each slice. The video packets (i.e., slices) of a GOP are divided into four equally populated classes based on their predicted distortion value, and priorities are assigned to each class. While transmitting through the channel, the packet is susceptible to channel noise and transmission errors. In this thesis, we have used an unequal error protection (UEP) scheme developed in our group, which protects the most important data against the channel induced losses based on their priority. For this, the low density parity check (LDPC) codes are applied to each packet based on its priority, for error correction. The LDPC code rates are computed for all the slices in a GOP by using a genetic algorithm (GA) based optimization scheme, which considers the video data rate, channel capacity and slice priorities. The performance of UEP scheme is evaluated for fading channels.
Rapid growth of video applications over wireless networks is overwhelming the wireless bandwidth. Since video applications demand large bandwidth and realtime transmission, supporting the rapidly increasing video traffic over the bandwidth-limited, error-prone, and time-varying wireless channels is very challenging. As a result, the video applications are likely to suffer packet losses over wireless networks which results in quality degradation. In this dissertation, we design a distortion prediction model for H.264/AVC compressed video streams, and use it for designing novel cross-layer protocols for enhancing the video quality by making more efficient use of the available wireless resources. The cumulative mean squared error (CMSE) is a widely used measure of video distortion. However, CMSE measurement is a time-consuming and computationally-intensive process which is not suitable for many video applications. A low-complexity and low-delay generalized linear model is proposed for predicting CMSE contributed by the loss of H.264 AVC encoded video slices. The model is trained over a video database by using a combination of video factors that are extracted during the encoding of the current frame, without using any data from future frames in the group of pictures (GOP). The slices are then prioritized within a GOP based on their predicted CMSE values. The accuracy of the CMSE prediction model is analyzed using cross-validation, analysis of variance, and correlation coefficients. The simulations are carried out to evaluate the performance of the CMSE prediction model for varying encoder configurations and bit rates of test videos. The CMSE slice prediction model is used to design an unequal error protection (UEP) scheme, using the rate-compatible punctured convolutional (RCPC) codes over wireless channels. This scheme provides protection to the video slices against the channel errors, based on their priority, in order to minimize the video distortion. An application of our slice prioritization is demonstrated by implementing a priority-aware slice discard scheme, where the low-priority slices are dropped from the router when the network experiences congestion. Additionally, the GOP-level slice prioritization is extended to the frame-level slice prioritization, and its performance is evaluated over the additive white Gaussian noise (AWGN) channels The idea of using slice CMSE prediction is extended to adapt the video packet size to the wireless channel conditions, in order to minimize the video distortion. A real-time, priority-aware joint packet fragmentation and error protection scheme for real-time video transmission over Rayleigh fading channels is presented. The fragment error rates (FERs) are simulated for a combination of different fragment sizes and RCPC code rates. These FERs are then used to determine the optimal fragment sizes and code rates for packets of each priority class by minimizing the expected normalized predicted CMSE per GOP in H.264 video bit stream. An improvement in the received video quality over the conventional and priority-agnostic packet fragmentation schemes is observed. Next, a cross-layer, priority-aware scheduling scheme for real-time transmission of multiple video applications over a time-varying channel is developed. Each video application considered has different characteristics such as user priority, latency, distortion, size, and encoding bit rate. A cost function is optimized to determine the scheduling order for video frames. The performance of our scheme is compared with that of the CMSE based scheme, where the frames are rank-ordered for transmission using its CMSE per bit values, and with the earliest deadline first (EDF) scheme in which each user takes turns to transmit a frame. A collaborative effort with other researchers and developed two additional cross-layer error protection schemes. In the first scheme, a cross layer UEP scheme that jointly assigned FEC at both the Application layer (using Luby Transform) and the Physical layer (using RCPC codes) for prioritized video transmission is developed. The video distortion function is minimized by using the genetic algorithm (GA). The performance of our scheme is evaluated for different channel SNR values. In the second UEP scheme, a framework that combined the RCPC codes and concatenated it with hierarchical quadrature amplitude modulation (QAM) is investigated. Employing RCPC codes and hierarchical modulation jointly resulted in greater flexibility as some parts of the data can be protected only by the hierarchical modulation while others may be protected by a low FEC code rate. The performance of the proposed scheme is compared to the standard 8-QAM with symmetric constellation.
The huge amount of data embodied in a video signal is by far the biggest burden on existing wireless communication systems. Adopting an efficient video transmission strategy is thus crucial in order to deliver video data at the lowest bit rate and the highest quality possible. Unequal error protection (UEP) is a powerful tool in this regard, whose ultimate goal is to wisely provide a stronger protection for the more important data, and a weaker protection for the less important data carried by a video signal. The use of efficient video delivery techniques becomes more important when 3D video content is transmitted over a wireless channel, since it contains twice as much data as 2D video. In this dissertation, we consider the UEP problem for transmission of 3D video over wireless channels. The proposed UEP techniques entail relatively high computational complexity which lend themselves to be more suitable for video-on-demand delivery, where the time-consuming computations are done offline at the transmitter/encoder side. To adopt UEP for 3D video, we consider a general problem of joint source-channel coding (JSCC). Solving the JSCC problem yields the optimum amount of 3D video compression as well as the optimum FEC (forward error correction) code rates exploited for UEP. We first need to estimate the perceived quality of the reconstructed video at the receiver. The lack of a good objective metric for 3D video makes adopting UEP a more challenging and problematic task compared to 2D video. Fortunately, for 3D video, some quality thresholds are derived in the literature based on the PSNR (peak-signal-to-noise-ratio) metric through experimental tests. These thresholds allow us to formulate the JSCC optimization problem using the PSNR in a straightforward but different way from the typical counterpart optimization problems in the literature. More precisely, we put the constraints of the optimization problem on the quality of the reconstructed 3D video and set our goal to minimize the total bit rate. We adopt the multiview coding (MVC) extension of the H.264/AVC. We also propose a scalable variant of MVC and formulate and solve the JSCC optimization problem for it. We show that significant gains are obtained if the proposed UEP scheme is combined with asymmetric coding. We also tackle the UEP problem for the video plus depth (V+D) format. We employ the SSIM (Structural SIMilarity) metric for designing UEP for V+D, since it has been shown that PSNR does not properly characterize the perceived quality of a 3D video represented in V+D format. Moreover, the synthesized right view always shows a huge PSNR loss (even in the absence of compression), which does not even allow us to use the asymmetric coding PSNR thresholds. This motivated us to adopt the classical JSCC problem formulation, where our goal is to maximize the quality of the reconstructed left and right views, given that there is a constraint on the sum of the number of source bits and the number of FEC bits. We show that UEP provides significant gains compared to equal error protection. We also derive several interesting results; some of them are in accordance with what have already been published in the literature and some of them are not. We show that the reason for this inconsistency is that we are solving the UEP problem in a more general situation, which yields novel solutions. Lastly, we focus on UEP for video broadcasting over wireless channels. Our goal here is to design a UEP-based video broadcasting system that well serves all the users within the service area of a base station. In a service area, there exist heterogeneous users with different display resolutions operating at different bit rates. Spatially scalable video is an excellent video compression format for this scenario, since it allows a user to decode that portion of the scalable bit stream that fits its operating bit rate as well as its display resolution. We tackle this problem for a MIMO (multi-input-multi-output) channel which enables us to exploit either spatial diversity or spatial multiplexing in a multipath fading channel to increase channel reliability or throughput, respectively. We employ spatial diversity techniques, in particular the Alamouti code, to encode the base layer. We also adopt spatial multiplexing techniques, in particular the V-BLAST, to encode the enhancement layer. By controlling the power allocation between the base layer and the enhancement layer, we can control the level of protection we provide to each of them. We also show that the adoption of scalable video in our system yields much higher gains compared to non-scalable video.
Wireless channels are prone to errors due to the presence of noise, fading effects and interference. The quality of communication is degraded by these channel errors. To counter the effect of these channel errors we use the Forward Error Correction (FEC) codes. The perceptual video quality is influenced by not only the channel errors caused during transmission, but also by the compression artifacts. We have aimed to reduce the amount of channel errors in this thesis. In this thesis, we have used a H.264/AVC bit stream. The video bit stream consists of high and low priority bits. The high priority bits are given more protection and the low priority bits are given less protection. This is called as Unequal Error Protection (UEP). H.264/AVC bit stream is then modulated and sent over the channel. The high priority data gets protection from the asymmetric modulation and the low priority data gets convolutionally encoded. In this way we have provided protection to both high and low priority. The bit stream is then recovered at the receiver side. The Peak Signal to Noise Ratio (PSNR) of the recovered video is calculated. PSNR is used to estimate the quality of the recovered video. This scheme gives a 3 dB improvement when compared with the previous techniques.
The quality of H.264/AVC compressed video delivery over time-varying and error-prone wireless channels is affected by packet losses. To support quality of service (QoS) for video delivery over wireless networks cross-layer schemes have been discussed in the literature. We introduce a cross-layer priority-aware packet fragmentation scheme at the medium access control (MAC) layer to enhance the quality of pre-encoded H.264/AVC compressed bitstreams over bit-rate limited error-prone links in wireless networks. Larger fragments are more likely to be in error but smaller fragments require more overhead. The H.264 slices are classified in four priorities at the encoder based on their cumulative mean square error (CMSE) contribution towards the received video quality. The slices of a priority class in each frame are aggregated into video packets of corresponding priority at the application (APP) layer. We derive the optimal fragment size for each priority class which achieves the maximum expected weighted goodput at different encoded video bit rates, slice sizes and bit error rates. Priority-aware packet fragmentation invokes slice discard in the buffer due to channel bit rate constraints on allocating fragment header bits. We propose a slice discard scheme using frame importance and slice CMSE contribution to control error propagation effects. Packet fragmentation is then extended to slice fragmentation by modifying the conventional H.264 decoder to handle partial slice decoding. Priority-aware slice fragmentation combined with the proposed slice discard scheme provides considerable peak signal-to-noise ratio (PSNR) and video quality metric gains as compared to priority-agnostic fragmentation. Distortion due to channel errors can be alleviated by assigning stronger channel code rates, at the cost of reduced rate for source coding. Besides MAC layer fragmentation, aggregating H.264/AVC slices at the APP layer to form video packets with sizes adapted to their importance can also improve transmission reliability. We present a cross-layer dynamic programming (DP) approach to minimize the expected received video distortion by jointly addressing the priority-adaptive packet formation at the APP layer and rate compatible punctured convolutional (RCPC) code rate allocation at the physical layer for pre-encoded prioritized slices of each group of pictures (GOP). Our scheme discards some low priority slices in order to improve protection to more important slices and meet the channel bitrate limitations, whenever necessary. Simulation results show that our proposed approach significantly improves received video quality compared to other error protection schemes. Further, we extend our cross-layer DP-based scheme to slices of each frame by predicting the expected channel bit budget per frame for real-time transmission. The prediction uses a generalized linear model developed over the parameters - CMSE per frame, channel SNR, and normalized compressed frame bit budget determined over a video dataset that spans high, medium and low motion complexity. This predicted frame bit budget is used to derive the packet sizes and their corresponding RCPC code rates for transmission using our DP-based approach. Simulation results show good correlation with the results of our DP-based scheme applied over the GOP. Unique characteristics of video traffic, such as the temporal and spatial dependencies between different video frames and their deadline constraints, pose a challenge in supporting the video quality rendered to the clients over time-varying, bandwidth-limited channels. Scalable Video Coding (H.264/SVC) enables the transmission and decoding of partial bit streams to provide video services with lower temporal or spatial resolutions or reduced fidelity while retaining a reconstruction quality that is high relative to the rate of the partial bit streams. We propose a sliding-window based flow control for scheduling the network abstraction layer (NAL) units in the post-encoding buffer of the streaming server for a real-time scalable video transmission scenario over a fast time-varying channel. Our scheduling scheme considers the importance of the NAL unit in terms of (i ) its CMSE distortion contributed to the received video quality, (ii ) its size in bits, and (iii ) its time-to-expiry in seconds. The scheduling problem of determining the appropriate order of transmission is formulated as a 0-1 knapsack problem and a DP solution is proposed which runs in polynomial time. Our scheduling approach significantly reduces the number of whole frames discarded as compared to (a) a CMSE-based scheme which considers the importance of the NAL units only in terms of their CMSE contribution, and (b) the earliest deadline first scheme which minimizes the dwelling time of the NAL units in the post-encoding buffer. Simulation results show significant PSNR gains for different video sequences at different pre-roll delays.
High efficiency video coding (HEVC) is the latest video coding standard that enables the transmission of high resolution videos over wireless networks. In this dissertation, HEVC based error resilient video coding is proposed to target real time video transmission applications. A feedback-based reference picture selection (RPS) method called hierarchical-P RPS is proposed such that it can adapt with the HEVC coding framework. Due to hierarchical coding structure of HEVC, the quality of each frame in the sequence varies depending on its location in the hierarchical structure. High quality pictures are used as reference for several subsequent pictures while low quality pictures are used for only one consecutive picture. When error is detected, hierarchical-P RPS algorithm will select suitable reference pictures subject to the trade-off of low quality picture with short temporal distance and high quality picture with long temporal distance. To enhance the performance of the proposed RPS, we combine feedback-based RPS and region-of-interest (ROI) based intra refresh method by using some novel features of HEVC. Each block in the ROI region is encoded with intra mode according to error status such that it can enhance the quality of picture in ROI area and can avoid the motion search process in this area. To enable block level intra refresh, rate control is also modified to fit in the framework. Experimental results demonstrate that the proposed framework can achieve better subjective and objective quality, compared to the hierarchical-P RPS algorithm. The average PSNR improvement of proposed algorithm over modified RPS algorithm is 1.04 dB and that over HEVC reference software with regular intra frame refresh is about 5.23 dB.
Today, the wide variety of devices in the digital world ranges from desktops to mobile phones. Within the currently available interactive multimedia applications, there are demands in terms of video quality and coding efficiency, the cost as well as scalability. That is why there is a need for scalable video coding schemes which provide fully progressive bit streams and supports scalability. Scalable Video Coding targets on seamless delivery of digital content and access to the same, enabling optimal user centered multi-channel and cross-platform media services, providing a straightforward solution for universal video delivery to a broad range of applications. Scalable video coding gives a nice way to perform rate shaping for video streams adapting to the available transmission resource. The work in this thesis deals with the overview and practical implementation of the H.264 Scalable Video Codec .All the major building blocks of H.264/SVC codec are discussed and implemented. Various kinds of Scalabilities and Error Concealment methods are achieved and comparative studies are performed. This thesis proposes and discusses Unequal Error Protection scheme in Scalable Video Coding. Unequal Error Protection method is an error resiliency scheme in which we protect the bit stream of the base and the enhancement layers based on priority levels. This priority information is based on the dependency, temporal, and quality scalabilities values. We have also provided an external protection to the bit stream in the form of Reed Solomon code. A detailed implementation of this scheme is done and results obtained through these simulations and video quality evaluation, are provided, showing the system performance under various network conditions.
