We exploit the power of turbo-like codes (mainly turbo codes) in the fields of source coding and joint source channel coding, with the objective of achieving performance close to the theoretical limits. Turbo codes are well known for their near Shannon limit performance in noisy channels. However, those "random-like" codewords also allow for compression rates close to the theoretical limits when applied to source coding.
This book presents the journey of Turbo-Codes from their first invention and initial design as error correcting codes to their application as video compression tools. This journey is presented in three milestones. First, Turbo-Codes are introduced as a channel coding tool. Different encoding structures and decoding algorithms are discussed from theoretical and practical aspects, for binary and non-binary Turbo-Codes. Slepian-Wolf and Wyner-Ziv theorems are then discussed, as they constitute the main theory behind distributed source coding (DSC). Turbo-Codes are then presented as a practical tool for distributed source compression. The study of Turbo-Codes application in DSC is also extended to the case of joint source-channel coding (JSCC), where these codes are jointly used for both source compression and error correction. Theoretical models for DSC and JSCC are thoroughly discussed along with the necessary modifications to the initial turbo encoder-decoder system. Different simulation setups are considered and results are presented and analyzed. Finally, Turbo-Code-based distributed video coding (DVC) techniques are discussed. The motivation behind DVC is first presented, followed by a general description of the DVC system model. Different techniques used to generate the side information needed for practical DVC systems are then discussed. Theoretical compression bounds are derived for both error-free and erroneous transmissions. Applications of DVC in the context of single user and multiuser setups are finally presented with different simulation scenarios and performance analysis.
PREFACE The increasing demand on high data rate and quality of service in wireless communication has to cope with limited bandwidth and energy resources. More than 50 years ago, Shannon has paved the way to optimal usage of bandwidth and energy resources by bounding the spectral efficiency vs. signal to noise ratio trade-off. However, as any information theorist, Shannon told us what is the best we can do but not how to do it [1]. In this view, turbo codes are like a dream come true: they allow approaching the theoretical Shannon capacity limit very closely. However, for the designer who wants to implement these codes, at first sight they appear to be a nightmare. We came a huge step closer in striving the theoretical limit, but see the historical axiom repeated on a different scale: we know we can achieve excellent performance with turbo codes, but not how to realize this in real devices.
Based on the encoding process, arithmetic codes can be viewed as tree codes and current proposals for decoding arithmetic codes with forbidden symbols belong to sequential decoding algorithms and their variants. In this monograph, we propose a new way of looking at arithmetic codes with forbidden symbols. If a limit is imposed on the maximum value of a key parameter in the encoder, this modified arithmetic encoder can also be modeled as a finite state machine and the code generated can be treated as a variable-length trellis code. The number of states used can be reduced and techniques used for decoding convolutional codes, such as the list Viterbi decoding algorithm, can be applied directly on the trellis. The finite state machine interpretation can be easily migrated to Markov source case. We can encode Markov sources without considering the conditional probabilities, while using the list Viterbi decoding algorithm which utilizes the conditional probabilities. We can also use context-based arithmetic coding to exploit the conditional probabilities of the Markov source and apply a finite state machine interpretation to this problem. The finite state machine interpretation also allows us to more systematically understand arithmetic codes with forbidden symbols. It allows us to find the partial distance spectrum of arithmetic codes with forbidden symbols. We also propose arithmetic codes with memories which use high memory but low implementation precision arithmetic codes. The low implementation precision results in a state machine with less complexity. The introduced input memories allow us to switch the probability functions used for arithmetic coding. Combining these two methods give us a huge parameter space of the arithmetic codes with forbidden symbols. Hence we can choose codes with better distance properties while maintaining the encoding efficiency and decoding complexity. A construction and search method is proposed and simulation results show that we can achieve a similar performance as turbo codes when we apply this approach to rate 2/3 arithmetic codes. Table of Contents: Introduction / Arithmetic Codes / Arithmetic Codes with Forbidden Symbols / Distance Property and Code Construction / Conclusion
Application of error correcting codes for data compression is first investigated by Shannon where he suggests that there is a duality between source coding and channel coding. This duality implies that good channel codes are likely to be good source codes (and vice versa). Recently the problem of source coding using channel codes is receiving increasing attention. The main application of this problem is when data are transmitted over noisy channels. Since standard data compression techniques are not designed for error correction, compressing data and transmitting over noisy channels may cause corruption of the whole compressed sequence. However, instead of employing standard compression techniques, like Huffman coding, one may compress data using error correcting codes that are suitable for both data compression and error correction purposes. Recently, turbo codes, repeat-accumulate codes, low density parity check codes, and fountain codes have been used as lossless source codes and have achieved compression rates very close to the source entropy. When a near-lossless compression is desired, i.e. a small level of distortion is acceptable, the source encoder generates fixed-length codewords and the encoding complexity is low. Theoretically, random codes could achieve near-lossless compression. In literature, this has been proved by presenting a random binning scheme. Practically, all powerful channel codes, e.g. turbo codes, can follow the same procedure as suggested in random binning and achieve compression rates close to the entropy. On the other hand, if a completely lossless compression is required, i.e. if the distortion must be forced to zero, the source encoding is a complicated iterative procedure that generates variable-length codewords to guarantee zero distortion. However, the large complexity of encoding imposes a large delay to the system. The iterative encoding procedure can be regarded as using a nested code where each codeword of a higher-rate code is formed by adding parities to a codeword of some lower-rate code. This iterative encoding is proposed for practical codes, e.g. turbo codes and low density parity check (LDPC) codes, in the literature. In contrast to near-lossless source coding, in the lossless case no random coding theory is available to support achievability of entropy and specify distribution of the compression rate. We have two main contributions in this thesis. Our first contribution is presenting a tree structured random binning scheme to prove that nested random codes asymptotically achieve the entropy. We derive the probability mass function of the compression rate and show how it varies when increasing the block length. We also consider a more practical tree structured random binning scheme, where parities are generated independently and randomly, but they are biased. Our second contribution is to decrease the delay in turbo source coding. We consider turbo codes for data compression and observe that existing schemes achieve low compression rates; but because of large block length and large number of iterations they impose a large delay to the system. To decrease this delay we look at the problem of source coding using short block length turbo codes. We show how to modify different components of the encoder to achieve low compression rates. Specifically we modify the parity interleaver and use rectangular puncturing arrays. We also replace a single turbo code by a library of turbo codes to further decrease the compression rate. Since the scheme is variable-length and also many codes are used, the codeword length along with the code index (index of the turbo code which is used for compression) are transmitted as an overhead. Transmission of this overhead increases the compression rate. We propose a detection method to detect this overhead from the codeword. Therefore, the overhead is no longer transmitted since it is detected from the codeword at the decoder. This detection method will reduce the compression rate for short block length systems but it becomes less attractive for large block length codes.
Treats joint source and channel decoding in an integrated way Gives a clear description of the problems in the field together with the mathematical tools for their solution Contains many detailed examples useful for practical applications of the theory to video broadcasting over mobile and wireless networks Traditionally, cross-layer and joint source-channel coding were seen as incompatible with classically structured networks but recent advances in theory changed this situation. Joint source-channel decoding is now seen as a viable alternative to separate decoding of source and channel codes, if the protocol layers are taken into account. A joint source/protocol/channel approach is thus addressed in this book: all levels of the protocol stack are considered, showing how the information in each layer influences the others. This book provides the tools to show how cross-layer and joint source-channel coding and decoding are now compatible with present-day mobile and wireless networks, with a particular application to the key area of video transmission to mobiles. Typical applications are broadcasting, or point-to-point delivery of multimedia contents, which are very timely in the context of the current development of mobile services such as audio (MPEG4 AAC) or video (H263, H264) transmission using recent wireless transmission standards (DVH-H, DVB-SH, WiMAX, LTE). This cross-disciplinary book is ideal for graduate students, researchers, and more generally professionals working either in signal processing for communications or in networking applications, interested in reliable multimedia transmission. This book is also of interest to people involved in cross-layer optimization of mobile networks. Its content may provide them with other points of view on their optimization problem, enlarging the set of tools which they could use. Pierre Duhamel is director of research at CNRS/ LSS and has previously held research positions at Thomson-CSF, CNET, and ENST, where he was head of the Signal and Image Processing Department. He has served as chairman of the DSP committee and associate Editor of the IEEE Transactions on Signal Processing and Signal Processing Letters, as well as acting as a co-chair at MMSP and ICASSP conferences. He was awarded the Grand Prix France Telecom by the French Science Academy in 2000. He is co-author of more than 80 papers in international journals, 250 conference proceedings, and 28 patents. Michel Kieffer is an assistant professor in signal processing for communications at the Université Paris-Sud and a researcher at the Laboratoire des Signaux et Systèmes, Gif-sur-Yvette, France. His research interests are in joint source-channel coding and decoding techniques for the reliable transmission of multimedia contents. He serves as associate editor of Signal Processing (Elsevier). He is co-author of more than 90 contributions to journals, conference proceedings, and book chapters. Treats joint source and channel decoding in an integrated way Gives a clear description of the problems in the field together with the mathematical tools for their solution Contains many detailed examples useful for practical applications of the theory to video broadcasting over mobile and wireless networks
This book is devoted to one of the essential functions of modern telecommunications systems: channel coding or error correction coding. Its main topic is iteratively decoded algebraic codes, convolutional codes and concatenated codes.
This book gives a review of the principles, methods and techniques of important and emerging research topics and technologies in Channel Coding, including theory, algorithms, and applications. Edited by leading people in the field who, through their reputation, have been able to commission experts to write on a particular topic. With this reference source you will: Quickly grasp a new area of research Understand the underlying principles of a topic and its applications Ascertain how a topic relates to other areas and learn of the research issues yet to be resolved Quick tutorial reviews of important and emerging topics of research in Channel Coding Presents core principles in Channel Coding theory and shows their applications Reference content on core principles, technologies, algorithms and applications Comprehensive references to journal articles and other literature on which to build further, more specific and detailed knowledge
Examines the problem of joint source-channel video transmission - the joint optimal allocation of resources at the application layer and the other network layers, such as data rate adaptation, channel coding, power adaptation in wireless networks, quality of service (QoS) support from the network, and packet scheduling, for efficient video transmission.
