MPEG: A video compression standard for multimedia applications

Lecture Notes in Computer Science, 1992

ABSTRACT We describe a system for storing, transmitting and presenting digital films in a computer network. The hardware used in the system is standard hardware, as found in typical workstations today; no special hardware is required. The movies are shown in a window of the X Window System. This allows full integration with the classical components of computer applications such as text, color graphics, menues and icons. The X-MOVIE system is based on color lookup table technology. We present a new algorithm for the gradual adaptation of the color lookup table during the presentation of the film.

Multimedia Tools and Applications, 1995

This paper presents the Distributed InterProcess Communication System (DIPCS) as a framework for managing communication in a distributed multimedia system. Within DIPCS, connection level management is provided through a novel distributed process group model called ADP-Group communication. The ADP-Group paradigm defines a new type of group message passing, called qos-reliable. Qos-reliable semantics are appropriate to controlling real-time multimedia communication, by allowing a spectrum of performance and reliability specifications to co-exist within one group. DIPCS also provides an abstract programming model of multimedia devices, easing control of a heterogeneous multimedia system. Distributed multimedia applications can be rapidly developed using simple group operation primitives. We show how ADP-Group message delivery semantics can be directly mapped into an efficient Integrated Services Network support policy.

ACM SIGCOMM Computer Communication Review, 1993

In this paper we show that modulating the source rate of a video encoder based on congestion signals from the network has two major benefits: the quality of the video transmission degrades gracefully when the network is congested and the transmission capacity is used efliciently. Source rate modulation techniques have been used in the past in designing k e d rate video encoders used over telephone networks. In such constant bitrate encoders, the source rate modulation is done using feedback information about the occupancy of a local buffer. Thus, the feedback information is available instantaneousiy to the encoder. In the scheme proposed in this paper, the feedback may be delayed by several frames because it comes from an intermediate switching node of a packet-switched network. This paper shows the proposed scheme performs quite well despite this delay in feedback. We believe the use of such schemes will simplify the architecture used for supporting real time video services in future nationwide gigabit networks.

ACM Transactions on Modeling and Computer Simulation, 2000

A performance comparison between an optimistic and a conservative parallel simulation kernel is presented. Performance of the parallel kernels is also compared to a central-eventlist sequential kernel. A spectrum of ATM network and traffic scenarios representative of those used by ATM networking researchers are used for the comparison. Experiments are conducted with a cell-level ATM network simulator and an 18-processor SGI PowerChallenge shared-memory multiprocessor.

Computer Networks and ISDN Systems, 1994

A transport protocol that supports real-time communication of audio/video frames across campus-area packet switched networks is presented. It is a "best effort" protocol that attempts to ameliorate the effect of jitter, load variation, and packet loss, to provide low latency, synchronized audio and video communications. This goal is realized through four transport and display mechanisms, and a real-time implementation of these mechanisms that integrates operating system services (e.g., scheduling and resource allocation, and device management) with network communication services (e.g., transport protocols), and with application code (e.g., display routines). The four mechanisms are: a facility for varying the synchronization between audio and video to achieve continuous audio in the face of jitter, a network congestion monitoring mechanism that is used to control audio/video latency, a queueing mechanism at the sender that is used to maximize frame throughput without unnecessarily increasing latency, and a forward error correction mechanism for transmitting audio frames multiple times to ameliorate the effects of packet loss in the network. The effectiveness of these techniques is demonstrated by measuring the performance of the protocol when transmitting audio and video across congested networks.

Journal of the Illuminating Engineering Society, 1993

The accuracy of the Content should not be relied upon and should be independently verified with primary sources of information. Taylor and Francis shall not be liable for any losses, actions, claims, proceedings, demands, costs, expenses, damages, and other liabilities whatsoever or howsoever caused arising directly or indirectly in connection with, in relation to or arising out of the use of the Content. This article may be used for research, teaching, and private study purposes. Any substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any form to anyone is expressly forbidden.

June 2020

We present a new approach to video compression for video surveillance by refining the shortcomings of conventional approach and substitute each traditional component with their neural network counterpart. Our proposed work consists of motion estimation, compression and compensation and residue compression, learned end-to-end to minimize the rate-distortion trade off. The whole model is jointly optimized using a single loss function. Our work is based on a standard method to exploit the spatio-temporal redundancy in video frames to reduce the bit rate along with the minimization of distortions in decoded frames. We implement a neural network version of conventional video compression approach and encode the redundant frames with lower number of bits. Although, our approach is more concerned toward surveillance, it can be extended easily to general purpose videos too. Experiments show that our technique is efficient and outperforms standard MPEG encoding at comparable bitrates while pr...

SpringerPlus, 2015

Distributed video coding (DVC) is a relatively new video coding architecture originated from two fundamental theorems namely, Slepian-Wolf and Wyner-Ziv. Recent research developments have made DVC attractive for applications in the emerging domain of wireless video sensor networks (WVSNs). This paper reviews the state-of-the-art DVC architectures with a focus on understanding their opportunities and gaps in addressing the operational requirements and application needs of WVSNs.

ACM Transactions on Multimedia Computing, Communications, and Applications, 2015

Even though people of all social standings use current mobile devices in the wide spectrum of purpose from entertainment tools to communication means, some issues with real-time video streaming in hostile wireless environment still exist. In this article, we introduce CoSA , a link-aware real-time video streaming system for mobile devices. The proposed system utilizes a 3D camera to distinguish the region of importance (ROI) and non-ROI region within the video frame. Based on the link-state feedback from the receiver, the proposed system allocates a higher bandwidth for the region that is classified as ROI and a lower bandwidth for non-ROI in the video stream by reducing the video's bit rate. We implemented CoSA in a real test-bed where the IEEE 802.11 is employed as a medium for wireless networking. Furthermore, we verified the effectiveness of the proposed system by conducting a thorough empirical study. The results indicate that the proposed system enables real-time video str...

Computer Vision – ECCV 2018

An ever increasing amount of our digital communication, media consumption, and content creation revolves around videos. We share, watch, and archive many aspects of our lives through them, all of which are powered by strong video compression. Traditional video compression is laboriously hand designed and hand optimized. This paper presents an alternative in an end-to-end deep learning codec. Our codec builds on one simple idea: Video compression is repeated image interpolation. It thus benefits from recent advances in deep image interpolation and generation. Our deep video codec outperforms today's prevailing codecs, such as H.261, MPEG-4 Part 2, and performs on par with H.264.

Signal Processing: Image Communication, 1995

We describe a technique for video compression based on a mosaic image representation obtained by aligning all frames of a video sequence, giving a panoramic view of the scene. We describe two types of mosaics, static and dynamic, which are suited for storage and transmission applications, respectively. In each case, the mosaic construction process aligns the images using a global parametric motion transformation, usually canceling the effect of camera motion on the dominant portion of the scene. The residual motions that are not compensated by the parametric motion are then analyzed for their significance and coded. The mosaic representation exploits large scale spatial and temporal correlations in image sequences. In many applications where there is significant camera motion (e.g., remote surveillance), it performs substantially better than traditional interframe compression methods and offers the potential for very low bit-rate transmission. In storage applications, such as digital libraries and video editing environments, it has the additional benefit of enabling direct access and retrieval of single frames at a time.

SIAM Journal on Imaging Sciences, 2013

Surface parameterizations and registrations are important in computer graphics and imaging, where 1-1 correspondences between meshes are computed. In practice, surface maps are usually represented and stored as three-dimensional coordinates each vertex is mapped to, which often requires lots of memory. This causes inconvenience in data transmission and data storage. To tackle this problem, we propose an effective algorithm for compressing surface homeomorphisms using Fourier approximation of the Beltrami representation. The Beltrami representation is a complex-valued function defined on triangular faces of the surface mesh with supreme norm strictly less than 1. Under suitable normalization, there is a 1-1 correspondence between the set of surface homeomorphisms and the set of Beltrami representations. Hence, every bijective surface map is associated with a unique Beltrami representation. Conversely, given a Beltrami representation, the corresponding bijective surface map can be exactly reconstructed using the linear Beltrami solver introduced in this paper. Using the Beltrami representation, the surface homeomorphism can be easily compressed by Fourier approximation, without distorting the bijectivity of the map. The storage requirement can be effectively reduced, which is useful for many practical problems in computer graphics and imaging. In this paper, we propose applying the algorithm to texture map compression and video compression. With our proposed algorithm, the storage requirement for the texture properties of a textured surface can be significantly reduced. Our algorithm can further be applied to compressing motion vector fields for video compression, which effectively improves the compression ratio.

Proc. ACM Comput. Graph. Interact. Tech., 2020

We propose a method to reduce the footprint of compressed data by using modified virtual address translation to permit random access to the data. This extends our prior work on using page translation to perform automatic decompression and deswizzling upon accesses to fixed rate lossy or lossless compressed data. Our compaction method allows a virtual address space the size of the uncompressed data to be used to efficiently access variable-size blocks of compressed data. Compression and decompression take place between the first and second level caches, which allows fast access to uncompressed data in the first level cache and provides data compaction at all other levels of the memory hierarchy. This improves performance and reduces power relative to compressed but uncompacted data. An important property of our method is that compression, decompression, and reallocation are automatically managed by the new hardware without operating system intervention and without storing compression...

RAIRO - Theoretical Informatics and Applications

L'accès aux archives de la revue « Informatique théorique et applications » implique l'accord avec les conditions générales d'utilisation (http://www.numdam. org/legal.php). Toute utilisation commerciale ou impression systématique est constitutive d'une infraction pénale. Toute copie ou impression de ce fichier doit contenir la présente mention de copyright. Article numérisé dans le cadre du programme Numérisation de documents anciens mathématiques http://www.numdam.org/ Informatique théorique et Apphcations/Theoretical Informaties and Applications (vol 28, n° 3-4, 1994, p 201 à 211) THEORY AS BASIS FOR ADVANCES IN HYPERMEDIA by H. MAURER C 1) and F. KAPPE C 1) I am dedicating tms paper to my fnend Karel Culik II on the occasion of his 60th birthday Hermann MAURER Abstract-Hypermedia is currently one of the big buzz words in computer science, it is indeed sometim.es ndiculed as an area with much too much "hype" Yet the basic idea to have large bodies of networked data of heterogeneous type (i e a mixture of data types such as text, pictures, videoclips, and other matenal) accessible by means of linking chunks of information together in an "associative fashion" will influence deeply how we live and work, and this statement is no exaggeration Yet much of current hypermedia work is severely limited due to the fact that the solution ofsome of the problems involved needs deep theoretical results We try to estabhsh thisfact by choosing two rather different problems, and the solution of one of them may well be obtained based on results of

TheScientificWorldJournal, 2015

A novel optimal structure for implementing 3D-integer discrete cosine transform (DCT) is presented by analyzing various integer approximation methods. The integer set with reduced mean squared error (MSE) and high coding efficiency are considered for implementation in FPGA. The proposed method proves that the least resources are utilized for the integer set that has shorter bit values. Optimal 3D-integer DCT structure is determined by analyzing the MSE, power dissipation, coding efficiency, and hardware complexity of different integer sets. The experimental results reveal that direct method of computing the 3D-integer DCT using the integer set [10, 9, 6, 2, 3, 1, 1] performs better when compared to other integer sets in terms of resource utilization and power dissipation.

Pattern Recognition, 2021

How to extract effective expression representations that invariant to the identity-specific attributes is a long-lasting problem for facial expression recognition (FER). Most of the previous methods process the RGB images of a sequence, while we argue that the off-the-shelf and valuable expression-related muscle movement is already embedded in the compression format. In this paper, we target to explore the inter-subject variations eliminated facial expression representation in the compressed video domain. In the up to two orders of magnitude compressed domain, we can explicitly infer the expression from the residual frames and possibly extract identity factors from the I frame with a pre-trained face recognition network. By enforcing the marginal independence of them, the expression feature is expected to be purer for the expression and be robust to identity shifts. Specifically, we propose a novel collaborative min-min game for mutual information (MI) minimization in latent space. We do not need the identity label or multiple expression samples from the same person for identity elimination. Moreover, when the apex frame is annotated in the dataset, the complementary constraint can be further added to regularize the feature-level game. In testing, only the compressed residual frames are required to achieve expression prediction. Our solution can achieve comparable or better performance than the recent decoded image-based methods on the typical FER benchmarks with about 3 times faster inference.

Real-Time Systems, 2020

The well-known model of Vestal aims to avoid excessive pessimism in the quantification of the processing requirements of mixed-criticality systems, while still guaranteeing the timeliness of higher-criticality functions. This can bring important savings in system costs, and indirectly help meet size, weight and power constraints. This efficiency is promoted via the use of multiple worstcase execution time (WCET) estimates for the same task, with each such estimate characterized by a confidence associated with a different criticality level. However, even this approach can be very pessimistic when the WCET of successive instances of the same task can vary greatly according to a known pattern, as in MP3 and MPEG codecs or the processing of ADVB video streams.

Journal of Systems Architecture, 2022

The multiframe mixed-criticality task model eliminates the pessimism in manysystems where the worst-case execution times (WCETs) of successive jobs varygreatly by design, in a known pattern. Existing feasibility analysis techniquesfor multiframe mixed-criticality tasks are shared-resource-oblivious, hence unsafe for commercial-o_the-shelf (COTS) multicore platforms with a memorycontroller shared among all cores. Conversely, the feasibility analyses thataccount for the interference on shared resource(s) in COTS platforms do notleverage theWCET variation in multiframe tasks. This paper extends the state-of-the-art by presenting analysis that incorporates the memory access stall inmemory-bandwidth-regulated multiframe mixed-criticality multicore systems.An exhaustive enumeration approach is proposed for this analysis to furtherenhance the schedulability success ratio. The running time of the exhaustiveanalysis is improved by proposing a pruning mechanism that eliminates thecombinations of interfering job sequences that subsume others. Experimentalevaluation, using synthetic task sets, demonstrates up to 72% improvement interms of schedulability success ratio, compared to frame-agnostic analysis.

Multimedia Tools and Applications, 2012

The Discrete Cosine Transform (DCT) is one of the most widely used techniques for image compression. Several algorithms are proposed to implement the DCT-2D. The scaled SDCT algorithm is an optimization of the DCT-1D, which consists in gathering all the multiplications at the end. In this paper, in addition to the hardware implementation on an FPGA, an extended optimization has been performed by merging the multiplications in the quantization block without having an impact on the image quality. A simplified quantization has been performed also to keep higher the performances of the all chain. Tests using MATLAB environment have shown that our proposed approach produces images with nearly the same quality of the ones obtained using the JPEG standard. FPGA-based implementations of this proposed approach is presented and compared to other state of the art techniques. The target is an an Altera Cyclone II FPGA using the Quartus synthesis tool. Results show that our approach outperforms the other ones in terms of processing-speed, used resources and power consumption. A comparison has been done between this architecture and a distributed arithmetic based architecture.

2002

We introduce a novel approach for sending messages over lossy packet-based networks. The new method, called Priority Encoding Transmission, allows a user to specify a di erent priority on each segment of the message. Based on the priorities, the sender uses the system to encode the segments into packets for transmission. The system ensures recovery of the segments in order of their priority. The priority of a segment determines the minimum number of packets su cient to recover the segment. We de ne a measure for a set of priorities, called the rate, which dictates how much information about the message must be contained in each bit of the encoding. We develop systems for implementing any set of priorities with rate equal to one. We also give an information-theoretic proof that there is no system that implements a set of priorities with rate greater than one. This work has applications to multi-media and high speed networks applications, especially in those with bursty sources and multiple receivers with heterogeneous capabilities.

Nature Neuroscience, 2019

Many behaviors rely on predictions derived from recent visual input, but the temporal evolution of those inputs is generally complex and difficult to extrapolate. We propose that the visual system transforms these inputs to follow straighter temporal trajectories. To test this 'temporal straightening' hypothesis, we develop a methodology for estimating the curvature of an internal trajectory from human perceptual judgments. We use this to test three distinct predictions: natural sequences that are highly curved in the space of pixel intensities should be substantially straighter perceptually; in contrast, artificial sequences that are straight in the intensity domain should be more curved perceptually; finally, naturalistic sequences that are straight in the intensity domain should be relatively less curved. Perceptual data validate all three predictions, as do population models of the early visual system, providing evidence that the visual system specifically straightens natural videos, offering a solution for tasks that rely on prediction.

arXiv (Cornell University), 2023

Today's large-scale scientific applications running on high-performance computing (HPC) systems generate vast data volumes. Thus, data compression is becoming a critical technique to mitigate the storage burden and data-movement cost. However, existing lossy compressors for scientific data cannot achieve a high compression ratio and throughput simultaneously, hindering their adoption in many applications requiring fast compression, such as in-memory compression. To this end, in this work, we develop a fast and highratio error-bounded lossy compressor on GPUs for scientific data (called FZ-GPU). Specifically, we first design a new compression pipeline that consists of fully parallelized quantization, bitshuffle, and our newly designed fast encoding. Then, we propose a series of deep architectural optimizations for each kernel in the pipeline to take full advantage of CUDA architectures. We propose a warplevel optimization to avoid data conflicts for bit-wise operations in bitshuffle, maximize shared memory utilization, and eliminate unnecessary data movements by fusing different compression kernels. Finally, we evaluate FZ-GPU on two NVIDIA GPUs (i.e., A100 and RTX A4000) using six representative scientific datasets from SDRBench. Results on the A100 GPU show that FZ-GPU achieves an average speedup of 4.2× over cuSZ and an average speedup of 37.0× over a multi-threaded CPU implementation of our algorithm under the same error bound. FZ-GPU also achieves an average speedup of 2.3× and an average compression ratio improvement of 2.0× over cuZFP under the same data distortion. • Theory of computation → Massively parallel algorithms.

Proceedings of the Fourth International Conference on World Wide Web

The architecture of World Wide Web (WWW) browsers and servers supports full-file transfer for docu ment retrieval. TCP is used for data transfers by Web browsers and their associated Hypertext Transfer Protocol (HTTP) servers. Full-file transfer and TCP are unsuitable for continuous media, such as real time audio and video. In order for the WWW to support continuous media, we require the transmission of video and audio on demand and in real time, as well as new protocols for real-time data. We extend the architecture of the WWW to encompass the dynamic, real-time information space of video and audio. Our WWW browser Vosaic, short for Video Mosaic, incorporates real-time video and audio into standard hypertext pages that are displayed in place. Video and audio transfers occur in real time; there is no file-retrieval latency. The video and audio result in compelling Web pages. Real-time video and audio data can be effectively served over the present day Internet with the proper transmission pro tocol. We have developed a real-time protocol called VDP that we specialized for handling real-time video over the WWW.-VDP reduces inter-frame jitter and dynamically adapts to the client CPU load and network congestion. Our WWW server dynamically changes transfer protocols, adapting to the request stream and the metainformation in requested documents. Experiments show a 44-fold increase in received video-frame rate (0.2 frames-per-second (Jps) to 9 fps) with the use of VDP in lieu of TCP, with a commensurate improvement in observed video quality. Our work enables a video-enhanced Web.