Video adaptation

This paper is among the first works to document experimental results for application-aware H.264 Scalable Video Coding (SVC) support over Wireless LANs. Application-aware support is achieved by introducing a bandwidth throttling device, called Virtual BottleNeck (VBN), before the WLAN Access Point. Throttling is set to a bandwidth slightly smaller than the actual WLAN capacity (either known or estimated), so that all packet/frame losses occur inside the VBN. Here, loss events are controlled by a scheduling mechanism devised to operate with information taken from the H.264 Network Abstraction Layer Units (NALUs). Despite its relative simplicity, the implemented scheduler exhibits effective video adaptation performance and close to optimal bandwidth efficiency. Setting up the trial was not trivial due to the lack of suitable publicly available tools. We filled this gap by implementing and integrating several separate software modules, e.g., streaming server, NALU dependency filtering, video frame concealment, etc. As a final result, the experimental trial supports the full delivery chain for an SVC stream with the only limitation of an off-line stream conversion for play-out and Peak Signal-to-Noise Ratio (PSNR) measurement purposes, due to the unavailability of real time SVC players. 1 The source code of the software as well as all the videos produced and discussed throughout the remainder of this paper can be found at the URL -for download purposes note that videos are 300-400 MB each.

With continued increase in the use of smartphones, user expectations of content access have also increased. Most of the content that exists today is not designed for mobile devices. Mobile devices cannot directly access most of the content due to the mismatch in device capability and content playback requirements. Content adaptation is an essential tool that bridges the gap between device capabilities and content formats. In this paper we present an overview of content adaptation and survey recent papers on content adaptation for mobile devices. We introduce the when, where, and what of content adaptation to help classify the content adaptation techniques and to select the appropriate techniques for a given content delivery environment.

Transcoding is a fast technique for video adaptation, which offers great flexibility for changing the characteristics of already coded video sequences in order to match the properties of the terminal display devices and the bandwidth availability of the transmission networks. For reduction of the bitrate, requantization transcoding is often used. In our previous work, we introduced a hybrid architecture for requantization transcoding, which is able to reduce the bitrate for H.264/AVC bitstreams with quality results approaching the rate-distortion optimal decoderencoder cascade. In this paper, we focus on the filters used for interpolation of the accumulated requantization errors, which are used for compensation of the current (sub-)macroblock partition. We show that by using reduced complexity filters, computational complexity can be significantly reduced without quality loss.

In this paper, we examine spatial resolution downscaling transcoding for H.264/AVC video coding. A number of advanced coding tools limit the applicability of techniques which were developed for previous video coding standards. We present a spatial resolution reduction transcoding architecture for H.264/AVC, which extends open-loop transcoding with a low-complexity compensation technique in the reduced-resolution domain. The proposed architecture tackles the problems in H.264/AVC and avoids visual artifacts in the transcoded sequence, while keeping complexity significantly lower than more traditional cascaded decoder-encoder architectures. The refinement step of the proposed architecture can be used to further improve rate-distortion performance, at the cost of additional complexity. In this way, a dynamiccomplexity transcoder is rendered possible. We present a thorough investigation of the problems related to motion and residual data mapping, leading to a transcoding solution resulting in fully compliant reduced-size H.264/AVC bitstreams.

By 2025, real-time communication has become a fundamental necessity for high-density urban centers around the world. In the Philippines-and in Metro Manila in particular-the rising demand for live streaming, interactive entertainment, remote work, and distance learning drives the evolution of communication technologies. This paper introduces an innovative WebRTC-based architecture engineered for ultra-low latency online streaming with robust server-side recording capabilities. We place special emphasis on the collaborative environment between major mobile network operators, including Globe and Smart, each managing a substantial network of cellular towers across a vast and densely populated region. Beyond detailing an in-depth evaluation of network performance metrics-bandwidth, Round-Trip Time (RTT), mobile tower densities, and user traffic patterns-this work demonstrates how advanced error correction, adaptive transcoding, and multi-factor buffering strategies can radically reduce latency. To showcase practical relevance, we include extensive numerical simulations focused on real-world throughput gains, cost savings, and improved Quality of Service (QoS). Our findings highlight a path forward for hyper-dense cities seeking to accommodate the exponential growth in real-time, high-fidelity streaming while ensuring data integrity through server-side capture and archiving.

This thesis presents a retro-engineering and an improvement of the admission control of the platform of Alcatel-Lucent (ASR). The admission control is managed by two algorithms : the flow control algorithm and the global load estimation algorithm of the system. On the one hand, both algorithms were improved. On the other hand, their behaviour has been formalised thanks to two models. Reliability, maintainability and good performance of the ASR require a redundant architecture in components, a division of these into containers, the implementation of strategies like the respect of the pattern Reactor/Proactor and the massive parallelisation of the treatment of the messages. Moreover, policy of the development team of Alcatel-Lucent consists in developing generic functionalities including the admission control. These principles complicates the improvement of a global load estimation algorithm of the system. A solution is presented in detail and a case study shows that the admission control is improved. Ce mémoire présente la rétro-ingénierie et une amélioration du contrôle d'admission. Le contrôle d'admission de la plateforme d'Alcatel-Lucent (ASR) est géré par deux algorithmes : l'algorithme du contrôle de flux et l'algorithme de l'évaluation de la charge globale du système. D'une part, l'algorithme de contrôle de flux et surtout l'évaluation de la charge globale du système ont été améliorés. D'autres parts, leur comportement est formalisé grâce à deux modèles. La fiabilité, la maintenabilité et les bonnes performances de l'ASR nécessitent une architecture redondante en composants, une découpe de ceux-ci en conteneurs, la mise en oeuvre de stratégies comme le respect du pattern Reactor/Proactor et la parallélisation massive du traitement des messages. La ligne de conduite de l'équipe de développement d'Alcatel-Lucent consiste à développer des fonctionnalités génériques dont le contrôle d'admission. Ces principes ont compliqué l'amélioration et la mise au point d'un algorithme d'évaluation de la charge globale du système. Une solution est présentée en détail et une étude de cas montre que le contrôle d'admission est amélioré. Au terme de cette année de travail au sein de deux équipes compétentes, en Belgique et en France, je tiens à remercier toutes les personnes qui m'ont permis de mener à bien ce mémoire. Je remercie particulièrement mes superviseurs, Laurent Schumacher et Dimitri Tombroff. À Namur, Laurent Schumacher et Antoine Roly ont été patients pendant la lecture de mes rapports et pendant la correction du mémoire, car mes idées étaient quelques fois confuses et difficiles à expliquer, mais les discussions fructueuses que nous avons eues pendant nos réunions avec Marie-Ange Remiche m'ont beaucoup aidé. Laurent Schumacher a rendu possible le stage à Alcatel-Lucent avec Dimitri Tombroff, leader d'une équipe de Recherche et Développement à Paris. J'ai eu la chance d'intégrer, pendant cinq mois à Paris, cette équipe dynamique. J'ai beaucoup apprécié travailler avec Dimitri : j'ai été marqué par sa volonté de formaliser rigoureusement certains processus très complexes, sa concentration sur la cause d'un problème et non sur ses symptômes et sa perpétuelle remise en question. J'espère acquérir dans ma future carrière la même méthodologie pour traiter les problèmes. Dans cette équipe, je remercie Arjun Penday et Pierre De Rop, ingénieurs du logiciel. Arjun, un grand voyageur, m'a toujours donné des explications claires et pédagogiques pour me dépanner. Pierre m'a rendu de fiers services, spécialement lorsque plus aucune commande n'arrivait à débloquer des serveurs. Merci à Jean-Francois Martin, manager de projet dans une autre équipe avec laquelle nous avons travaillé, nous avons beaucoup réfléchi et testé des solutions à nos problèmes communs dans une très bonne ambiance. Jean-François m'a également fourni les outils nécessaires pour procéder à des tests et m'en a expliqué leur fonctionnement. Je remercie également la Fondation Biermans-Lapôtre, la maison des étudiants belges et luxembourgeois, pour m'avoir logé à la Cité Universitaire de Paris, car elle m'a permis de rencontrer énormément de personnes et elle a contribué à rendre mon séjour en France agréable et unique. Je tiens à remercier ma famille et Tina pour leur soutien, ainsi que mes camarades de classe avec lesquels nous nous sommes motivés dans cette dernière ligne droite. Je remercie enfin Anne-Marie Breny, Jean-Marie Jacquet et Wim Vanhoof qui ont organisé sur mesure mon passage de la faculté des Sciences à la Faculté d'Informatique. Je suis très reconnaissant pour l'excellente formation que j'ai reçue à l'Université de Namur.

Ayman Abdalla as well for being my local supervisor. I thank him for his consistent support and encouraging comments during the research stages. I also thank Dr. Shakeel Ahmad for the useful discussion on the research and for his advice on dealing with problems. I thank my colleagues Dr. Muneeb Dawood and Mr. Shadan Khattak for their cooperation and help. I want to thank my beloved De Montfort University and all its familiar staff for the interesting time I spent with them. A special thanks to my colleagues and friends for their help and support: Dr. Osama Abdulhay for the statistical analysis and Dr. Mohammad Daoud and Mr. Hani al-Mimi for their help in the subjective evaluation. I also thank all staff and students of the Department of Multimedia Systems at Al-Zaytoonah University. v Dedication I dedicate this work to the soul of my father who supported me during all stages of my life. I dedicate this work to my mother for encouraging and giving me the power to tackle any task during the PhD. I also dedicate the work to my brothers, sisters and fiancé for their patience and prayers. I would like also to dedicate this thesis to my uncles Dr. Abdelfatah

Nowadays, the heterogeneity of networks, terminals, and users is growing. At the same time, the availability and usage of multimedia content is increasing, which has raised the relevance of content adaptation technologies able to fulfill the needs associated to all usage conditions. For example, mobile displays tend to be too small to allow one to see all the details of an image. This paper presents an innovative method to integrate low-level and semantic visual cues into a unique visual attention map that represents the most interesting contents of an image, allowing the creation of a video sequence that browses through the image displaying its regions of interest in detail. The architecture of the developed adaptation system, the processing solutions and also the principles and reasoning behind the algorithms that have been developed and implemented are presented in this paper. Special emphasis is given to the integration of low-level and semantic visual cues for the maximization of the image to video adapted experience.

In this paper, we present an integrated system that has been designed to support automatic semantic extraction of highlights in sports video and automatic video adaptation according to user's preferences. To analyze the user's satisfaction, we propose a new performance measure that explicitly takes into account the user's preferences and considers the number and type of errors produced by the annotation engine and the way in which these errors affect the compressed video quality and bandwidth allocation. We provide experimental results with application to soccer and swimming.

Le travail réalisé tout au long de cette thèse n'a été possible qu'avec l'aide et le soutien de nombreuses personnes. Je profite de l'occasion qui m'est donnée ici pour leur exprimer toute ma gratitude. Je tenais en premier lieu à remercier vivement Jean-Jacques Pansiot, mon directeur de thèse, pour son encadrement et sa disponibilité. Merci Jean-Jacques de m'avoir fait partager tes connaissances, ton expérience et ta bonne humeur. Ce fut un réel plaisir de travailler pendant ces années sous ta direction. Je souhaite remercier chaleureusement Dominique Grad et Benoît Hilt, mes coencadrants de thèse, pour m'avoir prodigué de précieux conseils et pour m'avoir apporté de nombreuses idées au cours de ce travail. Je remercie Messieurs Guy Leduc, Thomas Noël, Vincent Roca et François Spies pour l'intérêt qu'ils ont porté à mes travaux en acceptant de faire partie de mon jury de thèse. Je tiens également à remercier toute l'équipe Réseaux et Protocoles du LSIIT. En particulier les doctorants présents et passés, avec qui j'ai beaucoup appris, partagé et apprécié travailler. Merci à Alexander, Damien, Émil, Julien, Julien, Mickaël, Pascal et Romain, pour votre disponibilité et votre gentillesse. Enfin, je remercie également ma famille et mes amis qui m'ont toujours soutenu, encouragé et sans qui je n'écrirais pas ces lignes. i vi independently of the reception rate. To deal with these problems, we propose a sequencer and its API, which assure that the more important the data is, the lower its sending rate is. And thus acquired by the majority of receivers. Experiments have shown the sequencer optimal performances insuring that a receiver subscribed to N% of the source rate obtains the most important N% of the applicative data. Moreover, applications can use this API without knowing anything about the dynamic layering used by M2C. Therefore, this API is easy to use, which makes it suitable for various kind of applications. Indeed, we have demonstrated the usefulness of the API and the underlying congestion control protocol by implementing two experimental applications : a file transfer [MU-TUAL] and a video streaming software [MUST].

The Internet is facing a significant evolution from being a delivery network for static content to an efficient platform for multimedia content delivery. Well-known examples of applications driving this evolution are YouTube Video on Demand, Skype Audio/Video conference, IPTV and P2P video distribution. While YouTube streams videos using the Transmission Control Protocol (TCP), time-sensitive applications, such as Skype Audio/Video conference, employ the UDP because they can tolerate small loss percentages but not delays due to TCP recovery of lost packets via retransmissions. Since, differently from the TCP, the UDP does not implement congestion control, these applications must implement congestion control at the application layer in order to avoid congestion and preserve network stability. In this paper we investigate Skype Video congestion control in order to assess to what extent this application is able to throttle its sending rate to match the unpredictable Internet bandwidth while preserving resource for co-existing best-effort TCP traffic. We have found that: (1) Skype Video adapts its sending rate by varying frame rate, frame quality and video resolution; (2) in many scenarios a Skype Video call refrains from fully utilizing all available bandwidth thus not sending videos at the highest possible quality; (3) Skype Video employs an adaptive FEC action that is proportional to the experienced loss rate; (4) the sending rate matches a changing available bandwidth with a transient time as large as a hundred of seconds; (5) the minimum bandwidth required for a video call is 40 kbps at 5 frames per second.

The quality and bitrate modeling is essential to effectively adapt the bitrate and quality of videos when delivered to multiplatform devices over resource constraint heterogeneous networks. The recent model proposed by Wang et al. [1] estimates the bitrate and quality of videos in terms of the frame rate and quantization parameter. However, to build an effective video adaptation framework, it is crucial to incorporate the spatial resolution in the analytical model for bitrate and perceptual quality adaptation. Hence, this paper proposes an analytical model to estimate the bitrate of videos in terms of quantization parameter, frame rate, and spatial resolution. The model can fit the measured data accurately which is evident from the high Pearson correlation. The proposed model is based on the observation that the relative reduction in bitrate due to decreasing spatial resolution is independent of the quantization parameter and frame rate. This modeling can be used for rate-constrained bit-stream adaptation scheme which selects the scalability parameters to optimize the perceptual quality for a given bandwidth constraint.

Les propositions les plus avancées, comme Wave and Equation Based Rate Control (WEBRC) [Luby 02], utilisent de tels canaux dynamiques et prennent en compte l'équité des débits obtenus avec PROBLÉMATIQUE ETÉTAT DE L'ART 7 1. En SSM IGMPv3 ou MLDv2 est nécessaire, car seules ces versions permettent de spécifier une source pour un groupe multicast.

Video surveillance systems must support multiple streaming and prompt alert notification. We propose a two-tiered environment: a supervisor defines presentation layouts and model interface reactions to alerts; a surveillant watches synchronized videos, adapts layouts, and is notified with alerts.

Data encryption is one of the key information security technologies used for safeguarding multimedia content from unauthorised access and manipulation in end-to-end delivery and access chains. This technology, combined with the appropriate cryptographic methods, effectively prevents the content against malicious attacks, so as to protect its authenticity as well as integrity. While encryption based security is ensuring the authorised consumption of the multimedia content, content adaptation technologies have the primary goal of providing means for wider dissemination of the content across diverse networks, devices and users, and thus enriching user satisfaction and experience of the delivered content within a given set of usage environment constraints. Traditionally, protected contents can only be adapted at trusted adaptation engines residing between the source and end-users since they have to be fully decrypted before performing the necessary adaptation operations. The drawback of such a process is that it significantly limits the availability and flexibility of adaptation engines applicable for adapting protected contents on the fly. Thus, this paper proposes a novel scalable H.264/Advance Video Coding (AVC)-compatible video encryption technique, which is also transparent to adaptation engines in an end-to-end video delivery scenario. The proposed technology relies on keeping syntax elements required for performing the adaptation operations clear (i.e., not encrypted). The effectiveness of the proposed technique has been successfully verified in scenarios, where both conventional Joint Scalable Video Model (JSVM) bit stream extracting and random packet dropping mechanisms are used.

Semantic video adaptation improves traditional adaptation by taking into account the degree of relevance of the different portions of the content. It employs solutions to detect the significant parts of the video and applies different compression ratios to elements that have differentimportance. Performance of semantic adaptation heavily depends on the precision of the automatic annotation and the way of operation of the codec which is used to perform adaptation at the event or object level. In this paper, we discuss critical factors that affect performance of automatic annotation and define new performance measures of semantic adaptation, Viewing Quality Loss and Bitrate Cost Increase, that are obtained from classical PSNR and Bit Rate, but relate the results of semantic adaptation with the user's preferences and expectations. The new measures are discussed in detail for a system of sport annotation and adaptation with reference to different user profiles.

We present the Director's Lens, an intelligent interactive assistant for crafting virtual cinematography using a motiontracked hand-held device that can be aimed like a real camera. The system employs an intelligent cinematography engine that can compute, at the request of the filmmaker, a set of suitable camera placements for starting a shot. These suggestions represent semantically and cinematically distinct choices for visualizing the current narrative. In computing suggestions, the system considers established cinema conventions of continuity and composition along with the filmmaker's previous selected suggestions, and also his or her manually crafted camera compositions, by a machine learning component that adapts shot editing preferences from user-created camera edits. The result is a novel workflow based on interactive collaboration of human creativity with automated intelligence that enables efficient exploration of a wide range of cinematographic possibilities, and rapid production of computer-generated animated movies.

Video surveillance systems must support multiple streaming and prompt alert notification. We propose a two-tiered environment: a supervisor defines presentation layouts and model interface reactions to alerts; a surveillant watches synchronized videos, adapts layouts, and is notified with alerts.

This paper introduces an innovative Adaptive Video Call System (AVCS), designed to significantly enhance video communication quality in low bandwidth and weak signal environments, a challenge persistently faced in remote and disaster-stricken areas as of 2018. Traditional video call technologies often struggle under these conditions, leading to degraded video quality, high latency, and frequent interruptions. The AVCS addresses these limitations by seamlessly integrating advanced hardware and software components to dynamically adjust video call parameters in real-time, ensuring an optimized communication experience. Key features of the AVCS include intelligent bandwidth allocation, advanced signal enhancement algorithms, and a user-friendly interface. These elements work in concert to provide a robust solution that prioritizes critical aspects of video calls, such as voice clarity and visual fidelity, even in the most challenging network scenarios. The system also leverages predictive analytics to anticipate potential network fluctuations, enhancing its adaptive capabilities. A comparative analysis with existing technologies highlights the AVCS's superior performance, particularly its ability to facilitate seamless transitions between different network states and maintain consistent video call quality. The potential applications of the AVCS are vast, extending to emergency response, telemedicine, and remote education, indicating its capability to bridge the digital divide in communication technology. However, the implementation of such an advanced system is not without challenges. These include infrastructure dependencies, technological constraints, and user adoption barriers. Recognizing these challenges, the paper outlines future work and development directions, emphasizing the integration with emerging technologies, improving energy efficiency, and focusing on user-centric design. The AVCS, represents a groundbreaking advancement in telecommunications, offering a comprehensive solution to a longstanding problem in video communication technology. Its development and deployment could herald a new era of connectivity and accessibility in areas previously hindered by inadequate network infrastructure.

When viewing multimedia presentations, a user only attends to a relatively small part of the video display at any one point in time. By shifting allocation of bandwidth from peripheral areas to those locations where a user's gaze is more likely to rest, attentive displays can be produced. Attentive displays aim to reduce resource requirements while minimizing negative user perception-understood in this paper as not only a user's ability to assimilate and understand information but also his/her subjective satisfaction with the video content. This paper introduces and discusses a perceptual comparison between two region-of-interest display (RoID) adaptation techniques. A RoID is an attentive display where bandwidth has been preallocated around measured or highly probable areas of user gaze. In this paper, video content was manipulated using two sources of data: empirical measured data (captured using eye-tracking technology) and predictive data (calculated from the physical characteristics of the video data). Results show that display adaptation causes significant variation in users' understanding of specific multimedia content. Interestingly, RoID adaptation and the type of video being presented both affect user perception of video quality. Moreover, the use of frame rates less than 15 frames per second, for any video adaptation technique, caused a significant reduction in user perceived quality, suggesting that although users are aware of video quality reduction, it does impact level of information assimilation and understanding. Results also highlight that user level of enjoyment is significantly affected by the type of video yet is not as affected by the quality or type of video adaptation-an interesting implication in the field of entertainment. Index Terms-Attentive displays, eye tracking, perceptual quality, region of interest (RoI).

Multicast video streaming over multirate wireless LANs imposes strong demands on video codecs and the underlying network. It is not sufficient that only the video codec or only the underlying protocols adapt to changes in the wireless link quality. Research efforts should be applied in both and in a synchronized way. Cross layer design is a new paradigm that addresses this challenge by optimizing communication network architectures across traditional layer boundaries. In this paper we present a cross-layer architecture for adaptive video multicast streaming over multirate wireless LANs where layer-specific information is passed in both directions, top-down and bottomup. We jointly consider three layers of the protocol stack: the application, data link and physical layers. We analyze the performance of the proposed architecture and extensively evaluate it via simulations. Our results show that the real-time video quality of the overall system can be greatly improved by cross-layer signaling.

The satisfaction a user gets from watching a video in a resource limited device, can be formulated by Utility Theory. The resulting video adaptation is optimal in the sense that the adapted video maximizes the user satisfaction, which is modeled through subjective tests comprising of 3 independent utility components : crispness, motion smoothness and content visibility. These components are maximized in terms of coding parameters by obtaining a Pareto optimal set. In this manuscript, inclusion of transmission channel capacity into the subjective utility model of user satisfaction is addressed. It is proposed that using the maximum channel capacity as a restriction metric, certain members of the Pareto optimal solution set can be eliminated such that the remaining members are suitable for transmission through the given channel. Once the reduced solution set is obtained, an additional figure of merit can be used to pick a single solution from this set, depending on the application scenario. Figure 1. Delivery of the multimedia content from the media server to mobile terminal A novel approach to obtain the utility function for the problem above, is proposed. The problem is considered as a multiple objective utility formulation. The overall utility function is decomposed into 3 independent components, such that the satisfaction associated with any one of these terms can be considered as independent from every other component. These terms are determined as:

We present a brief summary of current approaches and challenges in the network traffic management area. In this contribution, we well cover the most recent experiments on the network traffic and behavior of applications under various network conditions. Quality of the users' experience is the most important factor being considered in this group of experiments.

This paper presents a novel algorithm designed to efficiently adapt scalable video using MPEG-21. The algorithm is able to take the optimal decision among all possible video adaptations. It takes into account information about network, user terminal and user preferences. This information is gathered together by means of MPEG-21 DIA descriptors. The decision on the optimal adaptation is achieved through the application of the Viterbi algorithm. The results prove that the presented decision-taking engine offers an optimal and efficient solution for personalized scalable video adaptation in heterogeneous environments.

The explosive growth of VoIP trac poses a potential challenge to the stability of the Internet that, up to now, has been guaranteed by the TCP congestion control. In this paper we investigate how Skype behaves in the presence of time-varying available bandwidth in order to discover if some sort of congestion control mechanism is implemented at the application layer to match the network available bandwidth and cope with congestion. We have found that Skype ows are somewhat elastic, i.e. they employ some sort of congestion control when sharing the bandwidth with unresponsive ows, but are inelastic in the presence of classic TCP responsive ows, which provoke extreme unfair use of the available bandwidth in this case. Finally, we have found that when more Skype calls are established on the same link, they are not able to adapt their sending rate to match correctly the available bandwidth, which would conrm the risk of network congestion collapse.

Video annotation provides a suitable way to describe, organize, and index stored videos. On the other hand, transcoding aims at adapting content to the user/client capabilities and requirements. Both cues are now mandatory, given the tremendous demand of multimedia access from remote clients, in particular nowadays that new terminals with limited resources (PDAs, HCCs, Smart phones) have access to the network. In this paper we propose an unified framework to define event-based and object-based semantic extraction from video to provide both semantic video annotation for video stored and semantic on-line transcoding from live cameras. Two case studies (highlights' extraction from soccer videos for the annotation and people behavior detection in domotic application for transcoding) and corresponding experimental results are reported.

The newly ratified video compression standard H.264/AVC provides tools and solutions for coding improvements of at least 2 dB, in terms of PSNR, and at least 40% in bit rate saving, as compared with the previous standards. The development of video transcoding, which is one of the critical components for multimedia adaptation and Universal Multimedia Access, can gain benefits by using the tools and solutions from the standard. In this paper, the new features of H.264/AVC are first briefly reviewed. The issues involved in video transcoding are addressed together with a general framework for video transcoding. Here we utilize an architecture of pixel domain video transcoding derived directly from H.264/AVC.

The explosive growth of compressed video streams and repositories which are accessible worldwide and the recent addition of new video-related standards, such as H.264/AVC, MPEG-7, and MPEG-21, have stimulated research for new technologies and applications in the area of multimedia architectures, processing, and networking. Current communication networks exhibit a wide range of capabilities, including various architectures, throughputs, and quality of service and protocols. The interconnection of different networks provides several advantages, but also poses major technical challenges. However, users employ heterogeneous videoenabled terminals such as computers, TVs, mobile phones, and personal digital assistants with a wide range of computational and display capabilities, energy resources, features, accessibilities, and user preferences. Such heterogeneity in networks and user devices has escalated the need for efficient and effective techniques for adapting compressed videos to better suit the different capabilities, constraints, and requirements of various transmission networks, applications, and end users. For instance, universal multimedia access (UMA) advocates the provision and adaptation of the same multimedia content for different networks, terminals, and user preferences. Video adaptation is an emerging field that offers a rich body of knowledge and techniques for handling the huge variation of resource constraints (e.g., bandwidth, display capability, processing speed, and power consumption) and the large diversity of user tasks in pervasive media applications. Video adaptation may apply to individual or multiple video streams and call for different means depending on the objectives and requirements of adaptation. Transcoding, transmoding (cross-modality transcoding), scalable content representation, and content abstraction and summarization are popular means for video adaptation. In addition, video con

Adaptive HTTP streaming has become a new trend to support adaptivity in video delivery. An HTTP streaming client needs to estimate exactly resource availability and resource demand. In this paper, we focus on the most important resource which is bandwidth. A new and general formulation for throughput estimation is presented taking into account previous values of instant throughput and round trip time. Besides, we introduce for the first time the use of bitrate estimation in HTTP streaming. The experiments show that our approach can effectively cope with drastic changes in connection throughput and video bitrate.

La famille de protocoles TCP/IP dispose d'un grand nombre de protocoles de transport. On connait bien sûr TCP mais il y en a d'autres comme UDP et SCTP et d'autres encore moins connus. Cette diversité peut compliquer la tâche des développeurs d'application, qui ne savent pas toujours bien lequel choisir. Et elle poseégalement problème pour les développements futursà l'IETF, par exemple la réflexion en cours sur les API. On a besoin d'une description claire des services que chaque protocole fournit effectivement. C'est le but du groupe de travail TAPS <https://datatracker.ietf.org/ wg/taps/about/> dont voici le deuxième RFC : une description de haut niveau des services que fournit la couche transport.À faire lireà tous les développeurs d'applications réseau,à tous les participants a la normalisation, et auxétudiants en réseaux informatiques. Ce RFC parle de TCP, MPTCP, SCTP, UDP et UDP-Lite, sous l'angle quels services rendent-ils aux applications ? (QUIC, pas encore normalisé, n'y figure pas. LEDBAT, qui n'est pas vraiment un protocole de transport, est par contre présent.) L'idée est de lister un ensemble d'opérations abstraites, qui pourront ensuiteêtre exportées dans une API. J'en profite pour recommander la lecture du RFC 8095 1 , premier RFC du groupe TAPS.

To enhance video streaming experience for mobile users, we propose an approach towards Quality-of-Experience (QoE) aware on-the-fly transcoding. The proposed approach relies on the concept of Mobile Edge Computing (MEC) as a key enabler in enhancing service quality. Our scheme involves an autonomic creation of a transcoding service as a Virtual Network Function (VNF) and ensures dynamic rate switching of the streamed video to maintain the desirable quality. This edge-assistive transcoding and adaptive streaming results in reduced computational loads and reduced core network traffic. The proposed solution represents a complete miniature content delivery network infrastructure on the edge, ensuring reduced latency and better quality of experience.

Video streaming over HTTP/2 is a new trend in multimedia delivery. Compared to the pull-based HTTP/1.1 protocol, the new HTTP/2 protocol's Server Push feature is very effective in reducing the overheads (e.g., in terms of energy, processing, bandwidth) for clients, servers, and network nodes. This paper presents an HTTP/2 push-based adaptation method for on-demand video streaming that reduces the number of requests and provides high video quality. In our method, for each client request, the server sends video segments continuously until it receives another client request with a new quality. Since a request is sent only if the client wants to update the video bitrate, our method can significantly reduce the request related overhead. For this context, a buffer based algorithm is proposed to provide high and smooth video quality while avoiding buffer underflows. Experiments show that the proposed method can provide a lower number of requests, higher average quality and better quality smoothness than existing methods.

The explosive growth of VoIP trac poses a potential challenge to the stability of the Internet that, up to now, has been guaranteed by the TCP congestion control. In this paper we investigate how Skype behaves in the presence of time-varying available bandwidth in order to discover if some sort of congestion control mechanism is implemented at the application layer to match the network available bandwidth and cope with congestion. We have found that Skype ows are somewhat elastic, i.e. they employ some sort of congestion control when sharing the bandwidth with unresponsive ows, but are inelastic in the presence of classic TCP responsive ows, which provoke extreme unfair use of the available bandwidth in this case. Finally, we have found that when more Skype calls are established on the same link, they are not able to adapt their sending rate to match correctly the available bandwidth, which would conrm the risk of network congestion collapse.

This letter presents an image to video adaptation system that transmodes images in order to be viewed on small displays without a significant loss of information. The followed approach is to automate the process of manual browsing and zooming through an image by simulating the movement of a virtual camera. The presented system proposes two modules: one for analysis (ROI Extraction: for defining the regions of interest (ROIs) in the image) and the second for transmoding (ROIs2Video: for creating a video from any predefined list of relevance annotated ROIs). The separation of the image analysis and transmoding steps allows to use the ROIs2Video module in different application domains, as the transmoding will only be driven by the available relevance values associated to the ROIs. Results are presented for automatic and manual ROI generation modules. User evaluation results show that the developed system offers a visually attractive solution. Index Terms-Browsing path, content adaptation, image browsing, image to video transmoding, regions of interest (ROIs), simulated camera movement.

This paper is among the first works to document experimental results for application-aware H.264 Scalable Video Coding (SVC) support over Wireless LANs. Application-aware support is achieved by introducing a bandwidth throttling device, called Virtual BottleNeck (VBN), before the WLAN Access Point. Throttling is set to a bandwidth slightly smaller than the actual WLAN capacity (either known or estimated), so that all packet/frame losses occur inside the VBN. Here, loss events are controlled by a scheduling mechanism devised to operate with information taken from the H.264 Network Abstraction Layer Units (NALUs). Despite its relative simplicity, the implemented scheduler exhibits effective video adaptation performance and close to optimal bandwidth efficiency. Setting up the trial was not trivial due to the lack of suitable publicly available tools. We filled this gap by implementing and integrating several separate software modules, e.g., streaming server, NALU dependency filtering, video frame concealment, etc. As a final result, the experimental trial supports the full delivery chain for an SVC stream with the only limitation of an off-line stream conversion for play-out and Peak Signal-to-Noise Ratio (PSNR) measurement purposes, due to the unavailability of real time SVC players. 1 The source code of the software as well as all the videos produced and discussed throughout the remainder of this paper can be found at the URL http://netgroup.uniroma2.it/iwcld09/-for download purposes note that videos are 300-400 MB each.

Video streaming over HTTP/2 is a new trend in multimedia delivery. Compared to the pull-based HTTP/1.1 protocol, the new HTTP/2 protocol's Server Push feature is very effective in reducing the overheads (e.g., in terms of energy, processing, bandwidth) for clients, servers, and network nodes. This paper presents an HTTP/2 push-based adaptation method for on-demand video streaming that reduces the number of requests and provides high video quality. In our method, for each client request, the server sends video segments continuously until it receives another client request with a new quality. Since a request is sent only if the client wants to update the video bitrate, our method can significantly reduce the request related overhead. For this context, a buffer based algorithm is proposed to provide high and smooth video quality while avoiding buffer underflows. Experiments show that the proposed method can provide a lower number of requests, higher average quality and better quality smoothness than existing methods.

Video streaming over HTTP/2 is a new trend in multimedia delivery. Compared to the pull-based HTTP/1.1 protocol, the new HTTP/2 protocol's Server Push feature is very effective in reducing the overheads (e.g., in terms of energy, processing, bandwidth) for clients, servers, and network nodes. This paper presents an HTTP/2 push-based adaptation method for on-demand video streaming that reduces the number of requests and provides high video quality. In our method, for each client request, the server sends video segments continuously until it receives another client request with a new quality. Since a request is sent only if the client wants to update the video bitrate, our method can significantly reduce the request related overhead. For this context, a buffer based algorithm is proposed to provide high and smooth video quality while avoiding buffer underflows. Experiments show that the proposed method can provide a lower number of requests, higher average quality and better quality smoothness than existing methods.

This paper describes the design and implementation of a multi-modal, multimedia capable sensor networking framework called SenseTK. SenseTK allows application writers to easily construct multi-modal, multimedia sensor networks that include both traditional scalar-based sensors as well as sensors capable of recording sound and video. The distinguishing features of such systems include the need to push application processing deep within the sensor network, the need to bridge extremely low power and low computation devices, and the need to distribute and manage such systems. This paper describes the design and implementation of SenseTK and provides several diverse examples to show the flexibility and unique aspects of SenseTK. Finally, we experimentally measure several aspects of SenseTK.

C'est avec un grand plaisir que je réserve ces lignes en témoignage de ma reconnaissance à tous ceux qui m'ont accompagné, soutenu et aidé pour accomplir les travaux présentés dans ce manuscrit. J'exprime mes profonds remerciements à M. Julien Bourgeois, mon directeur de thèse, qui a bien accepté de superviser mes travaux. Ensuite, à M. Eugen Dedu, qui a encadré mes travaux pendant cette thèse. Je tiens à lui exprimer ma gratitude pour les pistes de réflexion qu'il m'a fait prendre. J'adresse aussi mes remerciements à M. Congduc Pham et à M. Toufik Ahmed pour m'avoir fait l'honneur d'accepter d'être rapporteurs de cette thèse. Que M. Pascal Lorenz soit également remercié pour avoir accepté d'être membre de mon jury. Je suis très reconnaissant envers M. Jean-Laurent Hippolyte et Mlle Chloé Balzinge qui m'ont aidé à la relecture de ce document. Je ne manquerais pas de remercier l'ensemble des doctorants à Montbéliard Wahabou, Soumaya, Bogdan, Matteo et mes collègues de bureau Mouhannad et Raheel. J'exprime aussi ma gratitude à l'ensemble des membres de l'équipe OMNI (Optimisation, Mobility, Networking) de Montbéliard. Enfin, je tiens aussi à remercier toute ma famille en Syrie pour leur soutien inconditionnel, et surtout mon père et ma mère qui n'ont jamais cessé de m'encourager de si loin. À tous, mon infinie gratitude. .. notamment la technologie Wi-Fi qui permet à un dispositif (par exemple un ordinateur, une console de jeux, un smartphone) compatible de se connecter à Internet par le moyen d'un point d'accès sans avoir besoin d'y être relié par un câble (connexion dite sans-fil). Le Wi-Fi permet donc une très grande liberté de mouvement par rapport à une connexion filaire. Mais cette liberté est limitée car la portée du signal entre le mobile et le point d'accès est de l'ordre d'une dizaine ou d'une centaine de mètres. D'autres solutions s'appuyant sur les technologies de communication mobile comme par exemple la 3G ont fait leur apparition afin de pallier à cette limitation de portée et offrir aux utilisateurs une liberté de mouvement supérieure. Ainsi, un utilisateur connecté en 3G a la possibilité d'accéder au contenu d'Internet et d'être joignable n'importe où. Avec cette évolution vers une plus grande liberté de mouvement/mobilité, presque toute personne a la possibilité d'être connectée à Internet par le biais d'une de ces technologies. De plus, Le premier volet traite le problème des pertes dues aux erreurs du canal sans-fil qui ne sont pas traitées convenablement. En effet, comme nous l'avons déjà mentionné, les protocoles de transport considèrent toute perte de données comme due à la congestion alors que dans les réseaux sans-fil elles ne peuvent avoir d'autres causes. Nous proposons dans cette thèse une méthode permettant de différencier les pertes de congestion des pertes sans-fil et de les traiter de manière adéquate. Cette méthode est appelée RELD (RTT ECN Loss Differentiation). RELD est basé sur le RTT (Round Trip Time) et la signalisation ECN (Explicit Congestion Notification) afin de prendre la décision de différenciation. Le deuxième volet s'intéresse à la transmission de la vidéo en continu de manière adaptative. Pour le streaming vidéo sur un réseau qui a un débit variable, l'adaptation de contenu est un Plan du document Cette thèse est structurée en deux parties traitant les deux volets de nos contributions :-amélioration des protocoles de transport sur les réseaux sans-fil par la différenciation de pertes : cette partie est composée de trois chapitres. Le premier chapitre 4 présente l'état de l'art sur les méthodes utilisées pour différencier les pertes de congestion des pertes sans-fil. Le chapitre suivant 5 montre l'impact des pertes de congestion et sans-fil sur le RTT, ce qui aide à trouver une formule pour les différencier ; il présente également RELD comme une nouvelle méthode de différenciation de pertes. Le troisième chapitre 6 introduit l'environnement de simulation, en particulier le modèle d'erreur sans-fil utilisé et évalue les performances de RELD par la simulation.-amélioration du transfert du contenu multimédia par adaptation de la vidéo au niveau de la couche application : cette partie est composée de cinq chapitres. Le premier chapitre 8 présente des travaux antérieurs sur les méthodes d'adaptation de la vidéo. Le deuxième chapitre 9 présente d'abord un aperçu général des solutions possibles afin d'effectuer une adaptation de la vidéo. VAAL ainsi que son implémentation sur GNU/Linux sont expliqués en détail par la suite dans la section 9.3. Le chapitre 10 commence par introduire la plateforme de réseau réel utilisée pour effectuer les expérimentations, le modèle de limitation du trafic employé et les différents scénarios réalisés. Les performances de VAAL sont aussi évaluées dans la section suivante 10.2 qui, premièrement, montre l'adaptation réalisée en utilisant VAAL, et deuxièmement compare VAAL à une transmission classique avec des bitrates statiques sans adaptation. Le shadowing-pattern sera utilisé dans les chapitres 5 et 6 comme modèle de propagation sur le réseau sans-fil. Deuxième partie Amélioration des protocoles de transport sur les réseaux sans-fil par la différenciation de pertes Malgré cette prise en compte des caractéristiques des liaisons radio par les concepteurs des technologies sans-fil, les performances des protocoles de transport sur réseaux sans-fil sont tou

Cette thèse traite de l’amélioration du transfert de données, d’une part sur les réseaux sans-fils et d’autre part pour des données continues telles que la vidéo. Pour améliorer les transmissions sur les réseaux sans-fils nous nous sommes intéressés au contrôle de congestion des protocoles de transport mais nous avons également proposé une méthode pratique d’adaptation de la vidéo aux conditions du réseau.Cette thèse contient donc deux volets. La première porte sur la différenciation de pertes entre les pertes de congestion et les pertes sur le réseau sans fil. Il est connu que lors d’une perte, les protocoles de transport actuels réduisent le débit (par deux par exemple). Or, pour les pertes sans fil, cela n’a pas d’intérêt. Pour différencier ces pertes sur l’émetteur des données, nous proposons une méthode originale qui utilise à la fois ECN (Explicit Congestion Notification) et le changement sur le RTT du paquet qui suit la perte. La seconde propose une méthode originale d’adapta...

We describe the ADIMUS architecture which addresses the problem of maintaining the subjective quality of multimedia streaming for a mobile user. In contrast to other works, the entire end-to-end path of the video stream is considered. Adaptation mechanisms for maintaining quality include time-critical handovers, overlay routing and network estimation techniques. Our architecture is built on overlays that provides the necessary functionality for a video streaming service. The paper highlights the key components that ADIMUS advocates to support quality streaming from server to mobile client. 1) The work described in this paper has been conducted as a part of the ADIMUS (Adaptive Internet Multimedia Streaming) project which is funded by the NORDUnet-3 programme.

In this paper, we present an integrated system that has been designed to support automatic semantic extraction of highlights in sports video and automatic video adaptation according to user's preferences. To analyze the user's satisfaction, we propose a new performance measure that explicitly takes into account the user's preferences and considers the number and type of errors produced by the annotation engine and the way in which these errors affect the compressed video quality and bandwidth allocation. We provide experimental results with application to soccer and swimming.

In this paper we present a semi-automatic multi resolution video objects extraction and tracking algorithm well suited to scalable wavelet based object coding. Objects of interest are determined in the first frame through initial user intervention followed by a spatial segmentation algorithm. The specified objects are afterwards tracked in the subsequent frames. The tracking algorithm includes Multiresolution Markov Random Field (MMRF) based spatial segmentation with emphasis on border smoothness in different resolutions and multi resolution backward partition projection. An intensity change detector indicates newly appeared objects/regions. The proposed method produces well defined and visually pleasing objects as well as allowing for larger motion tracking, better noise tolerance and less computational complexity.

The newly ratified video compression standard H.264/AVC provides tools and solutions for coding improvements of at least 2 dB, in terms of PSNR, and at least 40% in bit rate saving, as compared with the previous standards. The development of video transcoding, which is one of the critical components for multimedia adaptation and Universal Multimedia Access, can gain benefits by using the tools and solutions from the standard. In this paper, the new features of H.264/AVC are first briefly reviewed. The issues involved in video transcoding are addressed together with a general framework for video transcoding. Here we utilize an architecture of pixel domain video transcoding derived directly from H.264/AVC.

We present a novel approach to optimally retarget videos for varied displays with differing aspect ratios by preserving salient scene content discovered via eye tracking. Our algorithm performs editing with cut, pan and zoom operations by optimizing the path of a cropping window within the original video while seeking to (i) preserve salient regions, and (ii) adhere to the principles of cinematography. Our approach is (a) content agnostic as the same methodology is employed to re-edit a wide-angle video recording or a close-up movie sequence captured with a static or moving camera, and (b) independent of video length and can in principle re-edit an entire movie in one shot. Our algorithm consists of two steps. The first step employs gaze transition cues to detect time stamps where new cuts are to be introduced in the original video via dynamic programming. A subsequent step optimizes the cropping window path (to create pan and zoom effects), while accounting for the original and new cuts. The cropping window path is designed to include maximum gaze information, and is composed of piecewise constant, linear and parabolic segments. It is obtained via L(1) regularized convex optimization which ensures a smooth viewing experience. We test our approach on a wide variety of videos and demonstrate significant improvement over the state-of-the-art, both in terms of computational complexity and qualitative aspects. A study performed with 16 users confirms that our approach results in a superior viewing experience as compared to gaze driven re-editing [JSSH15] and letterboxing methods, especially for wide-angle static camera recordings.

The presentation of multimedia data and especially of high resolution videos on small mobile devices is still a great challenge today. Both cropping of borders and scaling of frames may result in the removal of essential content of videos or lost details due to the reduced size of the visual content. Another major problem emerges if the aspect ratio of the original video and the display of the mobile device differ. User evaluations indicate that changing the aspect ratio may reduce the visual quality of videos significantly. In this paper, we present the new FSCAV algorithm (Fast Seam Carving for Size Adaptation of Videos) to adapt the size of videos to the limited display resolution and different aspect ratios of handheld mobile devices. The general idea of the seam carving algorithm for still images is to remove seams in images so that the essential content is preserved. We extended this technique which works very well for images to create videos without jitter or visible artifacts. A major feature of our FSCAV algorithm is the low computational complexity which enables an efficient adaptation of videos to small screens. Nevertheless, severe distortions are clearly visible in some shots of the adapted videos. We present a new heuristic to identify shots with such a low visual quality. If the quality drops below a threshold, a different adaptation technique is used for this shot (e.g., scaling or cropping). User evaluations confirm a very high visual quality of our approach.

Adaptive HTTP streaming has become a new trend to support adaptivity in video delivery. An HTTP streaming client needs to estimate exactly resource availability and resource demand. In this paper, we focus on the most important resource which is bandwidth. A new and general formulation for throughput estimation is presented taking into account previous values of instant throughput and round trip time. Besides, we introduce for the first time the use of bitrate estimation in HTTP streaming. The experiments show that our approach can effectively cope with drastic changes in connection throughput and video bitrate.

Recently, scalable video coding (SVC) has emerged as a promising video format for applications of multimedia communication. An SVC bitstream can be easily truncated (adapted) in spatial, temporal, and SNR dimensions to meet various characteristics of terminals and connections. Though SVC allows simple and flexible truncation of a coded bitstream, there still need to be efficient means to support the adaptation in different scenarios. In this paper, we review standard tools and supporting methods for SVC adaptation. We point out the practical questions and accordingly describe potential solutions to these questions. The paper also discusses two key standard tools in detail, namely MPEG-21 AdaptationQoS and SVC priority_id.