Search

The popularity of platforms for storing and transmitting video content has created a substantial volume of video data. Given a set of actors present in a video, generating metadata with the temporal determination of the interval in which each actor is present and their spatial 2D localization in each frame in these intervals can facilitate video retrieval and recommendation. In this work, we investigate Video Face Clustering for this spatio-temporal localization of actors in videos. We first describe our method for Video Face Clustering in which we take advantage of face detection, embeddings, and clustering methods to group similar faces of actors in different frames and provide the spatio-temporal localization of them. Then, we explore, propose, and investigate innovative applications of this spatio-temporal localization in three different tasks: (i) Video Face Recognition, (ii) Educational Video Recommendation and (iii) Subtitles Positioning in 360-video.

Recently the Brazilian DTV system standards have been upgraded, called TV 2.5, in order to provide a better integration between broadcast and broadband services. The next Brazilian DTV system evolution, called TV 3.0, will address more deeply this convergence of TV systems not only at low-level network layers but also at the application layer. One of the new features to be addressed by this future application layer is the use of Artificial Intelligence technologies. Recently, there have been practical applications using Artificial Intelligence (AI) deployed to improve TV production efficiency and correlated cost reduction. The success in operationalize and evaluate these applications is a strong indication of the interest and relevance of AI in TV. This paper presents TeleMídia Lab’s future vision on interactive and intelligent TV Systems, with particular focus on edge AI. Edge AI means use in-device capabilities to run AI applications instead of running them in cloud.

A learning object (LO) is an entity, digital or not, that can be used and reused or referenced during a technological support process for teaching and learning. Despite mainly being multimedia, with audio, video, text and images synchronized with each other, LOs can help disseminate knowledge even only in educational texts. However, creating these texts can be costly in time and effort, creating the need to seek new ways to generate this content. This article presents a solution for the generation of text-based LOs generated through summaries supported by Deep Learning models. The present work was evaluated in a supervised experiment in which volunteers rate computer educational texts generated by three types of summarizers. The results presented are positive and allow us to compare the performance of summaries as LO generators in text format. The findings also suggest that using post-processing in the output of models can improve the readability of generated content.

Ad hoc teamwork is a research topic in multi-agent systems whereby an agent (the "ad hoc agent") must successfully collaborate with a set of unknown agents (the "teammates") without any prior coordination or communication protocol. However, research in ad hoc teamwork is predominantly focused on agent-only teams, but not in agent-human teams, which we believe is an exciting research avenue and has enormous application potential in human-robot teams. This paper will tap into this potential by proposing HOTSPOT, the first framework for ad hoc teamwork in human-robot teams. Our framework comprises two main modules, addressing the two key challenges in the interaction between a robot acting as the ad hoc agent and human teammates. First, a decision-theoretic module that is responsible for all task-related decision making (task identification, teammate identification, and planning). Second, a communication module that uses natural language processing in order to parse all communication between the robot and the human. To evaluate our framework, we use a task where a mobile robot and a human cooperatively collect objects in an open space, illustrating the main features of our framework in a real-world task.

Audio quality degradation can have many causes. For musical applications, this fragmentation may lead to highly unpleasant experiences. Restoration algorithms may be employed to reconstruct missing parts of the audio in a similar way as for image reconstruction --- in an approach called audio inpainting. Current state-of-the art methods for audio inpainting cover limited scenarios, with well-defined gap windows and little variety of musical genres. In this work, we propose a Deep-Learning-based (DL-based) method for audio inpainting accompanied by a dataset with random fragmentation conditions that approximate real impairment situations. The dataset was collected using tracks from different music genres to provide a good signal variability. Our best model improved the quality of all musical genres, obtaining an average of 12.9 dB of PSNR, although it worked better for musical genres in which acoustic instruments are predominant.

A crucial task to overall video understanding is the recognition and localisation in time of different actions or events that are present along the scenes. To address this problem, action segmentation must be achieved. Action segmentation consists of temporally segmenting a video by labeling each frame with a specific action. In this work, we propose a novel action segmentation method that requires no prior video analysis and no annotated data. Our method involves extracting spatio-temporal features from videos in samples of 0.5s using a pre-trained deep network. Data is then transformed using a positional encoder and finally a clustering algorithm is applied with the use of a silhouette score to find the optimal number of clusters where each cluster presumably corresponds to a different single and distinguishable action. In experiments, we show that our method produces competitive results on Breakfast and Inria Instructional Videos dataset benchmarks.

NCL is the declarative programming language used to develop TV applications in IPTV systems and Terrestrial TV standardized by ITU and Brazilian TV Forum, respectively. Its main characteristics are: defining temporal synchronization among media assets and viewer interactions; layout reuse facilities; support multi-device presentation; support embed HTML code and scripts in the lightweight scripting language Lua; and an API for life-cycle controls (start, pause, resume, stop) and modifying applications on-the-fly called NCL editing command. This talk briefly introduces NCL, highlights its recent advances, and discuss the future of the language.

Discovering and accessing specific content within educational video bases is a challenging task, mainly because of the abundance of video content and its diversity. Recommender systems are often used to enhance the ability to find and select content. But, recommendation mechanisms, especially those based on textual information, exhibit some limitations, such as being error-prone to manually created keywords or due to imprecise speech recognition. This paper presents a method for generating educational video recommendations using deep face-features of lecturers without identifying them. More precisely, we use an unsupervised face clustering mechanism to create relations among the videos based on the lecturer's presence. Then, for a selected educational video taken as a reference, we recommend the ones where the presence of the same lecturers is detected. Moreover, we rank these recommended videos based on the amount of time the referenced lecturers were present. For this task, we achieved an mAP value of 99.165%.

Instant messaging apps have billions of monthly active users, which made this environment a good medium for purposes other than communication, such as education. In this context, one of the most appropriate technologies to encourage learning in these applications is chatbots. They consist of conversational applications that make use of artificial intelligence and natural language processing to interact with the user in a similar way to a human. In this scenario, this article presents the AGATE a chatbot developed for the Environmental Management Office (AGA / UEMA) in order to disseminate knowledge on environmental education, focusing on the problems of water and energy waste. The chatbot was developed for Telegram through the pyTelegramBotAPI implementation. In addition, conversation flows were created using DialogFlow. A multidisciplinary team composed of 14 people participated in the evaluation of the project, with a greater participation of professionals in the field of Biology. The initial results focused on the user experience were satisfactory.

People with visual impairments suffer from the incapacity to understand contextual information in videos, such as the place where characters are, or any other non-spoken actions in general. Some content creators address this issue by providing a secondary audio to describe such information, called Audio Descriptions (ADs). How- ever, some works in the literature have highlighted that people with visual impairment are usually not able to completely understand scene changes based only on characters’ voices or traditional ADs. Moreover, traditional ADs do not completely describe some of the important visual information, such as the background scenery (e.g. colors, furniture) and characters’ details (e.g. blond woman using a red dress). In this work, we propose incrementing the traditional AD techniques with the usage of interactive video features present in TV systems. More precisely, the proposed interactivity enables users to access specialized AD for different visual information (e.g., scene, scenario, character). To support the development of such interactive content, we present an application template, which helps to create the final interactive-enhanced video application. Asa proof of concept for our approach, we created an interactive AD for an independent video mainly composed of visual information, with only a few talks.

Face recognition systems are present in many modern solutions and thousands of applications in our daily lives. However, current solutions are not easily scalable, especially when it comes to the addition of new targeted people. We propose a cluster-matching-based approach for face recognition in video. In our approach, we use unsupervised learning to cluster the faces present in both the dataset and targeted videos selected for face recognition. Moreover, we design a cluster matching heuristic to associate clusters in both sets that is also capable of identifying when a face belongs to a non-registered person. Our method has achieved a recall of 99.435% and a precision of 99.131% in the task of video face recognition. Besides performing face recognition, it can also be used to determine the video segments where each person is present., Comment: 13 pages

Methods based on Machine Learning have become state-of-the-art in various segments of computing, especially in the fields of computer vision, speech recognition, and natural language processing. Such methods, however, generally work best when applied to specific tasks in specific domains where large training datasets are available. This paper presents an overview of the state-of-the-art in the area of Deep Learning for Multimedia Content Analysis (image, audio, and video), and describe recent works that propose The integration of deep learning with symbolic AI reasoning. We draw a picture of the future by discussing envisaged use cases that address media understanding gaps which can be solved by the integration of machine learning and symbolic AI, the so-called Neuro-Symbolic integration.

Before the COVID-19 pandemic, video was already one of the main media used on the internet. During the pandemic, video conferencing services became even more important, coming to be one of the main instruments to enable most social and professional human activities. Given the social distancing policies, people are spending more time using these online services for working, learning, and also for leisure activities. Videoconferencing software became the standard communication for home-office and remote learning. Nevertheless, there are still a lot of issues to be addressed on these platforms, and many different aspects to be reexamined or investigated, such as ethical and user-experience issues, just to name a few. We argue that many of the current state-of-the-art techniques of Artificial Intelligence (AI) may help on enhancing video collabo- ration services, particularly the methods based on Deep Learning such as face and sentiment analyses, and video classification. In this paper, we present a future vision about how AI techniques may contribute to this upcoming videoconferencing-age.

Conversational user interfaces have been an increasingly popular way to obtain quick information in natural language, especially with advances in the area of natural language processing (NLP) fueled by the surge of deep neural networks. Real-world cases include IBM Watson and Azure Microsoft chatbots, among others. Such examples, however, are most present in the commercial spectrum, with few being actively used by the scientific community. In this work, we develop an Interactive Question Answering System (IQA) for the Sucupira platform, Brazil's biggest open platform of postgraduate content, deployed as conversational interfaces for two popular messaging platforms: Telegram and Discord. We also propose the use of multilingual embeddings for ordering answers by a language-independent measuer of semantic similarity. Our IQA was evaluated with an study with 16 participants. The results were mostly positive, indicating that the use of conversational user interfaces, along with NLP techniques, may be a valid alternative for retrieving scientific information from the Sucupira plataform.

In 2020, COVID-19 pandemic is one of the most talked-about subjects on social networks. This subject has generated discussions of great importance about politics, economics, medical advances, people’s awareness, preventive techniques, etc. Using sentiment analysis and topic modeling techniques, in this paper, we aim to present an analysis of the messages from the social network Twitter during the pandemic of COVID-19. For this, we use a tweets dataset to train a sentiment classifier and then use the NMF algorithm to perform the interest topic generation.

Many recent works have successfully applied some types of Convolutional Neural Networks (CNNs) to reduce the noticeable distortion resulting from the lossy JPEG compression technique. Most of them are built upon the processing made on the spatial domain. In this work, we propose a JPEG image decoder that is purely based on the frequency-to-frequency domain: it reads the quantized DCT coefficients received from a low-quality image bitstream and, using a deep learning-based model, predicts the missing coefficients in order to recompose the same image with enhanced quality. In experiments with two datasets, our best model was able to improve from images with quantized DCT coefficients corresponding to a Qualityz Factor (QF) of 10 to enhanced quality images with QF slightly higher than 20.

A large number of videos are uploaded on educational platforms every minute. Those platforms are responsible for any sensitive media uploaded by their users. An automated detection system to identify pornographic content could assist human workers by pre-selecting suspicious videos. In this paper, we propose a multimodal approach to adult content detection. We use two Deep Convolutional Neural Networks to extract high-level features from both image and audio sources of a video. Then, we concatenate those features and evaluate the performance of classifiers on a set of mixed educational and pornographic videos. We achieve an F1-score of 95.67% on the educational and adult videos set and an F1-score of 94% on our test subset for the pornographic class.

Recent works have successfully applied some types of Convolutional Neural Networks (CNNs) to reduce the noticeable distortion resulting from the lossy JPEG/MPEG compression technique. Most of them are built upon the processing made on the spatial domain. In this work, we propose a MPEG video decoder that is purely based on the frequency-to-frequency domain: it reads the quantized DCT coefficients received from a low-quality I-frames bitstream and, using a deep learning-based model, predicts the missing coefficients in order to recompose the same frames with enhanced quality. In experiments with a video dataset, our best model was able to improve from frames with quantized DCT coefficients corresponding to a Quality Factor (QF) of 10 to enhanced quality frames with QF slightly near to 20.

The recent availability of consumer-level head-mounted displays and omnidirectional cameras has been driving an explosion of 360 video content. Transforming the original recorded 360 video content in meaningful interactive multimedia presentations that support viewers in tasks such as learning, entertainment, and telepresence, however, is not trivial and requires new tools. Such tools must provide an easy-to-use authoring model for the integration of different media objects and active user interface elements.In this paper, based on real-world scenarios for interactive 360 videos, we gather a set of requirements and propose a declarative authoring model to support authors in the process of designing and creating 360-degree interactive video presentations. As a case study, we implement different applications showing the expressiveness and completeness of the model in the scope of the target scenarios.

The Nested Context Language (NCL) are standards for multimedia application development for Digital TV. Some studies have indicated that NCL language is highly verbose. This factor increases the possibility of coding errors introduced by application developers. One way to reduce them is development based on reuse repeated elements in the code. On multimedia field literature, is common to use templates to active such goal. Templates describe a family of logically structured documents. Its adoption ends up in a reduction in the number of lines of code typed and thus make the final document less error-prone. On the web, templates are commonly employed in the development of Hypertext Markup Language (HTML) pages. In this scenario, developers use specific templates engines that can even run on the client-side, such as Jinja2 and Mustache. This work proposes provide an approach to support web template engines in the NCL applications development. More than that, it allows templates to be processed on the client-side, i.e. Ginga. By running on Ginga, developers can create applications with adaptable template-based content.

Anime character stickers consist of a detailed illustration of characters that represents emotions. Generally, message apps let users save received stickers, but in some cases, the task of manual searching for a specific sticker may be frustrating when a large amount of them are stored. In this work, we propose a CNN-based tool for emotion indexing of message stickers. We built a dataset with 12.668 labeled stickers with 3 classes (Sad, Happy and Angry). In experiments, our model achieves 84.01% of global f1-score. Additionally, we describe our proposed tool for emotion indexing of message stickers that has three main functionalities: (1) sticker recommendation function that classifies stickers and recommends the class desired; (2) filter function that takes out the stickers it considers mislabeled; (3) ranking function that orders the labeled stickers.

Vehicle’s license plate detection and recognition is a task with several practical applications. It can be applied, for example, in the security segment, identifying stolen cars and controlling cars entry/exit in private areas. This work presents a Deep Learning based tool that uses the cascaded YOLOv3 to simultaneously detect and recognize vehicle plate. In experiments performed, our tool got a recall of 95% in plate detection and 96.2% accuracy in the recognition of the 7 characters of the license plate.

Most of information technologies courses are focused on exposing the traditional static data (e.g., SGBD) processing model and lack on presenting the data stream model. Such model can be applied to fields such as Internet of Things, busyness and finances, logistics and smart cities) of the industry. Complex Event Processing consists of a programming approach to handle Data Stream Processing. It provide primitives to process and detect the occurrence of patterns in data streams. This short course has the objective of presenting the complex event processing (CEP) programming model as a means of dealing with the specificities of data flows.

The broad use of video capture and services for its storage and transmission has enabled the production of a massive volume of video data. This usage presents a challenge in controlling the type of content that is loaded for these video storage services. The Internet slang NSFW (Not Safe For Work) is often used as a warning for media that contain inappropriate content, such as nudity, intense sexuality, violence, gore or other potentially disturbing subject matter. Convolutional Neural Network (CNNs) architectures, or ConvNets, have become the primary method used for audio-visual pattern recognition. In this work, we intend to: (1) create a CNN based model for video features extraction; And (2), validate these video features with baselines models for NSFW video classification using a multi-modal approach. In initial experimentation, our best model achieves a recall of 96.6% for NSFW class.

Machine Learning field, methods based on Deep Learning (e.g. CNN, RNN) becomes the state-of-the-art in several problems of the multimedia domain, especially in audio-visual tasks. Typically, the training of Deep Learning Methods is done in a supervised manner, and it is trained on datasets containing thousands/millions of media examples and several related concepts/classes. During training, the Deep Learning Methods learn a hierarchy of filters that are applied to input data to classify/recognize the media content. In computer vision scenario, for example, given image pixels, the series of layers of the network can learn to extract visual features from it, the shallow layers can extract lower-level features (e.g. edges, corner, contours), while the deeper combine these features to produce higher-level features (e.g. textures, part of objects). These representative features can be clustered into groups, each one representing a specific concept. H.761 NCL currently lacks support for Deep Learning Methods inside their application specification. Because those languages still focus on presentations tasks such as capture, streaming, and presentation. They do not consider programmers to describe the semantic understanding of the used media and handle recognition of such under-standing. In this proposal, we aim at extending NCL to provide such support. More precisely, our proposal able NCL application support: (1) describe learning-based on structured multimedia datasets; (2) recognize content semantics of the media elements in presentation time. To achieve such goals, we propose, an extension that includes: (a) the new "knowledge" element describe concepts based on multimedia datasets; (b) "area" anchor with an associated "recognition" event that describes when a concept occurrences in multimedia content.

The H.761 standard defines that the NCL media object that is required in a Ginga-NCL implementation is the hypertext markup language (HTML)-based media object. Such HTML media object considerable use of memory and data transmission. Therefore, it is useful for devices with considerable memory and with a broadband connection. However, it is not recommended for scenarios with low bitrate and contrast memory, such as the digital radio device. In such cases, the NCL application used data should be minimum. For such scenarios, the use of graphic media, such as png and jpg, can consume considerable data. We propose that an SVG engine should also be required to support such cases.

Every NCL link relates two or more Node interfaces. When an NCL developer defi ne a link bind, however, it should always use BOTH component and interface attributes. This leads to two problems: (a) require defi ne two XML elements, which is tedious and error- prone; (b) require bad "Take It From There" workaround to access an interface from a Node participating in the link. We propose that component is optional. Then each interface can be accessed by an id which must be unique regarding the events of the same type in its container object. We propose a naming convention for identifying interfaces within a Document

Current consumer devices, such as TVs, smartphones, smartwatches,etc., often come with built-in support for radio-frequency (RF) wireless communication. This support requires RF hardware and de-ends on technologies such as Bluetooth, IEEE 802 WiFi, or 4Gmobile networks. There are situations, however, where the RF data network is not available, or where it is more advantageous to use some other form of wireless communication. One alternative communication through inaudible sound. The same consumer devices that come with RF support usually come equipped with speakers and microphones which enables them to produce and detect inaudible sound. By inaudible sound, we mean sound waves in frequencies that cannot be heard by humans—usually those above 19–22 kHz. Many works in the literature that show that communication through inaudible sound is feasible on consumer-grade PCs, laptops, and smartphones. Here we are mainly concerned with the applicability of this technique to enable inaudible sound applications in the Ginga-NCL engines. This way, we propose a new type of media object called SigGen.

There are many ways to build a person identification system and those systems can be used for authentication and security. The latest phones for example, bring fingerprint readers to enhance the user experience. From our perspective on Neural Networkswe determine that Machine Learning is enough to guarantee someone’s identity without the need of any specific sensors other than a camera and a microphone. It is achievable with pictures of their face, sounds of their voice and Deep Learning. This work presents a study to build an application to allow biometric authentication using only multimedia and Deep Learning.

Multimedia languages traditionally, they focus on synchronizing a multimedia presentation (based on media and time abstractions) and on supporting user interactions for a single user, usually limited to keyboard and mouse input. Recent advances in recognition technologies, however, have given rise to a new class of multimodal user interfaces (MUIs). In short, MUIs process two or more combined user input modalities (e.g. speech, pen, touch, gesture, gaze, and head and body movements) in a coordinated manner with output modalities . An individual input modality corresponds to a specific type of user-generated information captured by input devices (e.g. speech, pen) or sensors (e.g. motion sensor). An individual output modality corresponds to user-consumed information through stimuli captured by human senses. The computer system produces those stimuli through audiovisual or actuation devices (e.g. tactile feedback). In this proposal, we aim at extending the NCL multimedia language to take advantage of multimodal features.

Web developers commonly run HTML template engines at the client-side using JavaScript. One upper hand is to give more development flexibility comparing to the server-side. In this way, we may cite web frameworks webcomponents, ReactJS, Vue, and others. Those frameworks use the DOM API to access the template data inserted in the HTML document. Then the same DOM API is used to insert more HTML code. Currently, NCL not support access to the current document. But it can modify the document by NCL Editing commands in NCLua scripts. We propose NCLua scripts partially support the DOM Code API.

Methods based on Deep Learning became state-of-the-art in several Multimedia challenges. However, there is a gap of professionals to perform Deep Learning in the industry. Therefore, this tutorial aims to present the grounds and ways to develop applications that uses methods based on Deep Learning for video analysis tasks. Likewise, this tutorial is an opportunity for students and information technology professionals to qualify themselves. The main focus of this short course is to present fundamentals and technologies to develop such methods of DL.

An ergonomic evaluation is an observation of a person in order to identify musculoskeletal disorders (WMSDs) caused by prolonged or repeated harmful poses that a person adopts during work tasks. Nowadays, an ergonomist or other health professional perform such evaluations based on a set of posture rules and checklists, which can be subjective and thus lead to erroneous risk classifications. Moreover, this professional usually perform such evaluation in the patient work environment. In order to make those evaluations more objective and concise we propose a evaluation method using a mobile depth sensor. Different from other methods based in fixed depth sensors (e.g. Kinect), our method enable professionals easily perform it in the patient work environment. More precisely, we present an experiment that uses a smartphone from Google's Tango project and the Ovako Working Posture Analysing System (OWAS) method. To evaluate our approach, we also perform the ergonomic assessment using the Kinect sensor, a device that has a good reliability in the automated ergonomic evaluation. Both evaluations involved a set of 34 poses performed by 3 volunteers and annotated by an ergonomist. The Kinect has achieved accuracy of 57,08% on torso classification, 58,33% for arms and 25,00% for legs positions. While the approach using the mobile depth sensor has achieved 35,41% on torso classification, 93,05% for arms and 66,23% for legs positions on the same set of poses. Although the small sample, the achieved results may indicate that our mobile depth sensor approach can be as viable as methods based fixed depth sensor.

Recent advances in hardware and software technologies have given rise to a new class of human-computer interfaces that both explores multiple modalities and allows for multiple collaborating users. When compared to the development of traditional single-user WIMP (windows, icons, menus, pointer)-based applications, however, applications supporting the seamless integration of multimodal-multiuser interactions bring new specification and runtime requirements. With the aim of assisting the specification of multimedia applications that integrate multimodal-multiuser interactions, this paper: (1) proposes the MMAM (Multimodal-Multiuser Authoring Model); (2) presents three different instantiations of it (in NCL, HTML, and a block-based syntax); and (3) evaluates the proposed model through a task-based user study. MMAM enables programmers to design and ponder different solutions for applications with multimodal-multiuser requirements. The proposed instantiations served as proofs of concept about the feasibility of our model implementation and provided the basis for practical experimentation, while the performed user study focused on capturing evidence of both the user understanding and the user acceptance of the proposed model. We asked developers to perform tasks using MMAM and then answer a TAM (Technology Acceptance Model)-based questionnaire focused on both the model and its instances. As results, the study indicates that the participants easily understood the model (most of them performed the required tasks with minor or no errors) and found it both useful and easy to use. 94.47% of the participants gave positive answers to the block-based representation TAM questions, whereas 75.17% gave positive answers to the instances-related questions.

Methods based on Deep Learning became state-of-the-art in several Multimedia challenges. However, there is a gap of professionals to perform Deep Learning in the industry. Therefore, this short course aims to present the grounds and ways to develop multimedia applications using methods based on Deep Learning. Likewise, this short course is an opportunity for students and IT professionals can qualify yourselves.

In this paper, we report our efforts to add support for data transmission through inaudible sound to the Ginga-NCL Digital TV middleware. We present an algorithm for encoding a bitstream in an inaudible audio signal, and to do so reliably on consumer-grade hardware. We also discuss two attempts to implement this algorithm in NCL, the language in which Ginga-NCL applications are written. The first attempt was to transmit prerecorded inaudible audio signals in a Ginga-NCL-compatible set-top-box. And the second attempt was to use NCLua to generate at runtime the inaudible audio signal. For the second attempt we extended NCL with a novel media object type, called SigGen, which can be used to generate arbitrary audio signals. In the paper, we describe in detail the implementation of SigGen and the result of these attempts.

We describe how we modified an NCL player to accept as input, in addition to NCL documents, Lua scripts. These Lua scripts evaluate to a table in a particular format, called NCL-ltab, which is a Lua table encoding of the NCL player's internal model. One advantage of our modifications is that they allow the NCL parsing to occur in the Lua script, i.e., outside the formatter but integrated in its execution flow. The same applies for parsers of dialects of NCL or similar languages. Another advantage is that new parsers can be plugged dynamically into the formatter (if they are written in Lua or can be called from Lua). In this paper, we detail the internal model of the NCL player we are using and the NCL-ltab input format. To evaluate our proposal, we present two parser-integration use cases, one for NCL itself, using the DietNCL parsing toolkit, and another for sNCL, an alternative, user-friendlier syntax for NCL.

Deep Learning research has allowed significant advancement of various segments of multimedia, especially in tasks related to speech processing, hearing and computational vision. However, some video services are still focused only on the traditional use of media (capture, storage, transmission and presentation). In this paper, we discuss our ongoing research towards a DLaS, i.e. Deep Learning as a Service. This way, we present the state of art in video classification and recognition. Then we propose the VideoRecognition as DLaS to support the tasks such as: image classification and video scenes, object detection and facial recognition. We discuss the usage of the proposed service in the context of the video@RNP repository. Our main contributions consist on dicussussions over the state of art and it usage in nowdays multimedia services.

The usage of technologies for content authoring, storage, distribution and presentation supported the arise of Virtual Learning Environments (VLEs) such as Moodle and MOOCs (e.g., Udacity, Coursera, EdX). All students’ interaction in these environments can be stored in logs. For instance, video is one of the resources has a greater focus on VLEs. In this paper, we present a state-of-the-art overview of interaction analysis on video players. We noticed that teachers would like to know how and when students interact with videos. In order to fill this gap, we propose VideoViz, a tool to analyse logs from interaction on video players.

In this paper, we present the conversion of NCL to Smix and discuss its main implications. NCL is a declarative language for the specification of interactive multimedia presentations which was adopted by the ITU-T H.761 recommendation for interoperable IPTV services. Smix is a synchronous domain-specific language with a similar purpose, but with a simpler and more precise semantics. By implementing NCL over Smix, we bring to the former the notions of reaction and execution instants, and with them some benefits. From a practical perspective, we fix the semantics of the converted documents, get a leaner NCL player (the Smix interpreter), and simplify further conversions. From a systems-design perspective, the structured conversion of NCL to Smix helps us tame the complexity of mapping the user-oriented constructs of NCL into the machine-oriented primitives that realize them as a multimedia presentation. In the paper, we present NCL and Smix, discuss related work on document conversion, and detail the conversion process and a prototype implementation.

Nowadays, tablets and smartphones are commonly used by children for both entertainment and education purposes. In special, interactive multimedia eBooks running on those devices allow a richer experience when compared to traditional text-only books, being potentially more engaging and entertaining to readers. However, to explore the most exciting features in these environments, authors are currently left alone in the sense that there is no high level (less technical) support, and these features are usually accessible only through programming or some other technical skill. In this work, we aim at extracting the main features on enhanced children's eBooks and propose a model, named Fabulas - the Portuguese word for fables -that allows authors to create interactive multimedia children's eBooks declaratively. The model was conceived by taking, as a starting point, a systematic analysis of the common concepts, with the focus on identifying and categorizing recurring characteristics and pointing out functional and non-functional requirements that establish a strong orientation towards the set of desirable abstractions of an underlying model. Moreover, the paper presents a case study for the implementation of Fabulas on the Web, and discusses the authoring of a complete interactive story over it.

Learning Objects (LOs) are entities that can be used, reused, or referred during the teaching process. They are commonly embedded into documents that establish spatial and temporal relationships on their contents. Hypervideos LOs allow students to individualize their learning experience with non-linear browsing mechanisms and content adaptation. This paper presents a survey of features for a set of documents representing such LOs as well as desirable aspects that should be expressed during the authoring phase. Also, this paper presents a conceptual model that fits such requirements. The model is implemented by SceneSync, a domain specific language focused on the synchronization and temporal behavior of LOs. As a result of the work, we present a set of LOs specified in SceneSync and a discussion about the identified features, which confirm the expressiveness and applicability of the model.

Recent advances in technologies for speech, touch and gesture recognition have given rise to a new class of user interfaces that does not only explore multiple modalities but also allows for multiple interacting users. Even so, current declarative multimedia languages e.g. HTML, SMIL, and NCL?support only limited forms of user input (mainly keyboard and mouse) for a single user. In this paper, we aim at studying how the NCL multimedia language could take advantage of those new recognition technologies. To do so, we revisit the model behind NCL, named NCM (Nested Context Model), and extend it with first-class concepts supporting multiuser and multimodal features. To evaluate our approach, we instantiate the proposal and discuss some usage scenarios, developed as NCL applications with our extended features.

1 Abstract – The increasing demand for distributed multimedia applications makes evident the need for end-toend quality of service (QoS) provisioning. Particularly, operating systems, despite their location at end systems, switches or routers, must guarantee that resources under their control are adequately managed to fulfill the application requirements. This work proposes an architecture for adaptive QoS provisioning on network operating systems (QoSOS), focusing mainly on the packet queuing subsystem. The development of such architecture came after an analysis of solutions currently found in the literature and the perception of their functional similarities. QoSOS allows the reuse of common functions and the definition of an internal organization that is equivalent in different systems. In order to demonstrate how QoSOS can be applied in a real QoS provisioning scenario, the paper describes the modeling and implementation of an adaptable Intserv support, focusing on the management of the output queues of the Linux operating system. The architecture instantiation is based on few modifications introduced into the standard Linux kernel, that adds some desirable features such as runtime service adaptation.

Most hypermedia systems have been developed as a unique and self-contained application preventing information interchange and code reusability between applications. To address this problem, a generic layered architecture for hypermedia systems with four major interfaces is first introduced. The architecture adequately uses the MHEG proposal to provide the desired independence between hypermedia applications' conceptual models and storage strategies for hypermedia objects. Then, some design problems of the topmost layer, using as example the nested context hypermedia model, are discussed. Finally, motivated by the fact that a distributed environment may be necessary to meet the real time requirements of hypermedia applications, some issues concerning the design of a distributed system based on the proposed architecture are briefly addressed

This paper presents an abstract model for resource management mechanisms, called Virtual Resource Tree (VRT). The VRT model is developed aiming at meeting different end-to-end Quality of Service (QoS) requirements demanded by multimedia applications. Scalability plays an important role in this scenario, bringing new mandatory features for resource management, such as the distributed management of heterogeneous resources and the support for adapting the different underlying management algorithms. In addition, this paper discusses the main issues towards the definition of the VRT file system (VRT-FS). VRT-FS is a framework that shows how, with a simple and familiar design, the functionalities of the VRT model can be ported to network operating systems.