Search

Recently, IoT applications using Deep Neural Network (DNN) to embedded edge devices are increasing. Generally, in the case of DNN applications in the IoT system, training is mainly performed in the server and inference operation is performed on the edge device. The embedded edge devices still take a lot of loads in inference operations due to low computing resources, so proper customization of DNN with architectural exploration is required. However, there are few integrated frameworks to facilitate exploration and customization of various DNN models and their operations in embedded edge devices. In this paper, we propose an integrated framework that can explore and customize DNN inference operations of DNN models on embedded edge devices. The framework consists of the GUI interface part, the inference engine part, and the hardware Deep Learning Accelerator (DLA) Virtual Platform (VP) part. Specifically it focuses on Convolutional Neural Network (CNN), and provides integrated interoperability for convolutional neural network models and neural network customization techniques such as quantization and cross-inference functions. In addition, performance estimation is possible by providing hardware DLA VP for embedded edge devices. Those features are provided as web-based GUI interfaces, so users can easily utilize them.

Skin cancer is developed due to abnormal cell growth. These cells are grown rapidly and destroy the normal skin cells. However, it's curable at an initial stage to reduce the patient's mortality rate. In this article, the method is proposed for localization, segmentation and classification of the skin lesion at an early stage. The proposed method contains three phases. In phase I, different types of the skin lesion are localized using tinyYOLOv2 model in which open neural network (ONNX) and squeeze Net model are used as a backbone. The features are extracted from depthconcat7 layer of squeeze Net and passed as an input to the tinyYOLOv2. The propose model accurately localize the affected part of the skin. In Phase II, 13-layer 3D-semantic segmentation model (01 input, 04 convolutional, 03 batch-normalization, 03 ReLU, softmax and pixel classification) is used for segmentation. In the proposed segmentation model, pixel classification layer is used for computing the overlap region between the segmented and ground truth images. Later in Phase III, extract deep features using ResNet-18 model and optimized features are selected using ant colony optimization (ACO) method. The optimized features vector is passed to the classifiers such as optimized (O)-SVM and O-NB. The proposed method is evaluated on the top MICCAI ISIC challenging 2017, 2018 and 2019 datasets. The proposed method accurately localized, segmented and classified the skin lesion at an early stage. Qatar University [IRCC-2020-009].

Featured Application: System and method for continuously improving or adapting multi-access router path selection of autonomous moving vehicles in changing environments. Autonomous moving vehicles facilitate mining of ore in underground mines. The vehicles are usually equipped with many sensor-based devices (e.g., Lidar, video camera, proximity sensor, etc.), which enable environmental monitoring, and remote control of the vehicles at the control center. Transfer of sensor-based data from the vehicles towards the control center is challenging due to limited connectivity enabled by the multi-access technologies of the communication infrastructure (e.g., 5G, Wi-Fi) within the underground mine, and the mobility of the vehicles. This paper presents design, development, and evaluation of a concept and architecture enabling continuous machine learning (ML) for optimizing route selection of real-time streaming data in a real and emulated underground mining environment. Continuous ML refers to training and inference based on the most recently available data. Experiments in the emulator indicated that utilization of a ML-based model (based on the RandomForestRegressor) in decision making achieved ~5–13% lower one-way delay in streaming data transfers, when compared to a simpler heuristic model.

Cilj ovog rada je napraviti aplikaciju koju će korisnik koristiti kao pomoćni alat koji mu olakšava i ubrzava proces "oluje pisanja" (eng. Brainwriting) tako da automatizira pretragu web stranica s relevantnim informacijama, te integrira i prikazuje relevantne podatke na jednostavan način. Aplikacija se sastoji iz tri dijela koja čine cjelinu. Prvi dio je web stranica koja služi kao korisničko sučelje preko kojega korisnik može u slobodnoj formi unijeti zapise "oluje pisanja", te kasnije vidjeti rezultate pretrage sustava. Web stranica isto tako omogućuje korisniku da unosi svoje slike u sustava, te ih kasnije pregledava. Drugi dio aplikacije je model strojnog učenja koji na temelju zapisa "oluje pisanja", koju je unio korisnik, radi predviđanje filmskog žanra koji bi najbolje odgovarao tom zapisu. Treći dio sustava se odnosi na programe (botove) koji na temelju dobivenog filmskog žanra pretražuju relevantne izvore podataka na internetu kako bi prinašao pet najpopularnijih filmova za taj žanr. Nakon toga botovi pronalaze i dohvaćaju slike, citate i sažetak radnje za svaki od tih finova. Goal of this paper was to make web application that is used as a tool that will help speed up and automate process of acquiring relevant image templates based on user brainwriting input. After searching for images, quotes and plot summaries on relevant websites, our application integrates all this data and displays it in user friendly way. This application has three main components that work together as a whole. First component is website that enables users to enter their brainwriting input trough user interface and later displays search results in form of images and text. Second part of application is machine learning model that takes in as input brainwriting text and returns film genre that best fits users input. Third part of application is made of internet bots that, based on film genre, simulate (automate) users search the internet for five most popular films for given genre and then extract relevant data about them.

Exercising the body is a fundamental part of health. There are many re-sources freely available, such as audio-visual content on the internet, simplynature itself or outdoor gym structures in public parks. Using these resources,it has been proposed to implement a mobile application to encourage physicalactivity, using augmented reality to engage the user, providing a di?erent andmotivating experience that encourages them to explore and try new forms ofexercise. This thesis starts with the implementation of a mobile applicationthat detects and tracks outdoor gym structures, training neural networks tosegment and obtain the pose of the structures and project exercise sugges-tions using augmented reality. However, during the project we discover theunfeasibility of such an objective due to the fact that the platform used todevelop the project, Unity, is still in the process of evolving with respect todeep learning technologies.In these circumstances, this thesis documents the di?erent steps that havebeen taken, analyses the unfeasibility of the project and proposes a study toimprove the pose estimator. Ejercitar el cuerpo es una parte fundamental de la salud. Hay varios recursos gratuitos disponibles, como contenido audiovisual en internet, la naturaleza misma o estructuras de gimnasios exteriores en parques públicos. Utilizando estos recursos, se ha propuesto implementar una aplicación móvil que anima a cumplir actividad física, utilizando realidad aumentada para atraer al usuario, proporcionando una diferente y motivadora experiencia que anima a explorar y probar nuevas formas de ejercicio. Esta tesis comienza con una implementación de aplicación móvil que detecta y rastrea estructuras exteriores de gimnasios, entrena redes neuronales para segmentar y obtener la posición de las estructuras y proyectar sugerencias de ejercicios utilizando realidad aumentada. Aun así, durante el proyecto se ha descubierto que el objetivo es inviable, ya que la plataforma utilizada para desarrollar el proyecto, Unity, está aún en proceso de evolucionar respecto tecnologías de aprendizaje profundo. En estas circunstancias, la tesis documenta los diferentes pasos que se han hecho, analiza la inviabilidad del proyecto y propone un estudio para mejorar el pose estimator. Exercitar el cos és una part fonamental de la salut. Hi ha diversos recursos gratuïts disponibles, com contingut audiovisual a internet, la natura mateixa o estructures de gimnasos exteriors en parcs públics. Utilitzant aquests recursos, s'ha proposat implementar una aplicació mòbil que anima a acomplir activitat física, utilitzant realitat augmentada per atraure a l'usuari, proporcionant una diferent i motivadora experiència que anima a explorar i provar noves formes d'exercici. Aquesta tesi comença amb una implementació d'aplicació mòbil que detecta i rastreja estructures exteriors de gimnasos, entrena xarxes neuronals per segmentar i obtenir la posició de les estructures i projectar suggeriments d'exercicis utilitzant realitat augmentada. Així i tot, durant el projecte s'ha descobert que l'objectiu és inviable, ja que la plataforma utilitzada per desenvolupar el projecte, Unity, està encara en procés d'evolucionar respecte tecnologies d'aprenentatge profund. En aquestes circumstàncies, la tesi documenta els diferents passos que s'han fet, analitza la inviabilitat del projecte i proposa un estudi per millorar el pose estimator.

Popularnost korištenja neuronskih mreža u modernim računalnim sustavima svakim danom sve više raste, kao i kompleksnost problema koji se rješavaju uz pomoć tih neuronskih mreža. Poznatiji problemi koji se rješavaju uz pomoć neuronskih mreža uključuju prepoznavanje uzoraka na slikama ili video signalu, prepoznavanje teksta, pretvaranje teksta u govor ili obratno, klimatsko predviđanje i mnogi drugi manje poznati problemi. Međutim, kako istraživači povećavaju strukturu neuronskih mreža kako bi klasificirali komplicirane probleme uz veliku preciznost, moderne neuronske mreže mogu zauzimati nekoliko GB memorije u parametrima i zahtijevati nekoliko milijardi operacija za klasifikaciju jednog ulaza što je nepovoljno za izvođenje na računalnim sustavima koji imaju ograničenja u područjima memorije i potrošnje energije, primjerice ugradbeni računalni sustavi ili računalni sustavi koji zahtijevaju veliki stupanj skalabilnosti. Na sreću, dokazano je da neuronske mreže s parametrima u obliku brojeva s pomičnim zarezom sadrže veliki stupanj redundancije, odnosno da se isti problemi mogu, uz prihvatljiv gubitak na preciznosti, klasificirati uz pomoć neuronskih mreža s kvantiziranim parametrima. U sklopu ovog diplomskog rada opisan je postupak treniranja kvantiziranih neuronskih mreža korištenjem programske biblioteke Brevitas, prevođenje treniranog modela kvantizirane neuronske mreže iz ONNX podatkovnog formata u oblik pogodan za Vivado HLS sintezu uz pomoć skriptnog prevodioca FINN i izrada pokaznog sustava koji se izvodi na PYNQ-Z1 razvojnoj pločici. Pokazni sustav se sastoji od dvije Jupyter bilježnice od kojih prva služi za testiranje performansi dobivenog akceleratora, a druga omogućava klasifikaciju slike dohvaćene putem USB kamere. Korisnik upravlja s pokaznim sustavom tako da se spoji na poslužitelj Jupyter bilježnica koji je pokrenut na PYNQ-Z1 razvojnoj pločici koristeći web preglednik. The popularity of using neural networks in modern computing system grows with each passing day, as does the complexity of the problems which are solved with aforementioned neural networks. Some of the more known problems that are solved by using neural networks include pattern recognition in images or video signal, text recognition, text to speech conversion or speech to text conversion, climate change prediction along with many other less known problems. However, as researchers increase the structure of neural network sin order to classify more complex problems with great precision, modern neural networks can take up a few GB in memory for parameters and require a few billion floating point operations just to classify a single input which is not suitable for deployment on computing system with limitations regarding memory or energy consumption, such as embedded computing systems or computing systems with high scalability requirements. Luckily, it was proven that full precision floating point neural networks contain a lot of redundancy, or in other words it was shown that quantized parameter neural networks can be used to solve the same problems, albeit with acceptable accuracy loss. This masters thesis describes the training of quantized neural networks using Brevitas, exporting the neural networks from ONNX model format to a form which is suitable for Vivado HLS synthesis with the help of the FINN script compiler and construction of a demonstration system which is executed on a PYNQ-Z1 development board. The demonstration system consists of two Jupyter notebooks of which the first is used for testing the performance of the accelerator and the second is used to classify the image taken by the USB camera. Interfacing with the demonstration system is done by connecting to the running Jupyter notebook server on the PYNQ-Z1 development board with a web browser.

Exercising the body is a fundamental part of health. There are many re-sources freely available, such as audio-visual content on the internet, simplynature itself or outdoor gym structures in public parks. Using these resources,it has been proposed to implement a mobile application to encourage physicalactivity, using augmented reality to engage the user, providing a di?erent andmotivating experience that encourages them to explore and try new forms ofexercise. This thesis starts with the implementation of a mobile applicationthat detects and tracks outdoor gym structures, training neural networks tosegment and obtain the pose of the structures and project exercise sugges-tions using augmented reality. However, during the project we discover theunfeasibility of such an objective due to the fact that the platform used todevelop the project, Unity, is still in the process of evolving with respect todeep learning technologies.In these circumstances, this thesis documents the di?erent steps that havebeen taken, analyses the unfeasibility of the project and proposes a study toimprove the pose estimator. Ejercitar el cuerpo es una parte fundamental de la salud. Hay varios recursos gratuitos disponibles, como contenido audiovisual en internet, la naturaleza misma o estructuras de gimnasios exteriores en parques públicos. Utilizando estos recursos, se ha propuesto implementar una aplicación móvil que anima a cumplir actividad física, utilizando realidad aumentada para atraer al usuario, proporcionando una diferente y motivadora experiencia que anima a explorar y probar nuevas formas de ejercicio. Esta tesis comienza con una implementación de aplicación móvil que detecta y rastrea estructuras exteriores de gimnasios, entrena redes neuronales para segmentar y obtener la posición de las estructuras y proyectar sugerencias de ejercicios utilizando realidad aumentada. Aun así, durante el proyecto se ha descubierto que el objetivo es inviable, ya que la plataforma utilizada para desarrollar el proyecto, Unity, está aún en proceso de evolucionar respecto tecnologías de aprendizaje profundo. En estas circunstancias, la tesis documenta los diferentes pasos que se han hecho, analiza la inviabilidad del proyecto y propone un estudio para mejorar el pose estimator. Exercitar el cos és una part fonamental de la salut. Hi ha diversos recursos gratuïts disponibles, com contingut audiovisual a internet, la natura mateixa o estructures de gimnasos exteriors en parcs públics. Utilitzant aquests recursos, s'ha proposat implementar una aplicació mòbil que anima a acomplir activitat física, utilitzant realitat augmentada per atraure a l'usuari, proporcionant una diferent i motivadora experiència que anima a explorar i provar noves formes d'exercici. Aquesta tesi comença amb una implementació d'aplicació mòbil que detecta i rastreja estructures exteriors de gimnasos, entrena xarxes neuronals per segmentar i obtenir la posició de les estructures i projectar suggeriments d'exercicis utilitzant realitat augmentada. Així i tot, durant el projecte s'ha descobert que l'objectiu és inviable, ja que la plataforma utilitzada per desenvolupar el projecte, Unity, està encara en procés d'evolucionar respecte tecnologies d'aprenentatge profund. En aquestes circumstàncies, la tesi documenta els diferents passos que s'han fet, analitza la inviabilitat del projecte i proposa un estudi per millorar el pose estimator.

Transformer models perform well on Natural Language Processing and Natural Language Understanding tasks. Training and fine-tuning of these models consume a large amount of data and computing resources. Fast inference also requires high-end hardware for user-facing products. While distillation, quantization, and head-pruning for transformer models are well- explored domains in academia, the practical application is not straightforward. Currently, for good accuracy of the optimized models, it is necessary to fine-tune them for a particular task. This makes the generalization of the model difficult. If the same model has to be used for multiple downstream tasks, then it would require applying the process of optimization with fine-tuning for each task. This thesis explores the techniques of quantization and pruning for optimization of the Language-Agnostic BERT Sentence Embedding (LaBSE) model without fine-tuning for a downstream task. This should enable the model to be generalized enough for any downstream task. The techniques explored in this thesis are dynamic quantization, static quantization, quantize-aware training quantization, and head-pruning. The downstream performance is evaluated using sentiment classification, intent classification, and language-agnostic classification tasks. The results show that LaBSE can be accelerated on the CPU to 2.6x its original inference time without any loss of accuracy. Head-pruning 50% of the heads from each layer leads to 1.2x speedup while removing all heads but one leads to 1.32x speedup. A speedup of almost 9x is achieved by combining quantization with head-pruning with average 8% drop in accuracy on downstream evaluation tasks. Transformer-modeller ger bra resultat i uppgifter som rör behandling av och förståelse för naturligt språk. Träning och finjustering av dessa modeller kräver dock en stor mängd data och datorresurser. Snabb inferensförmåga kräver också högkvalitativ hårdvara för användarvänliga produkter och tjänster. Även om destillering, kvantisering och head-pruning för transformer-modeller är väl utforskade områden inom den akademiska världen är den praktiska tillämpningen inte okomplicerad. För närvarande är det nödvändigt att finjustera de optimerade modellerna för en viss uppgift för att uppnå god noggrannhet där. Detta gör det svårt att generalisera modellerna. Om samma modell skall användas för flera uppgifter i sekvens så måste man tillämpa optimeringsprocessen med finjustering för varje uppgift. I den här uppsatsen undersöks tekniker för kvantisering och prunering för optimering av LaBSE- modellen (Language-Agnostic BERT Sentence Embedding) utan finjustering för en downstream-uppgift. Detta bör göra det möjligt att generalisera modellen tillräckligt mycket för alla efterföljande uppgifter. De tekniker som undersöks är dynamisk kvantisering, statisk kvantisering, samt kvantisering för träning och head-pruning. Prestandan i efterföljande led utvärderas med hjälp av klassificering av känslor, avsiktsklassificering och språkagnostiska klassificeringsuppgifter. Resultaten visar att LaBSE kan öka effektiviteten hos CPU:n till 2,6 gånger sin ursprungliga inferenstid utan någon förlust av noggrannhet. Om 50% av huvudena från varje lager tas bort leder det till 1,2 gånger snabbare hastighet, medan det leder till 1,32 gånger snabbare hastighet om alla huvuden utom ett tas bort. Genom att kombinera kvantisering med head-pruning uppnås en ökning av hastigheten med nästan 9x, med en genomsnittlig minskning av noggrannheten med 8% i utvärderingsuppgifter nedströms.

Магістерська дисертація: 97 с., 33 рисунків, 22 табл., 29 джерел літератури, 2 додатки. Об’єкт дослідження: алгоритм автоматичного розпізнавання обличчя та розмиття. Предмет дослідження: розпізнавання обличчя на зображенні. Цілі дослідження: створення системи автоматичного розпізнавання обличчя та розмиття. Задачі роботи: розробити систему з використанням одного із сучасних методів розпізнавання обличчя, яка б могла в реальному часі у відео потоці розпізнавати обличчя людей та автоматично замилювати їх. Під час виконання роботи було проведено аналіз сучасних методів розпізнавання обличчя, був обраний метод з використанням нейронної мережі, а саме попередньо навчена модель FaceNet, як найбільш точна та швидка модель. Для роботи з потоковим відео було обрано бібліотеку OpenCV як найбільш гнучку та швидку бібліотеку для комп’ютерного зору. Була розроблена система для розпізнавання та розмиття облич у відео потоці, що виконує поставлену задачу. Актуальність проекту мотивується відсутністю аналогічних рішень і потенційною зацікавленістю з боку бізнесу. Результати роботи можуть бути використані як комерційними організаціями, так і ентузіастами, що прагнуть розробити щось подібне. Master's thesis: 97 p., 33 fig., 22 tabl., 29 ref., 2 appendixes. Object of research: an algorithm for automatic face recognition and blur. Subject of study: face recognition in the image. Objectives of the study: creation of an automatic face recognition and saponification system. Tasks of the work: to develop a system using one of the modern methods of face recognition that could recognize the faces of people in real time in the video stream and automatically blur them. During the work, the analysis of modern methods of face recognition was performed, the method was chosen using the neural network, namely the previously trained FaceNet model, as the most accurate and fastest model. OpenCV was selected as the most flexible and fastest computer vision library for streaming video. A system has been developed for recognizing and blurring faces in a video stream that accomplishes the task. The urgency of the project is motivated by the lack of similar solutions and potential interest from the business. The results of the work can be used by both commercial organizations and enthusiasts seeking to develop something similar.