Search

Simple Summary: Breast cancer is a significant cause of female cancer-related deaths in the US. Checking how severe the cancer is helps in planning treatment. Modern AI methods are good at grading cancer, but they are not used much in hospitals yet. We developed and utilized ensemble deep learning algorithms for addressing the tasks of classifying (1) breast cancer subtype and (2) breast cancer invasiveness from whole slide image (WSI) histopathology slides. The ensemble models used were based on convolutional neural networks (CNNs) known for extracting distinctive features crucial for accurate classification. In this paper, we provide a comprehensive analysis of these models and the used methodology for breast cancer diagnosis tasks. Cancer diagnosis and classification are pivotal for effective patient management and treatment planning. In this study, a comprehensive approach is presented utilizing ensemble deep learning techniques to analyze breast cancer histopathology images. Our datasets were based on two widely employed datasets from different centers for two different tasks: BACH and BreakHis. Within the BACH dataset, a proposed ensemble strategy was employed, incorporating VGG16 and ResNet50 architectures to achieve precise classification of breast cancer histopathology images. Introducing a novel image patching technique to preprocess a high-resolution image facilitated a focused analysis of localized regions of interest. The annotated BACH dataset encompassed 400 WSIs across four distinct classes: Normal, Benign, In Situ Carcinoma, and Invasive Carcinoma. In addition, the proposed ensemble was used on the BreakHis dataset, utilizing VGG16, ResNet34, and ResNet50 models to classify microscopic images into eight distinct categories (four benign and four malignant). For both datasets, a five-fold cross-validation approach was employed for rigorous training and testing. Preliminary experimental results indicated a patch classification accuracy of 95.31% (for the BACH dataset) and WSI image classification accuracy of 98.43% (BreakHis). This research significantly contributes to ongoing endeavors in harnessing artificial intelligence to advance breast cancer diagnosis, potentially fostering improved patient outcomes and alleviating healthcare burdens. [ABSTRACT FROM AUTHOR]

A longstanding line of research in urban studies explores how cities can be understood through their appearance. However, what remains unclear is to what extent urban dwellers' everyday life can be explained by the visual clues of the urban environment. In this paper, we address this question by applying a computer vision model to 27 million street view images across 80 counties in the United States. Then, we use the spatial distribution of notable urban features identified through the street view images, such as street furniture, sidewalks, building façades, and vegetation, to predict the socioeconomic profiles of their immediate neighborhood. Our results show that these urban features alone can account for up to 83% of the variance in people's travel behavior, 62% in poverty status, 64% in crime, and 68% in health behaviors. The results outperform models based on points of interest (POI), population, and other demographic data alone. Moreover, incorporating urban features captured from street view images can improve the explanatory power of these other methods by 5% to 25%. We propose "urban visual intelligence" as a process to uncover hidden city profiles, infer, and synthesize urban information with computer vision and street view images. This study serves as a foundation for future urban research interested in this process and understanding the role of visual aspects of the city. [ABSTRACT FROM AUTHOR]

A smart city is a city that binds together technology, society, and government to enable the existence of a smart economy, smart mobility, smart environment, smart living, smart people, and smart governance in order to reduce the environmental impact of cities and improve life quality. The first step to achieve a fully connected smart city is to start with smaller modules such as smart homes and smart buildings with energy management systems. Buildings are responsible for a third of the total energy consumption; moreover, heating, ventilation, and air conditioning (HVAC) systems account for more than half of the residential energy consumption in the United States. Even though connected thermostats are widely available, they are not used as intended since most people do not have the expertise to control this device to reduce energy consumption. It is commonly set according to their thermal comfort needs; therefore, unnecessary energy consumption is often caused by wasteful behaviors and the estimated energy saving is not reached. Most studies in the thermal comfort domain to date have relied on simple activity diaries to estimate metabolic rate and fixed values of clothing parameters for strategies to set the connected thermostat's setpoints because of the difficulty in tracking those variables. Therefore, this paper proposes a strategy to save energy by dynamically changing the setpoint of a connected thermostat by human activity recognition based on computer vision preserving the occupant's thermal comfort. With the use of a depth sensor in conjunction with an RGB (Red–Green–Blue) camera, a methodology is proposed to eliminate the most common challenges in computer vision: background clutter, partial occlusion, changes in scale, viewpoint, lighting, and appearance on human detection. Moreover, a Recurrent Neural Network (RNN) is implemented for human activity recognition (HAR) because of its data's sequential characteristics, in combination with physiological parameters identification to estimate a dynamic metabolic rate. Finally, a strategy for dynamic setpoints based on the metabolic rate, predicted mean vote (PMV) parameter and the air temperature is simulated using EnergyPlus™ to evaluate the energy consumption in comparison with the expected energy consumption with fixed value setpoints. This work contributes with a strategy to reduce energy consumption up to 15% in buildings with connected thermostats from the successful implementation of the proposed method. [ABSTRACT FROM AUTHOR]

Focuses on the computer vision research for undergraduates in the United States. Ability of students to prepare and present papers in conferences; Accomplishments of students in journal publications; Continuance of students on graduate studies at various institutions.

Evaluation was not a thing when the first author was a graduate student in the late 1970s. There was an Artificial Intelligence (AI) boom then, but that boom was quickly followed by a bust and a long AI Winter. Charles Wayne restarted funding in the mid-1980s by emphasizing evaluation. No other sort of program could have been funded at the time, at least in America. His program was so successful that these days, shared tasks and leaderboards have become common place in speech and language (and Vision and Machine Learning). It is hard to remember that evaluation was a tough sell 25 years ago. That said, we may be a bit too satisfied with current state of the art. This paper will survey considerations from other fields such as reliability and validity from psychology and generalization from systems. There has been a trend for publications to report better and better numbers, but what do these numbers mean? Sometimes the numbers are too good to be true, and sometimes the truth is better than the numbers. It is one thing for an evaluation to fail to find a difference between man and machine, and quite another thing to pass the Turing Test. As Feynman said, "the first principle is that you must not fool yourself–and you are the easiest person to fool." [ABSTRACT FROM AUTHOR]

The ecology of forest ecosystems depends on the composition of trees. Capturing fine-grained information on individual trees at broad scales provides a unique perspective on forest ecosystems, forest restoration, and responses to disturbance. Individual tree data at wide extents promises to increase the scale of forest analysis, biogeographic research, and ecosystem monitoring without losing details on individual species composition and abundance. Computer vision using deep neural networks can convert raw sensor data into predictions of individual canopy tree species through labeled data collected by field researchers. Using over 40,000 individual tree stems as training data, we create landscape-level species predictions for over 100 million individual trees across 24 sites in the National Ecological Observatory Network (NEON). Using hierarchical multi-temporal models fine-tuned for each geographic area, we produce open-source data available as 1 km2 shapefiles with individual tree species prediction, as well as crown location, crown area, and height of 81 canopy tree species. Site-specific models had an average performance of 79% accuracy covering an average of 6 species per site, ranging from 3 to 15 species per site. All predictions are openly archived and have been uploaded to Google Earth Engine to benefit the ecology community and overlay with other remote sensing assets. We outline the potential utility and limitations of these data in ecology and computer vision research, as well as strategies for improving predictions using targeted data sampling. Detailed maps of tree species can be used for forest analysis, biogeographic research and ecosystem monitoring. This study provides individual tree species maps for 24 sites in the United States National Ecological Observatory Network (NEON), covering 80 tree species and 100 million individuals. [ABSTRACT FROM AUTHOR]

Outlier detection plays a critical role in building operation optimization and data quality maintenance. However, existing methods often struggle with the complexity and variability of building energy data, leading to poorly generalized and explainable results. To address the gap, this study introduces a novel Vision-based Outlier Detection (VOD) approach, leveraging computer vision models to spot outliers in the building energy records. The models are trained to identify outliers by analyzing the load shapes in 2D time series plots derived from the energy data. The VOD approach is tested on four years of workday time-series electricity consumption data from 290 commercial buildings in the United States. Two distinct models are developed for different usage purposes, namely a classification model for broad-level outlier detection and an object detection model for the demands of precise pinpointing of outliers. The classification model is also interpreted via Grad-CAM to enhance its usage reliability. The classification model achieves an F1 score of 0.88, and the object detection model achieves an Average Precision (AP) of 0.84. VOD is a very efficient path to identifying energy consumption outliers in building operations, paving the way for the enhancement of building energy data quality, operation efficiency, and energy savings. [ABSTRACT FROM AUTHOR]

A significant percentage of bridges in the United States are serving beyond their 50-year design life, and many of them are in poor condition, making them vulnerable to fatigue cracks that can result in catastrophic failure. However, current fatigue crack inspection practice based on human vision is time-consuming, labor intensive, and prone to error. We present a novel human-centered bridge inspection methodology to enhance the efficiency and accuracy of fatigue crack detection by employing advanced technologies including computer vision and augmented reality (AR). In particular, a computer vision-based algorithm is developed to enable near-real-time fatigue crack detection by analyzing structural surface motion in a short video recorded by a moving camera of the AR headset. The approach monitors structural surfaces by tracking feature points and measuring variations in distances between feature point pairs to recognize the motion pattern associated with the crack opening and closing. Measuring distance changes between feature points, as opposed to their displacement changes before this improvement, eliminates the need of camera motion compensation and enables reliable and computationally efficient fatigue crack detection using the nonstationary AR headset. In addition, an AR environment is created and integrated with the computer vision algorithm. The crack detection results are transmitted to the AR headset worn by the bridge inspector, where they are converted into holograms and anchored on the bridge surface in the 3D real-world environment. The AR environment also provides virtual menus to support human-in-the-loop decision-making to determine optimal crack detection parameters. This human-centered approach with improved visualization and human–machine collaboration aids the inspector in making well-informed decisions in the field in a near-real-time fashion. The proposed crack detection method is comprehensively assessed using two laboratory test setups for both in-plane and out-of-plane fatigue cracks. Finally, using the integrated AR environment, a human-centered bridge inspection is conducted to demonstrate the efficacy and potential of the proposed methodology. [ABSTRACT FROM AUTHOR]

Due to the increasing consumption of food products and demand for food quality and safety, most food processing facilities in the United States utilize machines to automate their processes, such as cleaning, inspection and grading, packing, storing, and shipping. Machine vision technology has been a proven solution for inspection and grading of food products since the late 1980s. The remaining challenges, especially for small to midsize facilities, include the system and operating costs, demand for high-skilled workers for complicated configuration and operation and, in some cases, unsatisfactory results. This paper focuses on the development of an embedded solution with learning capability to alleviate these challenges. Three simple application cases are included to demonstrate the operation of this unique solution. Two datasets of more challenging cases were created to analyze and demonstrate the performance of our visual inspection algorithm. One dataset includes infrared images of Medjool dates of four levels of skin delamination for surface quality grading. The other one consists of grayscale images of oysters with varying shape for shape quality evaluation. Our algorithm achieved a grading accuracy of 95.0% on the date dataset and 98.6% on the oyster dataset, both easily surpassed manual grading, which constantly faces the challenges of human fatigue or other distractions. Details of the design and functions of our smart camera and our simple visual inspection algorithm are discussed in this paper. [ABSTRACT FROM AUTHOR]

Robotic weed control for vegetables is necessary to increase crop productivity, avoid intensive hand weeding as labour shortages in developed countries such as United States has led to a surge in food production costs. However, development of a reliable, intelligent robotic system for weed control in real-time for vegetables still remains a challenging task. The main issue arises while distinguishing crops from weeds in real-time. In this paper, a novel technique to crop signalling to distinguish crops from in-row weeds in complex natural scenarios, such as high weed densities commonly found on organic farms, in real-time using machine vision is presented. Crop signalling is a simple and low-cost technique in which a signalling compound is produced by or applied to the crop and where the signalling compound is machine readable and helps to create visual features that uniquely distinguish the crops from weeds. The crop and weed mapping algorithm presented here were specially designed and developed for a vision-based weeding robot equipped with a micro-jet herbicide-spraying system for weed control in a lettuce field. The proposed technique involves weed/crop mapping and decision making. Experimental results show that the crop detection accuracy was 99.75%, and 98.11% of sprayable weeds were detected. The proposed technique is highly accurate, reliable and more robust than other sensor-based techniques presented in the literature. Image 1 • A real-time in-row weed-crop detection and classification algorithm was proposed. • A novel crop signaling technique was introduced to distinguish crops from weeds. • It utilized computer vision to determine the spatial location of lettuce and weed. • The algorithm was designed to equipped with a micro-jet herbicide-spraying system. • The algorithm outperformed existing methods in terms classification accuracy. [ABSTRACT FROM AUTHOR]

Wide-baseline mapping technology has been applied in NASA's Mars Exploration Rover (MER) mission to improve the accuracy of mapping of far-range terrain from Rover stereo images. As a basic research topic in photogrammetry and computer vision, it is desirable to perform a comprehensive investigation on the accuracy and automation of wide- baseline mapping. This paper presents the results of a systematic accuracy analysis of wide-baseline mapping through theoretical derivation and Monte Carlo simulation. Automated bundle adjustment and 3D wide-baseline mapping techniques are developed and tested using wide-baseline images acquired by the Spirit Rover. Experimental results demonstrate the effectiveness of the proposed techniques and verify that the mapping capability of rover stereo images can be extended from tens of meters (hard-baseline) to hundreds of meters using wide-baseline mapping techniques. [ABSTRACT FROM AUTHOR]

In the year 1999 the world population reached 6 billion, doubling the previous census estimate of 1960. Recently, the United States Census Bureau issued a revised forecast for world population showing a projected growth to 9.4 billion by 2050 (US Census Bureau, ). Different research disci- plines have studied the crowd phenomenon and its dynamics from a social, psychological and computational standpoint respectively. This paper presents a survey on crowd analysis methods employed in computer vision research and discusses perspectives from other research disciplines and how they can contribute to the computer vision approach. [ABSTRACT FROM AUTHOR]

The ornamental crop industry is an important contributor to the economy in the United States. The industry has been facing challenges due to continuously increasing labor and agricultural input costs. Sensing and automation technologies have been introduced to reduce labor requirements and to ensure efficient management operations. This article reviews current sensing and automation technologies used for ornamental nursery crop production and highlights prospective technologies that can be applied for future applications. Applications of sensors, computer vision, artificial intelligence (AI), machine learning (ML), Internet-of-Things (IoT), and robotic technologies are reviewed. Some advanced technologies, including 3D cameras, enhanced deep learning models, edge computing, radio-frequency identification (RFID), and integrated robotics used for other cropping systems, are also discussed as potential prospects. This review concludes that advanced sensing, AI and robotic technologies are critically needed for the nursery crop industry. Adapting these current and future innovative technologies will benefit growers working towards sustainable ornamental nursery crop production. [ABSTRACT FROM AUTHOR]

Coastal hazard events such as hurricanes pose a significant threat to coastal communities. Disaster relief is essential to mitigating damage from these catastrophes; therefore, accurate and efficient damage assessment is key to evaluating the extent of damage inflicted on coastal cities and structures. Historically, this process has been carried out by human task forces that manually take post-disaster images and identify the damaged areas. While this method has been well established, current digital tools used for computer vision tasks such as artificial intelligence and machine learning put forth a more efficient and reliable method for assessing post-disaster damage. Using transfer learning on three advanced neural networks, ResNet, MobileNet, and EfficientNet, we applied techniques for damage classification and damaged object detection to our post-hurricane image dataset comprised of damaged buildings from the coastal region of the southeastern United States. Our dataset included 1000 images for the classification model with a binary classification structure containing classes of floods and non-floods and 800 images for the object detection model with four damaged object classes damaged roof, damaged wall, flood damage, and structural damage. Our damage classification model achieved 76 % overall accuracy for ResNet and 87 % overall accuracy for MobileNet. The F1 score for MobileNet was also 9 % higher than the F1 score of ResNet at 0.88. Our damaged object detection model achieved predominant predictions of the four damaged object classes, with MobileNet attaining the highest overall confidence score of 97.58 % in its predictions. The object detection results highlight the model's ability to successfully identify damaged areas of buildings and structures from images in a time span of seconds, which is necessary for more efficient damage assessment. Thus, we show that this level of accuracy for our damage assessment using artificial intelligence is akin to the accuracy of manual damage assessments while also completing the assessment in a drastically shorter time span. [ABSTRACT FROM AUTHOR]

This paper explains the use of a convolutional neural network (CNN) to segment solar panels in a satellite image containing solar arrays, and extract associated metadata from the arrays. A novel unsupervised technique is introduced to estimate the azimuth of each individual solar panel from the predicted mask of the convolutional neural network. This pipeline was developed with the aim of extracting necessary metadata for a solar installation, using only a set of latitude–longitude coordinates. Azimuth prediction results for 669 individual solar installations associated with 387 sites located across the United States are provided. A mean average error and median average error of 21.65 degrees and 1.0 degrees were obtained, respectively, when predicting the azimuth of the solar fleet data set, with about 80% of the results within an error of zero degrees of the ground truth azimuth value and about 85% within an error of 25 degrees. The predicted azimuth was then used to estimate the energy conversion of the solar arrays. Results show a 90.9 and 90.6 R-squared value for estimating alternating current (AC) and direct current (DC) energy, respectively, and a mean absolute percentage error (MAPE) of 1.70% in estimating the alternating current (AC) energy using the fully automated algorithm. [Display omitted] • A novel unsupervised technique is used to estimate the azimuth of solar arrays. • Segmentation of solar panels in the satellite image, using transfer learning and a CNN. • The proposed approach is can be used with high resolution satellite or aerial imagery. [ABSTRACT FROM AUTHOR]

Cervical cancer if detected early has an upward of 89% survival rate. The leading tool in identifying cervical cancer in its infancy is the Papanicolaou (Pap smear) test, which since its introduction dropped cervical cancer related deaths by 60%. The Pap smear test or Liquid-based Cytology (LBC) is a time-consuming procedure that requires a pathologist to manually identify cervical cells that may be in the middle of the processes that indicate cervical cancer. Unfortunately, due to the expenses related to conducting the Pap smear test many women are blocked from access to it and this leads to over 4000 women dying annually from cervical cancer in the United States alone. The aim of this research is to automate the methods used by pathologists in conducting the Pap smear or LBC. We show that using machine vision, ensemble learning and deep learning methods a significant portion of the Pap smear can be done automated. We set out to extract cells and cell clusters and classify those samples based on the Bethesda System for reporting cervical cytology. Achieving an accuracy of 90.4% and 91.6% for the ensemble learning and deep learning methods respectively when evaluated with a five-fold cross-validation demonstrates promise in the creation of an automated Pap smear screening test. [ABSTRACT FROM AUTHOR]

Premise: Apple orchards in the United States are under constant threat from a large number of pathogens and insects. Appropriate and timely deployment of disease management depends on early disease detection. Incorrect and delayed diagnosis can result in either excessive or inadequate use of chemicals, with increased production costs and increased environmental and health impacts. Methods and Results: We have manually captured 3651 high‐quality, real‐life symptom images of multiple apple foliar diseases, with variable illumination, angles, surfaces, and noise. A subset of images, expert‐annotated to create a pilot data set for apple scab, cedar apple rust, and healthy leaves, was made available to the Kaggle community for the Plant Pathology Challenge as part of the Fine‐Grained Visual Categorization (FGVC) workshop at the 2020 Computer Vision and Pattern Recognition conference (CVPR 2020). Participants were asked to use the image data set to train a machine learning model to classify disease categories and develop an algorithm for disease severity quantification. The top three area under the ROC curve (AUC) values submitted to the private leaderboard were 0.98445, 0.98182, and 0.98089. We also trained an off‐the‐shelf convolutional neural network on this data for disease classification and achieved 97% accuracy on a held‐out test set. Discussion: This data set will contribute toward development and deployment of machine learning–based automated plant disease classification algorithms to ultimately realize fast and accurate disease detection. We will continue to add images to the pilot data set for a larger, more comprehensive expert‐annotated data set for future Kaggle competitions and to explore more advanced methods for disease classification and quantification. [ABSTRACT FROM AUTHOR]

The article focuses on a study which examines large-scale observational evidence of cross-cultural differences in facial behavior using advanced in computer science. Topics discussed include the automated facial coding used, the reliability of the facial action unit detection algorithms across different testing subsets, multilevel regression estimates predicting each facial action's base rate percentage.

We present a pattern-based regionalization of the conterminous US – a partitioning of the country into a number of mutually exclusive and exhaustive regions that maximizes the intra-region stationarity of land cover patterns and inter-region disparity between those patterns. The result is a discretization of the land surface into a number of landscape pattern types (LPTs) – spatial units each containing a unique quasi-stationary pattern of land cover classes. To achieve this goal, we use a recently developed method which utilizes machine vision techniques. First, the entire National Land Cover Dataset (NLCD) is partitioned into a grid of square-size blocks of cells, called motifels. The size of a motifel defines the spatial scale of a local landscape. The land cover classes of cells within a motifel form a local landscape pattern which is mathematically represented by a histogram of co-occurrence features. Using the Jensen–Shannon divergence as a dissimilarity function between patterns we group the motifels into several LPTs. The grouping procedure consists of two phases. First, the grid of motifels is partitioned spatially using a region-growing segmentation algorithm. Then, the resulting segments of this grid, each represented by its medoid, are clustered using a hierarchical algorithm with Ward’s linkage. The broad-extent maps of progressively more generalized LPTs resulting from this procedure are shown and discussed. Our delineated LPTs agree well with the perceptual patterns seen in the NLCD map. [ABSTRACT FROM PUBLISHER]

Italian ryegrass (Lolium perenne ssp. multiflorum (Lam) Husnot) is a troublesome weed species in wheat (Triticum aestivum) production in the United States, severely affecting grain yields. Spatial mapping of ryegrass infestation in wheat fields and early prediction of its impact on yield can assist management decision making. In this study, unmanned aerial systems (UAS)-based red, green and blue (RGB) imageries acquired at an early wheat growth stage in two different experimental sites were used for developing predictive models. Deep neural networks (DNNs) coupled with an extensive feature selection method were used to detect ryegrass in wheat and estimate ryegrass canopy coverage. Predictive models were developed by regressing early-season ryegrass canopy coverage (%) with end-of-season (at wheat maturity) biomass and seed yield of ryegrass, as well as biomass and grain yield reduction (%) of wheat. Italian ryegrass was detected with high accuracy (precision = 95.44 ± 4.27%, recall = 95.48 ± 5.05%, F-score = 95.56 ± 4.11%) using the best model which included four features: hue, saturation, excess green index, and visible atmospheric resistant index. End-of-season ryegrass biomass was predicted with high accuracy (R2 = 0.87), whereas the other variables had moderate to high accuracy levels (R2 values of 0.74 for ryegrass seed yield, 0.73 for wheat biomass reduction, and 0.69 for wheat grain yield reduction). The methodology demonstrated in the current study shows great potential for mapping and quantifying ryegrass infestation and predicting its competitive response in wheat, allowing for timely management decisions. [ABSTRACT FROM AUTHOR]

Focuses on the Intelligent Ground Vehicle Competition (IGVC), a student competitions founded and sponsored by the Association for Unmanned Vehicle Systems International. Primary goal of IGVC; Integration of advanced control theory, machine vision, vehicular electronics and mobile platform fundamentals to design and build an unmanned system for competition against both the United States and international teams; Development of a suite of dual-use technologies to equip ground vehicles with intelligent driving capabilities; Three components of the IGVC;

The who, what, when, and how of the Arpanet is usually told in heroic terms—Licklider's vision, the fervor of his disciples, the dedication of computer scientists and engineers, the work of graduate students, and so forth. Told by one of the key actors in this salient part of US and Internet history, this article addresses why the Arpanet was built. [ABSTRACT FROM PUBLISHER]

Reports that Natick, Massachusetts-based Convex Corp. has formed two product teams within its Modular Vision Systems Division to focus engineering and marketing resources on new markets and applications for machine vision systems. Identification of products and working on expert sensors.