Search

This paper presents an approach to investigate distributed machine learning workloads on Spark. The work analyzes hardware and volume data factors regarding time and cost performance when applying machine learning (ML) techniques in big data scenarios. The method is based on the Design of Experiments (DoE) approach and applies randomized two-level fractional factorial design with replications to screening the most relevant factors. A Web Corpus was built from 16 million webpages from Portuguese-speaking countries. The application was a binary text classification to distinguish Brazillian Portuguese from other variations. Five different machine learning algorithms were examined: Logistic Regression, Random Forest, Support Vector Machines, Naive Bayes and Multilayer Perceptron. The data was processed using real clusters having up to 28 nodes, each composed of 12 or 32 cores, 1 or 7 SSD disks, and 3x or 6x RAM per core, totalizing a maximum computational power of 896 cores and 5.25 TB RAM. Linear models were applied to identify, analyze and rank the influence of factors. A total of 240 experiments were carefully organized to maximize the detection of non-cofounded effects up to the second-order, minimizing the experimental efforts. Our results include linear models to estimate time and cost performance, statistical inferences about effects, and a visualization tool based on parallel coordinates to aid decision making about cluster configuration.

The leaf of the jaborandi (Pilocarpus microphyllus) is the part of the plant of economic interest, from which the pilocarpine is extracted, therefore, leaf production is the main variable that determines the productivity of this species. Pilocarpus leaf production can be managed during its development phase. Thus, the objective of the present study was to evaluate branch regeneration of jaborandi submitted to two types of cutting/pruning and different cropping systems. Prunings were carried out in jaborandi plants from different accessions from the Janorandi Active Germoplasm Bank (BGA), that were under different cropping systems, full sun and shade. With pruning shears, cuts were made 10 and 20 cm from the branch bifurcations. The results showed that regardless of the distances of the cuts in the jaborandi plants, the prunings benefited new shoots production and the harvest and influence on leaf production could be anticipated without harming the plant?s development. The drastic pruning performed on the species also stimulated efficient regeneration without damaging the plant, resulting in bigger leaf production, regardless of the pruning direction. It was also observed that jaborandi cultivation becomes more productive when it is grown under full sun regardless of the germplasm used, and the leaves can be harvested in all the months of the year. Made available in DSpace on 2022-11-01T15:08:49Z (GMT). No. of bitstreams: 1 Pilocarpus-microphyllus.pdf: 661321 bytes, checksum: b929efa807e14310e4d5caf7defad62a (MD5) Previous issue date: 2022

Este trabalho analisa o desempenho com mineração de dados das escolas de São Paulo no exame SARESP, com dados da SEDUC-SP. A metodologia, baseada no CRISP-DM, propõe uma solução de aprendizagem de máquina para prever o desempenho das escolas e extrair padrões relevantes do desempenho educacional com técnicas de IA Explicativas. Sete classificadores alcançaram alta acurácia (93%) e AUC ROC (0.97) na previsão do desempenho das escolas, com dados do perfil de alunos, escolas e valores sócio-econômicos externos. O modelo não-linear e as técnicas SHAP e Counterfactual evidenciaram fatores relevantes que podem impactar o resultado educacional e a utilidade da metodologia no apoio à decisão.

Over-sampling algorithms are the most adopted approach to balance class distribution in imbalanced data problems, through random replication or synthesis of new examples in the minority class. Current over-sampling algorithms, however, usually use all available examples in the minority class to synthesise new instances, which may include noisy or outlier data. This work proposes k-INOS, a new algorithm to prevent over-sampling algorithms from being contaminated by noisy examples in the minority class. k-INOS is based on the concept of neighbourhood of influence and works as a wrapper around any over-sampling algorithm. A comprehensive experimentation was conducted to test k-INOS in 50 benchmark data sets, 8 over-sampling methods and 5 classifiers, with performance measured according to 7 metrics and Wilcoxon signed-ranks test. Results showed, particularly for weak classifiers (but not only), k-INOS significantly improved the performance of over-sampling algorithms in most performance metrics. Further investigations also allowed to identify conditions where k-INOS is likely to increase performance, according to features and rates measured from the data sets. The extensive experimentation framework evidenced k-INOS as an efficient algorithm to be applied prior to over-sampling methods.