Search

Twitter-based messages have presented challenges in the identification of features as applied to classification. This paper explores filtering techniques for improved trend detection and information extraction. Starting with a pre-filtered source (Twitter), we will examine the application of both information theory and Natural Language Processing (NLP) based techniques as a means of preprocessing for classification. Results demonstrate that both means allow for improved results in classification among highly idiosyncratic data (Twitter).

The MapReduce paradigm has become ubiquitous within Big Data Analytics. Within this field, Social Networks exist as an important area of applications as it relies on the large scale analysis of graphs. To enable the scalability of Social Networks, we consider the application of MapReduce design patterns for the determination of graph-based metrics. Specifically, we detail the application of a MapReduce-based solution for the metric of betweenness-centrality. The prevailing concept is separation of the graph topology from the actual graph analysis. Here, we consider the chaining of MapReduce jobs for the estimation of shortest paths in a graph as well as post processing statistics. Through our design pattern, we are able to leverage Big Data Technologies to determine metrics in the context of ever expanding internet-based data resources.