Search

In modern era, Cognitive Internet of Things (CIoT) in conjunction with IoT evolves which provides the intelligence power of sensing and computation for next-generation IoT (Nx-IoT) networks. The data scientists have discovered a large amount of techniques for knowledge discovery from processed data in CIoT. This task is accomplished successfully and data proceeds for further processing. The major cause for the failure of IoT devices is due to the attacks, in which web spam is more prominent. There seems a requirement of a technique which can detect the web spam before it enters into a device. Motivated from these issues, in this paper, Cognitive spammer framework (CSF) for web spam detection is proposed. CSF detects the web spam by fuzzy rule based classifiers along with machine learning classifiers. Each classifier produces the quality score of the webpage. These quality scores are then ensembled to generate a single score, which predicts the spamicity of the web page. For ensembling, fuzzy voting approach is used in CSF. The experiments were performed using standard dataset WEBSPAM-UK 2007 with respect to accuracy and overhead generated. From the results obtained, it has been demonstrated that CSF improves the accuracy by 97.3%, which is comparatively high in comparison to the other existing approaches in literature.

For next generation IoT applications, edge devices provides most of the computing resources close to the proximity of the end users. These devices having built-in intelligence using various AI techniques can take independent decisions in the environment where these are deployed. Motivated from these concerns, We suggest a cognitive intrusion security system to maintain the credibility of search engine results, which eliminates the advertising images from penetrating the image database of the web browser. The proposed framework provides edge intelligence for web data filteration and detects the web spam by considering three different layers, i.e., data collection services, edge computing services, and cloud services. The target is to detect the malicious images. Firstly, the features of an image such as mean, image gradient, entropy are fetched and then the retrieved data is processed in the proposed framework. Deep learning algorithms are used for the validation of the proposed system. By evaluating it on real-time collected dataset, it resulted in an accuracy of 98.77%.

In the modern digital era, the Web has been utilized for searching information by using different search engines (SE) as a tool. However, web spammers misuse the web for financial benefits by ranking the irrelevant and spam web pages higher than relevant pages in the search engine's results pages (SERPs) by using web spamming techniques. Furthermore, those top-ranked unrelated web pages contain insufficient or inappropriate information for the user. In addition, web spamming techniques dramatically affect the quality of the search engine. Researchers introduced several web spam detection techniques such as content-based features, link-based features, label propagation, label refinement, click-based web spamming detection, and real-time web spam detection. However, identifying all spam pages on the Web with high accuracy is still remains unsolved. This work proposes a content-based web spam detection framework, link-based web spam detection framework, and a combined approach to identify both types of web spams with high accuracy that can detect the newly evolved link pyramid. The content-based web spam detection framework uses three proposed and two improved content-based algorithms for web spam detection. The link-based web spam detection framework initially exposed the relationship network behind the link spamming and then used the paid-links database algorithm, spam signals algorithm, and improved link farms algorithm for link-based web spam identification. Finally, the combination of both content and link-based frameworks enhance the accuracy of web spam detection. The proposed combined approach's performance has been evaluated and compared with the J48 classifier, C4.5 decision tree classifier, SVM classifier, and heuristic combined approach. Some experiments were conducted to obtain the threshold values using the proposed collection architecture on well-known datasets WEB SPAM-UK2006 and WEB SPAM-UK2007. The results show that the proposed methods outperform other methods with 82.1% precision and an F-measure of 80.6% to illustrate the proposed framework's effectiveness and applicability.

Despite the great advances in spam detection, spam remains a major problem that has affected the global economy enormously. Spam attacks are popularly perpetrated through different digital platforms with a large electronic audience, such as emails, microblogging websites (e.g. Twitter), social networks (e.g. Facebook), and review sites (e.g. Amazon). Different spam detection solutions have been proposed in the literature, however, Machine Learning (ML) based solutions are one of the most effective. Nevertheless, most ML algorithms have computational complexity problem, thus some studies introduced Nature Inspired (NI) algorithms to further improve the speed and generalization performance of ML algorithms. This study presents a survey of recent ML-based and NI-based spam detection techniques to empower the research community with information that is suitable for designing effective spam filtering systems for emails, social networks, microblogging, and review websites. The recent success and prevalence of deep learning show that it can be used to solve spam detection problems. Moreover, the availability of large-scale spam datasets makes deep learning and big data solutions (such as Mahout) very suitable for spam detection. Few studies explored deep learning algorithms and big data solutions for spam detection. Besides, most of the datasets used in the literature are either small or synthetically created. Therefore, future studies can consider exploring big data solutions, big datasets, and deep learning algorithms for building efficient spam detection techniques.

Many score propagation based Web Spam Demotion Algorithms (WSDAs) have been proposed in last decade. There are two major challenges those algorithms suffer from. First, the non-incremental property of score propagation based WSDAs restricted their applications in real world since Web changes rapidly and running algorithm on the entire Web graph is computation consuming. Second, the score propagation based WSDAs adopt only link structure of the web graph to demote Web spam, so that they are vulnerable to some other kind of spamming techniques, such as content spam. In this paper, we propose a preference-based learning to rank method to address the above-mentioned issues confronted by score propagation based WSDAs. Our proposal consists of two components, a preference function and an ordering algorithm. The preference function is modeled by Deep Belief Network (DBN), which can benefit from unlabeled data for better generalization. The proposed Incremental Probabilistic Ordering Algorithm (IPOA) uses the trained preference function to calculate top-ranking probabilities of Web pages, and then uses those probabilities for final ranking. Therefore, the complex object (i.e. Web page) ranking problem is reduced to real number ranking problem, which can be solved efficiently by classical sorting algorithm. We conduct experiments to compare our proposal with conventional score propagation based WSDAs as well as some popular preference based learning to rank algorithms on two public available datasets, WEBSPAM-UK2006 and WEBSPAM-UK2007. Our experimental results demonstrate the superiority of our proposed method. Specifically, compared with score propagation based WSDAs, we obtain 0.0074 absolute improvement (0.7% relative improvement) on WEBSPAM-UK2006 and 0.065 absolute improvement (7.3% relative improvement) on WEBSPAM-UK2007 in terms of spam demotion score.

From the last few years, Internet of Things has revolutionized the entire world. In this, various smart objects perform the tasks of sensing and computing to provide uninterrupted services to the end users in different applications such as smart transportation, e-healthcare to name a few. With the inherent capabilities of these objects to take adaptive intelligent decisions, Cognitive Internet of Things is another paradigm of Internet of Things which emerges during this era. However, while accessing data from the Internet, web spam is one of the challenges to be handled. It has been observed from the literature review that for accessing data, search engines are preferred mostly by an individual. The search engine’s effective ranking can decrease the computational cost of objects during the data access. The current solutions to this issue are aimed to discover the spam in the search engine after its occurrence. So, in this proposal, we present a cognitive spammer framework that removes spam pages when search engines calculate the web page rank score. The framework detects web spam with the support of Long Short-Term Memory network by training the link features. This training resulted with an accuracy of 95.25, as more than 1,11,000 hosts are being correctly classified. However, the content features are trained by neural network. The proposed scheme has been validated with the WEBSPAM-UK 2007 dataset. Prior to processing, the dataset is pre-processed using a new technique called ‘Split by Over-sampling and Train by Under-fitting’. The ensemble and cross validation approach has been used for optimization of results with an accuracy of 96.96%. So, the proposed scheme outperforms the existing techniques.

Although a variety of techniques to detect malicious websites have been proposed, it becomes more and more difficult for those methods to provide a satisfying result nowadays. Many malicious websites can still escape detection with various Web spam techniques. In this paper, we first summarize three types of Web spam techniques used by malicious websites, such as redirection spam, hidden IFrame spam, and content hiding spam. We then present a new detection method that adopts the perspective of users and takes screenshots of malicious webpages to invalidate Web spams. The proposed detection method uses a Convolutional Neural Network, which is a class of deep neural networks, as a classification algorithm. In order to verify the effectiveness of the method, two different experiments have been conducted. First, the proposed method was tested based on a constructed complex dataset. We present comparison results between the proposed method and representative machine learning-based detection algorithms. Second, the proposed method was tested to detect malicious websites in a real-world Web environment for three months. These experimental results illustrate that the proposed method has a better performance and is applicable to a practical Web environment.

The ClueWeb09 dataset and its successor, the ClueWeb12 dataset, are two of the largest collections of Web pages released by Text REtrieval Conference (TREC). The ClueWeb datasets were used in various tracks of TREC ran through 2009 to 2017. For every year, approximately 50 new queries are released and a pool of Web pages are judged against these queries by human assessors as relevant, non-relevant or spam. In this article, a ground truth for binary classification (spam vs non-spam) is constructed from Web pages that are judged as spam or relevant under the assumption that a Web page judged as relevant for any query cannot be spam. Based on this ground truth, we evaluate classification performances of the Waterloo spam rankings (Fusion, Britney, GroupX and UK2006), which have been traditionally used to identify and filter spam pages in retrieval systems. The experimental results in terms of the universal binary classification evaluation measures suggest that the Fusion (with threshold = 11%) is the best for the ClueWeb09 dataset. Analysis of the frequency distributions of relevant/spam documents over spam scores reveals that the GroupX is the most powerful at identifying relevant documents, whereas the Fusion is the most powerful at identifying spam documents. It is also confirmed that the effectiveness of the Fusion spam ranking of the ClueWeb12 dataset is not as good as that of the ClueWeb09.

Background: With the rapid development of the Internet, the number of web spam has increased dramatically in recent years, which has wasted search engine storage and computing power on a massive scale. To identify the web spam effectively, the content features, link features, hidden features and quality features of web page are integrated to establish the corresponding web spam identification index system. However, the index system is highly correlation dimension. Methods: An improved method of autoencoder named stacked autoencoder neural network (SAE) is used to realize the reduction of the web spam identification index system. Results: The experiment results show that our method could reduce effectively the index of web spam and significantly improves the recognition rate in the following work. Conclusion: An autoencoder based web spam indexes reduction method is proposed in this paper. The experimental results show that it greatly reduces the temporal and spatial complexity of the future web spam detection model.

Web spam is an illegal and immoral way to increase the ranking of web pages by deceiving search engine algorithms. Therefore, different methods have been proposed to detect and improve the quality of results. Since a web page can be viewed from two aspects of the content and the link, the number of extracting features is high. Thus, selection of features with high separating ability can be considered as a preprocessing step in order to decrease computational time and cost. In this study, a new backward elimination approach is proposed for feature selection. The main idea of this method is measuring the impact of eliminating a set of features on the performance of a classifier instead of a single feature which is similar to the sequential backward selection. This method seeks for the largest feature subset that their omission from whole set features not only reduces the efficiency of the classifier but also improves it. Implementations on WEBSPAM-UK2007 dataset with Naive Bayes classifier show that the proposed method selects fewer features in comparison with other methods and improves the performance of the classifier in the IBA index about 7%.

From the past few years, there is an exponential increase in one of the most popular technologies of the modern era called as Internet of Things (IoT). In IoT, various objects perform the tasks of sensing, communication, and computation for providing uninterrupted services (e.g., e-health, e-transportation, security access, etc.) to the end users. In this era, Cognitive Internet of Things (CIoT) is an another paradigm of IoT developed to enhance the capabilities of intelligence in IoT objects where these objects can take independent decisions in any environment. IoT follows the service oriented architecture (SOA), in which the application layer is the topmost layer. It enables the IoT objects to interact with the other objects located across the globe. The power of learning, thinking, and understanding by these objects, can make the information access more accurate and reliable but Web spam is one of the challenges while accessing information from the web. It has been observed from the literature review that search engines are preferred mostly by the people for accessing information. The efficient ranking by the search engines can reduce the computational cost of information exchange by IoT objects. Search engines should be able to prevent the spam from being injected into the web. But, the existing techniques for this problem target in finding the spam after its occurrence in search engine result pages. So, in this proposal, we present an intelligent cognitive spammer framework, Cognitive spammer, which eliminates the spam pages during the web page rank score calculation by search engines. The framework update the Google’s ranking algorithm, PageRank in such a way that it automatically prevents link spam by considering the link structure of web for rank score computation. The updated PageRank algorithm provided the better ranking of web pages. The proposed framework is validated with the WEBSPAM-UK2007 dataset. Before processing, the dataset is preprocessed with a new technique, called as ‘Split by Over-sampling and Train by Under-fitting’ to remove the trade off between imbalanced instances of target class. After data cleaning, we applied machine learning techniques (Bagged model, Boosted linear model, etc) with the web page features to make accurate predictions. The detection classifiers only consider the link features of the web page irrespective of the page content. Out of the fifteen classifiers, best three are ensemble, which results in better performance with overall accuracy improvement. Ten-fold cross validation has also been applied with the resulted ensemble model, which results in getting the accuracy of 99.6% in the proposed scheme.

Feature selection is an important issue in data mining, and it is used to reduce dimensions of features set. Web spam detection is one of research fields of data mining. With regard to increasing available information in virtual space and the need of users to search, the role of search engines and used algorithms are important in terms of ranking. Web spam is an illegal method to increase mendacious rank of internet pages by deceiving the algorithms of search engines, so it is essential to use an efficient method. Up to now, many methods have been proposed to face with web spam. An ensemble feature selection method has been proposed in this paper to detect web spam. Content features of standard dataset of WEBSPAM-UK2007 are used for evaluation. Bayes network classifier is used along with 70-30% training-testing spilt of dataset. The presented results show that AUC of this method is higher than the other methods reported in this paper. Moreover, the best values of evaluation metrics in our proposed method are optimal in comparison to the other methods reported in this paper. In addition, it improves classification metrics in comparison to basic feature selection methods.

Researches on detecting Web spam are in full swing. However, very high feature dimension and sensitive information leakage restrict the mining. In this paper, a cascade feature selection for mining spam is proposed, which bases on Privacy Preservation (PP) method and Genetic Algorithm (GA). Two criteria, privacy protection degree and maximum classification reliability, are used to pick the representative features to form an optimal minimum feature subset. Discretization, data balancing, feature selection, and ensemble learning method are integrated to detect Web spam. The approach not only greatly reduces the data dimension but also protects the sensitive features from detection. Good spam detection performance is achieved by using only 22 features.

Web spam is one among the main problems of search engines because it reduces the standard of the online page. Web spam also effects economically because spammers / attackers provide an oversized free ad data or websites on the search engines that results in a rise within the web traffic. There are certain ways to tell apart such spam pages and one among them is using classification techniques. Relative examination of web spam detection using classification algorithms like, Random Forest and LAD Tree, C4.5 and Naive bayes is presented here during this paper. Analyses were completed on highlight sets of all around acknowledged dataset WEB SPAM UK-2007 utilizing WEKA. When classification was refrained from feature selection some classifier were high on false positive rate and time taken to create model but when feature selection was applied to datasets results were optimized and Random Forest outperformed on all the datasets altogether parameters that were selected.

Internet is an excellent and best way to represent and platform for sharing information for communication to each other in every field because without internet or technology not to share information as well. Using this technology user can facilitated by search engines in order to meet the user requirements in the hunt for in rank. Search engines are the “dragons” with the precious fortune that is useful information. It is a fact that web contains huge amount of information, but is customized to solve the user queries with less amount of results which is 10 to 20 pages. It is a responsibility of Search engines that they should rank the Web pages, as per the quality of data they contain. In network spam can extensively depreciate the excellence of result presented by search engine. It has been noticed that commercial search engines charge large amount of incentives for efficient spam detection. After using internet it has been considered as one of the major challenge in the industry of search engine. This is the problem of finding or reducing web spam over the internet that may be content spam or link spam. For this newly developed spam techniques, it is necessary to adapt Machine learning-based classification methods. After future technology we deal with the facts of uncertainty and vagueness one of the applications artificial intelligence with cognitive sciences is used as a tool for solving such kind of problem. Designing the series of algorithm to enhance the performance of the given problem is the main objective of this research.

PageRank algorithm is based on network link to calculate the importance of the web pages. It is successful for web search, but there are still many problems, such as web spam, invalid links etc. It can not be a good description of the user's behavior. In this paper, through the analysis of thousands of user's behavior data, a semi-Markov process method is proposed to simulate the user browsing behavior, and its stationary probability distribution can be used as the measure of page im-portance. At the taking the influence of web content and length into consideration and com-bining with traditional hyperlink analysis, the new web pages are reconsidered and the old web ones are ranked appropriately. The results show that degree of satisfaction by this method is increased by about24% compared with PageRank algorithm, which can be better to calculate the importance of web pages.

Web spam detection is one of the most important and challenging tasks in web search. Since web spam pages tend to have a lot of spurious links, many web spam detection algorithms exploit the hyperlink structure between the web pages to detect the spam pages. In this paper, we conduct a comprehensive analysis of the link structure of web spam using real-world web graphs to systemically investigate the characteristics of the link-based web spam. By exploring the structure of the page-level graph as well as the site-level graph, we propose a scalable site-level seeding methodology for the Anti-TrustRank (ATR) algorithm. The key idea is to map a website into a feature space where we learn a classifier to prioritize the websites so that we can effectively select a set of good seeds for the ATR algorithm. This seeding method enables the ATR algorithm to detect the largest number of spam pages among the competitive baseline methods. Furthermore, we design work-efficient asynchronous ATR algorithms which are able to significantly reduce the computational cost of the traditional ATR algorithm without degrading the performance in detecting spam pages while guaranteeing the convergence.

Web spam refers to some techniques, which try to manipulate search engine ranking algorithms in order to raise web page position in search engine results. In the best case, spammers encourage viewers to visit their sites, and provide undeserved advertisement gains to the page owner. In the worst case, they use malicious contents in their pages and try to install malware on the victims machine. Spammers use three kinds of spamming techniques to get higher score in ranking. These techniques are Link based techniques, hiding techniques and content-based techniques. Existing spam pages cause distrust to search engine results. This not only wastes the time of visitors, but also wastes lots of search engine resources. Hence spam detection methods have been proposed as a solution for web spam in order to reduce negative effects of spam pages. Experimental results show that some of these techniques are working well and can find spam pages more accurate than the others. This paper classifies web spam techniques and the related detection methods., Comment: 16 pages, 7 figures, 7 tables

To the modern Search Engines (SEs), one of the biggest threats to be considered is spamdexing. Nowadays spammers are using a wide range of techniques for content generation, they are using content spam to fill the Search Engine Result Pages (SERPs) with low-quality web pages. Generally, spam web pages are insufficient, irrelevant and improper results for users. Many researchers from academia and industry are working on spamdexing to identify the spam web pages. However, so far not even a single universally efficient method is developed for identification of all spam web pages. We believe that for tackling the content spam there must be improved methods. This article is an attempt in that direction, where a framework has been proposed for spam web pages identification. The framework uses Stop words, Keywords Density, Spam Keywords Database, Part of Speech (POS) ratio, and Copied Content algorithms. For conducting the experiments and obtaining threshold values WEBSPAM-UK2006 and WEBSPAM-UK2007 datasets have been used. An excellent and promising F-measure of 77.38% illustrates the effectiveness and applicability of proposed method.

Search engine are being used by the most of the World population as their basic information retrieval system for getting useful information from internet. As a service provider who uses internet for digital marketing it becomes mandatory to get high ranks from search engines. Search engines optimization (SEO) techniques are used for this purpose. Black hat SEO techniques are used for quick results but are prohibited by most of the search engines. Hence web space users or website developers should be well aware of SEO techniques and how to use them in optimal way. This paper presents some of the most commonly used black hat SEO techniques and the counter measures done by different search engines to prohibit them.

With the proliferation of web spam and infinite autogenerated web content, large-scale web crawlers require low-complexity ranking methods to effectively budget their limited resources and allocate bandwidth to reputable sites. In this work, we assume crawls that produce frontiers orders of magnitude larger than RAM, where sorting of pending URLs is infeasible in real time. Under these constraints, the main objective is to quickly compute domain budgets and decide which of them can be massively crawled. Those ranked at the top of the list receive aggressive crawling allowances, while all other domains are visited at some small default rate. To shed light on Internet-wide spam avoidance, we study topology-based ranking algorithms on domain-level graphs from the two largest academic crawls: a 6.3B-page IRLbot dataset and a 1B-page ClueWeb09 exploration. We first propose a new methodology for comparing the various rankings and then show that in-degree BFS-based techniques decisively outperform classic PageRank-style methods, including TrustRank. However, since BFS requires several orders of magnitude higher overhead and is generally infeasible for real-time use, we propose a fast, accurate, and scalable estimation method called TSE that can achieve much better crawl prioritization in practice. It is especially beneficial in applications with limited hardware resources.

In the modern era, Internet of Things(IoT) plays an important role in connecting the people across the globe. The IoT objects enable the communication and data exchange among each other irrespective of their geographical locations. In such an environment, the Web of Things (WoT) provides the Internet service to the IoT objects. The Internet is mostly accessed by the search engines. The success of search engine depends upon the ranking algorithm. Although, Google is preferred by the maximum Internet users, but still the Google’s ranking algorithm, PageRank experiences the occurrence of spam web pages. In this paper, the webpage filtering algorithm is proposed which automatically detects the spam web pages. The spam webpages are detected before these are processed by the ranking module of search engines. The machine learning model, i.e., decision tree is used for the validation of the proposed scheme. The ten fold cross validation approach is used to improve the accuracy of model, i.e., 98.2%. The results obtained demonstrate that the proposed scheme has the power of preventing the spam web pages in Cognitive Internet of Things (CIoT) environment.

Web spam is a serious threat for both end-users and search engines (w.r.t., query cost). Webgraphs can be exploited in detecting spam. In the past, several graph mining techniques were applied to measure metrics for pages and hyperlinks. In this paper, we justify the importance of webgraph to distinguish spam websites from non-spam ones based on several graph metrics computed for a labelled dataset (WEBSPAM-UK2007) and justify our model by testing on uk-2014 dataset, the most recently available dataset on the same (uk) domain. WEBSPAM-UK2007 dataset includes 0.1 million different hosts and four kinds of feature sets: Obvious, Link, Transformed Link and Content. We use five prominent machine learning (ML) techniques (i.e., Support Vector Machine (SVM), K-Nearest Neighbor (KNN), Logistic Regression, Naive Bayes and Random Forest) to build a ML-based classifier. To evaluate the performance of our classifier, we compute accuracy and F-1 score and perform 10-fold cross validation. We also compare graph based features with content based textual features and find that graph properties are similar or better than text properties. We achieve above 99% training accuracy for most of our machine learning models. We test our model with uk-2014 dataset with 4.7 million hosts for the graph-based feature sets and achieve accuracy in between 90-94% for most of the models. To the best of our knowledge, prior works on web spam detection with WEBSPAM-UK2007 dataset did not use different test dataset for their models. Our model classifier is capable of detecting web spam for any input webgraph based on its graph metrics features.

DBN- based classification model is proposed in this paper for web spam detection.SMOTE and DAE algorithms is applied in DBN to improve the classification performanceThe content and link feature is combined.The results obtained in this paper is better than the existing systems. With the development of the Internet, the number of web spam increases gradually, which has seriously affected the user experience of search engines. To improve the classification performance of web spam, the deep belief networks (DBN) is used for the first time, and it is effectively combined with the Synthetic Minority Over-Sampling Technique (SMOTE) and De-Noising Auto-Encoder (DAE) algorithm after the multi-aspect research and consideration. After multiple sets of experiments on WEBSPAM-UK2007 dataset, the results show that the classification method proposed in this paper improves the classification performance to a certain extent, which provides a good direction for the future classification of web spam.

This paper proposes Swarm based hybrid technique CFS-PSO, which combines the characteristics of Correlation Based Feature Selection Technique (CFS) and Particle Swarm Optimization (PSO) strategy. PSO is an optimization approach motivated by swarm conduct which uses the real-number randomness & the global communication among the swarm particles. Feature selection (pre-processing technique) is very crucial part of Data Mining & Machine Learning.The aims of feature selection includes building of simpler & more logical models and improving the performance in terms of reducing the time to build the learning model and increasing the accuracy. We assess the performance of CFS-PSO on WEBSPAM-UK2006 with five classifiers. Experimental results show reduction in original features & increasing the F-measure upto 88% & 45.83% respectively.

The Web spam identification problem can be modeled as an instance of the conventional classification problem. Web spams aim at deceiving web crawlers by advertising certain Web pages through elevation of their page rankings superficially than their actual weights. Web spams are intended to produce fraudulent results of web search queries and degenerate the client’s experience by directing users to fake Web pages. We present a fuzzy evidence-based methodology for identifying Web spams by which the spamicity of web hosts is formulated as a reasoning problem in the presence of uncertainty. However, any classification task intrinsically suffers from incomplete or vague evidence and ambiguity in the class assignment based on evidence. In this work, we combine fuzzy reasoning as the decision maker for selecting the most suitable evidence in a multi-source Dempster–Shafer (DS) based classification algorithm. The introduced approach has the benefit of providing more reliable solution to detect spams without any prior information. The evidence theory offers flexible support that takes into account the multi-dimensional nature of implementation decisions. The experimental results show that the fuzzy reasoning in combination with DS theory, reduces the conflicts among evidence leading to enhanced classification results. The aim of this paper is to describe the potential of fuzzy reasoning and the Dempster–Shafer Theory (DST) as a decision model for the web spams classification problem.

With Internet becoming an inevitable part of the global community and the huge amount of data resources present on the internet, the problem of malicious content present has become very serious. These spams can be introduced through many ways on the internet. Web spams involve the hike in the search engine index by manipulating the content of the web pages. To tailor the demands of the tricky spam algorithms, the hybrid use of crucial spamming features is inevitable.This paper presents the classification of web pages as spam, undecided and non-spam using a fuzzy classifier by evaluating the spamicity, the degree to which a web page is spammed. In order to gauge the performance of the proposed scheme, we evaluated the accuracy of our classifier and also performed comparative analysis with jrip classifier and naive Bayesian classifier through experiments.

Web spam pages exploit the biases of search engine algorithms to get higher than their deserved rankings in search results by using several types of spamming techniques. Many web spam demotion algorithms have been developed to combat spam via the use of the web link structure, from which the goodness or badness score of each web page is evaluated. Those scores are then used to identify spam pages or punish their rankings in search engine results. However, most of the published spam demotion algorithms differ from their base models by only very limited improvements and still suffer from some common score manipulation methods. The lack of a general framework for this field makes the task of designing high-performance spam demotion algorithms very inefficient. In this paper, we propose a unified score propagation model for web spam demotion algorithms by abstracting the score propagation process of relevant models with a forward score propagation function and a backward score propagation function, each of which can further be expressed as three sub-functions: a splitting function, an accepting function and a combination function. On the basis of the proposed model, we develop two new web spam demotion algorithms named Supervised Forward and Backward score Ranking (SFBR) and Unsupervised Forward and Backward score Ranking (UFBR). Our experiments, conducted on three large-scale public datasets, show that (1) SFBR is very robust and apparently outperforms other algorithms and (2) UFBR can obtain results comparable to some well-known supervised algorithms in the spam demotion task even if the UFBR is unsupervised.

Web spam is a negative practice carried out by spammers to produce fake searchengines results for improving rank position of their Web pages. It is available on arena of World Wide Web (WWW) in different forms and lacks a consistent definition. The search engines are struggling to eliminate spam pages through machine learning (ML) detectors. Mostly, search engines measure the quality of websites by using different factors (signals) such as, number of visitors, body text, anchor text, back link and forward link etc. information and, and spammers try to induce these signals into their desired pages to subvert ranking function of search engines. This study compares the detection efficiencyof different ML classifiers trained and tested on WebSpam UK2007 data set. The results of our study show that random forest has achieve higher score than other well-known classifiers.

Currently, web spamming is a serious problem for search engines. It not only degrades the quality of search results by intentionally boosting undesirable web pages to users, but also causes the search engine to waste a significant amount of computational and storage resources in manipulating useless information. In this paper, we present a novel ensemble classifier for web spam detection which combines the clonal selection algorithm for feature selection and under-sampling for data balancing. This web spam detection system is called USCS. The USCS ensemble classifiers can automatically sample and select sub-classifiers. First, the system will convert the imbalanced training dataset into several balanced datasets using the under-sampling method. Second, the system will automatically select several optimal feature subsets for each sub-classifier using a customized clonal selection algorithm. Third, the system will build several C4.5 decision tree sub-classifiers from these balanced datasets based on its specified features. Finally, these sub-classifiers will be used to construct an ensemble decision tree classifier which will be applied to classify the examples in the testing data. Experiments on WEBSPAM-UK2006 dataset on the web spam problem show that our proposed approach, the USCS ensemble web spam classifier, contributes significant classification performance compared to several baseline systems and state-of-the-art approaches.

Web spam uses numerous techniques to misguide Web search engines in exchange of financial profit. A myriad of semi-automatic propagation model has been proposed with the purpose of combating Web spam. In this paper, distrust propagation is used to detect Web spam. An automatic distrust seed set propagation algorithm (DSP), which acts as an extension to the seed set to propagate distrust further to detect more Web spam. Experiments are conducted on WEBSPAM-UK2006 and WEBSPAM-UK2007 dataset; the results have shown that DSP enhanced the baseline algorithms and detected 17.73 % more spam hosts in the former dataset and detected 8.59 % more spam hosts in later dataset.

In the mobile Internet era, users access interesting information in a continuous manner rather than as one-time results through search engines. The traditional link-based ranking algorithms typically return the relevant “popular”web pages. The current, most important web pages are ranked lower than these pages. Furthermore, most of the results are repeated when the user submits the same query days later. In this paper, we have described a novel service called tracking engine. The tracking engine allows users to enter and save queries, displays time-sensitive information, and notifies users when new, relevant information appears. To the best of our knowledge, this is the first solution seen in such a service in the mobile Internet era. First, our tracking engine called Tianji crawls the web pages based on time priority and constructs a new index structure, which enables a faster match of web pages to related keywords. Then, we develop a ranking model based on the correlation between time and importance. The experimental results show that the ranking model of Tianji has better performance than existing time-sensitive ranking methods in terms of timeliness and relevance.

Many machine-controlled Twitter accounts (also called "Sybils") are created each day to provide services, flood out messages for astroturf political campaigns, write fake product reviews, or produce an underground marketplace for purchasing Twitter followers, retweets, or URL advertisements. In addition, fake identities and user accounts in online communities are resources used by adversaries to spread malware, spam, and harmful links over social networks. In social networks, Sybil detectors rely on the assumption that Sybils will find it harder to befriend real users; thus, Sybils that are connected to each other form strongly connected subgraphs, which can be detected using the graph theory. However, a majority of Sybils have actually successfully integrated themselves into real social media user communities (such as Twitter and Facebook). In this study, we compared the current methods used for detecting Sybil accounts. We also explored the detection features of various types of Twitter Sybil accounts in order to build an effective and practical classifier. To evaluate our classifier, we collected and manually labeled a dataset of Twitter accounts, including human users, bots, and hybrids (i.e., tweets posted by both human and bots). We consider that this Twitter Sybils corpus will help researchers to conduct high-quality measurement studies. We also developed a browser plug-in, which utilizes our classifier and warns the user about possible Sybil accounts before accessing or following them after clicking on a Twitter account.

Traditional malicious website detection schemes no longer work effectively, for miscellaneous Web spam techniques are widely used and relentlessly innovative. This paper analyzes typical spam techniques and proposes a novel detection scheme, which origins from end-user perspective and adopts Convolutional Neural Network as a key element for precise image recognition.

Misspelled keywords have become an appealing target in search poisoning, since they are less competitive to promote than the correct queries and account for a considerable amount of search traffic. Search engines have adopted several countermeasure strategies, e.g., Google applies automated corrections on queried keywords and returns search results of the corrected versions directly. However, a sophisticated class of attack, which we term as linguistic-collision misspelling, can evade auto-correction and poison search results. Cybercriminals target special queries where the misspelled terms are existent words, even in other languages (e.g., "idobe", a misspelling of the English word "adobe", is a legitimate word in the Nigerian language). In this paper, we perform the first large-scale analysis on linguistic-collision search poisoning attacks. In particular, we check 1.77 million misspelled search terms on Google and Baidu and analyze both English and Chinese languages, which are the top two languages used by Internet users. We leverage edit distance operations and linguistic properties to generate misspelling candidates. To more efficiently identify linguistic-collision search terms, we design a deep learning model that can improve collection rate by 2.84x compared to random sampling. Our results show that the abuse is prevalent: around 1.19% of linguistic-collision search terms on Google and Baidu have results on the first page directing to malicious websites. We also find that cybercriminals mainly target categories of gambling, drugs, and adult content. Mobile-device users disproportionately search for misspelled keywords, presumably due to small screen for input. Our work highlights this new class of search engine poisoning and provides insights to help mitigate the threat.

Classifying imbalanced data with traditional classifiers is a huge challenge now-a-days. Imbalance data is a situation wherein the ratio of data within classes is not same. Many real life situations deal with such problems e.g. Web spam detection, Credit card frauds, and Fraudulent telephone calls. The problem exists everywhere when our objective is to identify exceptional cases. The problem is handled by researchers either by modifying the existing classifications methods or by developing new methods. This paper review ensemble based approaches (Boosting and Bagging based) designed to address imbalance in classes by focusing on binary classification. We compared 6 Boosting based, 7 Bagging based and 2 hybrid ensembles for their performance in imbalance domain. We use KEEL tool to evaluate the performance of these methods by implementing the methods on seven imbalance data having class imbalance ratio from 1.82 to as high as 129.44. Area Under the curve (AUC) parameter is recorded as the performance metric. We also statistically analyzed the methods using Friedman rank test and Wilcoxon Matched Pair signed rank test to strengthen the visual interpretations. After analysis, it is proved that RusBoost ensemble outperformed every other ensemble in the imbalanced data situations.

In modern era, Internet plays a key role in accessing and fetching web information and web resources from World Wide Web (WWW). The websites act as a medium for retrieving information from the web. Although it increases the data retrieval and users interactions, it also opens the gate for various types of attacks. For example, spams in the websites attract various Internet users. It has been observed from the literature that many authors attempted to detect the web spam using various machine learning techniques. However, none of these techniques used deep learning architecture for detection of hidden patterns. Hence, in this paper, a deep learning algorithm, i.e., Recurrent Neural Networks (RNN), has been used for the classification of spam nodes. We devise here a framework called FS2RNN: Feature Selection Scheme using Recurrent Neural Networks. In this framework, the dataset is preprocessed before applying RNN in which principal component analysis (PCA) is used for dimension reduction on the data set and recursive feature elimination (RFE) is used for feature selection. The accuracy of the proposed framework, when compared before and after preprocessing, is improved by 24.2 %, which is excellent result.

Spamdexing is the art of black hat SEO. Features which are more influential for high rank and visibility are manipulated for the SEO task. The motivation behind the work is utilizing the state of the art Website optimization features to enhance the performance of spamdexing detection. Features which play a focal role in current SEO strategies show a significant deviation for spam and non-spam samples. This paper proposes 44 features named as NLSDF (New Link Spamdexing Detection Features). Social media creates an impact in search engine results ranking. Features pertaining to the social media were incorporated with the NLSDF features to boost the recital of the spamdexing classification. The NLSDF features with 44 attributes along with 5 social media features boost the classification performance of the WEBSPAM-UK 2007 dataset. The one tailed paired t-test with 95% confidence, performed on the AUC values of the learning models shows significance of the NLSDF.

Purpose – The purpose of this paper is statistical examination of nearly 7,000 web directories and an analysis of factors which affect their quality measured by PageRank. Design/methodology/approach – The authors analysed 6,821 directories registered at www.katalogiseo.info/. The following information about the directories was examined: the year of registration on the website, the directory’s PageRank value, the existence of an active IP address, backlink requests, a fee charged for submission to the directory, as well as directory moderation and subject. Statistical analyses were performed with the use of Microsoft Excel, version 2010, and R software, version 3.0.0. The PageRank values were collected with a software written in Python. Findings – The study has shown a gradual increase in popularity of directories as one of the basic tools in search engine optimisation. The analysis has indicated a relatively high percentage of spam web directories. The evidence of this is the number of directories with undetermined PageRank values. The study revealed that careful management of a directory and its subject have key impact on directory quality measured by PageRank. Originality/value – Relatively few publications focus on the problem of web directories which represent a very large group of websites created solely to manipulate web search engine rankings. This paper discusses the phenomenon of web directories, reveals the percentage of spam directories, and factors which affect their quality measured by PageRank.

The Smart Drug Search is publicly accessible at http://sing.ei.uvigo.es/sds/. The BIOMedical Search Engine Framework is freely available for non-commercial use at https://github.com/agjacome/biomsef, Background and Objectives: Text mining and semantic analysis approaches can be applied to the construction of biomedical domain-specific search engines and provide an attractive alternative to create personalized and enhanced search experiences. Therefore, this work introduces the new open-source BIOMedical Search Engine Framework for the fast and lightweight development of domain-specific search engines. The rationale behind this framework is to incorporate core features typically available in search engine frameworks with flexible and extensible technologies to retrieve biomedical documents, annotate meaningful domain concepts, and develop highly customized Web search interfaces. Methods: The BIOMedical Search Engine Framework integrates taggers for major biomedical concepts, such as diseases, drugs, genes, proteins, compounds and organisms, and enables the use of domain-specific controlled vocabulary. Technologies from the Typesafe Reactive Platform, the AngularJS JavaScript framework and the Bootstrap HTML/CSS framework support the customization of the domain-oriented search application. Moreover, the RESTful API of the BIOMedical Search Engine Framework allows the integration of the search engine into existing systems or a complete web interface personalization. Results The construction of the Smart Drug Search is described as proof-of-concept of the BIOMedical Search Engine Framework. This public search engine catalogs scientific literature about antimicrobial resistance, microbial virulence and topics alike. The keyword-based queries of the users are transformed into concepts and search results are presented and ranked accordingly. The semantic graph view portraits all the concepts found in the results and the researcher may look into the relevance of different concepts, the strength of direct relations, and non-trivial, indirect relations. The number of occurrences of the concept shows its importance to the query, and the frequency of concept co-occurrence is indicative of biological relations meaningful to that particular scope of research. Conversely, indirect concept associations, i.e. concepts related by other intermediary concepts, can be useful to integrate information from different studies and look into non-trivial relations. Conclusions The BIOMedical Search Engine Framework supports the development of domain-specific search engines. The key strengths of the framework are modularity and extensibility in terms of software design, the use of open-source consolidated Web technologies, and the ability to integrate any number of biomedical text mining tools and information resources. Currently, the Smart Drug Search keeps over 1,186,000 documents, containing more than 11,854,000 annotations for 77,200 different concepts., This work was partially funded by the [14VI05] ContractProgramme from the University ofVigo and theAgrupamento INBIOMED from DXPCTSUG-FEDER unha maneira de facer Europa(2012/273).The research leading to these results has also received funding from the European Union Seventh Framework Programme FP7/REGPOT-2012-2013.1 under grant agreement n° 316265,BIOCAPS.This document reflects only the author’s views, and the European Union is not liable for any use that may be made of the information contained herein

world, with the increased use of internet the large volume of data is stored on World Wide Web. To use this large data the different search engines are provided. But the accuracy of the data is again based on the appropriate search query submitted by the user to search engine. Depending on the search query the search engine retrieves the massive amount of relevant data by using different algorithms such as page rank algorithm or relevancy algorithm. Further, the returned results decide the performance as well as the efficiency of the search engine. Search result clustering problem means clustering the search results returned by the search engine. In this paper a comparative analysis of Suffix Tree Clustering algorithms is done to decide the how accurately it clusters the search results i.e. an empirical analysis which is done by using standard datasets.

Human life easy with web search engines. But with day to day growth in web, proper information on the user search query is becoming difficult. User search some information on the web, search engines results in a large amount of web pages from his database in response of search related queries. For any user not possible to access all web pages. So need of a search engines (SE) that provide relevant information to a user search query. The technique called search engine optimization (SEO), to do so. SEO is a method that uses search principles for search engines to provide the higher ranking to the more suitable webpage. It can be done by applying various pages ranking algorithms. In this paper present a review on SEO techniques that are helpful for internet user to make available valuable data.

SEO stands for "search engine optimization." It is the process of getting traffic from the free, organic, editorial or natural search results on search engines. The earlier (or higher ranked on the search results page), and more frequently a site appears in the search results list, the more visitors it will receive from the search engine's users, and these visitors can be converted into customers. SEO may target different kinds of search, including image search, local search, video search, academic search, news search and industry-specific vertical search engines. As an Internet marketing strategy, SEO considers how search engines work, what people search for, the actual search terms or keywords typed into search engines and which search engines are preferred by their targeted audience. Optimizing a website may involve editing its content, HTML and associated coding to both increase its relevance to specific keywords and to remove barriers to the indexing activities of search engines. Promoting a site to increase the number of backlinks, or inbound links, is another SEO tactic. As of May 2015, mobile search has finally surpassed desktop search, Google is developing and pushing mobile search as the future in all of its products and many brands are beginning to take a different approach on their internet strategies.