Search

An IT system generates messages for other systems or users to consume, through direct interaction or as system logs. Automatically identifying the types of these machine-generated messages has many applications, such as intrusion detection and system behavior discovery. Among various heuristic methods for automatically identifying message types, the clustering methods based on keyword extraction have been quite effective. However, these methods still suffer from keyword misidentification problems, i.e., some keyword occurrences are wrongly identified as payload and some strings in the payload are wrongly identified as keyword occurrences, leading to the misidentification of the message types. In this paper, we propose a new machine language processing (MLP) approach, called P-gram, specifically designed for identifying keywords in, and subsequently clustering, machine-generated messages. First, we introduce a novel concept and technique, positional n-gram, for message keywords extraction. By associating the position as meta-data with each n-gram, we can more accurately discern which n-grams are keywords of a message and which n-grams are parts of the payload information. Then, the positional keywords are used as features to cluster the messages, and an entropy-based positional weighting method is devised to measure the importance or weight of the positional keywords to each message. Finally, a general centroid clustering method, K-Medoids, is used to leverage the importance of the keywords and cluster messages into groups reflecting their types. We evaluate our method on a range of machine-generated (text and binary) messages from the real-world systems and show that our method achieves higher accuracy than the current state-of-the-art tools.

We present GHTraffic, a dataset of significant size comprising HTTP transactions extracted from GitHub data and augmented with synthetic transaction data. The dataset facilitates reproducible research on many aspects of service-oriented computing. This paper discusses use cases for such a dataset and extracts a set of requirements from these use cases. We then discuss the design of GHTraffic, and the methods and tool used to construct it. We conclude our contribution with some selective metrics that characterise GHTraffic., Comment: 8 pages, 5 figures

Software service emulation is an emerging technique for creating realistic executable models of server-side behaviour. It is particularly useful in quality assurance and DevOps, replicating production-like conditions for large-scale enterprise software systems. Existing approaches can automatically build client-server and server-server interaction models of complex software systems directly from analysis of service interaction trace data. However, when these interaction traces become large, searching an entire trace library to generate a run-time responses can become very slow. In this paper we describe a new technique that utilises data mining, specifically clustering algorithms, to pre-process large amounts of recorded interaction trace data. With the obtained clusters we facilitate efficient yet well-formed runtime response generation in our Enterprise System emulation environment. We evaluate our approach using two common application-layer protocols: LDAP and SOAP. Our experimental results show that by utilising clustering techniques in the pre-processing step, the response generation time can be reduced by 99% on average compared with existing approaches.

When operating in volatile environments, service-based systems (SBSs) that are dynamically composed from component services must be monitored in order to guarantee timely and successful delivery of outcomes in response to user requests. However, monitoring consumes resources and very often impacts on the quality of the SBSs being monitored. Such resource and system costs need to be considered in formulating monitoring strategies for SBSs. The critical path of a composite SBS, i.e., the execution path in the service composition with the maximum execution time, is of particular importance in cost-effective monitoring as it determines the response time of the entire SBS. In volatile operating environments, the critical path of an SBS is probabilistic, as every execution path can be critical with a certain probability, i.e., its criticality. As such, it is important to estimate the criticalities of different execution paths when deciding which parts of the SBS to monitor. Furthermore, cost-effective monitoring also requires management of the trade-off between the benefit and cost of monitoring. In this paper, we propose CriMon, a novel approach to formulating and evaluating monitoring strategies for SBSs. CriMon first calculates the criticalities of the execution paths and the component services of an SBS and then, based on those criticalities, generates the optimal monitoring strategy considering both the benefit and cost of monitoring. CriMon has two monitoring strategy formulation methods, namely local optimisation and global optimisation. In-lab experimental results demonstrate that the response time of an SBS can be managed cost-effectively through CriMon-based monitoring. The effectiveness and efficiency of the two monitoring strategy formulation methods are also evaluated and compared.

Cloud computing is revolutionizing the IT industry by enabling them to offer access to their infrastructure and application services on a subscription basis. As a result, several enterprises including IBM, Microsoft, Google, and Amazon have started to offer different Cloud services to their customers. Due to the vast diversity in the available Cloud services, from the customer's point of view, it has become difficult to decide whose services they should use and what is the basis for their selection. Currently, there is no framework that can allow customers to evaluate Cloud offerings and rank them based on their ability to meet the user's Quality of Service (QoS) requirements. In this work, we propose a framework and a mechanism that measure the quality and prioritize Cloud services. Such a framework can make a significant impact and will create healthy competition among Cloud providers to satisfy their Service Level Agreement (SLA) and improve their QoS. We have shown the applicability of the ranking framework using a case study.

Collection of dynamic information requires that malware be executed in a controlled environment; the malware unpacks itself as a preliminary to the execution process. On the other hand, while execution of malware is not needed in order to collect static information, the file must first be unpacked manually. None-the-less, if a file has been executed, it is possible to use both static and dynamic information in designing a single classification method. In this paper, we present the first classification method integrating static and dynamic features into a single test. Our approach improves on previous results based on individual features and reduces by half the time needed to test such features separately. Robustness to changes in malware development is tested by comparing results on two sets of malware, the first collected between 2003 and 2007, and the second collected between 2009 and 2010. When classifying the older set as compared to the entire data set, our integrated test demonstrates significantly more robustness than previous methods by losing just 2.7% in accuracy as opposed to a drop of 7%. We conclude that to achieve acceptable accuracy in classifying the latest malware, some older malware should be included in the set of data.

Large enterprise software systems make many complex interactions with other services in their environment. Developing and testing for production-like conditions is therefore a very challenging task. Current approaches include emulation of dependent services using either explicit modelling or record-and-replay approaches. Models require deep knowledge of the target services while record-and-replay is limited in accuracy. Both face developmental and scaling issues. We present a new technique that improves the accuracy of record-and-replay approaches, without requiring prior knowledge of the service protocols. The approach uses Multiple Sequence Alignment to derive message prototypes from recorded system interactions and a scheme to match incoming request messages against prototypes to generate response messages. We use a modified Needleman-Wunsch algorithm for distance calculation during message matching. Our approach has shown greater than 99% accuracy for four evaluated enterprise system messaging protocols. The approach has been successfully integrated into the CA Service Virtualization commercial product to complement its existing techniques., In Proceedings of the 38th International Conference on Software Engineering Companion (pp. 202-211). arXiv admin note: text overlap with arXiv:1510.01421

Constructing testbeds for systems which are interconnected with large networks of other software services is a challenging task. It is particularly difficult to create testbeds facilitating evaluation of the non-functional qualities of a system, such as scalability, that can be expected in production deployments. Software service emulation is an approach for creating such testbeds where service behaviour is defined by emulate-able models executed in an emulation runtime environment. We present (i) a meta-modelling framework supporting emulate-able service modelling (including messages, protocol, behaviour and states), and (ii) Kaluta, an emulation environment able to concurrently execute large numbers (thousands) of service models, providing a testbed which mimics the behaviour and characteristics of large networks of interconnected software services. Experiments show that Kaluta can emulate 10,000 servers using a single physical machine, and is a practical testbed for scalability testing of a real, enterprise-grade identity management suite. The insights gained into the tested enterprise system were used to enhance its design., Comment: International Workshop on Continuous Software Evolution and Delivery (CSED'16), ICSE 2016 Austin, USA

Assuring quality of contemporary software systems is a very challenging task due to the often large complexity of the deployment environments in which they will operate. Service virtualization is an approach to this challenge where services within the deployment environment are emulated by synthesising service response messages from models or by recording and then replaying service interaction messages with the system. Record-and-replay techniques require an approach where (i) message prototypes can be derived from recorded system interactions (i.e. request-response sequences), (ii) a scheme to match incoming request messages against message prototypes, and (iii) the synthesis of response messages based on similarities between incoming messages and the recorded system interactions. Previous approaches in service virtualization have required a multiple sequence alignment (MSA) algorithm as a means of finding common patterns of similarities and differences between messages required by all three steps. In this paper, we present a novel MSA algorithm based on Generalized Suffix Trees (GSTs). We evaluated the accuracy and efficiency of the proposed algorithm against six enterprise service message trace datasets, with the proposed algorithm performing up to 50 times faster than standard MSA approaches. Furthermore, the algorithm has applicability to other domains beyond service virtualization., Comment: In Proceedings of 24th Australasian Software Engineering Conference (ASWEC2015), pp 48-57

The emergence of cloud computing has made dynamic provisioning of elastic capacity to applications on-demand. Cloud data centers contain thousands of physical servers hosting orders of magnitude more virtual machines that can be allocated on demand to users in a pay-as-you-go model. However, not all systems are able to scale up by just adding more virtual machines. Therefore, it is essential, even for scalable systems, to project workloads in advance rather than using a purely reactive approach. Given the scale of modern cloud infrastructures generating real time monitoring information, along with all the information generated by operating systems and applications, this data poses the issues of volume, velocity, and variety that are addressed by Big Data approaches. In this paper, we investigate how utilization of Big Data analytics helps in enhancing the operation of cloud computing environments. We discuss diverse applications of Big Data analytics in clouds, open issues for enhancing cloud operations via Big Data analytics, and architecture for anomaly detection and prevention in clouds along with future research directions., Comment: 10 pages, 2 figures, conference paper in Proceedings of the 21st IEEE International Conference on Parallel and Distributed Systems (ICPADS 2015, IEEE Press, USA), Melbourne, Australia, December 14-17, 2015

Software service emulation is an emerging technique for creating realistic executable models of server-side behaviour and is particularly useful in quality assurance: replicating production-like conditions for large-scale enterprise software systems. This allows performance engineers to mimic very large numbers of servers and/or provide a means of controlling dependencies on diverse third-party systems. Previous approaches to service emulation rely on manual definition of interaction behaviour requiring significant human effort. They also rely on either a system expert or documentation of system protocol and behaviour, neither of which are necessarily available. We present a novel method of automatically building client-server and server-server interaction models of complex software systems directly from interaction trace data, utilising longest common subsequence matching and field substitution algorithms. We evaluate our method against two common application-layer protocols: LDAP and SOAP. The results show that without explicit knowledge of the protocol specifications, our generated service models can produce well-formed responses for interactions. These responses can then be used within an emulation framework for large-scale enterprise system quality assurance purposes.

A Service Level Agreement (SLA) is a legal contract between parties to ensure the Quality of Service (QoS) are provided the providers to the customers. A SLA negotiation between participants assists in defining the QoS requirements of critical service-based processes. However, the negotiation process for customers is a significant task particularly when there are multiple SaaS providers in the Cloud market, as service cost and quality are constantly changing and consumers have varying needs. Therefore, we propose a novel automated negotiation framework where a SaaS broker is utilized as the one-stop-shop for customers to achieve the required service efficiently when negotiating with multiple providers. The automated negotiation framework facilitates intelligent bilateral bargaining of SLAs between a SaaS broker and multiple providers to achieve different objectives for different participants. To maximize profit and improve customer satisfaction levels for the broker, we propose the design of counter offer generation strategies and decision making heuristics that take into account time, market constraints and trade-off between QoS parameters. Our negotiation heuristics are evaluated by extensive experimental studies of our framework using data from a real Cloud provider.

Unit testing is a critical step in the development lifecycle of business processes for ensuring product reliability and dependability. Although plenty of unit testing approaches for WS-BPEL have been proposed, only a few of them designed and implemented a runnable unit testing framework, and none of them provides a technique to systematically specifying and testing the causal and temporal dependencies between the process-under-test and its partner services. In this paper, we propose a novel approach and framework for specifying and testing the inter-dependencies between the process-under-test and its partner services. The dependency constraints defined in the business protocol are declaratively specified using a pattern-based high-level language, and a FSA-based approach is proposed for detecting the violation of constraints. A testing framework that integrates with the Java Finite State Machine framework has been implemented to support the specification of both dependency constraints and test cases, and the execution and result analysis of test cases.

Cloud computing increases the level of connectivity between software applications. IT management applications delivered as a service may need to connect to tens of thousands of endpoint systems. In order to validate the application's reliability and performance at these very large scales, its scalability needs to be tested before being deployed in the cloud. We use an emulation approach, whereby endpoints are modelled and then executed in an emulation environment, which we call "Kaluta". The key aspect is to balance the modelling of the endpoint systems such that it is rich enough to "fool" an unmodified application-under-test into thinking that it is talking to real systems, but light-weight enough such that tens of thousands of instances of model systems can be executed simultaneously in the emulation engine. We present an industry case study - CA Identity Minder-as-a-Service - to demonstrate the effectiveness of using emulation to validate the scalability of a cloud hosted application.

Modern enterprise software systems often need to interact with a large number of heterogeneous systems in an enterprise IT environment. The distributedness, large-scale-ness, and heterogeneity of such environment makes it difficult to test a system's quality attributes such as performance and scalability before it is actually deployed in the environment. In this paper, we present a Coloured Petri nets (CPN) based system behaviour emulation approach and a lightweight virtual testing framework for provisioning the deployment testing environment of an enterprise system so that its quality attributes, especially scalability, can be evaluated without physically connecting to the real production environment. This testing environment is scalable and has a flexible pluggable architecture to support the emulation of the behaviour of heterogeneous systems in the environment. To validate the feasibility of this approach, a CPN emulation model for LDAP has been developed and applied in testing the scalability of a real-life identity management system. An in-lab performance study has been conducted to demonstrate the effectiveness of this approach.

When operating in volatile environments, service-based systems (SBSs) that are built through dynamic composition of component services must be monitored in order to guarantee the response times of the SBSs. In particular, the critical path of a composite SBS, i.e., the execution path in the service composition with the maximum execution time, should be prioritised in cost-effective monitoring as it determines the response time of the SBS. In volatile operating environments, the critical path of a SBS is probabilistic. As such, it is important to estimate the criticalities of the execution paths and the component services, i.e., the probabilities that they are critical, to decide which parts of the system to monitor. In this paper, we propose a novel approach to the identification of Probabilistic Critical Path for Service-based Systems (PCP-SBS). PCP-SBS takes into account the probabilistic nature of the critical path and calculates path criticalities in the context of service composition. We evaluate PCP-SBS experimentally using SBSs that are synthetically composed based on a real-world Web service dataset.

The critical path of a composite Web application operating in volatile environments, i.e., the execution path in the service composition with the maximum execution time, should be prioritised in cost-effective monitoring as it determines the response time of the Web application. In volatile operating environments, the critical path of a Web application is probabilistic. As such, it is important to estimate the criticalities of the execution paths, i.e., the probabilities that they are critical, to decide which parts of the system to monitor. We propose a novel approach to the identification of Probabilistic Critical Path for Service-based Web Applications (PCP-SWA), which calculates the criticalities of different execution paths in the context of service composition. We evaluate PCP-SWA experimentally using an example Web application. Compared to random monitoring, PCP-SWA based monitoring is 55.67% more cost-effective on average.

With the growth of Cloud Computing, more and more companies are offering different cloud services. From the customer's point of view, it is always difficult to decide whose services they should use, based on users' requirements. Currently there is no software framework which can automatically index cloud providers based on their needs. In this work, we propose a framework and a mechanism, which measure the quality and prioritize Cloud services. Such framework can make significant impact and will create healthy competition among Cloud providers to satisfy their Service Level Agreement (SLA) and improve their Quality of Services (QoS).

Monitoring is an essential part of web service (WS) systems. However, monitoring comes with a cost, including an impact on the quality of monitored services and systems. To deliver the best value to a service provider, it is important to balance between meeting monitoring requirements and reducing monitoring impacts when the system is implemented and as the system and its requirements evolve. To achieve this balance, we introduce a novel approach to measuring the Quality of Service (QoS) being delivered by a WS system and optimally reconfiguring a suite of WS monitors, both on a continuous basis at run-time. This includes an automated run-time management system and efficient (re)optimization algorithm for system monitoring configurations. Experiments have shown that our run-time optimization delivers significant benefit in terms of meeting monitoring requirements and minimizing monitoring impacts, and our optimization algorithm is fast enough to optimize large monitoring systems at run-time.

Whilst monitoring web services provides benefits in terms of demonstrating that Service Level Agreements (SLAs) have been met, monitoring comes with a cost on the QoS of delivered services.We present the application of our method for optimizing the configuration of a suite of web service monitors in order to minimise the QoS impacts of monitoring on a web service provider. We discuss the actions required for optimization, and benefits that were achieved. Through this case study, we highlight the possible benefits of optimization to a web service provider, and give details on how we achieve optimization.

This paper proposes a scalable approach for distinguishing malicious files from clean files by investigating the behavioural features using logs of various API calls. We also propose, as an alternative to the traditional method of manually identifying malware files, an automated classification system using runtime features of malware files. For both projects, we use an automated tool running in a virtual environment to extract API call features from executables and apply pattern recognition algorithms and statistical methods to differentiate between files. Our experimental results, based on a dataset of 1368 malware and 456 cleanware files, provide an accuracy of over 97% in distinguishing malware from cleanware. Our techniques provide a similar accuracy for classifying malware into families. In both cases, our results outperform comparable previously published techniques.

Information technology is playing a more and more critical role in many large organizations and enterprise software systems have become increasingly integrated to better support the goals of these organizations. As a consequence, it is a major engineering challenge to assure quality of a software system that is to be deployed in an enterprise environment containing many thousands of interconnected systems. To address this challenge, we propose the use of a scalable emulation environment enabling real-time system testing in large-scale settings. In this work, we discuss the main concepts of our approach, present Reac2o, a prototype implementation of such an emulation environment, and illustrate its applicability in the context of identity management.

Anti-malware software producers are continually challenged to identify and counter new malware as it is released into the wild. A dramatic increase in malware production in recent years has rendered the conventional method of manually determining a signature for each new malware sample untenable. This paper presents a scalable, automated approach for detecting and classifying malware by using pattern recognition algorithms and statistical methods at various stages of the malware analysis life cycle. Our framework combines the static features of function length and printable string information extracted from malware samples into a single test which gives classification results better than those achieved by using either feature individually. In our testing we input feature information from close to 1400 unpacked malware samples to a number of different classification algorithms. Using k-fold cross validation on the malware, which includes Trojans and viruses, along with 151 clean files, we achieve an overall classification accuracy of over 98%.

Current Web Service security standards have inadequate support for end-to-end protection of data when some receivers of the data are unknown to the sender. This paper presents an approach to aid collaborative partner services in properly protecting each other's data. Our approach allows each partner to derive an adequate protection mechanism with minimum performance overhead for each message it sends based on those of the corresponding messages it receives. We modify the message handling mechanisms of Web Service engines to dynamically gather protection requirements for a given outgoing message by aggregating requirements from original owners of message data.

Packing is the most common obfuscation method used by malware writers to hinder malware detection and analysis. There has been a dramatic increase in the number of new packers and variants of existing ones combined with packers employing increasingly sophisticated anti-unpacker tricks and obfuscation methods. This makes it difficult, costly and time-consuming for antivirus (AV) researchers to carry out the traditional static packer identification and classification methods which are mainly based on the packer's byte signature. In this paper1, we present a simple, yet fast and effective packer classification framework that applies pattern recognition techniques on automatically extracted randomness profiles of packers. This system can be run without AV researcher's manual input. We test various statistical classification algorithms, including k-Nearest Neighbor, Best-first Decision Tree, Sequential Minimal Optimization and Naive Bayes. We test these algorithms on a large data set that consists of clean packed files and 17,336 real malware samples. Experimental results demonstrate that our packer classification system achieves extremely high effectiveness (> 99%). The experiments also confirm that the randomness profile used in the system is a very strong feature for packer classification. It can be applied with high accuracy on real malware samples.

Distributed enterprise systems, which are comprised of multiple software systems, typically communicate using application-layer protocols. Being able to independently test each system’s conformance to the protocol specification is important to ensure the overall robustness and reliability of the distributed system. While there exists many modelling languages for network layer protocols, very few tools exist for modelling valid message sequences specifically for the application layer. In order to address this issue, we propose a concise formal modelling syntax for application-layer protocols, with clearly defined semantics. A key feature of our protocol model is dynamic extension, which is necessary for the concise modelling of subsidiary concurrent operations. We demonstrate the power of our modelling language by compactly specifying the valid message sequences of two common application-layer protocols. Trace conformance is defined for the model along with a prototype implementation demonstrating the practical utility of our modelling framework.

Protecting data in multi-stakeholder Web Services systems is a challenge. Using current Web Services security standards, data might be transmitted under-protected or blindly over-protected when the receivers of the data are not known directly to the sender. This paper presents an approach to aiding collaborative partner services to properly protect each other's data. Each partner derives an adequate protection mechanism for each message it sends based on those of the relevant messages it receives. Our approach improves the message handling mechanisms of Web Services engines to allow for dynamic data protection requirements derivation. A prototype is used to validate the approach and to show the performance gain.

A critical issue in developing Web Service-based business applications is the realization of business-level security requirements with system-level security mechanisms using the WS-∗ standards. Current practice has primarily relied on the engineer's experience and lacks consistency and methodological support. This paper introduces an approach to Web Services security engineering called SOABSE, which systematically models, designs and implements security for a WS-based application from a given set of business-oriented security requirements. It includes 1) a stepwise process that systematically transforms business-level security requirements into system-level WS-∗ security policies, and relies on 2) a security realization model that maps business-level security objectives to WS-∗ security realization mechanisms and 3) a security deployment model that sets out the security-oriented Web Service deployment information. A prototype tool supporting the approach is also introduced.

Classifying malware correctly is an important research issue for anti-malware software producers. This paper presents an effective and efficient malware classification technique based on string information using several well-known classification algorithms. In our testing we extracted the printable strings from 1367 samples, including unpacked trojans and viruses and clean files. Information describing the printable strings contained in each sample was input to various classification algorithms, including tree-based classifiers, a nearest neighbour algorithm, statistical algorithms and AdaBoost. Using k-fold cross validation on the unpacked malware and clean files, we achieved a classification accuracy of 97%. Our results reveal that strings from library code (rather than malicious code itself) can be utilised to distinguish different malware families.

Testing enterprise software that communicates with a large number of other software systems is a challenging task as it is often difficult to replicate the size and heterogeneity of large enterprise environments. In particular, it is challenging to conduct testing of non-functional properties related to scalability, performance, and robustness of enterprise software systems when deployed into such environments. In order to address this issue, we propose the use of a scalable emulation environment enabling non-functional testing of enterprise software in realistic and large-scale settings. In this paper, we illustrate our approach by using deterministic finite state machines to specify a scalable and interactive emulation of a modeled environment. To demonstrate the practicality and scalability of the approach, a prototype tool is presented that is used to emulate a large-scale environment of up to 10,000 endpoint systems for an enterprise class software system.

The proliferation of malware is a serious threat to computer and information systems throughout the world. Anti-malware companies are continually challenged to identify and counter new malware as it is released into the wild. In attempts to speed up this identification and response, many researchers have examined ways to efficiently automate classification of malware as it appears in the environment. In this paper, we present a fast, simple and scalable method of classifying Trojans based only on the lengths of their functions. Our results indicate that function length may play a significant role in classifying malware, and, combined with other features, may result in a fast, inexpensive and scalable method of malware classification.