Search

Despite tremendous advancements in large language models (LLMs) over recent years, a notably urgent challenge for their practical deployment is the phenomenon of hallucination, where the model fabricates facts and produces non-factual statements. In response, we propose PoLLMgraph, a Polygraph for LLMs, as an effective model-based white-box detection and forecasting approach. PoLLMgraph distinctly differs from the large body of existing research that concentrates on addressing such challenges through black-box evaluations. In particular, we demonstrate that hallucination can be effectively detected by analyzing the LLM's internal state transition dynamics during generation via tractable probabilistic models. Experimental results on various open-source LLMs confirm the efficacy of PoLLMgraph, outperforming state-of-the-art methods by a considerable margin, evidenced by over 20% improvement in AUC-ROC on common benchmarking datasets like TruthfulQA. Our work paves a new way for model-based white-box analysis of LLMs, motivating the research community to further explore, understand, and refine the intricate dynamics of LLM behaviors., Comment: 15 pages

Currently, the development of the metaverse lies in the hands of industry. Citizens have little influence on this process. Instead, to do justice to the pluralism of (digital) societies, we should strive for an open discourse including many different perspectives on the metaverse and its core technologies such as AI. We utilize a participatory speculative design (PSD) approach to explore Japanese citizens' perspectives on future metaverse societies, as well as social and ethical implications. Our contributions are twofold. Firstly, we demonstrate the effectiveness of PSD in engaging citizens in critical discourse on emerging technologies like the metaverse, offering our workshop framework as a methodological contribution. Secondly, we identify key themes from participants' perspectives, providing insights for culturally sensitive design and development of virtual environments. Our analysis shows that participants imagine the metaverse to have the potential to solve a variety of societal issues; for example, breaking down barriers of physical environments for communication, social interaction, crisis preparation, and political participation, or tackling identity-related issues. Regarding future metaverse societies, participants' imaginations raise critical questions about human-AI relations, technical solutionism, politics and technology, globalization and local cultures, and immersive technologies. We discuss implications and contribute to expanding conversations on metaverse developments.

In 2017, GitHub was the first online open source platform to show security alerts to its users. It has since introduced further security interventions to help developers improve the security of their open source software. In this study, we investigate and compare the effects of these interventions. This offers a valuable empirical perspective on security interventions in the context of software development, enriching the predominantly qualitative and survey-based literature landscape with substantial data-driven insights. We conduct a time series analysis on security-altering commits covering the entire history of a large-scale sample of over 50,000 GitHub repositories to infer the causal effects of the security alert, security update, and code scanning interventions. Our analysis shows that while all of GitHub's security interventions have a significant positive effect on security, they differ greatly in their effect size. By comparing the design of each intervention, we identify the building blocks that worked well and those that did not. We also provide recommendations on how practitioners can improve the design of their interventions to enhance their effectiveness.

In recent years, diffusion models have achieved tremendous success in the field of image generation, becoming the stateof-the-art technology for AI-based image processing applications. Despite the numerous benefits brought by recent advances in diffusion models, there are also concerns about their potential misuse, specifically in terms of privacy breaches and intellectual property infringement. In particular, some of their unique characteristics open up new attack surfaces when considering the real-world deployment of such models. With a thorough investigation of the attack vectors, we develop a systematic analysis of membership inference attacks on diffusion models and propose novel attack methods tailored to each attack scenario specifically relevant to diffusion models. Our approach exploits easily obtainable quantities and is highly effective, achieving near-perfect attack performance (>0.9 AUCROC) in realistic scenarios. Our extensive experiments demonstrate the effectiveness of our method, highlighting the importance of considering privacy and intellectual property risks when using diffusion models in image generation tasks.

Recently, bug-bounty programs have gained popularity and become a significant part of the security culture of many organizations. Bug-bounty programs enable organizations to enhance their security posture by harnessing the diverse expertise of crowds of external security experts (i.e., bug hunters). Nonetheless, quantifying the benefits of bug-bounty programs remains elusive, which presents a significant challenge for managing them. Previous studies focused on measuring their benefits in terms of the number of vulnerabilities reported or based on the properties of the reported vulnerabilities, such as severity or exploitability. However, beyond these inherent properties, the value of a report also depends on the probability that the vulnerability would be discovered by a threat actor before an internal expert could discover and patch it. In this paper, we present a data-driven study of the Chromium and Firefox vulnerability-reward programs. First, we estimate the difficulty of discovering a vulnerability using the probability of rediscovery as a novel metric. Our findings show that vulnerability discovery and patching provide clear benefits by making it difficult for threat actors to find vulnerabilities; however, we also identify opportunities for improvement, such as incentivizing bug hunters to focus more on development releases. Second, we compare the types of vulnerabilities that are discovered internally vs. externally and those that are exploited by threat actors. We observe significant differences between vulnerabilities found by external bug hunters, internal security teams, and external threat actors, which indicates that bug-bounty programs provide an important benefit by complementing the expertise of internal teams, but also that external hunters should be incentivized more to focus on the types of vulnerabilities that are likely to be exploited by threat actors.

Although researchers have characterized the bug-bounty ecosystem from the point of view of platforms and programs, minimal effort has been made to understand the perspectives of the main workers: bug hunters. To improve bug bounties, it is important to understand hunters' motivating factors, challenges, and overall benefits. We address this research gap with three studies: identifying key factors through a free listing survey (n=56), rating each factor's importance with a larger-scale factor-rating survey (n=159), and conducting semi-structured interviews to uncover details (n=24). Of 54 factors that bug hunters listed, we find that rewards and learning opportunities are the most important benefits. Further, we find scope to be the top differentiator between programs. Surprisingly, we find earning reputation to be one of the least important motivators for hunters. Of the challenges we identify, communication problems, such as unresponsiveness and disputes, are the most substantial. We present recommendations to make the bug-bounty ecosystem accommodating to more bug hunters and ultimately increase participation in an underutilized market.

Mobile devices encroach on almost every part of our lives, including work and leisure, and contain a wealth of personal and sensitive information. It is, therefore, imperative that these devices uphold high security standards. A key aspect is the security of the underlying operating system. In particular, Android plays a critical role due to being the most dominant platform in the mobile ecosystem with more than one billion active devices and due to its openness, which allows vendors to adopt and customize it. Similar to other platforms, Android maintains security by providing monthly security patches and announcing them via the Android security bulletin. To absorb this information successfully across the Android ecosystem, impeccable coordination by many different vendors is required. In this paper, we perform a comprehensive study of 3,171 Android-related vulnerabilities and study to which degree they are reflected in the Android security bulletin, as well as in the security bulletins of three leading vendors: Samsung, LG, and Huawei. In our analysis, we focus on the metadata of these security bulletins (e.g., timing, affected layers, severity, and CWE data) to better understand the similarities and differences among vendors. We find that (i) the studied vendors in the Android ecosystem have adopted different structures for vulnerability reporting, (ii) vendors are less likely to react with delay for CVEs with Android Git repository references, (iii) vendors handle Qualcomm-related CVEs differently from the rest of external layer CVEs.

Control-flow hijacking attacks are used to perform malicious com-putations. Current solutions for assessing the attack surface afteracontrol flow integrity(CFI) policy was applied can measure onlyindirect transfer averages in the best case without providing anyinsights w.r.t. the absolute calltarget reduction per callsite, and gad-get availability. Further, tool comparison is underdeveloped or notpossible at all. CFI has proven to be one of the most promising pro-tections against control flow hijacking attacks, thus many effortshave been made to improve CFI in various ways. However, there isa lack of systematic assessment of existing CFI protections. In this paper, we presentLLVM-CFI, a static source code analy-sis framework for analyzing state-of-the-art static CFI protectionsbased on the Clang/LLVM compiler framework.LLVM-CFIworksby precisely modeling a CFI policy and then evaluating it within aunified approach.LLVM-CFIhelps determine the level of securityoffered by different CFI protections, after the CFI protections weredeployed, thus providing an important step towards exploit cre-ation/prevention and stronger defenses. We have usedLLVM-CFIto assess eight state-of-the-art static CFI defenses on real-worldprograms such as Google Chrome and Apache Httpd.LLVM-CFIprovides a precise analysis of the residual attack surfaces, andaccordingly ranks CFI policies against each other.LLVM-CFIalsosuccessfully paves the way towards construction of COOP-like codereuse attacks and elimination of the remaining attack surface bydisclosing protected calltargets under eight restrictive CFI policies., Comment: Accepted for publication at ACSAC'19 conference. arXiv admin note: substantial text overlap with arXiv:1812.08496

In this paper, we perform a comprehensive study of 2,470 patched Android vulnerabilities that we collect from different data sources such as Android security bulletins, CVEDetails, Qualcomm Code Aurora, AOSP Git repository, and Linux Patchwork. In our data analysis, we focus on determining the affected layers, OS versions, severity levels, and common weakness enumerations (CWE) associated with the patched vulnerabilities. Further, we assess the timeline of each vulnerability, including discovery and patch dates. We find that (i) even though the number of patched vulnerabilities changes considerably from month to month, the relative number of patched vulnerabilities for each severity level remains stable over time, (ii) there is a significant delay in patching vulnerabilities that originate from the Linux community or concern Qualcomm components, even though Linux and Qualcomm provide and release their own patches earlier, (iii) different AOSP versions receive security updates for different periods of time, (iv) for 94% of patched Android vulnerabilities, the date of disclosure in public datasets is not before the patch release date, (v) there exist some inconsistencies among public vulnerability data sources, e.g., some CVE IDs are listed in Android Security bulletins with detailed information, but in CVEDetails they are listed as unknown, (vi) many patched vulnerabilities for newer Android versions likely also affect older versions that do not receive security patches due to end-of-life., Comment: The 2019 Workshop on the Economics of Information Security (WEIS 2019)

Cryptocurrency exchanges are frequently targeted and compromised by cyber-attacks, which may lead to significant losses for the depositors and closure of the affected exchanges. These risks threaten the viability of the entire public blockchain ecosystem since exchanges serve as major gateways for participation in public blockchain technologies. In this paper, we develop an economic model to capture the short-term incentives of cryptocurrency exchanges with respect to making security investments and establishing transaction fees. Using the model, we derive conclusions regarding an exchange's optimal economic decisions, and illustrate key features of these conclusions using graphs based on real-world data. Our security investment model exhibits horizontal scaling properties with respect to reducing exposure to losses, and may be of special interest to exchanges operating in markets with high price volatility.

Stack Overflow (SO) is the most popular online Q&A site for developers to share their expertise in solving programming issues. Given multiple answers to certain questions, developers may take the accepted answer, the answer from a person with high reputation, or the one frequently suggested. However, researchers recently observed exploitable security vulnerabilities in popular SO answers. This observation inspires us to explore the following questions: How much can we trust the security implementation suggestions on SO? If suggested answers are vulnerable, can developers rely on the community's dynamics to infer the vulnerability and identify a secure counterpart? To answer these highly important questions, we conducted a study on SO posts by contrasting secure and insecure advices with the community-given content evaluation. We investigated whether SO incentive mechanism is effective in improving security properties of distributed code examples. Moreover, we also traced duplicated answers to assess whether the community behavior facilitates propagation of secure and insecure code suggestions. We compiled 953 different groups of similar security-related code examples and labeled their security, identifying 785 secure answer posts and 644 insecure ones. Compared with secure suggestions, insecure ones had higher view counts (36,508 vs. 18,713), received a higher score (14 vs. 5), and had significantly more duplicates (3.8 vs. 3.0) on average. 34% of the posts provided by highly reputable so-called trusted users were insecure. Our findings show that there are lots of insecure snippets on SO, while the community-given feedback does not allow differentiating secure from insecure choices. Moreover, the reputation mechanism fails in indicating trustworthy users with respect to security questions, ultimately leaving other users wandering around alone in a software security minefield.

Modern operating systems such as Android, iOS, Windows Phone, and Chrome OS support a cooperating program abstraction. Instead of placing all functionality into a single program, programs cooperate to complete tasks requested by users. However, untrusted programs may exploit interactions with other programs to obtain unauthorized access to system sensors either directly or through privileged services. Researchers have proposed that programs should only be authorized to access system sensors on a user-approved input event, but these methods do not account for possible delegation done by the program receiving the user input event. Furthermore, proposed delegation methods do not enable users to control the use of their input events accurately. In this paper, we propose ENTRUST, a system that enables users to authorize sensor operations that follow their input events, even if the sensor operation is performed by a program different from the program receiving the input event. ENTRUST tracks user input as well as delegation events and restricts the execution of such events to compute unambiguous delegation paths to enable accurate and reusable authorization of sensor operations. To demonstrate this approach, we implement the ENTRUST authorization system for Android. We find, via a laboratory user study, that attacks can be prevented at a much higher rate (54-64% improvement); and via a field user study, that ENTRUST requires no more than three additional authorizations per program with respect to the first-use approach, while incurring modest performance (<1%) and memory overheads (5.5 KB per program).

Integer overflows have threatened software applications for decades. Thus, in this paper, we propose a novel technique to provide automatic repairs of integer overflows in C source code. Our technique, based on static symbolic execution, fuses detection, repair generation and validation. This technique is implemented in a prototype named IntRepair. We applied IntRepair to 2,052C programs (approx. 1 million lines of code) contained in SAMATE's Juliet test suite and 50 synthesized programs that range up to 20KLOC. Our experimental results show that IntRepair is able to effectively detect integer overflows and successfully repair them, while only increasing the source code (LOC) and binary (Kb) size by around 1%, respectively. Further, we present the results of a user study with 30 participants which shows that IntRepair repairs are more than 10x efficient as compared to manually generated code repairs, Comment: Accepted for publication at the IEEE TSE journal. arXiv admin note: text overlap with arXiv:1710.03720

Vendors in the Android ecosystem typically customize their devices by modifying Android Open Source Project (AOSP) code, adding in-house developed proprietary software, and pre-installing third-party applications. However, research has documented how various security problems are associated with this customization process. We develop a model of the Android ecosystem utilizing the concepts of game theory and product differentiation to capture the competition involving two vendors customizing the AOSP platform. We show how the vendors are incentivized to differentiate their products from AOSP and from each other, and how prices are shaped through this differentiation process. We also consider two types of consumers: security-conscious consumers who understand and care about security, and na\"ive consumers who lack the ability to correctly evaluate security properties of vendor-supplied Android products or simply ignore security. It is evident that vendors shirk on security investments in the latter case. Regulators such as the U.S. Federal Trade Commission have sanctioned Android vendors for underinvestment in security, but the exact effects of these sanctions are difficult to disentangle with empirical data. Here, we model the impact of a regulator-imposed fine that incentivizes vendors to match a minimum security standard. Interestingly, we show how product prices will decrease for the same cost of customization in the presence of a fine, or a higher level of regulator-imposed minimum security., Comment: 22nd International Conference on Financial Cryptography and Data Security (FC 2018)

Integer overflows in commodity software are a main source for software bugs, which can result in exploitable memory corruption vulnerabilities and may eventually contribute to powerful software based exploits, i.e., code reuse attacks (CRAs). In this paper, we present IntGuard , a tool that can repair integer overflows with high-quality source code repairs. Specifically, given the source code of a program, IntGuard first discovers the location of an integer overflow error by using static source code analysis and satisfiability modulo theories (SMT) solving. IntGuard then generates integer multi-precision code repairs based on modular manipulation of SMT constraints as well as an extensible set of customizable code repair patterns. We have implemented and evaluated IntGuard with 2052 C programs (approx. 1 Mil. LOC) available in the currently largest open- source test suite for C/C++ programs and with a benchmark containing large and complex programs. The evaluation results show that IntGuard can precisely (i.e., no false positives are accidentally repaired), with low computational and runtime overhead repair programs with very small binary and source code blow-up. In a controlled experiment, we show that IntGuard is more time-effective and achieves a higher repair success rate than manually generated code repairs., Comment: 20 pages

A core technique used by popular proxy-based circumvention systems like Tor, Psiphon, and Lantern is to secretly share the IP addresses of circumvention proxies with the censored clients for them to be able to use such systems. For instance, such secretly shared proxies are known as bridges in Tor. However, a key challenge to this mechanism is the insider attack problem: censoring agents can impersonate as benign censored clients in order to obtain (and then block) such secretly shared circumvention proxies. In this paper, we perform a fundamental study on the problem of insider attack on proxy-based circumvention systems. We model the proxy distribution problem using game theory, based on which we derive the optimal strategies of the parties involved, i.e., the censors and circumvention system operators. That is, we derive the optimal proxy distribution mechanism of a circumvention system like Tor, against the censorship adversary who also takes his optimal censorship strategies. This is unlike previous works that design ad hoc mechanisms for proxy distribution, against non-optimal censors. We perform extensive simulations to evaluate our optimal proxy assignment algorithm under various adversarial and network settings. Comparing with the state-of-the-art prior work, we show that our optimal proxy assignment algorithm has superior performance, i.e., better resistance to censorship even against the strongest censorship adversary who takes her optimal actions. We conclude with lessons and recommendation for the design of proxy-based circumvention systems.

While recognized as a theoretical and practical concept for over 20 years, only now ransomware has taken centerstage as one of the most prevalent cybercrimes. Various reports demonstrate the enormous burden placed on companies, which have to grapple with the ongoing attack waves. At the same time, our strategic understanding of the threat and the adversarial interaction between organizations and cybercriminals perpetrating ransomware attacks is lacking. In this paper, we develop, to the best of our knowledge, the first game-theoretic model of the ransomware ecosystem. Our model captures a multi-stage scenario involving organizations from different industry sectors facing a sophisticated ransomware attacker. We place particular emphasis on the decision of companies to invest in backup technologies as part of a contingency plan, and the economic incentives to pay a ransom if impacted by an attack. We further study to which degree comprehensive industry-wide backup investments can serve as a deterrent for ongoing attacks., Comment: 8th Conference on Decision and Game Theory for Security (GameSec 2017)

Games of timing aim to determine the optimal defense against a strategic attacker who has the technical capability to breach a system in a stealthy fashion. Key questions arising are when the attack takes place, and when a defensive move should be initiated to reset the system resource to a known safe state. In our work, we study a more complex scenario called Time-Based Security in which we combine three main notions: protection time, detection time, and reaction time. Protection time represents the amount of time the attacker needs to execute the attack successfully. In other words, protection time represents the inherent resilience of the system against an attack. Detection time is the required time for the defender to detect that the system is compromised. Reaction time is the required time for the defender to reset the defense mechanisms in order to recreate a safe system state. In the first part of the paper, we study the VERIS Community Database (VCDB) and screen other data sources to provide insights into the actual timing of security incidents and responses. While we are able to derive distributions for some of the factors regarding the timing of security breaches, we assess the state-of-the-art regarding the collection of timing-related data as insufficient. In the second part of the paper, we propose a two-player game which captures the outlined Time-Based Security scenario in which both players move according to a periodic strategy. We carefully develop the resulting payoff functions, and provide theorems and numerical results to help the defender to calculate the best time to reset the defense mechanism by considering protection time, detection time, and reaction time., Comment: Workshop on the Economics of Information Security (WEIS 2017)

Bug bounty programs offer a modern platform for organizations to crowdsource their software security and for security researchers to be fairly rewarded for the vulnerabilities they find. Little is known however on the incentives set by bug bounty programs: How they drive new bug discoveries, and how they supposedly improve security through the progressive exhaustion of discoverable vulnerabilities. Here, we recognize that bug bounty programs create tensions, for organizations running them on the one hand, and for security researchers on the other hand. At the level of one bug bounty program, security researchers face a sort of St-Petersburg paradox: The probability of finding additional bugs decays fast, and thus can hardly be matched with a sufficient increase of monetary rewards. Furthermore, bug bounty program managers have an incentive to gather the largest possible crowd to ensure a larger pool of expertise, which in turn increases competition among security researchers. As a result, we find that researchers have high incentives to switch to newly launched programs, for which a reserve of low-hanging fruit vulnerabilities is still available. Our results inform on the technical and economic mechanisms underlying the dynamics of bug bounty program contributions, and may in turn help improve the mechanism design of bug bounty programs that get increasingly adopted by cybersecurity savvy organizations., Comment: 19 pages, 3 figures, 1 table, forthcoming at Journal of Cybersecurity (2017)

Smartphones' cameras, microphones, and device displays enable users to capture and view memorable moments of their lives. However, adversaries can trick users into authorizing malicious apps that exploit weaknesses in current mobile platforms to misuse such on-board I/O devices to stealthily capture photos, videos, and screen content without the users' consent. Contemporary mobile operating systems fail to prevent such misuse of I/O devices by authorized apps due to lack of binding between users' interactions and accesses to I/O devices performed by these apps. In this paper, we propose Aware, a security framework for authorizing app requests to perform operations using I/O devices, which binds app requests with user intentions to make all uses of certain I/O devices explicit. We evaluate our defense mechanisms through laboratory-based experimentation and a user study, involving 74 human subjects, whose ability to identify undesired operations targeting I/O devices increased significantly. Without Aware, only 18% of the participants were able to identify attacks from tested RAT apps. Aware systematically blocks all the attacks in absence of user consent and supports users in identifying 82% of social-engineering attacks tested to hijack approved requests, including some more sophisticated forms of social engineering not yet present in available RATs. Aware introduces only 4.79% maximum performance overhead over operations targeting I/O devices. Aware shows that a combination of system defenses and user interface can significantly strengthen defenses for controlling the use of on-board I/O devices.

The amount of personal information contributed by individuals to digital repositories such as social network sites has grown substantially. The existence of this data offers unprecedented opportunities for data analytics research in various domains of societal importance including medicine and public policy. The results of these analyses can be considered a public good which benefits data contributors as well as individuals who are not making their data available. At the same time, the release of personal information carries perceived and actual privacy risks to the contributors. Our research addresses this problem area. In our work, we study a game-theoretic model in which individuals take control over participation in data analytics projects in two ways: 1) individuals can contribute data at a self-chosen level of precision, and 2) individuals can decide whether they want to contribute at all (or not). From the analyst's perspective, we investigate to which degree the research analyst has flexibility to set requirements for data precision, so that individuals are still willing to contribute to the project, and the quality of the estimation improves. We study this tradeoff scenario for populations of homogeneous and heterogeneous individuals, and determine Nash equilibria that reflect the optimal level of participation and precision of contributions. We further prove that the analyst can substantially increase the accuracy of the analysis by imposing a lower bound on the precision of the data that users can reveal.