Search

Circulation studies, as the theory of ecological spread of information, impacts public perception of knowledge-making, and digital circulation (i.e. online information sharing) impacts what people expect online knowledge-making and online education is or should be. Online education is becoming a new norm for students and universities at a time when economic pressure is pushing both to be more austere and expedient; at the same time, circulation collapses together the complex ways we communicate, making them harder to differentiate. This dissertation responds to these conditions by focusing on the labor behind circulation and Online Writing Instruction (OWI) in order to study knowledge-making online. Through focus groups with instructors, case studies, and surveys of students in online classes, this dissertation identifies strategies that benefit both teachers and students and improve Online Writing Classes. This work intersects with recent considerations of how mis- and dis-information spread online, the impact of Data Science and Information Theory on communication and knowledge-making, and how to make universities accessible to more people.Chapter 1 overviews the history of Distance Education (DE) and Online Education (OE) as well as the relevant disciplinary distinctions OWI makes for itself. Chapter 1 also identifies theoretical and practical challenges OE finds for itself and overviews recent shifts in OE student populations. Chapter 2 contextualizes the challenges OE and OWI face in a larger ecology of Information Theory, Rhetoric and Composition theory and practice, Technical Communication theory and practice, and Neoliberal economics, positing ecological links between modern data science, digital circulation, and economics. In doing so, Chapter 2 offers a rhetorical interpretation of the DIKW pyramid and definitions of data, information, knowledge, and wisdom (D, I, W, and W respectively) for rhetorical practitioners. Chapter 3 follows up on Chapter 2's arguments to respond with research on teacher and student labor in online classes with methods for such inquiry, through focus groups, case studies, and surveys. Chapter 4 presents data from all stages of that inquiry, and Chapter 5 connects together observations from the data with theory from Chapters 1 and 2 to draw more concrete conclusions. [The dissertation citations contained here are published with the permission of ProQuest LLC. Further reproduction is prohibited without permission. Copies of dissertations may be obtained by Telephone (800) 1-800-521-0600. Web page: http://www.proquest.com/en-US/products/dissertations/individuals.shtml.]

Red snapper, "Lutjanus campechanus," were sampled with hook and line at natural (n = 33) and artificial (n = 27) reef sites in the northern Gulf of Mexico from 2009-2011. Stomachs (n = 708) were extracted and their contents preserved for gut content analysis, and muscle tissue samples (n = 200) were dissected and frozen for stable isotope ratio-mass spectrometry (delta[superscript 13]C, delta[superscript 15]N, and delta[superscript 34]S) analysis. Forty-eight percent of sampled fish had identifiable prey which were grouped into 7 categories: fish, decapods, cephalopods, stomatopods, gastropods, zooplankton, and other invertebrates. A permutational multivariate analysis of variance (PERMANOVA) was computed to test the effect of fish size, habitat type, and the Deepwater Horizon Oil Spill (DHOS) on red snapper diet. Fish size (PERMANOVA, p = 0.021), and the DHOS (PERMANOVA, p = 0.001) were both significant effects in the model, but interactions between habitat type and the DHOS (PERMANOVA, p = 0.049) was also significant. Significant differences in diet among red snapper size categories were due to low trophic position prey items, such as pelagic zooplankton, being more abundant in the diet of larger (>500mm) red snapper, while red snapper in smaller size classes contained greater amounts of higher trophic level prey such decapods and fish. Although no significant difference existed in red snapper diet between habitat types, they consumed slightly higher amounts of decapods at artificial reefs (21.9% of total diet) than natural (14.8%). The significant DHOS effect was driven by an increase in fish and a decrease in zooplankton in red snapper diet following the spill; however, this effect cannot be interpreted independently given the significant interaction between habitat type and the DHOS effect. Zooplankton prey consumption declined and fish consumption increased for red snapper sampled at both artificial and natural reefs after the DHOS, but fish consumption increased to a greater extent at artificial reef sites following the spill. A significant increase in delta[superscript 15]N after the DHOS (ANOVA, p <0.001) also indicates red snapper sampled fed on higher trophic position prey following the spill, and lower delta[superscript 34]S values indicate an increase in benthic versus pelagic prey consumption following the DHOS (ANOVA, p <0.001). The ontogenetic shift in diet observed via gut content analysis was also apparent in the significant effect of fish size on delta[superscript 15]N (ANOVA, p = 0.015). Overall, results clearly indicate the DHOS affected red snapper diet and trophic position, and likely affected the abundance of prey resources as well. Study results also confirm the utility of stable isotope analysis to infer aspects of reef fish trophic ecology. [The dissertation citations contained here are published with the permission of ProQuest LLC. Further reproduction is prohibited without permission. Copies of dissertations may be obtained by Telephone (800) 1-800-521-0600. Web page: http://www.proquest.com/en-US/products/dissertations/individuals.shtml.]

How much information do natural systems store and process? In this work we attempt to answer this question in multiple ways. We first establish a mathematical framework where natural systems are represented by a canonical form of edge-labeled hidden fc models called e-machines. Then, utilizing this framework, a variety of measures are defined and algorithms for computing them from an e-machine are described. The first two measures defined are related to the length of time a system remembers. The first, the Markov order, is a well-known measure of the time one must observe a system in order to make accurate predictions. Despite its statistical nature, it is shown to be a topological property of the process's e-machine. The second, the recently defined cryptic order, quantifies the ability to retrodict a system's internal dynamics. It is also shown to be a topological property of the e-machine, and efficient algorithms for computing both quantities are given. The second batch of metrics quantify information generation and storage in a system by partitioning the observations. By considering the role of both the past and the future behavior of a system, a semantic understanding of information generation emerges, labeling some information generation as "ephemeral," having no lasting effects on the system, and the rest as "bound," playing a role temporal structure. Following through with this decomposition, other quantities of less straight-forward interpretation are also defined. This is followed by a thorough discussion of these quantities and other derived quantities. Lastly the decomposition of the entropy rate into ephemeral and bound components is applied to several standard chaotic systems through a duality between the entropy rate and the Lyapunov exponent. This exposes new structural behaviors hitherto unknown in these systems. These revolutions hint at a method for tuning natural or engineered systems so as to maximize the ability to harness their intrinsic computing abilities. [The dissertation citations contained here are published with the permission of ProQuest LLC. Further reproduction is prohibited without permission. Copies of dissertations may be obtained by Telephone (800) 1-800-521-0600. Web page: http://www.proquest.com/en-US/products/dissertations/individuals.shtml.]

This dissertation introduces a theory of information quality to explain macroscopic behavior observed in the systems engineering process. The theory extends principles of Shannon's mathematical theory of communication [1948] and statistical mechanics to information development processes concerned with the flow, transformation, and meaning of information. The meaning of requirements information in the systems engineering context is estimated or measured in terms of the cumulative requirements quality Q which corresponds to the distribution of the requirements among the available quality levels. The requirements entropy framework (REF) implements the theory to address the requirements engineering problem. The REF defines the relationship between requirements changes, requirements volatility, requirements quality, requirements entropy and uncertainty, and engineering effort. The REF is evaluated via simulation experiments to assess its practical utility as a new method for measuring, monitoring and predicting requirements trends and engineering effort at any given time in the process. The REF treats the requirements engineering process as an open system in which the requirements are discrete information entities that transition from initial states of high entropy, disorder and uncertainty toward the desired state of minimum entropy as engineering effort is input and requirements increase in quality. The distribution of the total number of requirements R among the N discrete quality levels is determined by the number of defined quality attributes accumulated by R at any given time. Quantum statistics are used to estimate the number of possibilities P for arranging R among the available quality levels. The requirements entropy H[subscript R] is estimated using R, N and P by extending principles of information theory and statistical mechanics to the requirements engineering process. The information I increases as HR and uncertainty decrease, and the change in information ÄI needed to reach the desired state of quality is estimated from the perspective of the receiver. The HR may increase, decrease or remain steady depending on the degree to which additions, deletions and revisions impact the distribution of R among the quality levels. Current requirements trend metrics generally treat additions, deletions and revisions the same and simply measure the quantity of these changes over time. The REF evaluates the quantity of requirements changes over time, distinguishes between their positive and negative effects by calculating their impact on H[subscript R], Q, and ÄI, and forecasts when the desired state will be reached, enabling more accurate assessment of the status and progress of the requirements engineering effort. Results from random variable simulations suggest the REF is an improved leading indicator of requirements trends that can be readily combined with current methods. The increase in I, or decrease in H[subscript R] and uncertainty, is proportional to the engineering effort E input into the requirements engineering process. The REF estimates the ÄE needed to transition R from their current state of quality to the desired end state or some other interim state of interest. Simulation results are compared with measured engineering effort data for Department of Defense programs published in the SE literature, and the results suggest the REF is a promising new method for estimation of ÄE. [The dissertation citations contained here are published with the permission of ProQuest LLC. Further reproduction is prohibited without permission. Copies of dissertations may be obtained by Telephone (800) 1-800-521-0600. Web page: http://www.proquest.com/en-US/products/dissertations/individuals.shtml.]

Elementary actions online establish an individual's existence on the web and her/his orientation toward different issues. In this sense, actions truly define a user in spaces like online forums and communities and the aggregate of elementary actions shape the atmosphere of these online spaces. This observation, coupled with the unprecedented scale and detail of data on user actions on the web, compels us to utilize them in understanding collective human behavior. Despite large investments by industry to capture this data and the expanding body of research on "big data" in academia, gaining insight into collective user behavior online has been elusive. If one is indeed able to overcome the considerable computational challenges posed by both the scale and the inevitable noisiness of the associated data sets, one could provide new automated frameworks to extract insights into evolving behavior at different scales, and to form an altogether different perspective of aggregated elementary user actions. This thesis addresses this fundamental and pressing problem and offers a "gestalt computing" approach when studying complex social phenomena in large datasets. This approach involves extracting macro structures from aggregated user actions, finding their possible meanings, and arranging data in layers so that it is iteratively explorable. The dissertation includes three major sections; first modeling and prediction of diffusion of information by users on the social web; next, detection of topics promoted by user communities; finally, presentation of the gestalt computing framework through a methodology that uses graph theory, language processing, and information theory to provide a top-down map of group dynamics on social news websites. What we find is not only statistical significance in the extracted structure, but also that the results are meaningful to human understanding. The efficacy of the proposed methodologies is established via multiple real-world data sets. [The dissertation citations contained here are published with the permission of ProQuest LLC. Further reproduction is prohibited without permission. Copies of dissertations may be obtained by Telephone (800) 1-800-521-0600. Web page: http://www.proquest.com/en-US/products/dissertations/individuals.shtml.]

The Internet, as a digital record of human discourse, provides an opportunity to directly analyze political communicative behavior. The rapid emergence of social online networks augurs a transformation in the quality and quantity of information people have to evaluate their political system. Digital formats instantiate new categories of actors and new capacities to intervene in political discourse. Existing theories of political communication fail to account for the time lag, speed, anonymity and nature of replication inherent in digital formats. To fully investigate the digital record, scholars need a new framework that apprehends the complex density of a multidirectional political discourse that defies traditional time controls. Using information and linguistic theory we develop and apply a functional, information flow model that illuminates the conceptual relationships and capacities of people to shape their information environment. Actors are defined in terms of their communicative role in idea exchange and the logic of communication is alloyed with political considerations to examine the scopes of action and immersion scenarios that affect communicants. Our modular and scalable theory offers a way to measure signal fidelity, node activation, and message branching. We define signaling and amplification schemes and effects and explore the benefits and disadvantages of amplification devices for speakers and recipients. We show how individual actors are linked by the micro information streams they receive and transmit. We are therefore able to model exposure conditions of a multiple input, noisy spectrum for citizens and provide a counterpoint to the simple, controlled environment of agenda-setting experiments. Finally, due to its foundation in information theory, our framework provides a natural platform by which to organize and develop a research program that uses computational linguistics and data mining techniques. [The dissertation citations contained here are published with the permission of ProQuest LLC. Further reproduction is prohibited without permission. Copies of dissertations may be obtained by Telephone (800) 1-800-521-0600. Web page: http://www.proquest.com/en-US/products/dissertations/individuals.shtml.]

Since information theory was developed by Claude E. Shannon, in addition to its primary role in communications and networking, it has broadened to find applications in many other areas of science and technology, such as microeconomics, statistics, and neuroscience. This thesis investigates the application of information theoretic viewpoints to two interesting and significant topics. One is the fundamental topic in the field of communications and networking. The other is a long-term open issue in the area of microeconomics. We study the capacity of multiuser networks, which has been a long-standing problem in the information theory. Recently, Avestimehr, Diggavi, and Tse have proposed a deterministic network model to approximate multiuser wireless networks. For wireless multicast relay networks, they have shown that the capacity for the deterministic model is equal to the minimal rank of the incidence matrix of a certain cut between the source and any of the sinks. Their proposed code construction, however, is not guaranteed to be efficient and may potentially involve an infinite block length. We propose an efficient linear code construction for the deterministic wireless relay network model. Unlike several previous coding schemes, we do not attempt to find flows in the network. Instead, for a layered network, we maintain an invariant where it is required that at each stage of the code construction, certain sets of codewords are linearly independent. We then apply ideas from information theory to solve a canonical problem in microeconomics with information constraints. Specifically, we analyze the problem of nonlinear pricing with limited information. Due to information constraints, the seller is limited to offering a menu with a finite number of choices to a continuum of buyers with a continuum of possible valuations. By revealing an underlying connection to the quantization in the information theory, we introduce the conditions which the optimal finite menu for the socially efficient and that for the revenue-maximizing mechanism must satisfy. In both cases, we provide an estimate of the loss resulting from using the finite menu. We then extend our nonlinear pricing model to multi-product environment where each buyer can purchase a variety of multiple goods at a time. His preference over these goods is represented by a multi-dimensional vector in some compact subset with a continuum of possible valuations. We generalize our results to multi-product environment via vector quantization. We discuss the benefit of bundling the consumer's preferences over multiple goods, and designing the finite menus jointly in higher dimensions. This benefit arises by using vector quantization methods that take advantage of the dimensionality, the shape of the marginal density function, and the correlation among the consumer's types over multiple products. [The dissertation citations contained here are published with the permission of ProQuest LLC. Further reproduction is prohibited without permission. Copies of dissertations may be obtained by Telephone (800) 1-800-521-0600. Web page: http://www.proquest.com/en-US/products/dissertations/individuals.shtml.]

This dissertation is an ethnography conducted with the Los Angeles-based community arts organization called Machine Project. Operating both a storefront gallery in Echo Park and as a loose association of contemporary artists, performers, curators, and designers, Machine Project seeks to make "rarefied knowledge accessible" through workshops, site-specific installations and performances, lectures, and various participatory projects. Machine Project exists as but one instantiation of a larger movement in contemporary art around "alternative spaces," or organizations and projects that resist and/or refigure the discursive structures imposed on art by institutions of cultural heritage and the art market. Alternative and artist-run spaces often operate with a Do-It-Yourself and independent ethos, and are often sustained by its communities of artists and the publics that support them. Many of the efforts of alternative spaces are process-based operations, whether as an exhibition space for experimental forms of contemporary art, forums and workshops on a range of topics, performances, participatory projects, among others. Records created about the events, programs, and operations of these alternative spaces are often elusive, if created at all. Historically, alternative and artist-run spaces have been invested in community building, the public circulation of aesthetic knowledge, the exposing of museums and other institutions of cultural heritage as discursive frames, and public participation. How does documentation serve to support such orientations? If an alternative or artist-run space describes its operations in terms of values like community participation and relational aesthetics, how might such values be folded into the production, circulation, and preservation of its records? This dissertation is comprised of two primary sections. The first section includes a critical review of the archival science literature, identifying fundamental concepts of archival theory and practice that are directly relevant to the research questions, such as the principle of provenance, evidence, and records creation. This section also includes a chapter devoted to describing and assessing ethnography as a methodological approach for archival research, drawing in insights culled from social systems theory and information theory. The second section of the dissertation is comprised of observations and reflections on Machine Project and its documentation practices at three levels of analysis. The first level explores documentation issues that emerge out of the organization's collaborations with arts institutions. The second level adopts a finer-grained view and looks at collaborative relationships between artists affiliated with Machine Project. The third level looks to notions of community as they are expressed in a selection of Machine Project's events and programs, and analyzes the documentation produced by audiences and shared in social media spaces. The dissertation concludes with reading of Machine Project's documentation practices through the theoretical lens of the records continuum, which forms the basis for a critique of the records continuum and the burgeoning area of research on community archiving. Following this critique, the dissertation then presents a series of recommendations for future research and describes the metaphorical figure of the "itinerant archivist" as a conceptual intervention and strategy for self-reflection among archival scholars and practitioners. [The dissertation citations contained here are published with the permission of ProQuest LLC. Further reproduction is prohibited without permission. Copies of dissertations may be obtained by Telephone (800) 1-800-521-0600. Web page: http://www.proquest.com/en-US/products/dissertations/individuals.shtml.]

This dissertation aims to investigate how advanced information technologies cope with the various demands of disaster response. It consists of three essays on the exploration of micro-blogging and FOSS environments. The first essay looks at the usage of micro-blogging in the aftermath of the massive 2008 China earthquake and explores the question whether micro-blogging could replace mainstream media information channels or merely supplement them in the manner of distributing disaster information. Twitter is used as a proxy for micro-blogging in general and the mainstream media information channel is used as the benchmark. In this essay, we focused on two sets of theories which are well known in the IS domain: Information Quality and Collective Intelligence. Information quality (IQ) has been defined as a measure of the "fitness for use" of information (Wang and Strong (1996)). Parker et al (2006) has adapted a framework of thirteen information quality dimensions, which includes: accessibility, accuracy, appropriateness and so on. Delone and Mclean (1992; 2003) also incorporated information quality as one of the determinants that affect the success of information systems in their proposed IS Success Model. In this essay, we focus on four dimensions: Timeliness, Accessibility, Accuracy, and Completeness. Regarding collective intelligence, Web 2.0 applications provide a platform or social network to facilitate communication, information sharing, and collaboration between its users. The concept of collective intelligence has been embraced in nearly all areas where extensive collaboration is a necessity. They have become a new means to collect the wisdom of different groups of people to enable greater productivity and facilitate more meaningful decisions than are possible by individuals working in isolation (Gregg 2010). We argue that micro-blogging can provide integrated and speedy information regarding disaster news. It serves as an excellent supplement to the traditional information channels, and if the circumstances of a situation were to render micro-blogging as the only effective communication tool available, it could certainly serve as a replacement in the short period of time after the disaster happens The second essay is a case study looking at the deployment of FOSS systems through the collaborative efforts of public and private entities in joint disaster response. Much previous IS research has fallen in the FOSS domain. This essay aims to identify the key factors that affect the potentials for response organizations to explore humanitarian FOSS systems as a novel and economically preferable approach for disaster management. From a collaborative system deployment perspective, it also helps to better understand how the government and private organization partnership in this area may be initiated, how to encourage government to better support their deployment, and how to coordinate with different organizations and leverage the FOSS advantage to the maximum extent. Technology-Organization-Environment (TOE) framework developed by Tornatzkey and Fleischer (1990) was the major theoretical foundation adopted in this essay. The third essay again touches on the micro-blogging environment and explores how communication patterns evolve on twitter in the midst of a disaster. This essay utilizes data from the period after the 2011 Japanese Earthquake and Tsunami with regards to the radiation threat from the Fukushima reactors. Risk communication is the major theory that we used in the third essay. The media plays an important role in risk communication. It not only provides information, but also brings the public's attention to urgent issues. Slovic (1999) examined risk assessment and the relationship to public perceptions and stated "risk is the socially constructed sum of hazards and public perceptions". Even though the most perfect risk communication may not solve all problems or conflicts, a poor one could lead to an overall failure of risk perception and risk management. We analyze the pattern of alarming, reassuring, and assuring coverage in the different time periods following the event, and check the different conditions under which the communication patterns change. Again tying in to our first paper where we compare Twitter to mainstream media coverage, findings here expand upon prior research regarding the content of traditional media coverage of a 'hot crisis' by looking at how the content of this new communications medium is similar and different. Finally, governmental information is also communicated in the micro-blogging environment. We examine how the public reacts to this information on Twitter under different circumstances. (Abstract shortened by UMI.) [The dissertation citations contained here are published with the permission of ProQuest LLC. Further reproduction is prohibited without permission. Copies of dissertations may be obtained by Telephone (800) 1-800-521-0600. Web page: http://www.proquest.com/en-US/products/dissertations/individuals.shtml.]

The context for this work is two-agent team decision systems. An "agent" is an intelligent entity that can measure some aspect of its environment, process information and possibly influence the environment through its action. In a collaborative two-agent team decision system, the agents can be coupled by noisy or noiseless interactions and cooperate to solve problems that are beyond the individual capabilities or knowledge of each agent. This thesis focuses on using "stochastic control" and "information theoretic" tools hand-in-hand in solving and analyzing an interactive two-agent sequential decision-making problem. "Stochastic control" techniques can help in identifying optimal strategies for sequential decision making based on observations. "Information-theoretic" tools address the fundamental limit of performance between two agents with noisy interaction--in the context of communication and rate-distortion. The motivation for this work comes from the quest for using stochastic control tools in identifying optimal policies for a two-agent team decision system with an objective of "maximizing the information rate." The resulting policies, if they exist, will involve decision making at each step based on observations, in contrast to existing communication schems that decide what to transmit over a long time-horizon, at the start of communication. However, there are many questions that have to be addressed: *How should we formulate a stochastic-control problem to capture information gains? *Suppose we can formulate such a control problem. Can we solve for explicit, non-random, optimal strategies that operate on sufficient statistics (thus resulting in a simple structure for optimal policies)? *Further, do these control-theory based policies assure reliability of communication in an information-theoretic sense? Consider a different problem where a third person has knowledge of the optimal policies of the two interacting agents, but is unaware of the cost function they are collaboratively optimizing. *Can he deduce what the two agents are trying to achieve based on their policies? In this thesis, we focus on addressing these questions using perspectives from both information and control theory. We consider an interacting two-agent decision-making problem consisting of a Markov source process, a causal encoder with feedback, and a causal decoder. We augment the standard formulation by considering general alphabets and a non-trivial cost function operating on current and previous symbols; this enables us to introduce the "sequential information gain cos"' function that can capture information gains accumulated at each time step. We emphasize how this problem formulation leads to a different style of coding scheme with a control-theoretic flavor. Further, we solve for structural results on these optimal policies using dynamic programming principles. We then demonstrate another interplay between information theory and control theory, at the level of reliability of message-point communication schemes, by establishing a relationship between "reliability" in feedback communication to the "stability" of the posterior belief's nonlinear filter. We also consider the two-agent inverse optimal control (IOC) problem, where a fixed policy satisfying certain statistical conditions is shown to be optimal for some cost function, using probabilistic matching. We provide examples of the applicability of this framework to communication with feedback, hidden Markov models and the nonlinear filter, decentralized control, brain-machine interfaces, and queuing theory. [The dissertation citations contained here are published with the permission of ProQuest LLC. Further reproduction is prohibited without permission. Copies of dissertations may be obtained by Telephone (800) 1-800-521-0600. Web page: http://www.proquest.com/en-US/products/dissertations/individuals.shtml.]

Many concepts of information have been proposed and discussed in library and information science. These concepts of information can be broadly categorized as empirical and situational information. Unlike nomenclatures in many sciences, however, the concept of information in library and information science does not bear a generally accepted definition and does not serve as the basis or assumptions for research studies. Nevertheless, every use of the term "information" and every conceptualization of information are based on certain epistemological assumptions. How do we understand "information" as a foundational concept of information science and what are the consequences? Based on the social ontology suggested in the theory of communicative action, this study shows how the understanding of the term 'information' could be communicatively constructed and how it may become "technicized" due to the systematic collection, storage, and organization of information. Technicized information not only means that the production of information may be subject to control and manipulation, it also entails that users may accept the cultural and social values embedded in information infrastructures without question and criticism. Epistemological and methodological issues are important and crucial because data or information are decontextualized and recontextualized at different levels in information infrastructures and hence should be a primary concern in information, information-related, and data-driven research. [The dissertation citations contained here are published with the permission of ProQuest LLC. Further reproduction is prohibited without permission. Copies of dissertations may be obtained by Telephone (800) 1-800-521-0600. Web page: http://www.proquest.com/en-US/products/dissertations/individuals.shtml.]

This thesis deals with "large-scale" detection problems that arise in many real world applications such as sensor networks, mapping with mobile robots and group testing for biological screening and drug discovery. These are problems where the values of a large number of inputs need to be inferred from noisy observations and where the transformation from input to measurement occurs because of a "physical" process. In particular, we focus on sparse measurement systems. We use the term sparse measurement system to refer to applications where every observation is a (stochastic) function of a small number of inputs. Here, small is relative to the total input size. Such a system can conveniently be represented by a (sparse) structured graphical model. We study the fundamental limits of performance of these sparse measurement systems through an information theoretic lens and analyze robustness to noise, model mismatch and parameter uncertainty. We also look at these problems from an algorithmic point of view and develop practical algorithms, aided by the representation of the system as a graphical model. We analyze how the computational cost of detection with sparse measurements changes with various system parameters. Finally, we will show how to use both these analyses to "design" sparse measurement systems. We show that, in addition to sensor parameters such as the measurement function and the noise model, which describe how the physical measurement transforms the inputs, the "structure" of the measurements critically effects the information theoretic and computational properties associated with a measurement "system". The term "structure" here refers to properties such as the number of times each input is measured and the pattern in which inputs are sensed--parameters that become important when information from many measurements has to be fused to reconstruct the input. The measurement "system" extends beyond the inputs and the measurements to include other factors such a show the inputs were generated, and how the measurements are fused to detect the input. We will show in this thesis that looking at a measurement system as a whole, in this way, leads to insights that can be used to improve system performance and to design better systems. The way we handle the correlations introduced by the fixed, sparse measurement structure and given measurement function, is what differentiates our use of the probabilistic method from other work in coding theory. The aim of this thesis is to develop tools for designers and users of sparse measurement systems. Given the characteristics of the inputs to be detected and constrained choices of measurement types and structures, we help answer the following types of questions for sparse measurement systems: (1) How many measurements are required to detect the input? (2) What algorithms can be used for detection? (3) How can the algorithm be modified to suit a particular application? (4) How should the system design be changed to improve performance and robustness? (5) How do these answers change with different sensing functions, measurement structures, application requirements and detection algorithms? Providing an answer to these questions allows an engineer to design a sparse measurement system that meets specified performance objectives while respecting various physical constraints. To demonstrate the impact of the ideas presented in this thesis we show results on two distinct real-world applications, by a combination of data driven modeling and simulation. We study two algorithms in some detail, the Cross Entropy (CE) method developed for rare event simulation and the Sequential Decoding (SD) algorithm from coding theory. In the first experiment we show how to use a sparse measurement system for a group testing based drug discovery application. We demonstrate how group testing with CE decoding can reduce the amount of noise while limiting the number of tests. In the second experiment we show that a simple Infra-Red sensor can be used to detect multiple hot targets--motivated by the important problem of finding people in disaster areas or in collapsed buildings. We validate the feasibility of our SD approach in an experiment using an actual Infra-Red temperature sensor. [The dissertation citations contained here are published with the permission of ProQuest LLC. Further reproduction is prohibited without permission. Copies of dissertations may be obtained by Telephone (800) 1-800-521-0600. Web page: http://www.proquest.com/en-US/products/dissertations/individuals.shtml.]

Consider a discrete bivariate random variable (X, Y) with possible values x[subscript 1], x[subscript 2],..., x[subscript I] for X and y[subscript 1], y[subscript 2],..., y[subscript J] for Y. Further suppose that the corresponding families of conditional distributions, for X given values of Y and of Y for given values of X are available. We specifically consider those situations where the above mentioned conditional distributions are not compatible. In such a case we seek a joint probability matrix (P), say, that is minimally incompatible with the given conditional distributions. The Kullback-Leibler Information function provides a convenient measure of (pseudo) distance between two distributions. However we will use a more general measure which is called the "power divergence criterion" which includes the Kullback-Leibler Function as a special case. We will, along with this measure, also consider some other measures of diversity which are widely used in the field of Information Theory. Using all these measures we have developed algorithms for finding the joint probability matrix which is minimally incompatible with the given conditionals. We will also propose here some alternative measures of compatibility related to discrepancy. Our main objective here is to find among the various measures of discrepancy (for example the power divergence test statistic, modified Renyi's measure etc.), along with the proposed measures, which one give us the minimally incompatible distribution with a faster convergence rate. This topic will be discussed in detail in chapter 1. Next we consider an alternative approach to determine whether or not any two given matrices (say, A and B) with non negative elements (where for the matrix A, the column sums add up to one and for the matrix B, the row sums add up to one) are compatible in the sense that there exists a joint probability matrix for which has the columns and rows, respectively, of A and B as its conditional distributions. We formulate the above problem as a homogeneous and consistent set of equations and consider the LPP (Linear Programming Problem) approach to solve for the unknown quantities. Furthermore we will also discuss briefly, under the condition of compatibility, how can we find some of the elements of the two given conditional matrices A and B, in case they are unknown. This is the subject matter for chapter 2. Next we consider the hidden truncation paradigm for the bivariate Pareto (type II) distribution when one variable is subject to hidden truncation from above. We consider the classical method of estimation and a reasonable testing procedure for the truncation parameter and also the other parameters involved in the model along with an application of the above mentioned model to a real life data. We will also focus on the estimation procedure under the Bayesian paradigm. This will be the subject matter in chapter 3. In chapter 4 we will consider the hidden truncation paradigm for a bivariate Pareto (type IV) distribution where both the marginals as well as the conditionals are again members of the Pareto (type IV) family. Here also we will consider inference for such a distribution under the classical approach as well as the Bayesian approach along with an application to a real life data. Next we will consider a possible extension of hidden truncation concept to the multivariate case for the Pareto (type II) family. In particular we will focus on single variable truncation as well as more than one variable truncation. We will discuss about the tractability of such type of models in the context of estimation and testing for the parameters involved in the model. In particular we focus on a trivariate Pareto (type II) set-up and we will consider estimation procedures under both the classical and Bayesian paradigm. Two specific situations are dealt with: (1) when one of the concomitant variables is truncated from above and (2) when more than one variable is truncated from above. This material is discussed in detail in chapter 5. In chapter 6 we provide a general discussion of the topics which are covered in chapter 1 through 5, together with a brief discussion of potential future work. [The dissertation citations contained here are published with the permission of ProQuest LLC. Further reproduction is prohibited without permission. Copies of dissertations may be obtained by Telephone (800) 1-800-521-0600. Web page: http://www.proquest.com/en-US/products/dissertations/individuals.shtml.]

This research takes steps towards developing a new theory of organizational information management based on the ideas that, first, information creates ordering effects in transactions and, second, that there are multiple centers of authority in organizations. The rationale for developing this theory is the empirical observation that hospitals have great difficulty in managing information relating to transactions with patients. The research illustrates the detailed workings of an initial conceptual framework based on an action research project into the revenue cycle of a hospital. The framework facilitates a deeper understanding of how information technology can help to transform information management practices in complex organizations, such as hospitals. At the same time, this research adds to the literature on Polycentricity Theory by linking its two core concepts--multiple nested centers of decision making and context-dependent governance--with Transaction Cost Theory and information management theories to establish a new foundation for understanding the role of information technology in organizational contexts. [The dissertation citations contained here are published with the permission of ProQuest LLC. Further reproduction is prohibited without permission. Copies of dissertations may be obtained by Telephone (800) 1-800-521-0600. Web page: http://www.proquest.com/en-US/products/dissertations/individuals.shtml.]

This thesis studies several problems in information theory where the notion of causality comes into play. Causality in information theory refers to the timing of when information is available to parties in a coding system. The first part of the thesis studies the error exponent (or reliability function) for several communication problems over discrete memoryless channels. In particular, it studies an upper bound to the error exponent, or equivalently, a lower bound to the error probability of general codes, called the Haroutunian exponent. The Haroutunian exponent is the best known upper bound to the error exponent for two channel coding problems: fixed blocklength coding with feedback and fixed-delay coding without feedback. For symmetric channels like the binary symmetric channel and the binary erasure channel, the Haroutunian exponent evaluates to the sphere-packing exponent, but for asymmetric channels like the Z-channel, the Haroutunian exponent is strictly larger than the sphere-packing exponent. The reason for the presumed looseness of the Haroutunian exponent is that it assumes, despite the inherent causality of feedback, a code might be able to predict future channel behavior based on past channel behavior and accordingly tune its input distribution. Intuitively, this kind of noncausal information should not be available to an encoder when the channel is memoryless. While we have not been able to tighten the Haroutunian exponent to the sphere-packing exponent for fixed blocklength codes with feedback, we describe some attempts made at bridging the gap. Additionally, we describe how to tighten the upper bound for two cases when the encoder is somehow limited: if the encoding strategy is constrained to use fixed type inputs regardless of output sequence and if there is a delay in the feedback path. The latter of these results leads to the insight that the Haroutunian exponent of a parallel channel constructed of independent uses of the original asymmetric channel approaches the sphere-packing exponent of the original channel after normalization. This fact can then be used to show that the error exponent for fixed-delay codes is upper bounded by the sphere-packing exponent. The second part of the thesis studies lossy compression of the arbitrarily varying sources introduced by Berger in his paper entitled "The Source Coding Game". An arbitrarily varying source is a model for a source that samples other subsources under the control of an agent called a switcher. Motivated by compression of active vision sources, we seek upper and lower bounds for the rate-distortion function of an arbitrarily varying source when the switcher has noncausal knowledge about the realizations of the subsources it samples from. We find that when the subsources are memoryless, noncausal knowledge of subsource realizations is strictly better than information about past subsource realizations only. [The dissertation citations contained here are published with the permission of ProQuest LLC. Further reproduction is prohibited without permission. Copies of dissertations may be obtained by Telephone (800) 1-800-521-0600. Web page: http://www.proquest.com/en-US/products/dissertations/individuals.shtml.]

An unprecedented amount of information encompassing almost every facet of human activities across the world is generated daily in the form of zeros and ones, and that is often the only form in which such information is recorded. A good fraction of this information needs to be preserved for periods of time ranging from a few years to centuries. Consequently, the problem of preserving digital information over a long-term has attracted the attention of many organizations, including libraries, government agencies, scientific communities, and individual researchers. In this dissertation, we address three issues that are critical to ensure long-term information preservation and access. The first concerns the core requirement of how to guarantee the integrity of preserved contents. Digital information is in general very fragile because of the many ways errors can be introduced, such as errors introduced because of hardware and media degradation, hardware and software malfunction, operational errors, security breaches, and malicious alterations. To address this problem, we develop a new approach based on efficient and rigorous cryptographic techniques, which will guarantee the integrity of preserved contents with extremely high probability even in the presence of malicious attacks. Our prototype implementation of this approach has been deployed and actively used in the past years in several organizations, including the San Diego Super Computer Center, the Chronopolis Consortium, North Carolina State University, and more recently the Government Printing Office. Second, we consider another crucial component in any preservation system--searching and locating information. The ever-growing size of a long-term archive and the temporality of each preserved item introduce a new set of challenges to providing a fast retrieval of content based on a temporal query. The widely-used cataloguing scheme has serious scalability problems. The standard full-text search approach has serious limitations since it does not deal appropriately with the temporal dimension, and, in particular, is incapable of performing relevancy scoring according to the temporal context. To address these problems, we introduce two types of indexing schemes--a location indexing scheme, and a full-text search indexing scheme. Our location indexing scheme provides optimal operations for inserting and locating a specific version of a preserved item given an item ID and a time point, and our full-text search indexing scheme efficiently handles the scalability problem, supporting relevancy scoring within the temporal context at the same time. Finally, we address the problem of organizing inter-related data, so that future accesses and data exploration can be quickly performed. We, in particular, consider web contents, where we combine a link-analysis scheme with a graph partitioning scheme to put together more closely related contents in the same standard web archive container. We conduct experiments that simulate random browsing of preserved contents, and show that our data organization scheme greatly minimizes the number of containers needed to be accessed for a random browsing session. Our schemes have been tested against real-world data of significant scale, and validated through extensive empirical evaluations. [The dissertation citations contained here are published with the permission of ProQuest LLC. Further reproduction is prohibited without permission. Copies of dissertations may be obtained by Telephone (800) 1-800-521-0600. Web page: http://www.proquest.com/en-US/products/dissertations/individuals.shtml.]

This dissertation investigates how distributional information in the speech stream can assist infants in the initial stages of acquisition of their native language phonology. An exploratory statistical analysis derives this information from the adult speech data in the corpus of conversations between adults and young children in Russian. Because the data were recorded in the traditional Russian orthography and spelling, the statistical analysis was performed on the symbolic data. International Phonetic Alphabet ensured uniformity of the speech sound representation as well as enabled the interpretation of the symbols as unique sets of distinctive feature values. Although the mechanism of information extraction employed in this work may vary with the representation, the objective of the study was to demonstrate that linguistically relevant information available to infants in the speech signal is not limited to the surface form, which they observe directly. The regularities in the distribution of the bigrams in the data reveal considerable predictability of occurrence in those characteristics of the speech sounds that information theory treats as unpredictable. Co-occurrence of distinctive features prominent in Russian provides a perceptually salient context for their identification and learning. In addition to suggesting a progression for the initial stages in the acquisition of the phonology of the Russian language, this dissertation contributes to the discussion of the degree to which human language is innate. While the innateness hypothesis is consistent with the data, the results suggest that it may be too strong. If the phonological features were inborn rather than acquired, humans would have had a specialized mechanism enabling their perception and production with a high degree of consistency across different languages. We would have been predisposed to recognize them in any phonetic environment, which means that their distribution in the speech signal can be quite random. An observed pattern is necessary from an empirical perspective, in which the distinctive features are learned from the input possibly through association. [The dissertation citations contained here are published with the permission of ProQuest LLC. Further reproduction is prohibited without permission. Copies of dissertations may be obtained by Telephone (800) 1-800-521-0600. Web page: http://www.proquest.com/en-US/products/dissertations/individuals.shtml.]

This dissertation deals with a multiterminal source model for secret key generation by multiple network terminals with prior and privileged access to a set of correlated signals complemented by public discussion among themselves. Emphasis is placed on a characterization of secret key capacity, i.e., the largest rate of an achievable secret key, and on algorithms for key construction. Various information theoretic security requirements of increasing stringency: weak, strong and perfect secrecy, as well as different types of sources: finite-valued and continuous, are studied. Specifically, three different models are investigated. First, we consider strong secrecy generation for a discrete multiterminal source model. We discover a connection between secret key capacity and a new source coding concept of "minimum information rate for signal dissemination," that is of independent interest in multiterminal data compression. Our main contribution is to show for this discrete model that structured linear codes suffice to generate a strong secret key of the best rate. Second, strong secrecy generation is considered for models with continuous observations, in particular jointly Gaussian signals. In the absence of suitable analogs of source coding notions for the previous discrete model, new techniques are required for a characterization of secret key capacity as well as for the design of algorithms for secret key generation. Our proof of the secret key capacity result, in particular the converse proof, as well as our capacity-achieving algorithms for secret key construction based on structured codes and quantization for a model with two terminals, constitute the two main contributions for this second model. Last, we turn our attention to perfect secrecy generation for fixed signal observation lengths as well as for their asymptotic limits. In contrast with the analysis of the previous two models that relies on probabilistic techniques, perfect secret key generation bears the essence of "zero-error information theory," and accordingly, we rely on mathematical techniques of a combinatorial nature. The model under consideration is the "Pairwise Independent Network" (PIN) model in which every pair of terminals share a random binary string, with the strings shared by distinct pairs of terminals being mutually independent. This model, which is motivated by practical aspects of a wireless communication network in which terminals communicate on the same frequency, results in three main contributions. First, the concept of perfect omniscience in data compression leads to a single-letter formula for the perfect secret key capacity of the PIN model; moreover, this capacity is shown to be achieved by linear noninteractive public communication, and coincides with strong secret key capacity. Second, taking advantage of a multigraph representation of the PIN model, we put forth an efficient algorithm for perfect secret key generation based on a combinatorial concept of maximal packing of Steiner trees of the multigraph. When all the terminals seek to share perfect secrecy, the algorithm is shown to achieve capacity. When only a subset of terminals wish to share perfect secrecy, the algorithm is shown to achieve at least half of it. Additionally, we obtain nonasymptotic and asymptotic bounds on the size and rate of the best perfect secret key generated by the algorithm. These bounds are of independent interest from a purely graph theoretic viewpoint as they constitute new estimates for the maximum size and rate of Steiner tree packing of a given multigraph. Third, a particular configuration of the PIN model arises when a lone "helper" terminal aids all the other "user" terminals generate perfect secrecy. This model has special features that enable us to obtain necessary and sufficient conditions for Steiner tree packing to achieve perfect secret key capacity. [The dissertation citations contained here are published with the permission of ProQuest LLC. Further reproduction is prohibited without permission. Copies of dissertations may be obtained by Telephone (800) 1-800-521-0600. Web page: http://www.proquest.com/en-US/products/dissertations/individuals.shtml.]

Quantum error-correcting codes have been the cornerstone of research in quantum information science (QIS) for more than a decade. Without their conception, quantum computers would be a footnote in the history of science. When researchers embraced the idea that we live in a world where the effects of a noisy environment cannot completely be stripped away from the operations of a quantum computer, the natural way forward was to think about importing classical coding theory into the quantum arena to give birth to quantum error-correcting codes which could help in mitigating the debilitating effects of decoherence on quantum data. We first talk about the six-qubit quantum error-correcting code and show its connections to entanglement-assisted error-correcting coding theory and then to subsystem codes. This code bridges the gap between the five-qubit (perfect) and Steane codes. We discuss two methods to encode one qubit into six physical qubits. Each of the two examples corrects an arbitrary single-qubit error. The first example is a degenerate six-qubit quantum error-correcting code. We explicitly provide the stabilizer generators, encoding circuits, codewords, logical Pauli operators, and logical CNOT operator for this code. We also show how to convert this code into a non-trivial subsystem code that saturates the subsystem Singleton bound. We then prove that a six-qubit code without entanglement assistance cannot simultaneously possess a Calderbank-Shor-Steane (CSS) stabilizer and correct an arbitrary single-qubit error. A corollary of this result is that the Steane seven-qubit code is the smallest single-error correcting CSS code. Our second example is the construction of a non-degenerate six-qubit CSS entanglement-assisted code. This code uses one bit of entanglement (an ebit) shared between the sender (Alice) and the receiver (Bob) and corrects an arbitrary single-qubit error. The code we obtain is globally equivalent to the Steane seven-qubit code and thus corrects an arbitrary error on the receiver's half of the ebit as well. We prove that this code is the smallest code with a CSS structure that uses only one ebit and corrects an arbitrary single-qubit error on the sender's side. We discuss the advantages and disadvantages for each of the two codes. In the second half of this thesis we explore the yet uncharted and relatively undiscovered area of quantum steganography. Steganography is the process of hiding secret information by embedding it in an "innocent" message. We present protocols for hiding quantum information in a codeword of a quantum error-correcting code passing through a channel. Using either a shared classical secret key or shared entanglement Alice disguises her information as errors in the channel. Bob can retrieve the hidden information, but an eavesdropper (Eve) with the power to monitor the channel, but without the secret key, cannot distinguish the message from channel noise. We analyze how difficult it is for Eve to detect the presence of secret messages, and estimate rates of steganographic communication and secret key consumption for certain protocols. We also provide an example of how Alice hides quantum information in the perfect code when the underlying channel between Bob and her is the depolarizing channel. Using this scheme Alice can hide up to four stego-qubits. [The dissertation citations contained here are published with the permission of ProQuest LLC. Further reproduction is prohibited without permission. Copies of dissertations may be obtained by Telephone (800) 1-800-521-0600. Web page: http://www.proquest.com/en-US/products/dissertations/individuals.shtml.]

The identification and examination of cultural information strategies and censorship patterns used to propagate the controversial issue of the caricatures in two separate cultural contexts was the aim of this dissertation. It explored discourse used for the coverage of this topic by one newspaper in a restrictive information context and two newspapers in a liberal information context. Message propagation in a restrictive information environment was analyzed using the English daily "Kuwait Times" from the Middle East; the liberal information environment of the US was analyzed using two major dailies, the "New York Times" and the "Philadelphia Inquirer." The study also concurrently identifies and elaborates on the themes and frames through which discourse was presented exposing the cultural ideologies and premises they represent. The topic was approached with an interdisciplinary position with the support and applicability testing of Chatman's insider-outsider theory within information science and Noelle-Neumann's spiral of silence theory and Herman and Chomsky's propaganda model based in the area of mass communication. The study has also presented a new model of information censorship--circle of information censorship, emphasizing conceptual issues that influence the selection and censorship of information. [The dissertation citations contained here are published with the permission of ProQuest LLC. Further reproduction is prohibited without permission. Copies of dissertations may be obtained by Telephone (800) 1-800-521-0600. Web page: http://www.proquest.com/en-US/products/dissertations/individuals.shtml.]

This dissertation proposes a model of phonological relationships, the Probabilistic Phonological Relationship Model (PPRM), that quantifies how predictably distributed two sounds in a relationship are. It builds on a core premise of traditional phonological analysis, that the ability to define phonological relationships such as contrast and allophony is crucial to the determination of phonological patterns in language. The PPRM starts with one of the long-standing tools for determining phonological relationships, the notion of predictability of distribution. Building on insights from probability and information theory, the model provides a way of calculating the precise degree to which two sounds are predictably distributed, rather than maintaining the traditional binary distinction between "predictable" and "not predictable." It includes a measure of the probability of each member of a pair in each environment they occur in, the uncertainty (entropy) of the choice between the members of the pair in each environment, and the overall uncertainty of choice between the members of the pair in a language. These numbers provide a way to formally describe and compare relationships that have heretofore been treated as exceptions, ignored, relegated to alternative grammars, or otherwise seen as problematic for traditional descriptions of phonology. The PPRM provides a way for what have been labelled "marginal contrasts," "quasi-allophones,""semi-phonemes," and the like to be integrated into the phonological system: There are phonological relationships that are neither entirely predictable nor entirely unpredictable, but rather belong somewhere in between these two extremes. The model, being based on entropy, which can be used to understand the cognitive function of uncertainty, provides insight into a number of phenomena in synchronic phonological patterning, diachronic phonological change, language acquisition, and language processing. Examples of how the model can be applied are provided for two languages, Japanese and German, using large-scale corpora to calculate the predictability of distribution of various pairs of sounds. An example of how empirical evidence for one of the predictions of the model, that entropy and perceptual distinctness are inversely related to each other, could be obtained is also provided. [The dissertation citations contained here are published with the permission of ProQuest LLC. Further reproduction is prohibited without permission. Copies of dissertations may be obtained by Telephone (800) 1-800-521-0600. Web page: http://www.proquest.com/en-US/products/dissertations/individuals.shtml.]

Closing the gap between theoretical reservoir operation and the real-world implementation remains a challenge in contemporary reservoir operations. Past research has focused on optimization algorithms and establishing optimal policies for reservoir operations. In this research, we attempt to understand operators' release decisions by investigating historical release data from 79 reservoirs in California and the Great Plains, using a data-mining approach. We use information theory--specifically, mutual information--to measure the quality of inference between a set of classic indicators and observed releases at the monthly and weekly timescales. Then we investigate the results obtained from the data mining step to discover knowledge about the behavior of reservoir operators under the various conditions. The central aim of this step is to understand the human behavior of reservoir operators in the face of many factors with particular attention to the value of hydrologic information and uncertainty. We also expand the state of art technique to determine the most relevant suite of hydrologic inputs to release datasets by introducing an input selection algorithm to measure the information relevance between a suite of inputs and a single output. We apply the proposed algorithm to 22 reservoirs in California to predict daily release based on a set from a 121 potential input variables. Results indicate that the proposed algorithm is a reliable measure of the competence of model inputs as reflected in enhanced model performance. Finally we couple the data mining procedure with conventional reservoir optimization techniques to build an enhanced stochastic dynamic programming (SDP) model. The enhanced SDP model is applied to the Shelbyville Reservoir, IL, and then compared to two classic SDP formulations. From a data mining procedure, past month's inflow, current month's inflow, past month's release, and past month's Palmer drought severity index are found to be important state variables in the enhanced SDP model formulations for Shelbyville Reservoir. The study indicates that the enhanced SDP model resembles historical records more closely yet provides lower expected average annual costs than either of the two classic formulations (25.4% and 4.5% reductions). [The dissertation citations contained here are published with the permission of ProQuest LLC. Further reproduction is prohibited without permission. Copies of dissertations may be obtained by Telephone (800) 1-800-521-0600. Web page: http://www.proquest.com/en-US/products/dissertations/individuals.shtml.]

This work attempts to explain the relationships between natural selection, information theory, and statistical inference. In particular, a geometric formulation of information theory known as information geometry and its deep connections to evolutionary game theory inform the role of natural selection in evolutionary processes. The goals of this work may be stated as (1) to show that a significant collection of important results in evolutionary dynamics can be interpreted elegantly in the context of information theory, and (2) to generalize the framework to study dynamics other than the replicator equation. [The dissertation citations contained here are published with the permission of ProQuest LLC. Further reproduction is prohibited without permission. Copies of dissertations may be obtained by Telephone (800) 1-800-521-0600. Web page: http://www.proquest.com/en-US/products/dissertations/individuals.shtml.]

This dissertation studies information extraction (IE), the problem of extracting structured information from unstructured data. Example IE tasks include extracting person names from news articles, product information from e-commerce Web pages, street addresses from emails, and names of emerging music bands from blogs. IE is all increasingly important problem in a broad range of applications that seek to utilize the growing amount of unstructured data available today. Such applications include structured community Web portals, data integration systems, and data mining applications over text data. However, despite significant progress, managing IE and building end-to-end IE applications still involves many difficult challenges, including writing complex IE programs and optimizing them, deciding how to store and process the large amounts of data the IE applications manage, and executing and obtaining meaningful results from partially specified or approximate IE programs (e.g., during the development process, or in scenarios where an approximate result may already be sufficient). In this dissertation, we develop solutions to the key challenges mentioned above. First, we develop a declarative framework that can help make it easier for developers to write and understand IE programs, and show how to automatically optimize IE programs written in this framework to reduce runtime. Next, given that relational database systems (RDBMSs) were designed to store and process large data sets, we study the benefits and limitations of employing RDBMSs for storing and processing data in IE applications. Finally, we extend our declarative framework to enable "best-effort IE," allowing developers to more easily write and refine approximate IE programs. A key idea underlying these solutions is that many of the principles behind RDBMSs for managing structured data can be extended to IE for managing unstructured data. [The dissertation citations contained here are published with the permission of ProQuest LLC. Further reproduction is prohibited without permission. Copies of dissertations may be obtained by Telephone (800) 1-800-521-0600. Web page: http://www.proquest.com/en-US/products/dissertations/individuals.shtml.]

This dissertation sought to answer the question: Is it possible to build a software tool that will allow teachers to write better multiple-choice questions? The thesis proceeded from the finding that the quality of teaching is very influential in the amount that students learn. A basic premise of this research, then, is that improving teachers will improve learning. With this foundation, the next question became what area of teaching to improve. The literature on educational assessment indicated that teachers lack competence at effective assessment, particularly in the area of multiple-choice question generation. It is likely that improvement in this area would yield large gains in educational achievement by students. Several areas of literature including teacher professional development, modification of health-related behaviors, and the information systems theories of captology and structuration theory were synthesized to develop a general model for designing systems to foster teacher professional development. This model was then applied to design and build a tool, QuesGen--a web-based system to help teachers write better multiple-choice questions. The tool was evaluated. Quantitative and qualitative results are presented, their implications discussed, and future steps are laid out. [The dissertation citations contained here are published with the permission of ProQuest LLC. Further reproduction is prohibited without permission. Copies of dissertations may be obtained by Telephone (800) 1-800-521-0600. Web page: http://www.proquest.com/en-US/products/dissertations/individuals.shtml.]

Implementation of information systems has lagged in many areas of clinical healthcare for a variety of reasons. Economics, data complexity and resistance are among the often quoted roadblocks. Research suggests that physicians play a major part in the adoption, use and diffusion of information technology (IT) in clinical settings. There are also other healthcare professionals, clinical and non-clinical, who play important roles in making decisions about the acquisition of information technology. In addition to these groups there are information technology professionals providing the services required within the healthcare field. Finally within this group are those F1 professionals who have sufficient cross training to understand specific needs. Each member of these groups brings a different perspective to both needs assessments as well as implementation of clinical systems. This study considers the idea that there are preconceived differences of opinion of the information needs of clinical healthcare by the clinical community and the information technology professionals. Are these differences significant enough to create a barrier to implementation? A questionnaire was developed from preliminary data to assess multiple parameters which could impact implementation of a clinical information technology solution. A Web of System Performance (WOSP) model was created to map each of the following eight areas of concern: functionality, usability, extendibility, connectivity, flexibility, reliability, privacy and security. Responses to the questions were related to professional roles, age and experience. There were no differences seen in the perceived need for secure systems by either healthcare workers or IT professionals. The variance of perceived need was greatest among the various non-physician healthcare workers when compared to physicians or information technology professions. This was a consistent pattern for the other parameters with the exception of the usability of the electronic health record. In this area all groups disagreed significantly. The study, though limited by its small sample, still suggests that the resistance by healthcare professionals is not a significant barrier to successful information technology implementation. [The dissertation citations contained here are published with the permission of ProQuest LLC. Further reproduction is prohibited without permission. Copies of dissertations may be obtained by Telephone (800) 1-800-521-0600. Web page: http://www.proquest.com/en-US/products/dissertations/individuals.shtml.]

Data Definition Languages (DDLs) have been created and used to represent data in programming languages and in database dictionaries. This representation includes descriptions in the form of data fields and relations in the form of a hierarchy, with the common exception of relational databases where relations are flat. Network computing created an environment that enables relatively easy and inexpensive exchange of data. What followed was the creation of new DDLs claiming better support for automatic data integration. It is uncertain from the literature if any real progress has been made toward achieving an ideal state or limit condition of automatic data integration. This research asserts that difficulties in accomplishing integration are indicative of socio-cultural systems in general and are caused by some measurable attributes common in DDLs. This research's main contributions are: (1) a theory of data integration requirements to fully support automatic data integration from autonomous heterogeneous data sources; (2) the identification of measurable related abstract attributes (Variety, Tension, and Entropy); (3) the development of tools to measure them. The research uses a multi-theoretic lens to define and articulate these attributes and their measurements. The proposed theory is founded on the Law of Requisite Variety, Information Theory, Complex Adaptive Systems (CAS) theory, Sowa's Meaning Preservation framework and Zipf distributions of words and meanings. Using the theory, the attributes, and their measures, this research proposes a framework for objectively evaluating the suitability of any data definition language with respect to degrees of automatic data integration. This research uses thirteen data structures constructed with various DDLs from the 1960's to date. No DDL examined (and therefore no DDL similar to those examined) is designed to satisfy the law of requisite variety. No DDL examined is designed to support CAS evolutionary processes that could result in fully automated integration of heterogenous data sources. There is no significant difference in measures of Variety, Tension, and Entropy among DDLs investigated in this research. A direction to overcome the common limitations discovered in this research is suggested and tested by proposing GlossoMote, a theoretical mathematically sound description language that satisfies the data integration theory requirements. The DDL, named GlossoMote, is not merely a new syntax, it is a drastic departure from existing DDL constructs. The feasibility of the approach is demonstrated with a small scale experiment and evaluated using the proposed assessment framework and other means. The promising results require additional research to evaluate GlossoMote's approach commercial use potential. [The dissertation citations contained here are published with the permission of ProQuest LLC. Further reproduction is prohibited without permission. Copies of dissertations may be obtained by Telephone (800) 1-800-521-0600. Web page: http://www.proquest.com/en-US/products/dissertations/individuals.shtml.

This study compared the relative effectiveness of two communication-skills training models: Ivey's Microcounseling Paradigm and Carkhuff's Systematic Human Relations Training Model. After 107 educational psychology students volunteered to participate in the group that fit their schedules, treatments were randomly assigned. Pretesting and posttesting of all subjects was conducted by administering the Index of Communication; responses scored on the scale for Gross Ratings of Facilitative Interpersonal Functioning yielded a function level for each subject before and after training. An analysis of covariance, evaluating the difference between groups with regard to levels of functioning, found significant differences between the two training models and between the experimental groups and the control group. (Author/RL)

Four hundred fifty subjects at Texas A&M University provided data, through self-administered questionnaires, to evaluate the influence on readers of qualifying statements regarding technical performance claims in magazine advertisements. Each respondent was exposed to a single treatment by viewing one of 30 versions of mock magazine advertisements. Results suggested the following: confidence in a claim was affected by use and intensity of qualification and by the reader's familiarity with the product; the image of the claim was affected by use of qualification and test source; the image of the product was affected by the intensity and the layout of the qualification; and the image of the advertiser was not appreciably affected by any of the independent variables. The final chapter of the study includes a discussion of various implications for regulatory policy makers and advertising strategists. Suggestions for further research are also presented. (Author/RL)

This study tests the relationship between a set of predictor variables that offer a possible explanation of communication apprehension and a set of criterion variables closely associated with communication apprehension. The predictor variables, emerging from "the double-bind theory of communication," which is concerned with the pathology-producing capability of dysfunctional communication patterns, consisted of the frequency of inconsistent messages received, the ability to recognize inconsistent messages, the ability to comment on the inconsistencies, and the intensity of the relationship between the parties involved. These variables focused on the parent/child relationship. Criterion variables included communication apprehension of the speaking situation, communication apprehension of the receiving situation, and the related personality traits of anxiety and introversion/extroversion. Results of the study indicate an interrelatedness of the criterion variables, while the double-bind variables are relatively independent of each other. Since the double-bind variables are not related, they serve as independent predictors of criterion variables, and the finding of a significant relationship between the double-bind variables and criterion variables is essentially an extension of the double-bind theory. (Author/RL)

Beginning in fall 1981, a course in communication and information theory was offered at the School of Natural Resources (University of Michigan). Throughout the course, emphasis was placed on comparing communication strategies and effectiveness as well as providing models for interdisciplinary research and data management. Case study materials and decision-making activities were frequently employed. Issues covered were derived from reports of United Nations Educational, Scientific and Cultural Organization (UNESCO) International Commission for the Study of Communication Problems and other UNESCO materials on the New International Information and Communication Order. A brief description and evaluation of the course, the 12 course units, and bibliography are provided in this document. Unit 1 is the course overview/introduction. Units 2 and 3 focus on the nature of information (objectivity/observables and interpretation). Units 4-6 focus on sources (indexing for management, information theory and integrative research, and information management models). Units 7-9 address information dissemination concerns (communication theory, mass media, and alternative media). Decision-making and effective communication, information technology and futures, and international information concerns are considered in units 10-12 respectively. Course outline, reading list, and course evaluation form are included in an appendix. Overall favorable student/teacher reaction to the course is reported. Revisions based on student/teacher evaluations are noted. (Author/JN)

The purposes of this study were twofold: (1) to test the hypothesis that a Similarity Rating Model based on a Classification Scheme of relationships between six words in physics is representative of human subjects' judgments of similarity between the six words, and (2) to test the hypothesis that the Classification Scheme could be used in a Word Association Model to generate responses in a word association test like those produced by human subjects. The Clsssification Scheme was in the form of a three-dimensional space with the six chosen physics words plotted as points in space. Pairwise similarity in the Classification Scheme was defined as distance between words. In this research stoachastic models of human response behavior were designed based on the Classification Scheme, in the form of FORTRAN computer programs. Ten independent sets of data were generated under each model. Strong and consistent correlations were found between data from the Similarity Rating Model and human data. These findings were interpreted as evidence that the Rating Model was a valid representation of human subjects' judgments of similarity, and the Classification Scheme could thus be used to generate these responses. Moderate to low correlations were found between model data and human data in word association tests, thus interpreted as neither a valid nor reliable representation of human word association behavior. (Author/EB)

It is a common misconception in data analysis that more data equates to more knowledge. However, as data becomes bigger, the methods required to decipher and visualise critical information become an ever more cumbersome task. A central problem in data analysis is identifying and understanding relationships in complex systems. Mutual information has proven valuable in this regard and is already a crucial measure in many data analysis and machine learning tasks. Nearest neighbour techniques are a classic approach in non-parametric statistics and have proven effective in entropy estimation. Unfortunately, it is well established that estimating entropy is fraught with difficulties, especially for discrete-continuous mixed cases. In this thesis, we develop an ensemble method to estimate information-theoretic measures using a novel noisy resampling technique. The method is empirically shown to be asymptotically unbiased and consistent. Moreover, through artificial and real-world experiments, we show that the approach repeatedly outperforms the current leading k-nearest-neighbour methods - the Kozachenko-Leonenko estimator and the KSG estimator - to achieve a more accurate and robust estimate for discrete and continuous random variables alike. This ability is essential in classification problems, where the class variable is often discrete, significantly widening the applicability of mutual information measures in data analysis. In real-world domains, the proposed method successfully identifies key variables supported by machine learning results. New algorithms are implemented in an exploratory analysis tool for multivariate data. We investigate the visualisations currently used for multi-dimensional data and introduce DataViewer, a visualisation software package for exploring patterns in modern data sets. We propose a variable interaction diagram for illustrating variable correlations with significant mutual information. The software is designed to aid interpretation of complex data structures, which in turn motivates intelligent feature selection. The techniques are illustrated by application to several real world data sets.

There are universal patterns in colour naming (Berlin & Kay, 1991; Cook et al., 2005) and the colour categories in a language influence colour perception during perceptual uncertainty (Bae et al., 2015). This suggests that that colour categorisation is not structured at random, but rather the structure is constrained to serve an adaptive purpose. Previous research has focused on identifying constraints on language and the human visual system to explain the origins of universal colour categories, but there was little evidence demonstrating ecological constraints on colour categories. This thesis proposes that the distribution of colours across objects in the environment place an ecological constraint on colour perception and categorisation. If there are systematic patterns in colouration across behaviourally relevant objects, then colour categories may be formed to guide perceptual processes towards visual stimuli which are the most important. To explore this, two tasks were used: a colour estimation task to measure the geometry of perceptual colour space, and an object recognition task to measure the distribution of colours across objects. Humans exhibited categorical biases in colour perception across hue and saturation towards universal colour categories, corresponding to red, blue, green, pink, orange and grey. These categorical biases were successfully estimated from the statistical regularities of colours across objects for two independent image datasets: colour perception was biased towards category foci which were most informative about objects, and away from category boundaries which were least informative about objects. These findings support a universal mechanism through which sensory systems adapt to environmental statistics, allowing for optimal representation of sensory features under uncertainty for both perception and language.

Progress towards more reliable weather and climate forecasts is limited by the resolution of numerical models and the complexity of simulated processes. Performance is therefore a major bottleneck and current models are not com- putationally efficient. High precision calculations are unnecessary, despite be- ing the standard, given the uncertainties in the climate system and the errors from discretisation, data assimilation and unresolved climate processes. In this thesis, we advance several aspects of low-precision climate computing to preserve information despite fewer bits: An information-preserving com- pression is developed that distinguishes between real and false information to reduce the very large volume of climate data produced by numerical mod- els, while minimising information loss. The bitwise real information content estimates the minimum required precision in climate data, which depends on the variable and is lower than the standard precision-levels of floating-point numbers. The impact of rounding errors introduced by different low-precision arithmetics with deterministic or stochastic rounding modes is analysed in chaotic dynamical systems. Standard floating-point numbers are not the best number format for weather and climate simulations. However, alternatives, such as posits, exist, but it is unclear whether the large effort needed to de- velop the respective hardware for future supercomputers is justified given the moderate advantage they provide in our applications. A much more central issue towards 16-bit climate models is the design of low precision-resilient al- gorithms. A naive transition to 16 bits either fails or was found to cause issues like amplified gravity waves, a change in geostrophy or rounding errors that grow as quickly as discretisation errors. However, many of these issues are found to be preventable with techniques such as scaling or a compensated time integration. Combining techniques, we develop a 16-bit fluid circulation model that approaches 4x speedups on Fujitsu's A64FX processor compared to 64 bits, despite minimal rounding errors. The result of this thesis show that there is little reason to assume that 16-bit weather and climate models are not possible. While the design of models to compute and output only the bitwise real information is challenging, it will be a major step towards computationally efficient digital twins of the Earth's climate system.

This thesis consists of three essays that explore the use of probabilistic machine learning techniques in combination with information-theoretic concepts to answer economic questions. Over the past years, economists have started applying machine learning methods to a wide range of topics. Probabilistic methods in the context of unsupervised learning represent one particular modelling approach at the intersection of computer science and statistics. While widely used in applied statistics, these models, however, do not necessarily provide relevant and interpretable outputs from an economist's perspective. In this thesis, I appeal to information-theoretic methods to summarise the probabilistic information inferred from such models and construct economically meaningful measures.

A key objective in an interventional study, such as a randomised clinical trial, is the evaluation of heterogeneity of treatment effect in the population. This allows us to identify the most promising intervention for a given observation. In this thesis we approach this by targeting two tightly coupled sub-problems. The first concerns the identification of covariates and the second the identification of subgroups associated with treatment effect heterogeneity. Regarding the first problem we study an information theoretic approach. This can be motivated by phrasing the predictive covariate selection problem in log-likelihood terms. We study the properties of this approach in the case of randomised studies and evaluate low-dimensional approximations that are better suited for small-sample and/or high-dimensional studies. We identify some limitations and propose extensions based on propensity score weighting and stratification that extend this criterion in scenarios when the treatment assignment depends on the covariates. Regarding the second problem, we discuss recursive partitioning approaches coupled with weighting methods for treatment effect estimation. The purpose of these methods is to tackle the problem of subgroup identification in the presence of confounders in the data. Finally, studying the literature of subgroup identification we identify a significant number of approaches. Given such a large number of methods to choose from, an important question is how to select the best for a given task. We introduce a framework that uses the subgroup stability as a measure to capture the variations in the identified subgroups due to small changes in the data.

This thesis presents a processual ontology of noise by virtue of which morphogenesis (in its most general understanding as the processes by which order/form is created) must be instantiated. Noise is here outlined as the far from equilibrium environment out of which metastable temporary 'solutions' can emerge as the system transitions through the pre-individual state space. While frequently addressed by humanities and arts studies on the basis of its supposed disruptive character (often in terms of aesthetics), this thesis aims to thoroughly examine noise's conceptual potencies. To explore and amplify the epistemic consequences not merely of the ineliminability of noise but of its originative power as well as within the course of the elimination of givenness by epistemology. This philosophical work is informed by many different fields of contemporary science (namely: statistical physics, information theory, probability theory, 4E cognition, synthetic biology, nonlinear dynamics, complexity science and computer science) in order to assess and highlight the problems of the metascientific and ideological foundations of diverse projects of prediction and control of uncertainty. From algorithmic surveillance back to cybernetics and how these rendered noise "informationally heretical". This conveys an analysis of how contemporary prediction technologies are dramatically transforming our relationship with the future and with uncertainty in a great number of our social structures. It is a philosophico-critical anthropology of data ontology and a critique of reductive pan-info-computationalism. Additionally, two practical examples of noise characterised as an enabling constraint for the functioning of complex adaptive systems are presented. These are at once biophysical and cognitive, : 1) interaction-dominance constituted by 'pink noise' and 2) noise as a source of variability that cells may exploit in (synthetic) biology. Finally, noise is posited as an intractable active ontological randomness that limits the scope of determinism and that goes beyond unpredictability in any epistemological sense due to the insuperability of the situation in which epistemology finds itself following the critique of the given.

Sparse regression codes (SPARCs) are a class of channel codes for efficient communication over the single-user additive white Gaussian noise (AWGN) channel at rates approaching the channel capacity. In a standard SPARC, codewords are sparse linear combinations of columns of an i.i.d. Gaussian design matrix, and the user message is encoded in the indices of those columns. Techniques such as power allocation and spatial coupling have been proposed to improve the performance of low-complexity iterative decoding algorithms such as approximate message passing (AMP). In this thesis we investigate spatially coupled SPARCs, where the design matrix has a block- wise band-diagonal structure, and modulated SPARCs, which generalise standard SPARCs by introducing modulation to the encoding of user messages. We introduce a base matrix framework which provides a unified way to construct power allocated and spatially coupled design matrices, and propose AMP decoders for modulated SPARCs constructed using base matrices. We prove that phase shift keying modulated and spatially coupled SPARCs with AMP decoding asymptotically achieve the capacity of the (complex) AWGN channel. We also show via numerical simulations that they can achieve lower error rates than standard coded modulation schemes at finite code lengths. A sliding window AMP decoder is proposed for spatially coupled SPARCs that significantly reduces the decoding latency and complexity. We then investigate coding schemes based on random linear models and AMP decoding for the multi-user Gaussian multiple access channel in the asymptotic regime where the number of users grows linearly with the code length. For a fixed target error rate and message size per user (in bits), we obtain the exact trade-off between energy-per-bit and the user density achievable in the large system limit. We show that a coding scheme based on spatially coupled Gaussian matrices and AMP decoding achieves near-optimal trade-off for a large range of user densities. To the best of our knowledge, this is the first efficient coding scheme to do so in this multiple access regime. Moreover, the spatially coupled coding scheme has a practical interpretation: it can be viewed as block-wise time-division with overlap.

Mathematical models form the basis of application in a multitude of processes and disciplines. With there being a recent and general trend of increased system complexity through added dimensionality and new innovations in technology, conventional characterisation methods fall short in many key areas. Consequently, it is necessary to address these shortcomings through the development of modelling methods which allow for the characterisation and investigations of the features of these systems. Local Model Networks (LMNs), a subset of the Neuro-Fuzzy modelling method, have become increasingly popular as a solution to this problem of Nonlinear System Identification. This thesis introduces a novel procedure for the identification of such systems, which returns a Pruned Hierarchical Network. Herein referred to as PRUHINET, the algorithm operates using hierarchical tree construction and returns a structure which is constituted of neurons containing local models each with an associated region of activation. The operational input space of the system is partitioned using an axis-oblique strategy, however unlike previous deployments, the employed partition method is predicated upon Maximum Likelihood Estimation (MLE). Analytical gradients are used to speed up the required nonlinear optimisation process. PRUHINET proposes various LMNs of varying complexity levels and utilises Information Theoretic Criterion (ITC) for the determination of the optimal network structure. This is addressed through the termination of the model build and the removal of redundant neurons via a Backward Elimination Pruning (BEP) approach. Multi-Model Inference (MMI) is used across the candidate LMNs to further mitigate model selection uncertainty and provide final response prediction. PRUHINET also allows for the consideration of systematic correlations within the supplied dataset by whitening the model errors through an iterated Feasible Generalised Least Squares (FGLS) approach external to the LMN build. The utility of the approach is shown through the identification of various dataset examples consisting of static and dynamic elements. The static simulation results illustrate the functionality of PRUHINET as an evolution of traditional approaches, providing superior performance and also being able to return results representative of classical methods under certain configurational assumptions. Validation results for the dynamic dataset showed the approach was able to identify the given system to an accuracy greater than 95% in all cases. Finally, the PRUHINET approach was shown to allow scrutinisation of the identified local models, which is of benefit in application.

Synthetic biology applies engineering principles to make progress in the study of complex biological phenomena. The aim is to develop understanding through the praxis of construction and design. The computational branch of this endeavour explicitly brings the tools of abstraction and modularity to bear. This thesis pursues two distinct lines of inquiry concerning the application of computational tools in the setting of synthetic biology. One thread traces a narrative through multi-paradigm computational simulations, interpretation of results, and quantification of biological order. The other develops computational infrastructure for describing, simulating and discovering, synthetic genetic circuits. The emergence of structure in biological organisms, morphogenesis, is critically important for understanding both normal and pathological development of tissues. Here, we focus on epithelial tissues because models of two dimensional cellular monolayers are computationally tractable. We use a vertex model that consists of a potential energy minimisation process interwoven with topological changes in the graph structure of the tissue. To make this interweaving precise, we define a language for propagators from which an unambiguous description of the simulation methodology can be constructed. The vertex model is then used to reproduce laboratory results of patterning in engineered mammalian cells. The assertion that the claim of reproduction is justified is based on a novel measure of structure on coloured graphs which we call path entropy. This measure is then extended to the setting of continuous regions and used to quantify the development of structure in house mouse (Mus musculus) embryos using three dimensional segmented anatomical models. While it is recognised that DNA can be considered a powerful computational environment, it is far from obvious how to program with nucleic acids. Using rule-based modelling of modular biological parts, we develop a method for discovering synthetic genetic programs that meet a specification provided by the user. This method rests on the concept of annotation as applied to rule-based programs. We begin with annotating rules and proceed to generating entire rule-based programs from annotations themselves. Building on those tools we describe an evolutionary algorithm for discovering genetic circuits from specifications provided in terms of probability distributions. This strategy provides a dual benefit: using stochastic simulation captures circuit behaviour at low copy numbers as well as complex properties such as oscillations, and using standard biological parts produces results that are implementable in the laboratory.

Mixed selectivity characterises neurons that simultaneously respond to different input stimuli. Neurons with mixed selectivity have been observed in multiple brain regions, and are hypothesised to play important roles in neural computation. Recently, both experimental and theoretical work demonstrated the importance of mixed selectivity in context-dependent decision tasks. This thesis extends existing theoretical work on mixed selectivity, arguing for a general and statistical role of mixed selectivity in learning complex dependencies of input stimuli. This role can be motivated from unsupervised learning of generative models, and is exhibited in increased mutual information between the stimuli and their neural representation. This argument is supported empirically using simulation of a sequence disambiguation task that incorporated key aspects of related behaviour experiments. Mixed selectivity neurons that resembled hippocampal place cells were emerged from models optimised only for behaviour. To understand these results, as well as to generalise the findings to a wider range of computations, I provided a formal connection between learning robust models and mixed selectivity.

"Predicting the future isn't magic, it's artificial intelligence" Dave Waters. In the last decades there has been an unprecedented growth in the field of machine learning, and particularly within deep learning models. The combination of big data and computational power has nurtured the evolution of a variety of new methods to predict and interpret future scenarios. These data centric models can achieve exceptional performances on specific tasks, with their prediction boundaries continuously expanding towards new and more complex challenges. However, the model complexity often translates into a lack of interpretability from a scientific c perspective, it is not trivial to identify the factors involved in final outcomes. Explainability may not always be a requirement for some machine learning tasks, specially when it comes in detriment of performance power. But for some applications, such as biological discoveries or medical diagnostics, understanding the output and determining factors that influence decisions is essential. In this thesis we develop both a supervised and unsupervised approach to map from genotype to phenotype. We emphasise the importance of interpretability and feature extraction from the models, by identifying relevant genes for cell differentiation. We then continue to explore the rules and mechanisms behind the models from a theoretical perspective. Using information theory to explain the learning process and applying perturbation theory to transform the results into a generalisable representation. We start by building a supervised approach to mapping cell profiles from genotype to phenotype, using single cell RNA-Seq data. We leverage non-linearities among gene expressions to identify cellular levels of differentiation. The ambiguity and even absence of labels in most biological studies instigated the development of novel unsupervised techniques, leading to a new general and biologically interpretable framework based on Variational Autoencoders. The application and validation of the methods has proven to be successful, but questions regarding the learning process and generative nature of the results remained unanswered. I use information theory to define a new approach to interpret training and the converged solutions of our models. The variational and generative nature of Autoencoders provides a platform to develop general models. Their results should extrapolate and allow generalisation beyond the boundaries of the observed data. To this extent, we introduce for the first time a new interpretation of the embedded generative functions through Perturbation Theory. The embedding multiplicity is addressed by transforming the distributions into a new set of generalisable functions, while characterising their energy spectrum under a particular energy landscape. We outline the combination of theoretical and machine learning based methods, for moving towards interpretable and generalisable models. Developing a theoretical framework to map from genotype to phenotype, we provide both supervised and unsupervised tools to operate over single cell RNA-Seq. data. We have generated a pipeline to identify relevant genes and cell types through Variational Autoencoders (VAEs), validating reconstructed gene expressions to prove the generative performance of the embeddings. The new interpretation of the information learned and extracted by the models de fines a label independent evaluation, particularly useful for unsupervised learning. Lastly, we introduce a novel transformation of the generative embeddings based on quantum and perturbation theory. Our contributions can and have been extended to new datasets, according to the nature of the tasks being explored. For instance, the combination of unsupervised learning and information theory can be applied to a variety of biological or medical data. We have trained several VAE models with additional cancer and metabolic data, proving to extract meaningful representations of the data. The perturbation theory transformation of the embedding can also lead to future research on the generative potential of Variational Autoencoders through a physics perspective, combining statistical and quantum mechanics. We believe that machine learning will only continue its fast expansion and growth through the development of more generalisable more interpretable models. "Prediction is very difficult, especially if it's about the future" Niels Bohr.

In the last 50 years computers have made dramatic progress in their capabilities, but at the same time their failings have demonstrated that we, as designers, do not yet understand the nature of intelligence. Chess playing, for example, was long offered up as an example of the unassailability of the human mind to Artificial Intelligence, but now a chess engine on a smartphone can beat a grandmaster. Yet, at the same time, computers struggle to beat amateur players in simpler games, such as Stratego, where sheer processing power cannot substitute for a lack of deeper understanding. The task of developing that deeper understanding is overwhelming, and has previously been underestimated. There are many threads and all must be investigated. This dissertation explores one of those threads, namely asking the question "How might an artificial agent decide on a sensible course of action, without being told what to do?". To this end, this research builds upon empowerment, a universal utility which provides an entirely general method for allowing an agent to measure the preferability of one state over another. Empowerment requires no explicit goals, and instead favours states that maximise an agent's control over its environment. Several extensions to the empowerment framework are proposed, which drastically increase the array of scenarios to which it can be applied, and allow it to evaluate actions in addition to states. These extensions are motivated by concepts such as bounded rationality, sub-goals, and anticipated future utility. In addition, the novel concept of strategic affinity is proposed as a general method for measuring the strategic similarity between two (or more) potential sequences of actions. It does this in a general fashion, by examining how similar the distribution of future possible states would be in the case of enacting either sequence. This allows an agent to group action sequences, even in an unknown task space, into 'strategies'. Strategic affinity is combined with the empowerment extensions to form soft-horizon empowerment, which is capable of composing action policies in a variety of unknown scenarios. A Pac-Man-inspired prey game and the Gambler's Problem are used to demonstrate this selfmotivated action selection, and a Sokoban inspired box-pushing scenario is used to highlight the capability to pick strategically diverse actions. The culmination of this is that soft-horizon empowerment demonstrates a variety of 'intuitive' behaviours, which are not dissimilar to what we might expect a human to try. This line of thinking demonstrates compelling results, and it is suggested there are a couple of avenues for immediate further research. One of the most promising of these would be applying the self-motivated methodology and strategic affinity method to a wider range of scenarios, with a view to developing improved heuristic approximations that generate similar results. A goal of replicating similar results, whilst reducing the computational overhead, could help drive an improved understanding of how we may get closer to replicating a human-like approach.

Sparse regression codes (SPARCs) are a recently introduced coding scheme for the additive white Gaussian noise channel, for which polynomial time decoding algorithms have been proposed which provably achieve the Shannon channel capacity. One such algorithm is the approximate message passing (AMP) decoder. However, directly implementing these decoders does not yield good empirical performance at practical block lengths. This thesis develops techniques for improving both the error rate performance, and the time and memory complexity, of the AMP decoder. It focuses on practical and efficient implementations for both single- and multi-user scenarios. A key design parameter for SPARCs is the power allocation, which is a vector of coefficients which determines how codewords are constructed. In this thesis, novel power allocation schemes are proposed which result in several orders of magnitude improvement to error rate compared to previous designs. Further improvements to error rate come from investigating the role of other SPARC construction parameters, and from performing an online estimation of a key AMP parameter instead of using a pre-computed value. Another significant improvement to error rates comes from a novel three-stage decoder which combines SPARCs with an outer code based on low-density parity-check codes. This construction protects only vulnerable sections of the SPARC codeword with the outer code, minimising the impact to the code rate. The combination provides a sharp waterfall in bit error rates and very low overall codeword error rates. Two changes to the basic SPARC structure are proposed to reduce computational and memory complexity. First, the design matrix is replaced with an efficient in-place transform based on Hadamard matrices, which dramatically reduces the overall decoder time and memory complexity with no impact on error rate. Second, an alternative SPARC design is developed, called Modulated SPARCs. These are shown to also achieve the Shannon channel capacity, while obtaining similar empirical error rates to the original SPARC, and permitting a further reduction in time and memory complexity. Finally, SPARCs are implemented for the broadcast and multiple access channels, and for the multiple description and Wyner-Ziv source coding models. Designs for appropriate power allocations and decoding strategies are proposed and are found to give good empirical results, demonstrating that SPARCs are also well suited to these multi-user settings.

In 1935 the linguist George Kingsley Zipf made a now classic observation about the relationship between a word's length and its frequency: the more frequent a word is, the shorter it tends to be. He claimed that this 'Law of Abbreviation' is a universal structural property of language. The Law of Abbreviation has since been documented in a wide range of human languages, and extended to animal communication systems and even computer programming languages. Zipf hypothesised that this universal design feature arises as a result of individuals optimising form-meaning mappings under competing pressures to communicate accurately but also efficiently - his famous Principle of Least Effort. In this thesis, I present a novel set of studies which provide direct experimental evidence for this explanatory hypothesis. Using a miniature artificial language learning paradigm, I show in Chapter 2 that language users optimise form-meaning mappings in line with the Law of Abbreviation only when pressures for accuracy and efficiency both operate during a communicative task. These results are robust across different methods of data collection: one version of the experiment was run in the lab, and another was run online, using a novel method I developed which allows participants to partake in dyadic interaction through a web-based interface. In Chapter 3, I address the growing body of work suggesting that a word's predictability in context may be an even stronger determiner of its length than its frequency alone. For instance, Piantadosi et al. (2011) show that shorter words have a lower average surprisal (i.e., tend to appear in more predictive contexts) than longer words, in synchronic corpora across many languages. We hypothesise that the same communicative pressures posited by the Principle of Least Effort, when acting on speakers in situations where context manipulates the information content of words, can give rise to these lexical distributions. Adapting the methodology developed in Chapter 2, I show that participants use shorter words in more predictive contexts only when subject to the competing pressures for accurate and efficient communication. In a second experiment, I show that participants are more likely to use shorter words for meanings with a lower average surprisal. These results suggest that communicative pressures acting on individuals during language use can lead to the re-mapping of a lexicon to align with 'Uniform Information Density', the principle that information content ought to be evenly spread across an utterance, such that shorter linguistic units carry less information than longer ones. Over generations, linguistic behaviour such as that observed in the experiments reported here may bring entire lexicons into alignment with the Law of Abbreviation and Uniform Information Density. For this to happen, a diachronic process which leads to permanent lexical change is necessary. However, crucial evidence for this process - decreasing word length as a result of increasing frequency over time - has never before been systematically documented in natural language. In Chapter 4, I conduct the first large-scale diachronic corpus study investigating the relationship between word length and frequency over time, using the Google Books Ngrams corpus and three different word lists covering both English and French. Focusing on words which have both long and short variants (e.g., info/information), I show that the frequency of a word lemma may influence the rate at which the shorter variant gains in popularity. This suggests that the lexicon as a whole may indeed be gradually evolving towards greater efficiency. Taken together, the behavioural and corpus-based evidence presented in this thesis supports the hypothesis that communicative pressures acting on language-users are at least partially responsible for the frequency-length and surprisal-length relationships found universally across lexicons. More generally, the approach taken in this thesis promotes a view of language as, among other things, an evolving, efficient code for information transfer.

This thesis aims to find general principles governing the behaviour of biological systems, with a particular emphasis in the communicational (social) aspect of these systems. Communication between biological entities plays a major role in their evolution, enabling them to exchange information about their environment and thereby improving their chances of survival. Communication also plays a pivotal role in the organisation of populations of organisms, clearly observed in social insects, but present also at least in bacteria, plants, fungi, animals and humans. It is also theorised that the genetic code is a by-product of the establishment of an innovation-sharing protocol between primitive cells [Vetsigian et al., 2006]. This thesis is mainly concerned with identifying necessary conditions for the emergence of communicational codes, and the problems that arise with their establishment. For this purpose, we introduce an information-theoretic framework where species maximise their growth rate by following a Kelly-gambling strategy to bet on environmental conditions. Information theory provides a powerful tool for abstracting away mechanisms and for focusing on hard limits of a system's dynamics which cannot be circumvented. We begin by exploring the relation between information exchange and limited resources. We show that a transition from cooperation to antagonism in the exchange of environmental information follows from a change in the availability of resources, from abundant to scarce. We then assume a non-competitive scenario with abundance of resources, where conflicts in a population occur only at a communicational (informational) level, rather than on the physical level, such as competing for (physical) resources. However, traditional Shannon communication is non-semantic, as opposed to the semantic communication observed in biological systems, which is necessary for capturing conflicts in communication. In the traditional use of information theory, it is assumed that every organism knows how to \interpret" the information offered by other organisms. However, this assumes that one \knows" which other organisms one observes, and thus which code they use. In our model, however, we wish to preclude that: namely, we will do away with the assumption that the identity of the organisms who send the messages and those who receive them is known, and the resulting usable information is therefore influenced by the universality of the code used and by which organisms an organism is \listening" to. We introduce a model which captures semantic communication in information-theoretic terms, where organisms talk to each other in a communication network. We show that, for particular population structures, when organisms cannot identify which other organisms they talk to, the adoption of a universal code emerges as a solution for full interpretation of the shared information. However, the evolution and establishment of universal codes for communication introduces vulnerabilities: organisms can be exploited by parasites. We de ne two types of parasites whose strategies have different levels of complexity and study the co-evolution of a host (the population) and a parasite by optimising their respective objective functions in stages. First, we consider a disruptive parasite (a troll) that inflicts harm in a host by minimising a population's mutual understanding, and then a more complex parasite, which manipulates the members of the population via their codes (the puppetmaster). We show emergent characterisations of both parasites, as well as which host configurations are robust against parasites and show adaptive properties. This thesis introduces a framework which allows the study of informational properties in the host-parasite co-evolution, where the rules of the parasite's habitat, the host, are the outcome of an evolutionary process, and where these very same rules are those that allow the parasite to exploit the host.

Human language is both a cognitive and a cultural phenomenon. Any evolutionary account of language, then, must address both biological and cultural evolution. In this thesis, I give a mainly cultural evolutionary answer to two main questions: firstly, how do working systems of learned communication arise in populations in the absence of external or internal guidance? Secondly, how do those communication systems take on the fundamental structural properties found in human languages, i.e. systematicity at both a meaningless and meaningful level? A large, multi-disciplinary literature exists for each question, full of apparently conflicting results and analyses. My aim in this thesis is to survey this work, so as to find any commonalities and bring this together in order to provide a minimal account of the cultural evolution of language. The first chapter of this thesis takes a number of well-established models of the emergence of signalling systems. These are taken from several different fields: evolutionary linguistics, evolutionary game theory, philosophy, artificial life, and cognitive science. By using a common framework to directly compare these models, I show that three underlying commonalities determine the ability of any population of agents to reliably develop optimal signalling. The three requirements are that i) agents can create and transfer referential information, ii) there is a systemic bias against ambiguity, and iii) some mechanism leading to information loss exists. Following this, I extend the model to determine the effects of including referential uncertainty. I show that, for the group of models to which this applies, this places certain extra restrictions on the three requirements stated above. In the next chapter, I use an information-theoretic framework to construct a novel analysis of signalling games in general, and rephrase the three requirements in more formal terms. I then show that we can use these 3 criteria as a diagnostic for determining whether any given signalling game will lead to optimal signalling, without the requirement for repeated simulations. In the final, much longer, chapter, I address the topic of duality of patterning. This involves a lengthy review of the literature on duality of patterning, combinatoriality, and compositionality. I then argue that both levels of systematicity can be seen as a functional adaptation which maintains communicative accuracy in the face of noisy processes at different levels of analysis. I support this with results from a new, minimally-specified model, which also clarifies and informs a number of long-fought debates within the field.

The brain processes information sensed from the environment and guides behaviour. A fundamental component in this process is the storage and retrieval of past experiences as memories, which relies on the hippocampal formation. Although there has been a great progress in understanding the underlying neural code by which neurons communicate information, there are still open questions. Neural activity can be measured extracellularly as either spikes or field potentials. Isolated spikes and bursts of high-frequency spikes followed by silent periods can transmit messages to distant networks. The local field potential (LFP) reflects synaptic activity within a local network. The interplay between the two has been linked to cognitive functions, such as memory, attention and decision making. However, the code by which this neural communication is achieved is not well understood. We investigated a mechanism by which local network information contained in LFP rhythms can be transmitted to distant networks in the formof spike patterns fired by bursting neurons. Since rhythms within different frequency bands are prevalent during behavioural states, we studied this encoding during different states within the hippocampal formation. In the first paper, using a computational model we show that bursts of different size preferentially lock to the phase of the dominant rhythm within the LFP.We also present examples showing that bursting activity in the subiculum of an anaesthetised rat was phase-locked to delta or theta rhythms as predicted by the model. In the second paper, we explored possible neural codes by which bursting neurons can encode features of the LFP.We used the computational model reported in the first paper and analysed recordings from the subiculum of anaesthetised rats and the medial entorhinal cortex of an awake behaving rat. We show that bursting neurons encoded information about the instantaneous voltage, phase, slope and/or amplitude of the dominant LFP rhythm (delta or theta) in their firing rate. In addition, some neurons encoded about 10-15% of this information in intra-burst spike counts. We subsequently studied how the interactions between delta or theta rhythms can transfer information between different areas within the hippocampal formation. In the third paper, we show that delta and theta rhythms can act as separate routes for simultaneously transferring segregate information between the hippocampus and the subiculum of anaesthetised mice. We found that the phase of the rhythms conveyed more information than amplitude. We next investigated whether neurodegenerative pathology affects this information exchange. We compared information transfer within the hippocampal formation of young transgenic mice exhibiting Alzheimer’s disease-like pathology and healthy aged-matched control mice and show that at early stages of the disease the information transmission by LFP rhythm interactions appears to be intact but with some differences. The outcome of this project supports a burst code for relaying information about local network activity to downstream neurons and underscores the importance of LFP phase, which provides a reference time frame for coordinating neural activity, in information exchange between neural networks.

There have been a number of computational modelling studies that aim to replicate the cerebellar Purkinje cell, though these typically use the morphology of rodent cells. While many species, including rodents, display intricate dendritic branching, it is not a universal feature among Purkinje cells. This study uses morphological reconstructions of 24 Purkinje cells from seven species to explore the changes that occur to the cell through evolution and examine whether this has an effect on the processing capacity of the cell. This is achieved by combining several modes of study in order to gain a comprehensive overview of the variations between the cells in both morphology and behaviour. Passive and active computational models of the cells were created, using the same electrophysiological parameters and ion channels for all models, to characterise the voltage attenuation and electrophysiological behaviour of the cells. These results and several measures of branching and size were then used to look for clusters in the data set using machine learning techniques. They were also used to visualise the differences within each species group. Information theory methods were also employed to compare the estimated information transfer from input to output across each cell. Along with a literature review into what is known about Purkinje cells and the cerebellum across the phylogenetic tree, these results show that while there are some obvious differences in morphology, the variation within species groups in electrophysiological behaviour is often as high as between them. This suggests that morphological changes may occur in order to conserve behaviour in the face of other changes to the cerebellum.