Search

Usability testing has long been considered a gold standard in evaluating the ease of use of software and websites-producing metrics to benchmark the experience and identifying areas for improvement. However, logistical complexities and costs can make frequent usability testing infeasible. Alternatives to usability testing include various forms of expert reviews that identify usability problems but fail to provide task performance metrics. This case study describes a method by which multiple teams of trained evaluators generated task usability ratings and compared them to metrics collected from an independently run usability test on three software products. Although inter-rater reliability ranged from modest to strong and the correlation between actual and predicted metrics did establish fair concurrent validity, opportunities for improved reliability and validity were identified. By establishing clear guidelines, this method can provide a useful usability rating for a range of products across multiple platforms, without costing significant time or money.

Deep unification- (constraint-)based grammars are usually hand-crafted. Scaling such grammars from fragments to unrestricted text is time-consuming and expensive. This problem can be exacerbated in multilingual broad-coverage grammar development scenarios. Cahill et al. (2002, 2004) and O’Donovan et al. (2004) present an automatic f-structure annotation-based methodology to acquire broad-coverage, deep, Lexical-Functional Grammar (LFG) resources for English from the Penn-II Treebank. In this paper we show how this model can be adapted to a multilingual grammar development scenario to induce robust, wide-coverage, PCFG-based LFG approximations for German from the TIGER Treebank. We show how the architecture of LFG, in particular the distinction between c-structure and f-structure representations, facilitates multilingual, treebank-based unification grammar induction, allowing us to cross-linguistically reuse the lexical extraction and parsing modules from O’Donovan et al. (2004) and Cahill et al. (2004), respectively. We evaluate our grammars against the PARC 700 Dependency Bank (King et al., 2003), against dependency structures for 2000 held-out sentences from the TIGER Corpus as well as against a hand-crafted dependency gold standard for 100 TIGER trees. Currently, our resources achieve 81.79% f-score against the PARC 700, a 2.19% improvement over the best result reported for a hand-crafted grammar in Kaplan et al. (2004), 74.6% against the 2000 held-out TIGER dependency structures and 71.08% against the 100-sentence TIGER gold standard, with substantially improved coverage compared to hand-crafted resources. We have since applied our methodology to induce wide-coverage LFG resources for Chinese (Burke et al., 2004b) from the Penn Chinese Treebank (Xue et al., 2002) and for Spanish from the CAST3LB Treebank (Civit, 2003).

This paper reports on the TIGER Treebank, a corpus of currently 40,000 syntactically annotated German newspaper sentences. We describe what kind of information is encoded in the treebank and introduce the different representation formats that are used for the annotation and exploitation of the treebank. We explain the different methods used for the annotation: interactive annotation, using the tool ANNOTATE, and LFG parsing. Furthermore, we give an account of the annotation scheme used for the TIGER treebank. This scheme is an extended and improved version of the NEGRA annotation scheme and we illustrate in detail the linguistic extensions that were made concerning the annotation in the TIGER project. The main differences are concerned with coordination, verb-subcategorization, expletives as well as proper nouns. In addition, the paper also presents the query tool TIGERSearch that was developed in the project to exploit the treebank in an adequate way. We describe the query language which was designed to facilitate a simple formulation of complex queries; furthermore, we shortly introduce TIGER in, a graphical user interface for query input. The paper concludes with a summary and some directions for future work.

Mobile application app stores are a critical source of information about risk in an uncertain environment. App stores ought to assess and communicate the risk associated with an installation so that users are discouraged from installing risky or harmful apps in app stores. However, only a limited number of studies offer designers information about how to communicate risk effectively. We focused on the user's trust associated with security information stemming from crowd-sourced evaluations compared to those generated from an automated system. Both of these sources of security information are pervasively used to indicate possible risk associated with an app. We investigated whether biases exist for a particular source of information given similar amount of security information being available. We found that participants preferred to install apps rated by automation to those rated by humans despite equivalence in stated risk. Further, we found evidence of a gender difference in trust in automation.

© Exact and reproducible dispensing of liquids is a prerequisite for many applications in medical and molecular biology laboratories. The performance of test systems is largely dependent upon the exact proportions of the individual reaction components in an assay. Modern pipettes enable the user to manually dispense amounts of liquid down to about 0.1 μL. Using this mature technology, even users with little experience can dispense such small amounts of liquid without fearing the effects on the precision of the results. Despite the great importance of precise liquid handling in research and diagnostics, little attention is often paid to the hardware responsible for it, namely the pipettes. Here, I will give an overview of the technical principles of pipettes and discuss the characteristics of different systems defined by different methods of construction. I will also classify them by their applications. Other factors that affect the precision and accuracy of dispensing liquids will also be examined. Some practical tips will be given outlining even more precise methods of working.

In this paper we present a method for greatly reducing parse times in LFG parsing, while at the same time maintaining parse accuracy. We evaluate the methodology on data from English, German and Norwegian and show that the same patterns hold across languages. We achieve a speedup of 67% on the English data and 49% on the German data. On a small amount of data for Norwegian, we achieve a speedup of 40%, although with more training data we expect this figure to increase.

We present a log-linear model that is used for ranking the string realisations produced for given corpus f-structures by a reversible broad-coverage LFG for German and compare its results with the ones achieved by the application of a language model (LM). Like other authors that have developed log-linear models for realisation ranking, we use a hybrid model that uses linguistically motivated learning features and a LM (whose score is simply integrated into the log-linear model as an additional feature) for the task of realisation ranking. We carry out a large evaluation of the model, training on over 8,600 structures and testing on 323. We observe that the contribution that the structural features make to the quality of the output is slightly greater in the case of a free word order language like German than it is in the case of English. The exact match metric improves from 27% to 37% when going from the LM-based realisation ranking to the hybrid model, BLEU score improves from 0.7306 to 0.7939.

We report on the Parallel Grammar (ParGram) project which uses the XLE parser and grammar development platform for six languages: English, French, German, Japanese, Norwegian, and Urdu.

The SFB starts from two assumptions: on the one hand a theoretically well-founded linguistic analysis is a necessary prerequisite for the development of computational linguistics and its applications. On the other hand problems and results of natural language processing provide new insights and challenges for theoretical linguistics.

a) A parser which assigns a tree structure to sentences. The fragment which the parser can analyse corresponds roughly to the fragment of English described by R. Montague in 'The proper treatment of quantification in ordinary English'. Our fragment contains however more tense forms, temporal adverbs, and temporal conjunctions. In particular we treat the notoriously difficult pair of tense forms pass~ simple and imparfait, frame adverbials like aujourd'hui, hier, demai____~n,ce lour-l~ , u_nn ~our plus tard, etc.,as well as the .zonjunctions quand,pendant ~[u_e,depuis que,a r ~ ~ .

My aim in organizing this panel is to stimulate the discussion between researchers working on MT and linguists interested in formal syntax and semantics. I am convinced that a closer cooperation will be fruitful for both sides. I will be talking about experimental MT or MT as a research project and not as a development project.[1]