Search

Vision-based Interfaces (VIs) are pivotal in advancing Human-Computer Interaction (HCI), particularly in enhancing context awareness. However, there are significant opportunities for these interfaces due to rapid advancements in multimodal Artificial Intelligence (AI), which promise a future of tight coupling between humans and intelligent systems. AI-driven VIs, when integrated with other modalities, offer a robust solution for effectively capturing and interpreting user intentions and complex environmental information, thereby facilitating seamless and efficient interactions. This PhD study explores three application cases of multimodal interfaces to augment context awareness, respectively focusing on three dimensions of visual modality: scale, depth, and time: a fine-grained analysis of physical surfaces via microscopic image, precise projection of the real world using depth data, and rendering haptic feedback from video background in virtual environments., Comment: 3 pages, MOBILEHCI Adjunct '24 26th International Conference on Mobile Human-Computer Interaction, September 30-October 3, 2024, Melbourne, VIC, Australia

Integrating physiological signals such as electroencephalogram (EEG), with other data such as interview audio, may offer valuable multimodal insights into psychological states or neurological disorders. Recent advancements with Large Language Models (LLMs) position them as prospective ``health agents'' for mental health assessment. However, current research predominantly focus on single data modalities, presenting an opportunity to advance understanding through multimodal data. Our study aims to advance this approach by investigating multimodal data using LLMs for mental health assessment, specifically through zero-shot and few-shot prompting. Three datasets are adopted for depression and emotion classifications incorporating EEG, facial expressions, and audio (text). The results indicate that multimodal information confers substantial advantages over single modality approaches in mental health assessment. Notably, integrating EEG alongside commonly used LLM modalities such as audio and images demonstrates promising potential. Moreover, our findings reveal that 1-shot learning offers greater benefits compared to zero-shot learning methods., Comment: 6 pages; UbiComp Companion '24, Companion of the 2024 ACM International Joint Conference on Pervasive and Ubiquitous Computing, October 5--9, 2024}{Melbourne, VIC, Australia

Surface sensing is widely employed in health diagnostics, manufacturing and safety monitoring. Advances in mobile sensing affords this potential for context awareness in mobile computing, typically with a single sensing modality. Emerging multimodal large-scale language models offer new opportunities. We propose MultiSurf-GPT, which utilizes the advanced capabilities of GPT-4o to process and interpret diverse modalities (radar, microscope and multispectral data) uniformly based on prompting strategies (zero-shot and few-shot prompting). We preliminarily validated our framework by using MultiSurf-GPT to identify low-level information, and to infer high-level context-aware analytics, demonstrating the capability of augmenting context-aware insights. This framework shows promise as a tool to expedite the development of more complex context-aware applications in the future, providing a faster, more cost-effective, and integrated solution., Comment: 6 pages. MOBILEHCI Adjunct '24, 26th International Conference on Mobile Human-Computer Interaction, September 30-October 3, 2024, Melbourne, VIC, Australia

The primary focus of this research is the discreet and subtle everyday contact interactions between mobile phones and their surrounding surfaces. Such interactions are anticipated to facilitate mobile context awareness, encompassing aspects such as dispensing medication updates, intelligently switching modes (e.g., silent mode), or initiating commands (e.g., deactivating an alarm). We introduce MicroCam, a contact-based sensing system that employs smartphone IMU data to detect the routine state of phone placement and utilizes a built-in microscope camera to capture intricate surface details. In particular, a natural dataset is collected to acquire authentic surface textures in situ for training and testing. Moreover, we optimize the deep neural network component of the algorithm, based on continual learning, to accurately discriminate between object categories (e.g., tables) and material constituents (e.g., wood). Experimental results highlight the superior accuracy, robustness and generalization of the proposed method. Lastly, we conducted a comprehensive discussion centered on our prototype, encompassing topics such as system performance and potential applications and scenarios., Comment: 28 pages

Large Language Models (LLMs) have demonstrated exceptional capabilities across various natural language processing tasks. Yet, many of these advanced LLMs are tailored for broad, general-purpose applications. In this technical report, we introduce AcademicGPT, designed specifically to empower academic research. AcademicGPT is a continual training model derived from LLaMA2-70B. Our training corpus mainly consists of academic papers, thesis, content from some academic domain, high-quality Chinese data and others. While it may not be extensive in data scale, AcademicGPT marks our initial venture into a domain-specific GPT tailored for research area. We evaluate AcademicGPT on several established public benchmarks such as MMLU and CEval, as well as on some specialized academic benchmarks like PubMedQA, SCIEval, and our newly-created ComputerScienceQA, to demonstrate its ability from general knowledge ability, to Chinese ability, and to academic ability. Building upon AcademicGPT's foundation model, we also developed several applications catered to the academic area, including General Academic Question Answering, AI-assisted Paper Reading, Paper Review, and AI-assisted Title and Abstract Generation., Comment: Technical Report. arXiv admin note: text overlap with arXiv:2310.12081, arXiv:2310.10053 by other authors

Language tests measure a person's ability to use a language in terms of listening, speaking, reading, or writing. Such tests play an integral role in academic, professional, and immigration domains, with entities such as educational institutions, professional accreditation bodies, and governments using them to assess candidate language proficiency. Recent advances in Artificial Intelligence (AI) and the discipline of Natural Language Processing have prompted language test providers to explore AI's potential applicability within language testing, leading to transformative activity patterns surrounding language instruction and learning. However, with concerns over AI's trustworthiness, it is imperative to understand the implications of integrating AI into language testing. This knowledge will enable stakeholders to make well-informed decisions, thus safeguarding community well-being and testing integrity. To understand the concerns and effects of AI usage in language tests, we conducted interviews and surveys with English test-takers. To the best of our knowledge, this is the first empirical study aimed at identifying the implications of AI adoption in language tests from a test-taker perspective. Our study reveals test-taker perceptions and behavioral patterns. Specifically, we identify that AI integration may enhance perceptions of fairness, consistency, and availability. Conversely, it might incite mistrust regarding reliability and interactivity aspects, subsequently influencing the behaviors and well-being of test-takers. These insights provide a better understanding of potential societal implications and assist stakeholders in making informed decisions concerning AI usage in language testing.

Generative Artificial Intelligence (GenAI) has emerged as a fundamental component of intelligent interactive systems, enabling the automatic generation of multimodal media content. The continuous enhancement in the quality of Artificial Intelligence-Generated Content (AIGC), including but not limited to images and text, is forging new paradigms for its application, particularly within the domain of Augmented Reality (AR). Nevertheless, the application of GenAI within the AR design process remains opaque. This paper aims to articulate a design space encapsulating a series of criteria and a prototypical process to aid practitioners in assessing the aptness of adopting pertinent technologies. The proposed model has been formulated based on a synthesis of design insights garnered from ten experts, obtained through focus group interviews. Leveraging these initial insights, we delineate potential applications of GenAI in AR.

Let $D$ be the non-split quaternion algebra over $\mathbb{Q}_p$. We prove that a class of admissible unitary Banach space representations of $D^{\times}$ of global origin are topologically of finite length., Comment: 38 pages. Comments welcome !

Let $p$ be a prime number, $K$ a finite unramified extension of $\mathbb{Q}_p$ and $\mathbb{F}$ a finite extension of $\mathbb{F}_p$. Using perfectoid spaces we associate to any finite-dimensional continuous representation $\overline{\rho}$ of ${\rm Gal}(\overline K/K)$ over $\mathbb{F}$ an \'etale $(\varphi,\mathcal{O}_K^\times)$-module $D_A^\otimes(\overline{\rho})$ over a completed localization $A$ of $\mathbb{F}[\![\mathcal{O}_K]\!]$. We conjecture that one can also associate an \'etale $(\varphi,\mathcal{O}_K^\times)$-module $D_A(\pi)$ to any smooth representation $\pi$ of $\mathrm{GL}_2(K)$ occurring in some Hecke eigenspace of the mod $p$ cohomology of a Shimura curve, and that moreover $D_A(\pi)$ is isomorphic (up to twist) to $D_A^\otimes(\overline{\rho})$, where $\overline{\rho}$ is the underlying $2$-dimensional representation of ${\rm Gal}(\overline K/K)$. Using previous work of the same authors, we prove this conjecture when $\overline{\rho}$ is semi-simple and sufficiently generic., Comment: Minor modifications

Let $F$ be a totally real field in which $p$ is unramified and $B$ be a quaternion algebra which splits at at most one infinite place. Let $\overline{r}:\mathrm{Gal}(\overline{F}/F)\to \mathrm{GL}_2(\overline{\mathbb{F}}_p)$ be a modular Galois representation which satisfies the Taylor-Wiles hypotheses. Assume that for some fixed place $v|p$, $B$ ramifies at $v$ and $F_v$ is isomorphic to $ \mathbb{Q}_p$ and $\overline{r}$ is generic at $v$. We prove that the admissible smooth representations of the quaternion algebra over $\mathbb{Q}_p$ coming from mod $p$ cohomology of Shimura varieties associated to $B$ have Gelfand-Kirillov dimension $1$. As an application we prove that the degree two Scholze's functor vanishes on supersingular representations of $\mathrm{GL}_2(\mathbb{Q}_p)$. We also prove some finer structure theorem about the image of Scholze's functor in the reducible case., Comment: Final version, to appear in Compos. Math

Let $p$ be a prime number and $K$ a finite extension of $\mathbb{Q}_p$. We state conjectures on the smooth representations of $\mathrm{GL}_n(K)$ that occur in spaces of mod $p$ automorphic forms (for compact unitary groups). In particular, when $K$ is unramified, we conjecture that they are of finite length and predict their internal structure (extensions, form of subquotients) from the structure of a certain algebraic representation of $\mathrm{GL}_n$. When $n=2$ and $K$ is unramified, we prove several cases of our conjectures, including new finite length results., Comment: Minor updates after the referee's report

Let $F$ be a totally real field unramified at all places above $p$ and $D$ be a quaternion algebra which splits at either none, or exactly one, of the infinite places. Let $\bar{r}:\mathrm{Gal}(\bar{F}/F)\to \mathrm{GL}_2(\bar{\mathbb{F}}_p)$ be a continuous irreducible representation which, when restricted to a fixed place $v|p$, is non-semisimple and sufficiently generic. Under some mild assumptions, we prove that the admissible smooth representations of $\mathrm{GL}_2(F_v)$ occurring in the corresponding Hecke eigenspaces of the mod $p$ cohomology of Shimura varieties associated to $D$ have Gelfand-Kirillov dimension $[F_v:\mathbb{Q}_p]$. We also prove that any such representation can be generated as a $\mathrm{GL}_2(F_v)$-representation by its subspace of invariants under the first principal congruence subgroup. If moreover $[F_v:\mathbb{Q}_p]=2$, we prove that such representations have length $3$, confirming a speculation of Breuil and Pa\v{s}k\=unas., Comment: Final version

Let $p$ be a prime number, $F$ a totally real number field unramified at places above $p$ and $D$ a quaternion algebra of center $F$ split at places above $p$ and at no more than one infinite place. Let $v$ be a fixed place of $F$ above $p$ and $\overline{r} : {\rm Gal}(\overline F/F)\rightarrow \mathrm{GL}_2(\overline{\mathbb{F}}_p)$ an irreducible modular continuous Galois representation which, at the place $v$, is semisimple and sufficiently generic (and satisfies some weak genericity conditions at a few other finite places). We prove that many of the admissible smooth representations of $\mathrm{GL}_2(F_v)$ over $\overline{\mathbb{F}}_p$ associated to $\overline{r}$ in the corresponding Hecke-eigenspaces of the mod $p$ cohomology have Gelfand--Kirillov dimension $[F_v:\mathbb{Q}]$, as well as several related results., Comment: Final version, to appear in Inventiones Mathematicae

We prove a new upper bound for the dimension of the space of cohomological automorphic forms of fixed level and growing parallel weight on $\mathrm{GL}_2$ over a number field which is not totally real, improving the one obtained by Marshall. The main tool of the proof is the mod $p$ representation theory of $\mathrm{GL}_2(\mathbb{Q}_p)$ as started by Barthel-Livne and Breuil, and developed by Paskunas., Comment: Final version

We prove that certain crystabelline deformation rings of two dimensional residual representations of $\mathrm{Gal}(\overline{\mathbb{Q}}_p/\mathbb{Q}_p)$ are Cohen-Macaulay. As a consequence, this allows to improve Kisin's $R[1/p]=\mathbb{T}[1/p]$ theorem to an $R=\mathbb{T}$ theorem., Comment: 57 pages; with an appendix by Jack Shotton; v2 minor revisions after a referee report

Let $F$ be a totally real field, $\mathfrak{p}$ an unramified place of $F$ dividing $p$ and $\overline{r}: \mathrm{Gal}(\overline{F}/F)\rightarrow\mathrm{GL}_2(\overline{\mathbb{F}}_p)$ a continuous irreducible modular representation. The work of Buzzard, Diamond and Jarvis associates to $\overline{r}$ an admissible smooth representation of $\mathrm{GL}_2(F_\mathfrak{p})$ on the mod $p$ cohomology of Shimura curves attached to indefinite division algebras which split at $\mathfrak{p}$. When $\overline{r}|_{\mathrm{Gal}(\overline{F_\mathfrak{p}}/F_\mathfrak{p})}$ is tamely ramified and generic (and under some technical assumptions), we determine the subspace of invariants of this representation under the principal congruence subgroup of level $\mathfrak{p}$. In particular, it depends only on $\overline{r}|_{\mathrm{Gal}(\overline{F_\mathfrak{p}}/F_\mathfrak{p})}$ and verifies a multiplicity one property., Comment: 21 pages. Title changed; revised version

Let $p$ be a prime and $L$ be a finite extension of $\mathbb{Q}_p$. We study the ordinary parts of $\mathrm{GL}_2(L)$-representations arised in the mod $p$ cohomology of Shimura curves attached to indefinite division algebras which splits at a finite place above $p$. The main tool of the proof is a theorem of Emerton \cite{Em3}., Comment: 20 pages

We prove the Breuil-Mezard conjecture for split non-scalar residual representations of Gal(Qp/Qp) by local methods. Combined with the cases previously proved in [18] and [24], this completes the proof of the conjecture (when p>3). As a consequence, the local restriction in the proof of the Fontaine-Mazur conjecture in [18] is removed., Comment: Final version. To appear in Ann. Scient. de l'E.N.S

Let p>3 be a prime, f a positive integer and Q_{p^f} the unramified extension of Q_p of degree f. After Breuil and Paskunas, to a generic semi-simple continue modulo p representation of the absolute Galois group of Q_{p^f}, we can associate a parameterized family of smooth admissible modulo p representations of GL_2(Q_{p^f}). In this article, we prove that there are more parameters than those known., Comment: 40 pages, in French. Final version, to appear in J. Algebra.

Let $p$ be a prime number and $F$ a local field with residual characteristic $p$. In this article, to an irreducible smooth representation of $GL_2(F)$ over $\bar{\mathbf{F}}_p$ with central character, we associate canonically a diagram which determines the original representation up to isomorphism. We also determine it in some cases., Comment: 47 pages in French; v3 final version

Let L be a finite extension of Q_p and d a positive integer. A conjecture, due to C. Breuil and P. Schneider, says that the existence of invariant norms on certain locally algebraic representations of GL_{d+1}(L) should be equivalent to the existence of certain (d+1)-dimensional de Rham representations of Gal(\bar{L}/L). We prove the easy direction of this conjecture: the existence of invariant norms implies the existence of admissible filtrations, by generalizing an idea of M.Emerton., Comment: 30 pages