Criterion validity of five open-source app-based cognitive and sensory tasks in an Australian adult life course sample aged 18 to 82: Labs without walls.

With recent technical advances, many cognitive and sensory tasks have been adapted for smartphone testing. This study aimed to assess the criterion validity of a subset of self-administered, open-source app-based cognitive and sensory tasks by comparing test performance to lab-based alternatives. An in-person baseline was completed by 43 participants (aged 21 to 82) from the larger Labs without Walls project (Brady et al., 2023) to compare the self-administered, app-based tasks with researcher-administered equivalents. 4 preset tasks sourced from Apple's ResearchKit (Spatial Memory, Trail Making Test, Stroop Test, and dBHL Tone Audiometry) and 1 custom-built task (Ishihara Color Deficiency Test) were compared. All tasks except the Spatial Memory task demonstrated high comparability to the researcher-administered version. Specifically, the Trail Making Tests were strongly correlated (.77 and .78 for parts A and B, respectively), Stroop correlations ranged from .77 to .89 and the Ishihara tasks were moderately correlated (r = .69). ICCs for the Audiometry task ranged from .56 to .96 (Moderate to Excellent) with 83% sensitivity and 100% specificity. Bland-Altman plots revealed a mean bias between -5.35 to 9.67 dB for each ear and frequency with an overall bias of 3.02 and 1.98 for the left and right ears, respectively, within the minimum testing interval. Furthermore, all app-based tasks were significantly correlated with age. These results offer preliminary evidence of the validity of four open-source cognitive and sensory tasks with implications for effective remote testing in non-lab settings.
Competing Interests: Declarations. Funding: Apple,Investigator Support Program,University of New South Wales,Ageing Futures Institute,Australian Research Council,FL190100011 Ethics approval: The Labs without Walls project was approved by the UNSW Human Research Ethics Committee (Ethics approval number: HC200792). All participants provided informed consent to participate in the study and could freely withdraw at any point. Open practices statement: This study was pre-registered through OSF and can be found under the registrations tab in the main project folder ( https://osf.io/3gk4f/ ). The materials for the app-based Ishihara task are freely available in the public domain (Commons) and the code for the app-based task is available on request to the study PI (Anstey). All other app-based tasks mentioned in the paper are freely available through Apple’s ResearchKit ( http://researchkit.org/ ). The datasets generated and analyzed during the current study are not publicly available as, according to the approved ethics protocol, participants did not consent to their data being shared outside the immediate research team without ethical approval. However, authors can be contacted with any questions related to the data and de-identified data are available on reasonable request with ethical approval. Because of this limitation, analysis code is illustrated with a synthetic dataset, which allows readers to check the correctness of their implementation. The analysis code can be accessed at https://osf.io/2nswx/ . Conflicts of interest: Apple Watch devices were provided by Apple Inc through a successful application to their Investigator Support Program. However, Apple Inc was not involved in conducting the study, cleaning, and analyzing the data.
(© 2025. The Author(s).)