Data Synthesis and Simulation for Modeling Cognitive Abilities

Abstract

Recent advancements in the methodology of cognitive assessment and development rely on various cognitive models. Determining the underlying abilities tapped by individual tasks involves different procedures, which presuppose dependencies on specific subskills, considerations of statistical distributions, and a substantial amount of measurement data for accurate estimation of latent factors. Addressing these bottlenecks, various deep learning (DL) models show promising performance. Despite their initial success, it is evident that DL models are hindered by the requirement for significant quantities of annotated and labeled data to experiment and refine these models. Synthetic data offer a solution to this challenge by being easily generated, error-free, inexhaustible, pre-annotated, and circumventing various ethical and practical concerns. The past decade has witnessed remarkable progress in data synthesis and domain adaptation techniques, narrowing the statistical gap between synthetic and real data. Beyond sustaining the DL revolution, synthetic data will pave the way for the next generation of DL models, capable of understanding the physical composition of the world and learning continually, multimodally, and interactively. This paper clarifies the models emerging from prevalent cognitive models, statistical methodologies, and psychometric research regarding the subjects and their subskills, as well as how to model the parameter and subskill dependencies of individual tasks independent of the limitations posed by current solutions. Building upon these insights, an environment for data synthesis and simulation is developed, suitable for validating various analysis solutions.