Proyecto ROCF Automatic Scoring
A Benchmark for Rey-Osterrieth Complex Figure Test Automatic Scoring
The Rey–Osterrieth complex figure (ROCF) test is a neuropsychological task that can be useful for early detection of cognitive decline in the elderly population. Several computer vision systems have been proposed to automate this complex analysis task, but the results have not been sufficiently robust. However, deep learning advances in computer vision augur good results shortly. To advance in that direction, we present a benchmark for the automatic scoring of the ROCF test that provides (i) the ROCFD528 dataset, which is the largest open dataset of ROCF line drawings made by the elderly; (ii) a straightforward scoring method that correlates well with the standard method but does not require prior subdivision of the ROCF; (iii) a proposal of metrics to evaluate the performance of scoring systems as a regression task; and finally, (iv) the experimental results obtained by several modern deep learning models and two human raters, which can be used as a baseline for comparing new proposals. We evaluate different state-of-the-art convolutional neural networks (CNNs) under traditional and transfer learning paradigms. Experimental quantitative results indicate that a CNN specifically designed for sketches outperforms other state of the art CNN architectures when the number of examples available is limited. This benchmark can also be a paradigmatic example within the broad field of machine learning for the development of efficient and robust models for analysis of line drawings and sketches not only in classification but also regression tasks.
Source code: https://github.com/SIMDA-UNED/rocf_automatic_scoring
Datasets:
QDSD414k (The subset of Quick, Draw! dataset used to pre-train the deep learning architectures)
We give credit to the original Quick, Draw! dataset which is available under the Creative Commons Attribution 4.0 International license.