CNR-ICAR – Institute of Computing and High Performance Networks

Personalized Human-Computer Interaction

Keywords

Augmented reality (AR), Mixed Reality (MR), Artificial Intelligence (AI), Holograms, Natural User Interfaces (NUIs), 3D User Interfaces

Operation of the proposed solution

The main goal is to introduce AI techniques in the interaction design to: i) enable mechanisms multi-modal natural interaction (gestural, vocal, bio-feedback, etc.) in augmented, virtual and mixed reality environments; ii) allow the learning of user interaction patterns and therefore the design of highly personalized interaction solutions based on user preferences or any cognitive and/or motor disabilities. Potential applications: i) precision human-machine interaction, i.e. identifying the optimal tailoring of the interface to specific users based on their personal preferences, emotional state, current situation, tasks to be performed; ii) Virtual/Augmented reality with AI-powered sensing & actuation, to improve the user’s perception within the virtual environment by optimizing the visual and sound feedback that the environment returns (e.g., bio-feedback, to encourage the user to implement control strategies); iii) Human-AI collaboration, with “human-in-the-loop” approaches in the AI learning phase.

Challenges

1. AI on wearable devices (on-device ML);
2. Human-in-the-loop approaches for AI;
3. Precision Human-Computer Interaction.

Bibliography

Caggianese, G., De Pietro, G., Esposito, M., Gallo, L., Minutolo, A., & Neroni, P. (2020). Discovering Leonardo with artificial intelligence and holograms: A user study. Pattern Recognition Letters, 131, 361-367.