Robot-controlling Glove - Elisabet Delgado Mas

From RoboWiki
Revision as of 11:38, 10 March 2023 by Robot (talk | contribs) (→‎References:)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to: navigation, search

back to projects


Result

Wearable technology prototype consisting of a robot-controlling glove.

Wearables can be used to remotely control physical (i.e. robotic hand) or virtual elements (i.e. 3D hand model in VR). With this concept, we integrate the glove with Neuro-Inspired COmpanion (NICO), an open-source antropomorphic robot for multi-modal research available at the Faculty of Mathematics, Physics and Informatics at the Comenius University in Bratislava [1].

Video

Photos

Frontal View Side View Setting


Motivation

Keywords: Human-robot interaction (HRI), glove controller, anthropomorphic design, embodiment.

Human-robot interaction (HRI) gathers contributions from AI, robotics, design and psychology. It offers a splendid opportunity for interdisciplinary research aiming to understand how we interact with robotic systems, and how this technology integrates in our daily life and can be useful for us. The application of anthropomorphic features to robots is generally considered beneficial for HRI [1]. In this line of research, there are noteworthy findings such a better rating of both social interaction and physically embodied agents [2], especially when they are in the room [3]. Therefore, the influence of embodiment must be taken into account when designing HRI. Questions such as how does VR alter the way we perceive physically embodied agents compared to physical reality, or how does the human perception of an android change when a human can control it in a natural way as if it was his or her body come into play. However, to conduct research on this area and begin to answer these questions on a more empirical way, we need to develop the apparatuses required for this interaction.

In this research project, it is proposed to investigate natural controllers for HRI interaction to build a wearable glove and armbands that can be used to remotely control both physical (i.e. robotic hand) and virtual elements (i.e. 3D hand model). With this concept, it will be possible to build a glove to work in a more natural way with Neuro-Inspired COmpanion (NICO) [4], an open-source anthropomorphic robot for multi-modal research.

Our main criteria has been to create a project that is useful for further research. An artificial hand controlled by a wearable glove offers that possibility in multiple areas of interest such as: - Cognitive Robotics: E.g. to register, understand and identify movement. - Human-Robot Interaction (HRI): E.g. user perception of the robotic limb before and after engaging with the wearable controller. - Body Schema Illusions: E.g. extending the Rubber Hand Illusion (RHI) to further domains by the emulation of motor action by the robotic model.

Lastly, secondary relevant criteria have been to build this project on a low-cost accessible and affordably budget. Therefore, the components (bending sensors, accelerometers and Arduino boards) are chosen and purchased accordingly.

References

[1] L. Onnasch, C. Hildebrandt, “Impact of Anthropomorphic Robot Design on Trust and Attention in Industrial Human-Robot Interaction”, ACM Transactions on Human-Robot Interaction, vol. 11, no. 1, https://doi.org/10.1145/3472224

[2] K. Lee, Y. Jung, J. Kim and S. Kim, "Are physically embodied social agents better than disembodied social agents?: The effects of physical embodiment, tactile interaction, and people's loneliness in human–robot interaction", International Journal of Human-Computer Studies, vol. 64, no. 10, pp. 962-973, 2006.

[3] Li, "The benefit of being physically present: A survey of experimental works comparing copresent robots, telepresent robots and virtual agents", International Journal of Human-Computer Studies, vol.77, pp. 23-37, 2015

[4] M. Kerzel, E. Strahl, S. Magg, N. Navarro-Guerrero, S. Heinrich, and S. Wermter, “NICO — Neuro-inspired companion: A developmental humanoid robot platform for multimodal interaction,” IEEE Xplore, Aug. 01, 2017. https://ieeexplore.ieee.org/abstract/document/8172289

Source Code

GitHub

Process

Achieved:

Fase 0. Nico’s robot hands movement (21.12.03)

Fase 1. Potentiometers controller, v1 prototype (21.12.10)

  • Controlling Nico hand with Arduino wooden console
  • Wooden console up
    Wooden console bottom

Fase 2. Minimal glove, v2 prototype (21.12.17)

  • Minimal glove

Fase 3.0 Wearable (sewn) glove, v3 prototype (22.01.04)

  • Fase 3.0 + Sewing, v3.0
Glove and marks for the bending sensors
Sewn Glove
  • Fase 3.1 + Soldering, v3.1
Soldering
Soldering
Soldering
Soldering
  • Fase 3.2 + Wireless component, v3.2

Achieved milestone : control of Nico’s hand via a wireless wearable glove

______________________________________________________________


Pending:

Fase 4. Gyroscope armbands, v1 prototype

  • Fase 4.1 + Fine-tune integration, v4.1

Pending milestone: control of Nico’s arm

Progress

  • 22.06 Pictures, learning contract and report
  • 22.02-22.03 Gyroscope testing. Documentation writing.
  • 22.01.07 Sewn glove with improvements.
  • 22.01.03 Sewn glove first version.
  • [Christmas Break]
  • 21.12.17 Code and control via bending sensors.
  • 21.12.10 Code and control via potentiometres.
  • 21.12.03 Order necessary materials. Explore NICO’s code.
  • 21.11.26 Redesign of the project using NICO (Neuro-Inspired COmpanion Robot).
  • 21.11.20-21 Research of robotic and prosthetic hand 3D models, biomimetism, animatronics, human augmentation.
  • 21.11.19 Kick-off meeting & FabLab visit.
  • 21.11.10-17 Project Proposal

Note: Date format is 'yy.mm.dd'

Report

The idea of a glove controller started as Elisabet Delgado Mas' personal proposal for Pavel Petrovič's practical seminar on Introduction to Robotics during the winter semester of the academic year 2021-2022 in Bratislava, where the author of the report did her Erasmus as part of the Master's programme in Cognitive Science. The initial purpose of the seminar assignment was to create a project that would allow to apply the knowledge of educational robotics learned in class (Lego Mindstorms, Arduino etc.), although we could extend it according to personal preferences and research interests. Despite the fact that this seminar project was supposed to be something quite uncomplicated and the present proposal was a much larger project, our supervisor Kristína Malinovská and Pavel Petrovič were extremely supporting in its expansion, so it gradually became the essential core of Delgado Mas' HRI mobility project.

The evolution of this research project has been very favourable, as it has made it possible to build a tool that can be useful for the university in terms of empirical research: a glove controller integrated with the robot NICO (Neuro-Inspired COmpanion). Proof of this is that soon afterwards another student in Slovakia, Jakub Mišovský, has already taken over the project for a bachelor thesis on cognitive robotics.

The result of our collaboration under Petrovič and Malinovska's guidance has been the aforementioned glove controller, whose development has required integrating lessons learned in diverse fields and has allowed to take advantage of the equipment and opportunities of the cognitive science consortium member Comenius University in Bratislava, where our master degree is framed in the Faculty of Mathematics, Physics and Informatics. This project has allowed the author to doing research in new methods and the integration of interdisciplinary knowledge from fields such as engineering and applied computer science to build tools for conducting research in cognitive robotics and HRI. Besides, it has required to acquire familiariarity with electrical components (bending sensors to track the flexion of the fingers, gyroscopes to track their orientation, Arduino boards etc.) and skills (i.e. how to design, connect and solder Arduino boards).

In this project we first got familiar with HRI and focused on researching the state of the art of prosthetics and robotic arms. Once achieved a closer understanding of such technologies, the approaches to build them, the basic inner working of their components and the pros and cons of each approach, we focused in the few do-it-yourself (DIY) and low-cost options available. Finally, the decision was made on conceptualizing a glove that could operate NICO’s hand.

Given that most work was of a developmental nature, it has required a lot of testing, troubleshooting and iterative design. Therefore, the process was not defined by a list of tasks but by milestones and the consecution of the different prototypes: a wooden console of potentiometers to recontrol NICO (prototype v1), a minimal glove (v2) and a sewn glove (v3) with its own batteries (v3.2) and Bluetooth (v3.3).

In the end, the hours required for project proved to be beyond the scope of a regular mobility project (10ECTS) and a seminar altogether (5ECTS). Consequentially, further research is necessary to fine-tune and extend the last prototype of the glove to better integrate it with arm bands that could enable to move the whole arm. The current glove controller is capable of making the robotic hand perform a grasp as the human user does and move the robotic fingers independently, as shown in the videos attached. However, tracking the movement of the internal side of the thumb and the different degrees of freedom of the wrist proved very challenging, and further research is required, for example, to: • Figure out a way to hold the bending sensor of the internal side of the hand closer to the skin at any time so the bending and the response accordingly is accurate, and do so without restricting the free movement of a hand. • Find and purchase bending sensors that don't eventually break at the contact points, an issue that has not yet become a problem but it has been detected as one once the glove is more extensively used with participants. • Find and code gyroscopes that can track simultaneously the different degrees of freedom of the different parts of an arm, perhaps with a digital model of it.

Once further fine-tuning phases make the glove more robust to technical malfunctioning and interindividual anatomical human differences, the author believes the glove could also be used in empirical HRI and embodiment research. An example of such would be the to test the Rubber Hand Illusion with a robotic or prosthetic hand responsive to our own movement. Overall, the final milestones of the project were achieved and we believe the most significant outcome is that our project has brought real value to the academic community that assisted in making it a reality.


Links

Basic wireless glove controllers and low-cost robotic hands

More advanced prosthetic-like robotic hands

Biomimetic Anthropomorphic
Exo hand

Other ideas

Extra fingers:

Exo hand:

Robotic arms:

+AI:

  • OpenAI - Learning Dexterity, robot hand reinforcement learning to manipulate physical objects.

Wearable ASL gloves:

Animatronic Eyes:

Humanoid Robots:

Self-made Behance list with several glove and medical technology designs


Retrieved from "https://wiki.robotika.sk/robowiki/index.php?title=Robot-controlling_Glove_-_Elisabet_Delgado_Mas&oldid=10572"