Robot-controlling Glove - Elisabet Delgado Mas

From RoboWiki
Revision as of 16:04, 1 August 2022 by Robot (talk | contribs) (→‎Process)
Jump to: navigation, search

back to projects


Result

Wearable technology prototype consisting of a robot-controlling glove.

Wearables can be used to remotely control physical (i.e. robotic hand) or virtual elements (i.e. 3D hand model in VR). With this concept, we integrate the glove with Neuro-Inspired COmpanion (NICO), an open-source antropomorphic robot for multi-modal research available at the Faculty of Mathematics, Physics and Informatics at the Comenius University in Bratislava [1].

Video

Photos

Frontal View Side View Setting


Source Code

GitHub

Process

Achieved:

Fase 0. Nico’s robot hands movement (21.12.03)

Fase 1. Potentiometers controller, v1 prototype (21.12.10)

  • Controlling Nico hand with Arduino wooden console
  • Wooden console up
    Wooden console bottom

Fase 2. Minimal glove, v2 prototype (21.12.17)

  • Minimal glove

Fase 3.0 Wearable (sewn) glove, v3 prototype (22.01.04)

  • Fase 3.0 + Sewing, v3.0
Glove and marks for the bending sensors
Sewn Glove
  • Fase 3.1 + Soldering, v3.1
Soldering
Soldering
Soldering
Soldering
  • Fase 3.2 + Wireless component, v3.2

Achieved milestone : control of Nico’s hand via a wireless wearable glove

______________________________________________________________


Pending:

Fase 4. Gyroscope armbands, v1 prototype

  • Fase 4.1 + Fine-tune integration, v4.1

Pending milestone: control of Nico’s arm

Progress

  • 22.06 Pictures, learning contract and report
  • 22.02-22.03 Gyroscope testing. Documentation writing.
  • 22.01.07 Sewn glove with improvements.
  • 22.01.03 Sewn glove first version.
  • [Christmas Break]
  • 21.12.17 Code and control via bending sensors.
  • 21.12.10 Code and control via potentiometres.
  • 21.12.03 Order necessary materials. Explore NICO’s code.
  • 21.11.26 Redesign of the project using NICO (Neuro-Inspired COmpanion Robot).
  • 21.11.20-21 Research of robotic and prosthetic hand 3D models, biomimetism, animatronics, human augmentation.
  • 21.11.19 Kick-off meeting & FabLab visit.
  • 21.11.10-17 Project Proposal

Note: Date format is 'yy.mm.dd'

Report

The idea of a glove controller started as Elisabet Delgado Mas' personal proposal for Pavel Petrovič's practical seminar on Introduction to Robotics during the winter semester of the academic year 2021-2022 in Bratislava, where the author of the report did her Erasmus as part of the Master's programme in Cognitive Science. The initial purpose of the seminar assignment was to create a project that would allow to apply the knowledge of educational robotics learned in class (Lego Mindstorms, Arduino etc.), although we could extend it according to personal preferences and research interests. Despite the fact that this seminar project was supposed to be something quite uncomplicated and the present proposal was a much larger project, our supervisor Kristína Malinovská and Pavel Petrovič were extremely supporting in its expansion, so it gradually became the essential core of Delgado Mas' HRI mobility project.

The evolution of this research project has been very favourable, as it has made it possible to build a tool that can be useful for the university in terms of empirical research: a glove controller integrated with the robot NICO (Neuro-Inspired COmpanion). Proof of this is that soon afterwards another student in Slovakia, Jakub Mišovský, has already taken over the project for a bachelor thesis on cognitive robotics.

The result of our collaboration under Petrovič and Malinovska's guidance has been the aforementioned glove controller, whose development has required integrating lessons learned in diverse fields and has allowed to take advantage of the equipment and opportunities of the cognitive science consortium member Comenius University in Bratislava, where our master degree is framed in the Faculty of Mathematics, Physics and Informatics. This project has allowed the author to doing research in new methods and the integration of interdisciplinary knowledge from fields such as engineering and applied computer science to build tools for conducting research in cognitive robotics and HRI. Besides, it has required to acquire familiariarity with electrical components (bending sensors to track the flexion of the fingers, gyroscopes to track their orientation, Arduino boards etc.) and skills (i.e. how to design, connect and solder Arduino boards).

In this project we first got familiar with HRI and focused on researching the state of the art of prosthetics and robotic arms. Once achieved a closer understanding of such technologies, the approaches to build them, the basic inner working of their components and the pros and cons of each approach, we focused in the few do-it-yourself (DIY) and low-cost options available. Finally, the decision was made on conceptualizing a glove that could operate NICO’s hand.

Given that most work was of a developmental nature, it has required a lot of testing, troubleshooting and iterative design. Therefore, the process was not defined by a list of tasks but by milestones and the consecution of the different prototypes: a wooden console of potentiometers to recontrol NICO (prototype v1), a minimal glove (v2) and a sewn glove (v3) with its own batteries (v3.2) and Bluetooth (v3.3).

In the end, the hours required for project proved to be beyond the scope of a regular mobility project (10ECTS) and a seminar altogether (5ECTS). Consequentially, further research is necessary to fine-tune and extend the last prototype of the glove to better integrate it with arm bands that could enable to move the whole arm. The current glove controller is capable of making the robotic hand perform a grasp as the human user does and move the robotic fingers independently, as shown in the videos attached. However, tracking the movement of the internal side of the thumb and the different degrees of freedom of the wrist proved very challenging, and further research is required, for example, to: • Figure out a way to hold the bending sensor of the internal side of the hand closer to the skin at any time so the bending and the response accordingly is accurate, and do so without restricting the free movement of a hand. • Find and purchase bending sensors that don't eventually break at the contact points, an issue that has not yet become a problem but it has been detected as one once the glove is more extensively used with participants. • Find and code gyroscopes that can track simultaneously the different degrees of freedom of the different parts of an arm, perhaps with a digital model of it.

Once further fine-tuning phases make the glove more robust to technical malfunctioning and interindividual anatomical human differences, the author believes the glove could also be used in empirical HRI and embodiment research. An example of such would be the to test the Rubber Hand Illusion with a robotic or prosthetic hand responsive to our own movement. Overall, the final milestones of the project were achieved and we believe the most significant outcome is that our project has brought real value to the academic community that assisted in making it a reality.


Links

Basic wireless glove controllers and low-cost robotic hands

More advanced prosthetic-like robotic hands

Biomimetic Anthropomorphic
Exo hand

Other ideas

Extra fingers:

Exo hand:

Robotic arms:

+AI:

  • OpenAI - Learning Dexterity, robot hand reinforcement learning to manipulate physical objects.

Wearable ASL gloves:

Animatronic Eyes:

Humanoid Robots:

Self-made Behance list with several glove and medical technology designs


Retrieved from "https://wiki.robotika.sk/robowiki/index.php?title=Robot-controlling_Glove_-_Elisabet_Delgado_Mas&oldid=10160"