publications

2023

1. BioIn-Tacto: A compliant multi-modal tactile sensing module for robotic tasks de Oliveira, Thiago Eustaquio Alves, and da Fonseca, Vinicius Prado HardwareX 2023
2. A multimodal tactile dataset for dynamic texture classification Lima, Bruno Monteiro Rocha, Danyamraju, Venkata Naga Sai Siddhartha, de Oliveira, Thiago Eustaquio Alves, and da Fonseca, Vinicius Prado Data in Brief 2023
3. Exploring Tactile Temporal Features for Object Pose Estimation during Robotic Manipulation Galaiya, Viral Rasik, Asfour, Mohammed, Oliveira, Thiago Eustaquio, Jiang, Xianta, and Fonseca, Vinicius Sensors 2023

2022

1. A Dashboard Tool for Mobility Data Mining Preprocessing Tasks Haranwala, Yaksh J., Haidri, Salman, Tricco, Terrence S., Fonseca, Vinicius P., and Soares, Amilcar In 2022 23rd IEEE International Conference on Mobile Data Management (MDM) 2022 [Abs]

   Mobility data mining has received significant interest in the literature in the last few years since social media, sensor networks, IoT, and GPS devices generate a vast amount of data. Its growth was also boosted by the growing availability of machine learning algorithms and Python libraries for trajectory analysis. However, we believe that a proper tool that supports trajectory data preprocessing tasks using a dashboard-like application is missing. Such a tool helps users visualize the effects of preprocessing techniques and adequately select the ones that have a desired effect on the data. This demo proposes a tool that combines state-of-the-art Python trajectory analysis libraries to preprocess trajectory data and visualize their effect using a dashboard with maps, tables, and charts that will assist the user through this challenging process.

2. BioIn-Tacto: tactile sensing module design files and source code. Oliveira, Thiago Eustaquio Alves, and Fonseca, Vinicius Prado 2022
3. Integrating computer vision to prosthetic hand control with sEMG: Preliminary results in grasp classification Wang, Shuo, Zheng, Jingjing, Huang, Ziwei, Zhang, Xiaoqin, Prado da Fonseca, Vinicius, Zheng, Bin, and Jiang, Xianta Frontiers in Robotics and AI 2022 [Abs]

   The myoelectric prosthesis is a promising tool to restore the hand abilities of amputees, but the classification accuracy of surface electromyography (sEMG) is not high enough for real-time application. Researchers proposed integrating sEMG signals with another feature that is not affected by amputation. The strong coordination between vision and hand manipulation makes us consider including visual information in prosthetic hand control. In this study, we identified a sweet period during the early reaching phase in which the vision data could yield a higher accuracy in classifying the grasp patterns. Moreover, the visual classification results from the sweet period could be naturally integrated with sEMG data collected during the grasp phase. After the integration, the accuracy of grasp classification increased from 85.5% (only sEMG) to 90.06% (integrated). Knowledge gained from this study encourages us to further explore the methods for incorporating computer vision into myoelectric data to enhance the movement control of prosthetic hands.

4. PTRAIL — A python package for parallel trajectory data preprocessing Haidri, Salman, Haranwala, Yaksh J., Bogorny, Vania, Renso, Chiara, Fonseca, Vinicius Prado, and Soares, Amilcar SoftwareX 2022
5. Tactile object recognition in early phases of grasping using underactuated robotic hands Fonseca, Vinicius Prado, Jiang, Xianta, Petriu, Emil M, and Oliveira, Thiago Eustaquio Alves Intelligent Service Robotics 2022
6. Evaluating Data Representations for Object Recognition During Pick-and-Place Manipulation Tasks Carvalho, Humberto Navarro De, Castro, Lucas Pontes, Rego, Thais G. Do, Filho, Telmo M. Silva, Barbosa, Yuri de A. M., Batista, Leonardo Vidal, Soares, Amilcar, and Fonseca, Vinicius Prado Da In 2022 IEEE International Systems Conference (SysCon) 2022
7. Neuro-Fuzzy Grasp Control for a Teleoperated Five Finger Anthropomorphic Robotic Hand Welyhorsky, Maxwell, Prado Da Fonseca, Vinicius, Zhu, Qi, Rocha Lima, Bruno Monteiro, Alves De Oliveira, Thiago Eustaquio, and Petriu, Emil M In 2022 IEEE International Systems Conference (SysCon) 2022

2021

1. Classification of Textures using a Tactile-Enabled Finger in Dynamic Exploration Tasks Rocha Lima, Bruno Monteiro, Alves de Oliveira, Thiago Eustaquio, and Fonseca, Vinicius Prado In 2021 IEEE Sensors 2021 [Abs]

   Reproducing human-like dexterous manipulation in robots requires identifying objects and ures. In unstructured settings, robots equipped with tactile sensors can detect ures by using touch-related characteristics. The use of a tactile-enabled finger for ure categorization is investigated in this article. Four machine learning methods are used to recognize ures from the data of pressure, gravity, angular rate, magnetic field sensors embedded in the compliant structure of a multimodal tactile sensing module. Machine learning models trained on data retrieved during 2-dimensional exploration of sample ures achieved classification accuracy rates of more than 90% for all features.

2. Small Scale Unmanned Aircraft System and Photogrammetry Applied for 3D Modeling of Historical Buildings Boente, Alexandre, Baldivieso, Thiago, Oliveira, Thiago, Fonseca, Vinicius, and Rosa, Paulo In SENSORDEVICES 2021, The Twelfth International Conference on Sensor Device Technologies and Applications 2021
3. Tactile Sensor Analysis during Early Stages of Manipulation for Single Grasp Identification of Daily Objects Prado da Fonseca, Vinicius Engineering Proceedings 2021 [Abs]

   Dexterous robotic manipulation in unstructured environments is still challenging, despite the increasing number of robots entering human settings each day. Even though robotic manipulation provides complete solutions in factories and industries, it still lacks essential techniques, displaying clumsy or limited operation in unstructured environments. Daily objects typically aim at the human hand, and the human somatosensory system is responsible for solving all the complex calculations required for dexterous manipulations in unstructured settings. Borrowing concepts of the human visuotactile system can improve dexterous manipulation and increase robotics usage in unstructured environments. In humans, required finger and wrist joint adjustments occur after fast identification of the object in the initial stages of manipulation. Fast object identification during those phases may increase robotic dexterous manipulation performance. The present paper explores human-inspired concepts such as haptic glance to develop robotic single-grasp object identification. This concept can assist early phases of robotic manipulation, helping automated decision-making, such as type of grasp and joint position, during manipulation tasks. The main stages developed here are detecting sensor activation and sample collection using signal-to-noise and z-score filtering on tactile data. This procedure automates touch detection and reduces the sensor space for classification. Experiments on a daily objects dataset presented compelling results that will assist in the later stages of the early phases of robotic grasping.

2020

1. Tactile Sensors for Robotic Applications Pirozzi, Salvatore Sensors 2020 [Abs]

   In recent years, tactile sensing has become a key enabling technology to implement complex tasks by using robotic systems [...]

2. Dynamic Tactile Exploration for Texture Classification using a Miniaturized Multi-modal Tactile Sensor and Machine Learning Lima, Bruno Monteiro Rocha, Fonseca, Vinicius Prado, Oliveira, Thiago Eustaquio Alves, Zhu, Qi, and Petriu, Emil M. In 2020 IEEE International Systems Conference (SysCon) 2020
3. Teleoperated Grasping Using a Robotic Hand and a Haptic-Feedback Data Glove Zhu, Qi, Fonseca, Vinicius Prado, Lima, Bruno Monteiro Rocha, Welyhorsky, Maxwell, Goubran, Miriam, Oliveira, Thiago Eustaquio Alves, and Petriu, Emil M In 2020 IEEE International Systems Conference (SysCon) 2020

2019

1. Heart Rate Detection Using a Miniaturized Multimodal Tactile Sensor Rocha Lima, Bruno Monteiro, Eustaquio Alves de Oliveira, Thiago, Fonseca, Vinicius Prado, Zhu, Qi, Goubran, Miriam, Groza, Voicu Z., and Petriu, Emil M. In 2019 IEEE International Symposium on Medical Measurements and Applications (MeMeA) 2019
2. End-Effector Approach Flexibilization in a Surface Approximation Task Using a Bioinspired Tactile Sensing Module Alves de Oliveira, Thiago Eustaquio, Fonseca, Vinicius Prado, Lima, Bruno Monteiro Rocha, Cretu, Ana-Maria, and Petriu, M. In 2019 IEEE International Symposium on Robotic and Sensors Environments (ROSE) 2019 [Abs]

   The flexibilization of the end-effector orientation constraint is critical in various robotic tasks, in particular when approaching surfaces of unknown objects and when unexpected contacts occur. This paper presents the use of a bioinspired multimodal tactile sensing module for tasks requiring knowledge about the inclination of a surface. The advantage of this module is that it can detect the pitch of the surface even though the pitch of the end-effector that carries the module during exploration is kept constant. Such flexibilization is achieved due to the compliant nature of the sensing module, its inner organization and the placement of embedded sensors.

3. In-Hand Telemanipulation Using a Robotic Hand and Biology-Inspired Haptic Sensing Fonseca, Vinicius Prado, Monteiro Rocha Lima, Bruno, Alves de Oliveira, Thiago Eustaquio, Zhu, Qi, Groza, Voicu Z., and Petriu, Emil M. In 2019 IEEE International Symposium on Medical Measurements and Applications (MeMeA) 2019 [Abs]

   "Part Number: CFP19MEA-ART" "IEEE Catalog Number: CFP19MEA-USB"–PDF copyright page

4. Estimating the Orientation of Objects from Tactile Sensing Data Using Machine Learning Methods and Visual Frames of Reference Prado da Fonseca, Vinicius, Alves de Oliveira, Thiago Eustaquio, and Petriu, Emil M. Sensors 2019 [Abs]

   Underactuated hands are useful tools for robotic in-hand manipulation tasks due to their capability to seamlessly adapt to unknown objects. To enable robots using such hands to achieve and maintain stable grasping conditions even under external disturbances while keeping track of an in-hand object’s state requires learning object-tactile sensing data relationships. The human somatosensory system combines visual and tactile sensing information in their “What and Where” subsystem to achieve high levels of manipulation skills. The present paper proposes an approach for estimating the pose of in-hand objects combining tactile sensing data and visual frames of reference like the human “What and Where” subsystem. The system proposed here uses machine learning methods to estimate the orientation of in-hand objects from the data gathered by tactile sensors mounted on the phalanges of underactuated fingers. While tactile sensing provides local information about objects during in-hand manipulation, a vision system generates egocentric and allocentric frames of reference. A dual fuzzy logic controller was developed to achieve and sustain stable grasping conditions autonomously while forces were applied to in-hand objects to expose the system to different object configurations. Two sets of experiments were used to explore the system capabilities. On the first set, external forces changed the orientation of objects while the fuzzy controller kept objects in-hand for tactile and visual data collection for five machine learning estimators. Among these estimators, the ridge regressor achieved an average mean squared error of 0.077 ∘ . On the second set of experiments, one of the underactuated fingers performed open-loop object rotations and data recorded were supplied to the same set of estimators. In this scenario, the Multilayer perceptron (MLP) neural network achieved the lowest mean squared error of 0.067 ∘ .

2018

1. Teaching a robot sign language using vision-based hand gesture recognition Zhi, Da, De Oliveira, Thiago E.Alves, Da Fonseca, Vinicius Prado, and Petriu, Emil M. CIVEMSA 2018 - 2018 IEEE International Conference on Computational Intelligence and Virtual Environments for Measurement Systems and Applications, Proceedings 2018 [Abs]

   This paper presents a novel vision-based hand gesture recognition (HGR) and training system for a human-like robot hand. We implemented and trained a multiclass-SVM classifier and N-Dimensional DTW (ND-DTW) classifier for static posture recognition and dynamic gesture recognition. Training features were extracted from the raw gestures depth data captured by Leap Motion Controller. The experimental results show that multiclass SVM method has an average 98.25% recognition rates and the shortest run time when compared to k-NN and ANBC. For dynamic gestures, ND-DTW classifier displays a better performance than DHMM with an average 95.5% recognition rate and significantly shorter run time. In conclusion, the combination of SVMs and DTW proves the efficiency and high accuracy in proposed human-robot interaction system.

2. Object Recognition Through Manipulation Using Tactile Enabled Prosthetic Fingers and Feedback Glove - Experimental Study Kucherhan, Daniel J., Goubran, Miriam, Da Fonseca, Vinicius P., Alves De Oliveira, Thiago E., Petriu, Emil M., and Groza, Voicu MeMeA 2018 - 2018 IEEE International Symposium on Medical Measurements and Applications, Proceedings 2018
3. Object Recognition Through Manipulation Using Tactile Enabled Prosthetic Fingers and Feedback Glove - Experimental Study Kucherhan, Daniel J., Goubran, Miriam, Da Fonseca, Vinicius P., Alves De Oliveira, Thiago E., Petriu, Emil M., and Groza, Voicu MeMeA 2018 - 2018 IEEE International Symposium on Medical Measurements and Applications, Proceedings 2018

2017

1. Fuzzy controlled object manipulation using a three-fingered robotic hand Fonseca, Vinicius Prado, Kucherhan, Daniel John, Oliveira, Thiago E. Alves, Zhi, Da, and Petriu, Emil M. In 2017 Annual IEEE International Systems Conference (SysCon) 2017 [Abs]

   \textcopyright 2017 IEEE. Use of underactuated fingers to conduct precision, in-hand manipulation is a common topic of recent robotics research, mostly due to their relatively light weight and simplicity of use. Grasping operations are facilitated by compliant joints however precise, in-hand manipulation is more challenging since post-grasp orientation of an object varies. Underactuated, robotic-fingered hands that are capable of predictable grasping are one step closer to human-like end-effectors. This paper presents a new effort towards effective robotic manipulation using two underactuated fingers and one fully actuated robotic thumb with 3 degrees of freedom (DOF). Fuzzy grasping using tactile feedback is used to provide an enhanced stable grasp solution. The system comprises tactile feedback, orientation of underactuated phalanges using flexible joints, and thumb trajectory planning.

2. Stable grasping and object reorientation with a three-fingered robotic hand Prado da Fonseca, Vinicius, Oliveira, Thiago E. Alves, Eyre, Katerina, and Petriu, Emil M. In 2017 IEEE International Symposium on Robotics and Intelligent Sensors (IRIS) 2017

2015

1. Protocolo de comunicação para localização de objetos na casa inteligente Prado da Fonseca, Vinicius, and Rosa, Paulo Fernando Ferreira Revista Militar de Ciência e Tecnologia 2015
2. Touch sensing for humanoid robots Oliveira, Thiago Eustaquio Alves, Cretu, Ana-Maria, Fonseca, Vinicius Prado, and Petriu, Emil M. IEEE Instrumentation & Measurement Magazine 2015 [Abs]

   A new generation of humanoid robots is emerging to work together with, or even replace, human operators performing complex dextrous manipulation operations in a variety of applications such as health and elder care, hazardous or high-risk environments, telemedicine, or manufacturing. To meet the challenging operational requirements of such applications, this new generation of humanoid robots should not only look as humans, but should also behave like them, being able to sense and perceive the external world and perform tasks as humans do. Touch sensing and perception is essential when handling objects while working on such complex activities in unstructured environments. The major challenges encountered when replicating the human touch sensing mechanisms are due to the inherently low resolution of the tactile images produced by the artificial sensors, to the complexity of interpreting the sensor data, and to the fact that robot hand technology is still clumsy when compared with the nimble dexterity of the human hand and fingers. This paper presents practical touch sensing solutions for humanoid robots (Fig. 1) that mimic the complex sensing mechanisms occurring in a human hand while exploring by touch 3D objects.

3. Computational intelligence and mechatronics solutions for robotic tactile object recognition Cretu, Ana-Maria, Oliveira, Thiago Eustaquio Alves, Prado da Fonseca, Vinicius, Tawbe, Bilal, Petriu, Emil M., and Groza, Voicu Z. In 2015 IEEE 9th International Symposium on Intelligent Signal Processing (WISP) Proceedings 2015 [Abs]

   The paper discusses signal processing and mechatronics solutions for tactile object recognition using robotic hands. The two important aspects analyzed are the recognition of objects from tactile displacement profiles obtained by force sensing transducers and the recognition of textures using a rubbing motion executed by a robotic finger equipped with a dynamic tactile fingertip. Neural network solutions are used as an intelligent approach to recognize symbols, such as embossed numbers and letters, as well as for the recognition of different texture profiles. The conception issues, the challenges as well as the experimental results obtained are discussed from both a biological and a technological perspective.

4. Data-driven analysis of kinaesthetic and tactile information for shape classification Eustaquio Alves de Oliveira, Thiago, Prado da Fonseca, Vinicius, Huluta, Emanuil, Rosa, Paulo F F, and Petriu, Emil M. In 2015 IEEE International Conference on Computational Intelligence and Virtual Environments for Measurement Systems and Applications (CIVEMSA) 2015 [Abs]

   Humans sense of touch consists in a complexity of sensors and nervous\nsystem. The information inferred by this system enables the daily\ndexterous manipulation tasks. In biological systems, there is no\nconscious prioritization of sensors while performing tactile exploration\nand the selection of exploratory movements is driven by learning\ninstincts and data gathered by previous movements. The development of\nartificial systems tries to mimic such systems with engineered sensors\nand strategies for movement selection. This paper presents a data-driven\nanalysis to the problem of sensor selection in the contour following for\nshape discrimination task. This task consists of a 4-DOF robotic finger\nexploring a set of 7 synthetic shapes. The data collected from the\nmotors, inertial measurement unit, and magnetometer was analyzed\napplying principal component analysis and a multilayer perceptron neural\nnetwork. Results show the variation of classification rate depending on\nthe fingertip material and sensor considered. It is worth to observe\nthat the magnetometer was the most robust in both cases.