Publications of Dr. Oscar Meruvia-Pastor & Collaborators

I work at the interphase between computer graphics and human-computer interaction. More precisely, I have done research in telepresence systems, multimedia applications, medical imaging, non-photorealistic rendering and visibility pre-processing algorithms.

My current research interest is in human-centered solutions for 3D telepresence, remote training, immersive Virtual Reality, Augmented Reality and telecollaboration. Together with several colleagues, Computer Science Graduate Students under my supervision are working or have worked on gesture-based interaction using depth-sensing devices, 3D-enhanced telecollaboration systems and algorithms for obtaining depth from stereoscopic imagery.


3D Telepresence for Education, Training and Games

A Window to your Smartphone: Exploring Interaction and Communication in
Immersive VR with Augmented Virtuality

A major drawback of most Head Mounted Displays (HMDs) used in immersive Virtual Reality (VR) is the visual and social isolation of users from their real-world surroundings when wearing these headsets. To address this issue, we present a new method to allow people wearing VR HMDs to use their smartphones or tablets without removing their HMDs. We augment the scene inside the VR HMD with a view of the user's smartphone so that the user can interact with the device without removing the headset using additional cameras, such as the Leap Motion sensor, on top of the Oculus Rift. Our systems allows VR users to have a window to their smartphone from within the virtual world and make user of much of the functionality provided by their smartphones.
(accepted manuscript; online article @ IEEE Xplore; reference).

Operating Virtual Panels with Hand Gestures in Immersive VR Games - Experiences with the Leap Motion Controller

In this research, we explore gesture-based interaction in immersive VR games and present a novel platform for research and rapid application development at an affordable cost, while avoiding constant Android deployment during development. Using the Unity game engine, we implemented a prototype to interact with a numeric dial panel and play a game in a virtual world and we compared placements of the leap motion sensor on the desktop and on the HMD. Our results suggest that even though the tracking quality of the Leap Motion sensor was rather limited when used in the HMD for pointing and selection tasks, its performance was much better in the desk-mounted setup.
(accepted manuscript; online article @ Springer; reference).

OMARC: An Online Multimedia Application for the Assessment of Respiratory Conditions

OMARC is a multimedia application designed to support the training of health care providers in the identification of common lung sounds heard in a patient's thorax as part of a health assessment session or group of sessions. The main goal of OMARC is to effectively help health-care students and professionals to become familiar with lung sounds that might be heard during the assessment of respiratory conditions. OMARC was developed using an online platform to facilitate access to users in remote locations and is designed in a way that is easy to use and accessible to students and practitioners over the internet.
(accepted manuscript; online article @ IJMI; reference).

Evaluation of Depth-Sensing Cameras against Off-the-Shelf Controllers for Pick and Drop Tasks Using a Simulated Robot Arm Manipulator

In this article, we investigate a depth-sensing camera controller based on inverse-kinematics for placement tasks using a robot arm simulator, and describe a user study to evaluate participants' performance using short-range depth-sensing camera controllers against comparable off-the-shelf controllers.
(conference article @ HCII; reference).

Automatic and Adaptable Registration of Live RGBD Video Streams

A novel method for automatic registration of video streams originated from two depth-sensing cameras is shown. The motivation of this work is to facilitate the use of RGBD sensors for non-expert users, so that cameras need not to be calibrated, and if cameras are moved, the system will automatically recover the alignment of the video streams. DeReEs is used, since it is fast and successful in registering scenes with small overlapping sections.
(conference article @ SIGGRAPH's MIG; poster @ SIGGRAPH; video (36.5MB); reference).

*Student Research Competition Semi-finalist Poster at SIGGRAPH 2015 *

Robot Arm Manipulation Using Depth-Sensing Cameras and Inverse Kinematics

This article presents a new method in which the user just needs to point towards the target and a robotic arm will reach the target itself using inverse kinematics algorithms and a depth-sensing camera to capture the user commands. The advantage of this approach is that manipulation of the arm needs less training and is easier to learn compared to more complex input devices. We test our approach with a robot arm simulator similar to those used in industrial applications.
(conference article @ MTS/IEEE's Oceans ; poster abstract @ GI; poster @ OERA-NS; reference).

*Best HCI Poster at Graphics Interface (AI/GI/CRV) 2014 *

A Comprehensive Framework for Evaluation of Stereo Correspondence Solutions in Immersive Augmented and Virtual Realities

In this work we present a comprehensive evaluation scheme for stereo correspondence methods in outdoor Augmented Reality applications based on the processing speed requirements in AR and the necessary accuracy of depth results for the human visual system (HVS) at different ages. For a given stereo-correspondence solution, various metrics are estimated and reported in the framework of outdoor augmented reality: Average disparity error, average number of outliers, average stereoacuity and execution times.
(Journal article @ JVRB, reference; conference article @ VISAPP, reference,Graphics Interface poster).

DeReEs: Real-Time Registration of RGBD Images Using Image-Based Feature Detection And Robust 3D Correspondence Estimation and Refinement

DeReEs, an algorithm to align color + depth image pairs obtained from commercial depth cameras to build an extended model of the scene captured by both cameras is presented. The challenge is to work with input images taken from the same scene from different positions, with small overlapping regions, with the potential to support real-time processing.
(conference article @ ICVNZ; Graphics Interface poster; reference).


Visualization and Processing of Medical Data

Estimating Cell Distribution in Histological Samples Based on Incremental Search and Color Filtering

Incremental Cell Search (ICS) is a software that expedites the analysis of relationships between morphological outgrowth and cell proliferation in embryonic tissues. Based on an estimated average cell size and stain color, ICS rapidly indicates the approximate location and amount of cells in histological images of labeled embryonic tissue and even provides estimates of cell counts in regions with saturated fluorescence and blurred cell boundaries.
(open access article @ IJBI, ICS software distribution , reference).

 

Integrative Visualization of Temporally Varying Medical Image Patterns

We present software for the visualization of temporal changes of disease patterns using stacks of medical images collected in time-series experiments. With this software users can generate 3D surface models representing disease patterns and observe changes over time in size, shape, and location of clinically significant image patterns. Statistical measurements of the volume of the observed disease patterns can also be obtained simultaneously.
(journal article, online article @JIB, reference).

Generation of Connectivity-Preserving Surface Models of Multiple Sclerosis Lesions

In this article we introduce a software tool to build 3D surface models of Multiple Sclerosis (MS) brain lesions from 2D image stacks typically obtained through Magnetic Resonance Imaging (MRI). This tool allows users to create 3D models of MS lesions quickly and visualize the lesions and brain tissue using various visual attributes and configurations.
(journal article, online version @ google books, online version @ SHTI, reference).

A Novel MRI Visualization Tool for White Matter Pathology in Multiple Sclerosis

We developed a tool to visualize MS lesions and their 3D surface models to show changes in the lesions over time. These can be shown as an animation to elucidate differences across scanning sessions. With this software, a volumetric sub-region can be selected from the 3D model for zooming or animation, and a point on the 3D model can be selected to highlight all lesions connected to it. The total volume of lesions can be calculated, displayed as a chart, and exported.
(poster, online abstract @ SfN, reference).

Concurrent Visualization of and Mapping between 2D and 3D Medical Images for Disease Pattern Analysis

We present a software tool to highlight and display regions of interest in 2D medical images and their 3D mesh model counterparts. This tool can help researchers visualize and compare brain lesions and tissues in 2D and 3D at the same time.
(conference article, online article @ VBL, reference).

Building generic anatomical models using virtual model cutting and iterative registration

Using 3D generic models to statistically analyze trends in biological structure changes is an important tool in morphometrics research. In this article we show how the creation of a generic model that captures spatial information exploitable in statistical analyses is facilitated by coupling our generalized segmentation method to existing automatic image registration algorithms.
(open access article @ BMC-MI, reference).

Fast Interactive Integration of Cross-sectional Image Datasets and Surface Data for Morphometric Analysis

A system to align a stack of images in parallel for the reconstruction of a 3D data structure and surface model from a set of histological cross-sections is presented, along with complementary tools for interactive alignment. introduced.
(journal article, online article @ SHTI, poster @ iCore, reference).

An Efficient Virtual Dissection Tool to Create Generic Models for Anatomical Atlases

A method is presented to combine 3D mesh cutting tools to extract a particular object from a polygonal mesh of amalgamated structures. With this technique it is possible to extract individual objects, such as the jawbone from a skull, from a polygonal model.
(journal article, online article @ SHTI, reference).


Non-Photorealistic Rendering

Adaptive Incremental Stippling Using the Poisson Disk Distribution

This paper presents a method to produce high-quality stippled renditions with an emphasis in detail enhancement, using edge information contained in the input image.
(full text paper, online article @ JGGPUGT, reference).

 

Poisson-Disk Distributed Dots and its Applications in Non-Photorealistic Rendering (Puntos con Distribución Poisson Disk y su Uso en NPR)

This paper, written in spanish and presented in Mexico, presents two techniques to produce stippled renditions faster than traditional methods at the expense of quality.
(full text paper, reference).

 

OPENNPAR: A System for Developing, Programming, and Designing Non-
Photorealistic Animation and Rendering

In this article, co-written with Nick Halper et al, we present an Open-Source project to implement a library of algorithms commonly used in non-photorealistic rendering.
(full text paper, reference).

Graph-Based Point Relaxation for 3D Stippling

A new way to distribute point particles on the surface of a model that also satisfy the requirement that they belong in a point hierarchy of well-distirbuted points on several levels of detail is proposed. We also introduce the concept of using stipples to convey deformable semitransparent surfaces.
(full text paper, videos, reference).

 

Real-Time Animated Stippling

A point hierarchy mesh is created on the surface of a deformable 3D model. The point hierarchy is used to convey the stippling style used by many artists. Because the point hierarchy is attached to the surface of the mesh, the points enable frame-coherent rendering on the particle level
(full text paper, videos, reference).

 

Frame-Coherent Stippling

The foundations for creating a hierarchy of points based on mesh simplification and mesh subdivision of steps that can be used to convey a 3D-stippling style is introduced here. The technique is applied on models with fixed geometry.
(full text paper,poster, reference).

 


Visibility Precomputation Techniques

Visibility Preprocessing Using Spherical Sampling of Polygonal Patches

This paper builds up on the concept of visibility precomputation, and presents a more robust approach, where polygons are organized in contiguous regions of polygonal patches. The size of this bitfields is relatively small and the technique yields a dramatic reduction in the number of polygons sent to the graphics processor, so the response time is improved significantly for models with many folds or self-occluding surfaces.

(full text paper, poster, videos, reference).

Approximated View Reconstruction Using Precomputed ID-Bitfields

A method to pre-compute visibility of the polygons of a 3D mesh in order to cull away the non-visible polygons is presented. The advantage of visibility precomputation is that, for any arbitrary viewpoint, a reduced numer of polygons is sent to the graphics processor. A list of pre-computed polygons is encoded in a long bit list called ID-bitfield.
(full text paper, presentation, reference).


Dissertation & Thesis

Frame-Coherent 3D Stippling for Non-photorealistic Computer Graphics
Oscar Meruvia-Pastor, Dr. Ingenieur (Ph.D.) Dissertation

The foundations for frame-coherent 3D stippling are described here, as well as methods for rendering animated and static models in real time. In addition, several applications for 3D stippling in archaeology, anatomy and animated semi-transparent surfaces are presented.
(full text of dissertation (25MB), videos, videos referred, reference).

Level of Detail Selection and Interactivity
Oscar Meruvia-Pastor, M.Sc. Thesis

A new method to switch between different levels of detail is proposed. The method aims at improving the quality of interaction by reducing the response time based on a measure of divergence between the expected and the actual positions of a subject.
(full text of thesis, presentation, reference).


Theses of the Graduates of

the 3D Telepresence & Telecollaboration Group

DeReEs: Real-Time Registration of RGBD Images Using Image-Based Feature Detection and Robust 3D Correspondence Estimation and Refinement
Afsaneh Rafighi, M.Sc. Thesis

This thesis describes in detail a new method in which the user just needs to point towards the target and a robotic arm will reach the target itself using inverse kinematics algorithms and a depth-sensing camera to capture the user commands. In addition, a user study to evaluate participants' performance using short-range depth-sensing camera controllers against comparable off-the-shelf controllers is described in detail.
(full text, reference).

DeReEs: Real-Time Registration of RGBD Images Using Image-Based Feature Detection and Robust 3D Correspondence Estimation and Refinement
Akhilesh K. Mishra, M.Sc. Thesis

This thesis describes in detail a new method in which the user just needs to point towards the target and a robotic arm will reach the target itself using inverse kinematics algorithms and a depth-sensing camera to capture the user commands. In addition, a user study to evaluate participants' performance using short-range depth-sensing camera controllers against comparable off-the-shelf controllers is described in detail.
(full text, reference).

DeReEs: Real-Time Registration of RGBD Images Using Image-Based Feature Detection and Robust 3D Correspondence Estimation and Refinement
Sahand Seifi, M.Sc. Thesis

This thesis describes in detail DeReEs, an algorithm to align color + depth image pairs obtained from commercial depth cameras to build an extended model of the scene captured by both cameras is presented. The challenge is to work with input images taken from the same scene from different positions, with small overlapping regions, with the potential to support real-time video processing.
(full text, reference).

A Comprehensive Approach for Evaluation of Stereo Correspondence Solutions in Outdoor Augmented Reality
Bahar Pourazar, M.Sc. Thesis

This thesis describes in detail a comprehensive evaluation scheme for stereo correspondence methods in outdoor Augmented Reality applications based on the processing speed requirements in AR and the necessary accuracy of depth results for the human visual system (HVS) at different ages. For a given stereo-correspondence solution, various metrics are estimated and reported in the framework of outdoor augmented reality: Average disparity error, average number of outliers, average stereoacuity and execution times..
(full text, reference).


References