Combining Sight and Touch - Research Review Blog

Lab Devices Report

2010-02-15T17:08:00.001-08:00

Video Goggles:
I used the video goggles to complete some basic walking, reading and object manipulation tasks. In my opinion, the stereo goggles were a little hard to adjust for my eyes. I found it difficult to see through both eyes without a double vision. Maybe if I had some more experience this problem would go away. The second thing to note is that the field of vision even after placing the goggles as close to my eyes as possible, are a little narrow compared to normal vision. But that din't seem to be a problem while using the device. Another problem I faced is the color sensitivity of the cameras in the front of the goggles. A yellow book that I was looking at disappeared when it was placed on the table, due to a slight reflection and a glossy cover. Again, some more adjustment might have removed this problem to a certain extent, but it is important to know the limitation of the device while designing systems that use it.

AccelGlove:
This was the lighter weight (and cheaper?) glove that only produced acceleration data on the fingers and wrist as compared to joint angle data of the other glove. I saw the utility that displayed the graph of the various 3 axis accelerometers from the glove. One of the primary 'features' or 'bugs' depending on which way you look at it, is that this glove only reports relative movements. Static poses cannot be determined by looking at values of the accelerometers at a given moment in time. On the other hand, this glove would ideally be used in contexts where dynamic movements of the hand are more important than their static postures. For example, the american sign language can be interpreted through the change in finger positions as opposed to their static posture.

Eye Tracking:
This was the device I was forward to use. Unfortunately, my expectations were a little too high. At it's current state, the device seems to be able to obtain eye movement data, but not transform them accurately to screen coordinates. One theory I have is that even a slight unconscious movement of the head would cause the calibration to be skewed. Even though the setup of the device allows for free rotation of the head, I think we either need to use some head tracking to overcome head movement, or for now, use a chin-rest to stablize the head. Atleast with a chin-rest, we can confirm whether the problem is in the software or in the way the device is used.

CyberGlove:
I used the cyber glove only as part of the user study to collect data. It was comfortable to wear, and did not interfere with any posture or movement that I had to make. The weight of the vibrators were initially noticeable, but after a minute of use, they weren't a hindrance at all. I have yet to complete the programming task for this device.

Motion Editing with Data Glove

2010-02-15T16:19:00.000-08:00

Summary

This paper describes techniques for 'editing captured human motion data' using a data glove as the input device. The problem statement is the current state of techniques for editing human motion data are slow and cumbersome. The industry standard are keyframe based approaches that require the animator to define a posture at multiple key frames. The data-glove being used in the P5 glove, with a resolution of 64 intermediate positions between a flat and fisted hand, although it cannot capture adduction/abduction of the fingers.

The research is described in two stages: capture and reproduction. On capturing data from the glove, first, a topological matching of the human body to the index and middle fingers is performed. The movement of the hands and shoulders is matched to the finger representing the opposite leg. Next, the trajectories of finger and hand movement is mapped to body movement, while overcoming problems with the noisy data. Finally the reproduction stage resolves any values that are outside the possible range of motion by the human body. The paper concludes with some simple observations from a study. The lack of the same physical constraints of the hand as compared to the body, leads to some impossible movements such as jumping with both legs. The authors note that additional information from the environment such as whether or not the fingertips are in contact with the surface, would help make the system more robust.

Discussion

This paper is aimed at a very specific problem, and brings up an interesting solution to it. This is a fairly recent paper, and the prior work shows that the use of a glove for human body motion editing is novel. The interaction that this system allows is very intuitive. I believe that even though there are some flaws in the system, this would definitely replace a LOT manual labor involved in keyframe based editing. The fact that the human hand does not exactly mimic the human body should have been considered obvious, but is stated only in the discussion section.

Comments : here

3DM

2010-02-15T16:09:00.000-08:00

Summary

A head mounted display (a 'VPL eyephone') is used to create a 3D visual experience, and a Polhemus trackers are used for head and hand tracking. A 6D, 2-button mouse is the primary input device. A flat toolbox is rendered in the 3D space providing functionality of creating and manipulating pre-defined objects from the library. The manipulation includes scaling, rotating and translating. The cursor is changed to provide visual feedback on the current mode. Additional features are the surface creation by extrusion, and picking from typical shapes. Grouping of shapes and objects is also allowed. A simple user study showed that a larger feature set is required to enable modeling in the form of CAD tools.

Discussion

This was a very early paper, being almost 20 years since this research was performed. Just for that, this paper deserves accolades for their technical achievement. However, the focus of the research was unclear. As a 3D modeler, the feature set was reported to be lesser than expected by the users. However, as a novel interface for modeling, it seemed to have succeeded, by the fact that users were able to use this system to create some scenes. For a short paper, the technical achievement itself seems to merit publication, even though they could have elaborated more on the implementation and user feedback.

Comments here and here

HoloSketch: A virtual reality Sketchin/Animation Tool

2010-01-26T09:30:00.000-08:00

Deering, M. F. 1995. HoloSketch: a virtual reality sketching/animation tool. ACM Trans. Comput.-Hum. Interact. 2, 3 (Sep. 1995), 220-238.

Summary

This journal paper summarizes the design, development and evaluation of HoloSketch, a 3D VR tool for sketching and animation. The hardware is a SPARCStation 10 ZX with a 960x680 20inch CRT display, acheiving 3D capability with a pair of head-tracked field sequential stereo shutter glasses, obtaining input from a hand-held six-axis mouse with a protuding rod which the authors call a wand. The hardware is claimed to be sub-centimeter accurate, a major acheivement in it's time, critical in the success of the system. The holosketch system makes full use of the 3D by designing a pie-menu that displays hierarchical menu items arranged along the z-axis. The various modes of the system can be accessed through the menu, while drawing is performed by holding down the left button on the wand. Text input is performed through a traditional keyboard. Color is accessed by a 3D pop-up RGB color cube through the center of the pie menu. Selection of an item is made by clicking the middle button while hovering over the item. A multi-select can be made by creating an inclusion cube that is initiated by middle clicking on an empty region of the 3D space. The side button of the wand is used exclusively for the often required task of movement. Three modes of movement are provided : positional panning, change in orientation and combined orientation/panning. Keyboard modifiers ctrl, ctrl+shift and shift are used to select the different modes. Grouping and scaling operations are defined. Property sheets are used to edit attributes of an object, such as the text size, color etc. Animations are described in detail, with various supported modes. Head movement is primarily used for controlling the view of the 3D space. The main user study describes an artist (who has never used HoloSketch) using the system without documentation over a period of a month. Valuble feedback is obtained from the only user in the study over the time period. Finally, the authors elaborate on their future directions of the research : developing a feature filled 3d modelling system; studying novice users while they interact with HoloSketch.

Discussion

This is a fascinating paper on the details of a 3D sketching system. What makes it more fascinating is the fact that this research is more than 15 years old. It's a sobering thought that what we consider future interaction techniques have been a major topic of research for decades. The technical acheivements of developing an accurate 3D pointing device in a VR environment itself is worthy of a reading in this class, but that's just section 1 of this paper. We could nitpick on various details of the design of the menu, but its obvious from this paper that it worked for a novice user, albeit with a few days of practice. Another important point that authors mention is that any such novel device would take more than five minutes of the users time for them to 'internalize' the interaction techniques. That says a lot about the kinds of user studies that are actually relevant to obtaining useful information for such research. How long should we have study participants use novel systems that we develop in class ? Can one hour be sufficient to train them for an elaborate use case ? HoloSketch is a non-trivial application to attempt short user studies, benefits of the system will only become apparant after long term use of the system in the context for which it was developed. Since the application context of the system is 3D modeling, it is not feasible to have 'novice' users test the system. What they really mean is active 3D modelers and designers who are novice at VR interaction. The limitations of the paper are not really limitations given the progress of hardware and graphics capabilities of systems. We can see that we have FAR more powerful graphics systems to render billions of polygons on screen at real time with very acceptable framerates. Adding features to the sketching system is only a matter of time, although each feature could warrant a user study and publication. The immediate question that comes to my mind on reading this paper is : What's the current state-of-the-art for VR modelling systems ?

Comments on Blogs: Josh, Drew

Wearable EOG Goggles

2010-01-26T08:28:00.000-08:00

Wearable EOG goggles: eye-based interaction in everyday environments (Bulling, et al.)

Summary

This paper describes the design and evaluation of a wearable eye-gestrure tracking device using unobtrusive Electrooculography. An aim of the paper is to produce a portable, wearable eye tracking system that is unobtrusive and lightweight. The authors claim that the novelty of their research is the on-board recording and analysis of eye-motion. The goggles holds dry electrodes that make contact with the skin at positions above and below the left eye to track vertical movements (with a reference electrode above the right eye), and to either side of each eye to track horizontal movements. Saccades of the eye are detected using a 'Continuous Wavelet Transform - Saccade Detection' algorithm. Eye gestures are encoded with the symbols L, R, U and D for directional movement and 1, 3, 7, and 9 for diagonal movements. Recognition is performed by string matching.

Discussion

I like the simplicity of the system design, although I am unsure how novel the technology is compared to the cited prior work. This is a good paper to learn about how electrooculography can be used for tracking eye movements without vision. Another important point to note is that the tracking is only relative, meaning that only Up-Down motions can be captured without knowing exactly how much of each direction was performed. In a mobile setting, real-time calibration is a big problem that would need to be addressed if such a device were to be used for interaction purposes. I fail to see a context in which a user would perform eye-gestures sans visual stimuli for interacting with a system. The conclusion of the paper makes a more realistic point about using the system purely for analysis of unconsious eye movement in terms of understanding the 'internal aspects of visual perception such as memory and learning'.

Comments on Blogs: Marat, Josh

Noise Tolerant Selection by Gaze-Controlled Pan and Zoom in 3D

2010-01-21T09:37:00.000-08:00

Paper
Hansen, D. W., Skovsgaard, H. H., Hansen, J. P., and Møllenbach, E. 2008. Noise tolerant selection by gaze-controlled pan and zoom in 3D. In Proceedings of the 2008 Symposium on Eye Tracking Research & Applications (Savannah, Georgia, March 26 - 28, 2008). ETRA '08. ACM, New York, NY, 205-212.

Summary

The paper describes a 'gaze typing system' that enables users to type words with eye-tracking as the only input. A with simple selection techniques such as dwell-time activation is that users are forced to wait for a period of time before selection is made. Common eye-tracking techniques include zooming, expanding in motor space, and enlarging using a fish-eye lens. Zooming was selected for it's seemingly natural visualization without rapid changes of onscreen elements, and ability to make unambiguous selections because of eventually large target sizes. In the StarGazer system, the user is presented with a circular layout of the alphabet, and asked to enter their names as fast as possible. After initial saccade of the eye, the focus settles on a letter of choice, causing the screen to first pan, making the selected letter appear at the center of the screen. On continued focus, the zoom operation takes place, and a plane of activation triggers selection of a letter. A study was run with 48 participants, for three size conditions, some of whom tested the system with added noise and latency.

Discussion

One of the key points that I find disturbing about the paper is the fact that they report lower WPM scores than a paper that they cite. In their discussion section, they try to dismiss this saying that it was due to the fact that they used novice users, and their alphabet layout was not QWERTY. This by itself would have warranted further discussion by the authors. The authors also start the paper not for typing tasks, but for general gaze-based interaction for selection, typing happened to be the use-case that was tested for this study ? Most of the future work only deals with improving typing speed with prediction, dictionaries and user-customization.

Comment at : Murat's blog

Distant Freehand Pointing

2010-01-21T08:36:00.000-08:00

Distant Freehand Pointing and Clicking on Very Large, High Resolution Displays
Daniel Vogel, Ravin Balakrishnan.

Summary

Two clicking techniques are compared: AirTap and ThumbTrigger. The pointing techniques user are RayCasting, Relative Pointing with Clutching and Hybrid RayToRelative Pointing.

Hardware
The screen is 5m x 1.8m large, powered by 18 projectors. The user wears glove with 6 passive reflective markers to track the movement of the hand in 3D at 120Hz using a Vicon motion tracking system.

Design Goals
The authors seek to develop techniques that enable fast, accurate and comfortable interaction while being able to transition from distant to close distances. The interaction should also be easily turned on or off for casual interaction.

Clicking
Visual and audio 'click' feedback is provided at two states of the click (down and up).
AirTap: Down-Up movement of the index finger for a click. Velocity and Distance thresholds are placed to filter noisy movement. ThumbTrigger: Thumb to index finger contact, filtered by using relative and absolute features.

Pointing
Visual feedback is provided on ambiguous positions of the hand to teach participants of accepted input postures. RayCasting: Ray emanating from index finger intersecting with the screen defines the cursor position. A dynamicrecursive low pass filter for 3D marker positions is used to reduce jitter. Relative Pointing with Clutching: Hand for relative cursor control, Clutch to reposition reference frame. Hybrid: Uses ray casting to reposition reference frame instead of a clutch.

Study Results
No significant effects on: order of presentation of techniques, selection time. RayCasting was slower for small targets, but faster for large targets. RayCasting is significantly more error prone (for small and medium targets). Obviously, distance had a significant effect on recalibration (reposition of reference frame).

Discussion

It's interesting that they had to force their participants to keep comfortable positions instead of moving their hands to extreme postures. The techniques are well explained, especially details regarding filtering out noisy movements. This is a very focused study specific to the interaction techniques described, however, as mentioned in the paper this is aimed at casual interaction over prolonged periods of time. It would be interesting to study the effects of the techniques for a contextualized interaction for an extended period of an hour, say selection and organization of photos of an event, or navigating through an information collection.

Comments on Blog: Josh

Introduction

2010-01-21T08:11:00.000-08:00

d.sashikanth ( a t ) gmail ( d o t ) com

I'm in my 3rd Year of the Phd program in Computer Science.

Before College Station, I was based in Hyderabad, India

This course seems relevant to my research in multitouch, and I would benefit from a healthy discussion in the related fields of haptics and gaze tracking. I'm interested in using eye-tracking technologies not as a primary input device, but to aid our understanding of how people perceive various experimental interfaces that we build.

A decade later, ideally, I would continue to be tackling problem in my areas of interest, whatever they may be

Computer science is too vast a field to make claims on (64k enough for anyone ?), but I imagine that the 'next big thing' would be how the next generation of people interact with information from personal devices.

I'm interested in building applications for large and medium sized (multi)touch screens. I believe that there's an untapped potential of our dextrous human hands that is not used by input devices such as the mouse.

I bet a lunch conversation with Genghis Khan would be interesting.

It's tough to single it down to one movie, but for this list, I'd pick City of God.