"Those Look Similar" Issues in Automating Gesture Design Advice

Published in 2001, Long et al's paper, as suggested by the title, discusses issues found related to the timing and content of feedback on gesture design advice within the context of their program, "Quill."

Quill is a gesture design tool that allows its users to create gesture sets and determine their computational and visual similarity so that the gestures can be accurately recognized as well as easily remembered. Quill attempted to use 'unsolicited advice' to warn the gesture designer that their gesture was too hard to recognize so that the designer to take steps to correct their gestures.

The problems with the unsolicited advice were broken down into three categories:

1. Timing
• When to show the advice
• As soon as problem is detected
• Benefits:
• Can (potentially) fix the problem as soon as it occurs
• Drawbacks:
• Interrupts ideation and focus on task
• Advice may no longer be relevant later
• => Confusion from Incorrect Advice
• May be ignored anyway
• Delayed until designer is testing the gesture set
• Benefits:
• Won't produce advice that is incorrect / no longer relevant
• Drawbacks:
• Won't see the problems until testing
• Won't see what caused the problems as it happens
• When to remove the advice
• When told to be ignored by the user
• Trivial
• When the problem is fixed
• "Fixed" is not well defined
• How dissimilar do gesture need to be to be unambiguous and easy to remember?
• Need to evaluate whole gesture set when a change occurs
2. Volume
• How much advice to give
• Concise Info First, Detail-on-demand
3. Content
• Expert User:
• Less info
• automatically generated
• a little more cryptic
• Beginner: 
• Links to more detailed info
• handcrafted information, examples, and illustrations

Long et al. also needed to determine how to perform the more computationally heavy analysis in the background of the application. Since the Quill tool attempts to display the analysis and advice info in the GUI dynamically and because the content of that information depends on the analysis, the policy on when to perform the analysis affects the user's mental model of the nature of and the connection between gesture form and gesture identifiability.

5 schemes for analysis computation were devised:

1. Lock all user actions during advice computation
• Benefits: Easy to implement and understand
• Drawbacks: Frustrates user with delays
2. Disable any action that affects the advice  <= Used for User Initiated Analyses
• Benefits: Distinct separation between action and effect on analysis
• Drawbacks: Frustrates user with delays; potentially confuses users who don't see the connection between the action and the effect
3. Allow any action but cancel if it affects the advice <= Used for System Initiated Analyses
• Benefits: Some freedom
• Drawbacks: User can inadvertently cancel the calculation and cause delays and must determine when to restart calculation
4. Allow all actions
• Benefits: High User Freedom
• Drawbacks: Potentially incorrect advice
5. Disable user actions that would change state in use by current analysis
• Benefits: Most efficient in that it allows non-conflicting operations to occur
• Drawbacks: High confusion because of the differing availability of options for different sets of gestures

Long et al. also noted that having error in the advice either greatly confused the gesture designers or caused them to ignore the advice entirely. Therefore, correctness of advice and quality of similarity metrics is a priority.

Personal thoughts:
Quill's unsolicited advice system sounds like a nice idea, but it reminds me a lot of the 'clippy' characters that Microsoft tried to advocate in the early to mid 1990s. People don't like being interrupted and they certainly don't like being given obvious or unhelpful advice. In this respect, the admittance at the end of the character that there the similarity metric challenges produced negative results in the user groups was a little underwhelming despite how important the observation may be.


Long et al. made an interesting bullet point on their list of future work when they said that they wanted, "To (partially) automate the repair of human similarity and recognition problems by morphing gestures."
This to me seems like the best solution to their problem of giving clear feedback on what constitutes a good gesture set. Providing an intelli-sense menu of possible transformations may have been useful.


Cited Work
A. Chris Long, James A. Landay, and Lawrence A. Rowe. 2001. "Those look similar!" issues in automating gesture design advice. In Proceedings of the 2001 workshop on Perceptive user interfaces (PUI '01). ACM, New York, NY, USA, 1-5. DOI=10.1145/971478.971510 http://doi.acm.org/10.1145/971478.971510