Week 6: Interaction Design Frameworks & Experience Analysis

Virtual, Augmented and Spatial Computing

Week 6 Overview

  • Interaction design frameworks for XR
  • Analysing XR experiences critically
  • Common interaction patterns and anti-patterns
  • Evaluation methods
  • Assessment 2 briefing

Why Frameworks?

“A framework is a lens — it doesn’t tell you what to see, but it tells you where to look.”

Frameworks help us: - Analyse experiences systematically - Communicate design decisions - Identify problems before building - Compare across different experiences

Framework 1: Norman’s Action Cycle

Donald Norman’s 7-stage action cycle applies directly to XR:

  1. Form a goal — what does the user want to do?
  2. Form an intention — how will they do it?
  3. Specify an action — which gesture/input?
  4. Execute the action — perform the input
  5. Perceive the state — what changed?
  6. Interpret the state — what does it mean?
  7. Evaluate the outcome — did it work?

Norman in XR: Failure Points

Stage Common XR Failure
Form intention No affordance — user doesn’t know what’s interactive
Specify action Gesture not discoverable
Execute action Tracking failure, fatigue
Perceive state No feedback (visual/audio/haptic)
Interpret state Ambiguous feedback
Evaluate outcome No confirmation of success

Framework 2: Milgram’s Reality-Virtuality Continuum

Real ←————————————————————————→ Virtual
Environment    AR    MR    AV    Environment
  • AR: Real world + virtual overlays (Snap Spectacles, Hololens 2)
  • MR: Real and virtual interact (Hololens 2, Quest 3 passthrough)
  • AV: Mostly virtual + some real elements
  • VR: Fully virtual (Quest 2, Vive Pro)

Design implication: Interaction design must match the continuum position.

Framework 3: Presence Model (Slater)

Two components of presence:

  • Place Illusion (PI): “I am there” — the sense of being in the virtual space
  • Plausibility Illusion (Psi): “This is real” — events feel like they’re really happening

Interaction design affects both: - Poor interaction breaks PI (you notice the interface) - Inconsistent physics breaks Psi (the world feels fake)

Interaction Patterns

Pattern: Progressive Disclosure

Reveal complexity gradually. Don’t show all options at once.

Pattern: Spatial Consistency

Objects stay where you put them. The world has memory.

Pattern: Embodied Metaphor

Map virtual actions to real-world physical actions. Example: Pull a lever to open a door (not press a button).

Pattern: Graceful Degradation

If tracking fails, fall back gracefully. Don’t crash or freeze.

Interaction Anti-Patterns

Anti-Pattern Problem
Floating menus Break immersion, cause eye strain
Tiny targets Violate Fitts’ Law, cause frustration
No feedback User doesn’t know if action worked
Gorilla arm UI Causes fatigue in minutes
Forced smooth locomotion Causes sickness for sensitive users
Invisible affordances User can’t discover interactions

Evaluation Methods for XR

Method When What it measures
Heuristic evaluation Early Design problems against principles
Think-aloud Mid User mental model
SUS (System Usability Scale) Post-use Overall usability score
SSQ (Simulator Sickness Questionnaire) Post-use Comfort/sickness
Presence questionnaire Post-use Sense of presence
Eye tracking heatmap During Attention patterns

Heuristic Evaluation for XR

Nielsen’s 10 heuristics adapted for XR:

  1. Visibility of system status → feedback on every interaction
  2. Match between system and real world → embodied metaphors
  3. User control and freedom → undo, cancel, exit
  4. Consistency and standards → consistent gesture vocabulary
  5. Error prevention → confirm before destructive actions
  6. Recognition over recall → visible affordances
  7. Flexibility and efficiency → shortcuts for expert users
  8. Aesthetic and minimalist design → diegetic UI
  9. Help users recognise errors → clear error states
  10. Help and documentation → in-world tutorials

Assessment 2: Case Study Brief

Due: End of Week 8 | Weight: 30% | LO2, LO3, LO4

Three Parts:

  1. Experience Analysis (40%)
    • Choose a VR or AR experience
    • Apply 2 frameworks from today’s lecture
    • Identify 3 interaction strengths and 3 weaknesses
  2. Interaction Prototype (40%)
    • Redesign one weak interaction
    • Build a low-fi prototype (Unity or paper)
    • Justify design decisions using frameworks
  3. Evaluation (20%)
    • Test with 2 users
    • Report findings
    • Reflect on what you would change

Assessment 2: Choosing an Experience

Good choices: - Beat Saber (interaction design, rhythm, feedback) - Tilt Brush / Open Brush (creative tools, spatial UI) - Microsoft Flight Simulator VR (complex UI, comfort) - Hololens 2 demos (AR interaction, spatial anchoring) - Snap Spectacles experiences (AR, gesture) - Any experience from our lab hardware

Key Takeaways

  • Frameworks give you a systematic language for analysis
  • Norman’s Action Cycle identifies where interactions fail
  • Milgram’s continuum positions your design in the XR spectrum
  • Presence theory explains why interaction quality matters
  • Evaluation methods should match your design stage

References

  • Norman, D. (2013) The Design of Everyday Things (revised ed.). Basic Books.
  • Milgram, P. & Kishino, F. (1994) “A taxonomy of mixed reality visual displays.” IEICE Transactions.
  • Slater, M. (2009) “Place illusion and plausibility can lead to realistic behaviour in immersive virtual environments.” Philosophical Transactions of the Royal Society B.
  • Nielsen, J. (1994) “Heuristic evaluation.” In Nielsen & Mack (eds.) Usability Inspection Methods. Wiley.