Methods to study UX snapshots

The methods in this category evaluate momentary experiences while the participant is interacting with the system. An example of an evaluation case is assessment of emotions during game play.

Affect Grid

Affect Grid is a scale designed as a quick means of assessing affect along the dimensions of pleasure-displeasure and arousal-sleepiness.

Affective Diary

During a field study, a participant wears a sensor that registers their physical states. User's mobile phone logs the activities on the phone. The data from both sources is combined to scraps of person's life. The scraps data is presented to the participant who can scribble their reflected thoughts on them.

Controlled observation

Individual participants are invited to a controlled environment (not real context) to test e.g. colors or audio of the system. The target is to gain insights of design details that would be hard to test in real contexts (e.g. controlled lighting conditions, background noise).

Differential Emotions Scale (DES)

The Differential Emotions Scale (DES) is a standardized instrument that reliably divides the individual's description of emotion experience into validated, discrete categories of emotion. The DES was formulated to gouge the emotional state of individuals at that specific point in time when they are responding to the instrument.

e) Theoretical background: theories/models underlying the tool/method
Izard, 1972, 1977

Emotion Cards

Provide a way for users to quickly document emotions at a specific moment

Experience Sampling Method (ESM)

Experience sampling is a method for collecting experiences "in situ" and immediately, thus there is no disturbance because of memory effects.

Experience clip

When collecting data about user experience evoked by mobile acpplications, one user uses the mobile application to be evaluated, and another uses a mobile phone to shoot clips about usage and expressions of experiences. The two users know each others, and interact in natural situations and environment without presence of researcher. The method takes use of the social interaction between the user and someone the user knows well.

Experiential Contextual Inquiry

Observing the user in real context, researcher taking a role of an apprentice. The method was originally developed for understanding work practices*. When focus is on UX, the researcher pays attention to the emotional aspects of product use: not only the behavior but also the affective aspects of product use.
* "Contextual Inquiry: Field interviews with customers in their work places while they work, observing and inquiring into the structure of their own work practice."

Facial EMG

Does natural motion behavior during gaming enhance positive UX regarding the intensity of measured dimensions of emotional arousal and valence. Comparison of two products.

Geneva Appraisal Questionnaire

The Geneva Appraisal Questionnaire (GAQ) can be used to assess, as much as is possible through recall and verbal report, the results of an individual's appraisal process in the case of a specific emotional episode (as based on Scherer's Component Process Model of Emotion). The files available for download contain the current English, French, and German versions (and information on utilization).
This is a tool that can be used to describe emotional experiences (i.e. not a description as such). Tool is available in English, French and German

Geneva Emotion Wheel

Based on Scherer's Component Process Model, the Geneva Emotion Research Group has developed this new instrument to obtain self-report of felt emotions elicited by events or objects.

Hedonic Utility scale (HED/UT)

The HED/UT is developed to both address hedonic aspects of product/website interaction and the utitility and usability aspects. HED/UT is an attitude measure from the consumer behaviour literature, and consists of 12 items measuring hedonic value, and 12 items measuring utilitarian value of a service or concept.

Human Computer trust

The Human Computer Trust scale is a psychometric instrument specifically designed to measure human-computer. Both cognitive and affective components of trust are measured; the affective components are the strongest indicators of trust.

I.D. Tool

(From the ENGAGE Web site description:) I.D. Tool identifies the physical design attributes that a product has in order to evoke the desired experience by the target customers. This is uncovered by mapping the user's mental reactions that creates the immediate affective impressions of the product as well as the long term opinions towards it.

Intrinsic motivation inventory (IMI)

The Intrinsic Motivation Inventory (IMI) is a multidimensional measurement device intended to assess participants' subjective experience related to a target activity in laboratory experiments. It has been used in several experiments related to intrinsic motivation and self-regulation (see weblink for references).

Mental effort

Zijlstra's mental effort scale is an easy and quick to use scale that helps to determine how much (perceived) mental effort was required to complete a task; depending on the setting, product and task, and in combination with other measures, this will help in getting a clearer picture of the overall quality of a product or service - too much mental effort will be stressful, and scary, too little mental effort will be boring....

Mindmap

Lab study: Combination of qualitative and quantitative measures for tasks.

PAD

General Tests of Emotion or Affect for Evaluating Consumer Reactions to Products and Services, Including User Interface.

Paired comparison

Very easy to use technique to rank order stimuli (products) with respect to some quality (e.g. enjoyment); also easy to do for children; goes back to early (1920's) test and scale development techniques; paired comparison data can be transformed in ordering stimuli.

Physiological arousal via electrodermal activity

The method utilises physiological arousal as an indicator for involvement and emotional state (arousal not valence). It is meant to also capture unconscious processes. Unobtrusive, not distracting, continuous measure, in situ

Playability heuristics

Playability heuristics evaluate the playability aspect within games. Apart from usability problems, the heuristics can reveal the experiential aspects of game play.

Positive and Negative Affect Scale (PANAS)

The Positive Affect Negative Affect Schedule (PANAS) is a psychometric scale developed to measure the largely independent constructs of positive and negative affect both as states and traits. Positive and negative affect have been shown to relate to other personality states and traits, such as anxiety. PANAS was originally developed for more clinical settings, but is also used now in evaluation studies in which moods of users might be affected (e.g. studies around effects of lighting, of entertainment content, etc).

PrEmo

Emotional responses elicited by consumer products might be difficult to verbalize because their nature is subtle (low intensity) and often mixed (i.e. more than one emotional response at the same time). So, emotional responses to products might be difficult to measure with verbal questionnaires. Instead of relying on the use of words, respondents can report their emotions with the use of expressive cartoon animations. In PrEmo, 14 emotions are portrayed by an animation of dynamic facial, bodily, and vocal expressions.

Presence questionnaire

Presence is defined as the subjective experience of being in one place or environment, even when one is physically situated in another.) The authors of the scale state that presence is a normal awareness phenomenon that requires directed attention and is based in the interaction between sensory stimulation, environmental factors that encourage involvement and enable immersion, and internal tendencies to become involved. Focus of the PQ is to measure presence in virtual environments and games. In addition the immersive tendencies questionnaire (ITQ) was developed to measure differences in the tendencies of individuals to experience presence.

Psychophysiological measurements

E.g. heart beat, skin perspiration, facial muscles tell about the emotional state of the user. The physiological reactions are recorded with sensors attached to the participant. This objective data can be used in combination with self-report data to find out what the user experienced.

Resonance testing

Resonance testing is a method to validate product concepts againts a set of experiential goals. It is primarily intended for use with physical products

SUMI

Satisfaction is a part of classic definition of usability since a long time. SUMI (Software Usability Measurement Inventory) has been developed to provide an authoritative, standardised measurement of user satisfaction with software. It can be used for the evaluation and comparison of products (or versions of a product) and to set and track verifiable targets regarding satisfaction. SUMI is a classical Likert-type measure of attitude toward a software package. The questionnaire comprises five subscales: efficiency, affect, helpfulness, control and learnability. SUMI analysis also provides a “global” satisfaction score.

Self Assessment Manikin (SAM)

SAM is an emotion assessment tool that uses graphic scales, depicting cartoon characters expressing three emotion elements: pleasure, arousal and dominance. SAM has been used often in evaluations of advertisements, and increasingly also in evaluations of products. SAM is based on the PAD emotion model of Mehrabian.

Semi-structured experience interview

Face to face or online interview. Ask people what they think, feel, experience

Sensual Evaluation Instrument

Different shaped objects are used by users during a usability test to express how they feel. After the test, an interview is conducted to interpret the results.

TUMCAT

Remote, automated, online sampling of user experiences, based on automatic logging of user actions or on timing over the network.

This-or-that

Pairwise comparison testing aimed at young children (preschoolers).

Timed ESM

Experience sampling during field studies, so that the participant reports their experience at a certain point of time.

UTAUT

UTAUT is based on Technology Acceptance Model (TAM), but addressing some of TAM's shortcomings; includes now also more or less affective aspects

UX Expert evaluation

UX experts use their expertise of users and UX theories to evaluate UX of a system.

UX laddering

UX Laddering is an adapted interview method and adapted data analysis process for investigating the user experience, adapted from Laddering in consumer research and based upon Means-end Theory. The goal of laddering -as with all means-end approaches- is to identify and understand the linkages between key perceptual elements across the range of attributes, consequences and values. Therefore, UX Laddering helps researchers and designers understand how concrete product attributes benefit personal values for end users.

Valence method

The valence method is based on the user experience model of Hassenzahl (2008) defining user experience as a primary evaluative feeling during the usage of a product or service. A positive user experience the consequence of fullfinging human needs. The valence method captures positive and negative feelings (valence) during the explorative usage of an interactive product. In a subsequent retrospective interview phase users indicate for each instance of a positive or negative feeling (valence marker) the product design aspects inducing it. This phase further employs the laddering interview technique [Reynolds & Gutman, 1988] to reveal the personal meaning of product design aspects to the user and the underlying fulfilled or frustrated needs. The generated information helps designers to understand and optimize the user experience potential of a product.