UX Research & Click Test Analysis

Which design wins?
Testing scan-button discoverability, not assuming it.

Users couldn't reliably find the scan button in the PBL app during live shopping events. Rather than pick a fix by instinct, we designed two competing nav bar variants and ran a moderated click test, 200 participants, real time-on-task data, heatmaps, to let evidence decide.

RoleUX Researcher
MethodClick test, Userzoom
Participants200, ages 18–80
2-minute read
Problem
The scan button wasn't consistent across screens, users couldn't reliably find it.
Approach
Designed two nav bar variants and ran a click test to see which improved discoverability.
Execution
200 participants via Userzoom; caught and corrected a data contamination issue mid-study.
Result
Variant 1 won on speed (16s vs 20s) and preference (76%), the opposite of our hypothesis.
The Problem

Real events surfaced a real usability gap

During the Atmore and Montgomery shopping events, live usage exposed friction that internal review hadn't caught.

01

Scan button placement wasn't consistent from screen to screen

02

Sign-in button was hidden by the keyboard on open

03

Users reported general difficulty locating and using the scan feature

04

No data yet on which of two proposed fixes actually performs better

Hypothesis

We expected Variant 2 to win. It uses a floating action button (FAB) for the scan icon, a shape that breaks the pattern of the rest of the nav bar, which the Von Restorff effect suggests should make it easier to notice and remember. That gave us a clear, falsifiable prediction to test rather than debate.

The Two Variants

Two nav bar designs, one shared goal

Both variants redesigned the navigation bar to emphasize the scan feature, Variant 2 leaned on the FAB pattern, Variant 1 kept it inline but relabeled and repositioned.

Variant 1
Variant 1 nav bar
Inline scan icon, relabeled and repositioned in the nav bar
Variant 2
Variant 2 nav bar
Floating action button (FAB), round shape breaks pattern with rest of nav bar
Before / After, Actual Test Data

What participants actually did

Heatmaps and click maps from the corrected retest, the same task, run against each variant with a fresh, unexposed set of participants.

Task 1, Click the QR scan button
Variant 1 · Median time on task: 16s
Heat map
Variant 1 heatmap
Click map
Variant 1 clickmap
Task 2, Click the QR scan button
Variant 2 · Median time on task: 20s
Heat map
Variant 2 heatmap
Click map
Variant 2 clickmap
200
Participants recruited
93%
Completed assigned tasks
16s
Variant 1 median time-on-task
20s
Variant 2 median time-on-task
Task 3, Questionnaire

Closing the loop with participant feedback

Beyond time-on-task, we asked participants directly, a rating scale, a forced choice, and an open-ended question, to round out the quantitative data with intent and preference.

44% (18)
12% (5)
17% (7)
12% (5)
15% (6)
0%10%20%30%40%50%60%70%80%90%100%
Agree — Disagree
1 2 3 4 5
44% agreed the scan button was easy to discover
Version 1
76% (31)
Version 2
24% (10)
0%10%20%30%40%50%60%70%80%90%100%
76% found Variant 1 the most visually appealing
Word frequency, open-ended responses41 responses
Easier
Scan
Button
Find
Navigate
Labeled
Clear
One
Bar length reflects relative mention frequency, not exact counts. Lighter mentions included "much," "better," "thought," "like," and "use" - all reinforcing the same theme: the scan button needed to be easier to find.
Most-used words in open-ended responses, "easy," "scan," "button," "find"
The Approach

Letting the retest correct the first result

Hypothesize

Made the prediction falsifiable before testing

Backed Variant 2's FAB pattern with the Von Restorff effect, a specific, testable reason to expect it would win, rather than a vague preference.

Recruit

200 participants, screened for age 18+

Recruited through Userzoom across a broad age range (18–80), with a screening question gating entry so results reflected real adult users.

Catch & correct

Caught a contamination issue mid-study, retested

The first pass tested both variants on the same participants, which made the results unreliable. Re-ran the study with separate, unexposed participant groups per variant before drawing conclusions.

Contaminated pass: V1 23s · V2 12s → Clean retest: V1 16s · V2 20s
Analyze

Combined behavioral and stated-preference data

Cross-referenced time-on-task and heatmaps against the questionnaire results, including an open-ended question, to check whether what people did matched what they said.

Outcome

The data overturned the hypothesis

Key insight

Variant 1 won on both measures we trusted, faster median time-on-task and stronger stated preference, despite Variant 2's FAB pattern being the theoretically stronger bet going in. Heatmaps also showed participants instinctively looking for the scan feature in the header, near search, likely a habit carried over from other e-commerce apps, a separate, reusable insight for future navigation work.