Articles > Conversion Optimization Designer, Test Thyself – Five-Second Usability Tests

These days, most web designers know the value of testing designs with representative users. And most have heard that there’s a lot to be gained from conducting even small, informal usability tests to validate page layout, branding ethos, and messaging. But the logistics of usability testing can get in the way. Even if a test is easy to do and doesn’t require a huge number of test participants, it’s still difficult to set it up, run it, and gather results properly. This challenge has driven the development of numerous remote usability testing tools, including this new entrant in the field.

A Five Second Background

In early October I had the pleasure of chatting with Matt Milosavljevic of MayhemMethod.com about their cool little usability testing tool, “Five Second Test.” At that time, the tool had just launched and was already receiving thousands of visits a day. The test itself was devised by Jared Spool, but it was a comment by Robert Hoekman, Jr. that sparked the idea for the online version.

“Robert mentioned the test, how useful it was, and also pointed out the difficulty of gathering people in one place to conduct the test,” Matt said. “We thought the web was the perfect answer to the problem.” And so the idea was born, and initial coding was completed in under two days.

Since its inception, the Five Second Test has grown to include an administrative area, nifty downloadable reports, two new variations on the original test, and, coming soon, Premium tests.

What It Does

The Five Second Test focuses on those crucial first few seconds of a visitor’s interaction with a web page. It’s during this time that visitors decide

  • am I in the right place?
  • can I trust this site / company?
  • does it look as though what I want will be here?

If a web page fails to pass this initial few seconds of scrutiny, the visitor is gone.

The premise of the test, then, is simple: Have test participants provide reactions to a design after viewing it for five seconds. What’s great about having this test online is that it greatly simplifies the test logistics. The timing is automatically taken care of. Test participants can be located anywhere with a web connection. Results are compiled into a neat, downloadable .csv file.

The tests currently include:

  • The original test – now called “Classic” – asks participants to list what they remember seeing.
  • The Compare test shows two design variations and asks participants which they prefer.
  • The Sentiment test asks participants to list their most and least favorite elements.

There’s also another new feature, the “do a random test” button that allows anyone coming to Five Second Test to become a test participant and, in the words above the button, “help out a designer”.

For each of the tests, you simply upload an image (two for the Compare test, of course), such as a screenshot of your web page or interface design. Five Second Test stores your image, provides you with a unique URL to send out to test participants, and handles all the display timing for you. There is more information about each test, as well as some good references for further study, available on the Five Second Test site.

Randomly Testing the Tests – Some Observations

I couldn’t help but do a number of Random Tests myself, and I looked at perhaps 20 different designs and interfaces. About five tests in, I had these epiphanies which the remainder of tests bore out:

  • For most web pages, all I remember is the company name and the main image.
  • Therefor, all the pages I saw essentially failed the five-second test.
  • Why did they fail? First, the web pages were all far, far too busy to truly capture and hold visitor attention.
  • Second, the web pages all tried to do far too many different things.

This small experiment reinforced for me the need to simplify, simplify, simplify. It should be the web designer’s mantra. If it’s not crucial, remove it. If it’s unclear, thrash it mercilessly until it is.

It’s Not the Violin, It’s the Violinist – Getting Truly Useful Results

So now it’s time for a curmudgeonly reminder: As with any tool, the Five Second Test will be most valuable in experienced hands. Talking about formal usability research and testing procedures may sound grumpy and obsessive amongst all the user experience enthusiasm, but there are reasons behind the “how” of running tests. Quite simply – the better your tests are, the cleaner and more useful your results will be.  Poorly-designed and poorly-run tests, on the other hand, produce undependable, misleading results that can make your interfaces worse, not better.

For example, here are just a couple of foundational practices to include in any usability test:

Get the Right Audience: The participants must be part of the intended or target audience. Otherwise the feedback you get is at best muddled, at worst biased and completely irrelevant. Yes, there are some issues that will be common across all types of users, but as a user experience designer you should already know and address most of these as part of your best practice. For more information on recruitment, see Jakob Nielsen’s post on recruiting participants and UIE’s “Recruiting without Fear” report.

Assign a Meaningful Task: What you tell the participant to do prior to the test will change their behavior. For example, a classic eye tracking study conducted by Alfred Yarbus showed differences in how participants look at a family photo depending on the task they were given. What does that mean for you? If you’re using the Five Second Test, the primary “Classic” task is to “Remember what you see.” While this can provide helpful insights into what elements are memorable, remember that it primes the participant with a specific intent, which in turn influences his or her behavior.

What Next?

It’s exciting to see how tools like the Five Second Test are making usability testing easier than ever to do. That’s a good thing – testing is crucial in today’s pressurized, competitive business environment. But it makes it more important than ever for designers and developers to learn how to get the most out of these tools, with a solid grounding in usability methodology.