We've known how to test TV and print for years, but when it comes to digital we just pretend like we don't know how or that it's too expensive. We do know how. We just choose not to. Why? (Hint: It's not because it's too expensive either.)
It would be really nice to test iterations of a website or application on a sample that was statistically significant. And you don't need pay a firm tens of thousands of dollars to do that kind of work. There are desktop applications like Silverback that do a lot of the work for you. All you've got to do is get a few people in front of a computer and test whatever it is you've created. You could find 30 people in your target demographic and pay them $100 each, for a total cost of $3,000, and gain real insight into how your information architecture and user interface work. More importantly, you'll find out what parts of your overall user experience are completely flawed.
The reason we don't do this kind of testing as a matter of practice in digital is because creatives don't want to change their designs. They don't like agile and iterative development – which is something that really doesn't exist for TV or print. If you spend a bucket of money on a TV spot and it turns out to be terrible, you fire the agency that made it. You can't rearrange all the elements in the shot. On the web, however, it
may be as easy as chaining a few lines in a CSS template.
At the very least, more web developers and agencies should be using things like
Five Second Test to get some idea around how people actually interact with your site, application or digital experience. I absolutely love the idea of a simple, five second test as a way to gain insight into what parts of a design are most memorable. Too often creatives run wild creating concepts where the only one's who "get it" end up being people who have spent the last six weeks engulfed by it. And it's easy to sell a product team on that kind of work, because they live and breathe it day in and day out. But when consumers actually have to use it… well, if it can't pass a five second test,
you're probably doing it wrong.