USA Today usability study: Better than most news websites but could use some iterating

This Fall I conducted a usability study of USA Today’s new website (at the time it was still in beta). I developed a task list for participants to complete and observed them attempting the tasks, which were designed to test different parts and functionality of the USA Today website. I wrote down observations while users were completing the tasks and asked them a series of questions about their experiences once the tasks were completed. Surveys were also conducted before and after the study. Below is part of the report that I created, focusing on comments and recommendations:

Participants’ successes and struggles were observed and recorded, and users provided insight during post-test questionnaires and interviews.

Key observations:

Design comments: Users found the site to be clean, easy to navigate, airy and well laid out. One participant compared the design to CNN.com, a website she goes to frequently and said USA Today’s website was “well organized,” and added, “It doesn’t seem like the same content would be repurposed into multiple sections. It was clean. CNN is better than it used to be, but USA Today didn’t have the cluttered look that others have with ads. If the ads were there, it was subtle. The space was airy. They weren’t afraid of requiring a scroll. CNN is very much in the above-the-fold mindset. This felt like they didn’t fear people scrolling.”

Another user found USA Today’s new website to be much better than other news websites she uses on a regular basis. She was very impressed and said she may become a regular user of the website, “I might start looking at this; it’s pretty user friendly.”

This user compared USA Today’s new website to her most-used news website, Cleveland.com: “Unlike USA Today, there is no rhyme or reason of what [Cleveland.com] is putting on the top of the website. They have way too many ads. It’s horrible. It’s not setup so that you can easily find things. A lot of times I give up and just stop using it.”

Unclear wording and labeling: All of the participants were able to quickly find the most popular stories but none were quite sure which was supposed to be the most popular. Some thought it was the story in the top left, while others thought it was in the top right. The site isn’t clear as to what users should assume are the most popular of the most popular stories, and there isn’t a numerical ranking of top stories similar to what other sites have.

One subject said, “I don’t understand this page at all,” and then thought maybe the different sized blue orbs/moons were meant to symbolize which stories were the most popular. She eventually assumed the stories with the more colored in orbs/moons were the more popular ones (Users weren’t quite sure what this iconography was supposed to be. A ball, a moon, an orb?). Note: This unclear iconography has since been removed from the site.

Labeling issues carried over to other areas. One user struggled finding the homepage because USA Today doesn’t have a navigation linked called “home,” unlike a lot of websites. He couldn’t figure out how to get to the homepage without hitting the back button. USA Today does, however, have a logo that takes users back to the homepage, a common website paradigm.

USA Today also puts a X in the right-hand corner of story pages, allowing users to click on it to get back to the previous page. If a user clicked on a story from the homepage, clicking the X would take that user back to the homepage, but if she clicked on a story from the sports page, for instance, clicking on the X would take her back to the sports page.

None of the users noticed the X, and none made use of it. When I asked one user about the X on story pages, she said, “I don’t care about the X, because it’s just an option. Maybe it’s something that I’d realize later was there, but just because there is a feature there that I don’t use doesn’t mean it’s not worth having.”

The X on story pages may be one of those features that users need to be educated about. Once they learn how this feature works, they may begin to like it. All the users I studied, however, were not used to this concept on a website.

Search engine issues: Two users were not able to discern that the magnifying glass icon at the top of the website was meant to be clicked on to pop up a search box. They were looking around for a big search box that most websites offer. When they couldn’t find it, they were lost with how to search.

One participant said, “Almost every website I go to has a box, and that’s what I use.” That participant also found issues with the search engine. When he was trying to find out how to subscribe to the paper for a year, his first instinct was to use the search engine. The search engine did not index this information, and wasn’t helpful for find this information. For users accustomed to using search, the new USA Today website has a lot of information still to index.

The search icon in question. Users have to click it for a search box to appear.

The search icon in question. Users have to click it for a search box to appear. Users were used to see a search box, often with the word “search” in the middle of it.

The USA Today search engine does a good job of indexing stories, and is fairly straightforward, but it appears to mainly index news stories and not the other areas of the website.

After assuring two participants that there was a search engine on the site and that all tasks could be performed, they were able to discover the search engine. One said, “I just wasn’t looking for an icon; I was looking for a box that says search. I then realized that if I clicked on the icon that I could search.”

The youngest participant, however, did not have issues finding the search engine. She knew that the magnifying glass meant search and understood that the new USA Today website relies heavily on iconography. “I knew that the website was using symbols,” she said. “There were icons for all different things, and I realized that’s how you use the site.”

Iconography: This website makes heavy use of iconography. After users became adjusted to this fact, they were better able to make use of the website. One area where two of the participants struggled was setting their local weather.

Users can mouse over the weather and set it via a gear icon, similar to the icon Apple uses on iOS for preferences. Two of the participants were unaware that this icon meant settings or preferences and thought nothing of it. The younger participant was the only one who quickly understood what that icon meant.

USA Today does have a fallback, however, for setting the weather. By clicking on the weather page, users can set their weather on that page. This page, however, also gave users issues. There is no set location button, which confused users as to how to actually set their location.

This page is supposed to work by either the user entering in their city or zip code and then pressing enter. This wasn’t clear to users, and didn’t even work every time when users did try it. This page also begins to pop up suggested cities when users type in where they live, and if they click on the location from a drop-down menu that appears, they can set their weather. This wasn’t immediately clear to users and doesn’t always appear quickly, which confused users.

Two users requested a “set” button, and one said she wouldn’t have had a problem if there was just a button to click after she entered in her location.

Another participant said, “I don’t like the fact that it doesn’t have something to click on.” This participant also found the x button in the enter city box confusing. He clicked on it a few times assuming it was a way to set his location, when in reality it’s a button to clear the data he has entered. “Where the X button is is usually where you would click [a set button],” he said.

Every user was confused by cover view (now called Big Page), USA Today’s new feature that allows users to view the website with one image taking up the entire page with a link to a story at the bottom with a design similar to apps such as Flipboard. One user said of the cover view icon, “That’s a stupid icon. What is it?” and then said, “The weather is obvious. The movie thing is obvious.”

Another participant said, “I would say cover view is pointless.”

As mentioned above, this website relies solely on an icon to alert users to where to search. The younger user, however, understood USA Today’s iconography for search and preferences based on her experience with other websites and operating systems, suggesting that these icons may be better understood by younger audiences.

System strengths

  • Clean layout: Users were able to easily navigate the website and found news stories easy to read. Pages were not weighed down by access chrome and ads, which is an issue that plagues a lot of news websites. Pages also loaded quickly because of their lack of chrome.
  • Ads: Users found them unobtrusive and said they didn’t interfere with the ability to enjoy the news content. The ads also fit in with the visual design of the website.
  • Big icons at bottom of page: These at the bottom of the page help users find staff info, how to get the newspaper, where to find apps, contact info and more. Many news websites put small links at the bottom of the page for this info, but USA Today uses large icons with text that makes it very easy on users.

System weaknesses

  • Discoverability of iconography: It takes users awhile to adjust to the iconography of the website, especially the icons that don’t have text with them. Cover view in particular perplexed users.
  • Search engine: It doesn’t index all of site and lacks granularity and advanced search parameters, such as the ability to search by date, author, etc. The search engine is hard to find for users due to reliance on iconography over a traditional search box.
  • Lack of buttons to click:  Several areas of the site would be easier to use if the site had more buttons to click as a fallback for users. Users struggled to set the weather, in particular, and this section of the website seemed to suffer from glitches.

Suggested improvements

Search box:  After watching two of the participants struggle to find how to search USA Today’s website, it became clear that some people are more accustomed to seeing a search box — often with the word search in it — than they are of seeing a magnifying glass. Google.com, for instance, is a large search box with buttons for searching, and that’s the dominant way that people understand search.

This is what an average user thinks search looks like.

This is what an average user thinks search looks like.

USA Today’s current way of presenting search with just a magnifying glass is a succinct way of displaying a search box and visually appealing. The magnifying glass is also recognized as iconography representing searching or finding, but my participants all agreed that the dominant search metaphor was the search box. Without adding a search box, USA Today will need to educate users on what their icons mean

Consistency of buttons: Users are used to clicking buttons to set a preference, perform a search and various other tasks on computers. Searching on USA Today requires a button to be pressed, but setting the weather does not on one of the two areas it can be set. In one area for weather, users are explicitly given a “set” button to set their local weather. In another area, they are not given this button.

Setting the weather was a problem for two of the participants. Two of them did not recognize the gear icon as a way to change settings or preference. Since they missed this, they clicked over to the weather page itself. This page did not have a “set” button. This button would have prevented a lot of frustration for users.

Search improvements: One participant found the search engine lacking. It searches articles fine, but doesn’t search some other parts of the site. This participant was looking for how much it would cost to sign up for the paper for a year. Searching yielded nothing, even though people are used to this way of navigating the Web because of search engines.

There also is no way to do an “advanced search” or to use search parameters. These are common ways that people who are fluent with search engines find information. On a site with so much information, providing some ways to narrow down a search or search by date or author would make searching quicker and easier.

Clarifying design elements: Every user was unclear as to which stories were the most popular on the most popular story page. There was no numbers ranking the stories. There is a blue orb/ball/moon icon that appears to be more full on some stories. Does the fullness of the orb/ball/moon mean the popularity of the story? Many of the most popular stories had completely empty blue orbs/balls/moons. Does this mean that they’re not really popular?

Explain new, divergent features: Cover view is not a common metaphor found on other news websites. All users were confused by it. USA Today needs to find a way to tell users what it is and why they might like it. As it stands now, cover view is just a random icon that a user could press that radically changes how the website looks. USA Today could have a page to explain how to use the site. Even having text pop up as people mouse over icons would be helpful.

Conclusion

Users found USA Today’s new website to be better than most news websites, particularly with regards to design and usability. Users found the site easy to navigate, and all commented how the top navigation bar was easy to follow and that they were comfortable that they would be able to find all sections of the website via the top navigation, despite the navigation bar only having eight broad sections to click on, which is a lot less than many news websites present in their navigation. Users also liked the visual nature of the website, with its focus on graphic design and photos and video. Despite the sites focus on visuals, users did not find the site cluttered.

Users also liked how story pages were not bogged down with excess chrome — too many ads, social media icons all over the place, side-columns filled with lists of stories from the website, etc. — like many news websites. Users found story pages easy and enjoyable to read with ample white space and focus on the content.

A few quirks remain, however, and the heavy reliance on iconography over better known metaphors confuses some users. The lack of a traditional search box could prove to be a pretty big usability issue. Once users realize that the magnifying glass icon is how you bring up the search box, using the search engine is pretty straightforward, but most users learned what a search engine looks like from Google, and Google has a big search box.

Quirks and frustrations aside, users were generally happy with the site and two users indicted that they may begin to use USA Today’s website more after seeing the new website.