So Where's Your Site Search Engine, Fractal?
I'll let Jared explain why I think a site like FyrenIyce is especially ill-suited to those trendy, online search engines popping up everywhere. You cannot afford to miss information about this illness, and I really don't want something on the site that might be responsible for you missing something critical. Read on and hear what Jared has to say about his research into the efficacy of onsite search engines.
From: Jared M. Spool
Organization: User Interface Engineering
Subject: On-Site Searching Results
In a recent study, we sent users off on scavenger hunts of
information on various web sites. While all of the information
that they sought could be found just by using the links provided,
users would often use the search functionality provided by the
site. (We saw users use search on 75% of the sites in our study.)
This makes sense. Theoretically, if you know the keywords, you
should just be able to type them in and instantly find the page
you're looking for. However, in practice, it doesn't work that well.
Search engines didn't make finding information easier, they made
it harder. When users found information without a search engine,
they did 50% better than when they tried to use a search engine.
Overall, having a search engine on a site doesn't seem to make
the site more usable. Of the different sites we tested, the best
search engine helped users find the target information only 50%
of the time. Some were as low as 25%.
We think there are four major reasons for this:
1) Users don't know how to narrow searches
- Few users did anything other than simple keyword searches.
(Almost no one used booleans or other features of the search
engines.) They would often type in very broad terms, such as
"Videos" when looking for how a video about the Wild West to get a friend. This is the equivalent of walking up to a librarian
and just saying 'Travel' and expecting they'll instantly find the
book on Hawaii that you are interested in.
When users got back a result set that was too large to be
practical, they often tried completely different keywords instead
of adding to the set they had. They didn't seem to know any
strategies that would have allowed them to take the stuff already
found and narrow it down further (for instance, "regular tire rotation").
None of the sites that we tested provided any useful information
on how to narrow a search. In fact, most seemed to assume that
users knew how to search effectively and didn't provide any clues
2) Full-text Search engines are not indexed
- Users didn't seem to understand what a full-text search is. The
dynamics of full-text searches are different than looking something
up in an index, but users didn't seem to grasp this.
For instance, they were surprised when they typed in the word
'Tire' on the Car Talk site to find results that contained the word 'entire' or the phrase "I'm tired." Although the site did present the option to search for entire words or partial words, users didn't change the setting (the default was partial).
We also saw that users didn't understand that plurals and singular
words would produce different results and were surprised. Users
didn't know that typing errors would produce poor results and
couldn't tell that it was a typo, instead of a lack of content, that
got them the "nothing found" message. (For example, one user mistyped 'Videos' as 'Vidoes' and got zero hits -- and then
assumed that there weren't any videos on the site.)
Full-text searching produces a lot of irrelevant information for
users. For instance, one of the tasks we had for the Smithsonian
Magazine site was, "Your six-year-old son has to do a report on dinosaurs for school. You remember an excellent article in an old issue of the Smithsonian Magazine that you think would be a fine reference. Go find it."
For this task, users naturally typed in the keyword 'dinosaur.'
The first article returned was on the American steel industry --
one of the great American industrial dinosaurs. (Go try this
yourself, it's great!)
Indexing is a craft that takes a lot of skill. No professional
indexer with any self-respect would ever put an article about the
steel industry under the topic of dinosaurs. As sites get bigger,
this problem will only become more of an issue. Full-text
searches will get more noisy and irrelevant as more words are
introduced without any sense of what makes them important to
Successful searching is essentially an indexing task. To help
user search more effectively, this intelligence must be designed
into the site. We think that professional indexers and others who
have these skills will become more valuable in the years to come.
3) Multiple search areas are not clear
- Many of the sites provided the ability to search different types of content or different areas of the site. This capability differed from site to site. For instance, the Smithsonian Magazine site lets
users search either Feature stories, Columns, or Back Issues
'89-'94. Users didn't know which one to pick.
Car Talk has four search areas that do not overlap. The designers
have scattered the search screens in several different locations
(usually with the content it searches). Users not finding information in one area didn't know that they should search other areas.
Disney lets users narrow the areas to search only after they've done an initial search. There is no explanation to users as to how the results from the other areas
will be different from the search they just conducted.
4) Search engines are short cuts
- As a short cut, the search engine is intended to get users quickly to their content. But like other types of short cuts, users first need to know how to get there "the long way."
- The short cut requires that users understand how the search engine works, how the content has been segmented and indexed, and how content has been labeled (page titles). This is a lot to ask of users and most are not up to the task.
Our data shows that the site uses a search engine, there is a
strong correlation to users failing to find the right information
(r-squared > .7). It also shows that users will gravitate to a
search engine when the links are not clear. On sites where no search option was provided, users complained, but did 50% better than the best site that had a search engine. (This is more proof that users don't necessarily know what's good for them.) And with all the times that users failed using search engines, they never seemed to associate the failure with the search engine -- it never occured to them to try a different tack. Instead, they gave up.
At this point, based on this information, our recommendation is for
site designers to focus on making the long way -- the links of the
site -- work effectively for users.
Over the next few weeks, we'll be publishing other findings from
our most recent study of web usability in UIETips, which you can subscribe to by sending the word SUBSCRIBE in the body of a message to UIETips-Request@uie.com.
Jared M. Spool
User Interface Engineering
800 Turnpike Street, Suite 101
North Andover, MA 01845
fax: (978) 975-5353