Eye tracking: Investigating snippet length, target position and task types

Posted on January 14, 2007
Filed Under The WorldWideWeb |

Just found this on the Blog Searchtank of Marina Garrison Research Manager & Project Manager at Enquiro Search Solutions Inc. Where she does a summary on the findings of the MSN study and points to a interesting question:
"should the URL be placed above the snippet right below the listing title?"

Microsoft released a new eye tracking study that looked at the effects of using various snippet lengths in organic search results. The study used three different snippet lengths (short, medium, long) for users performing two types of search: informational (find a specific piece of information) and navigational (find a specific site).

The study found that there was NO effect on people’s search strategies due to the length of the snippet shown. However, the study did uncover some other interesting behaviors.

Users looked at a minimum of 4 listings regardless of where the “best” listing ranked on the first page
Users looked at more listings as the “best” listing moved down the first page, indicating that the user knew that they had not found the information/site they should be looking for (regardless of snippet length)
The success rate of the task decreased dramatically as the “best” listing moved down the page (position 1 = 78%, position 2 = 83%, position 8 = 11%)
For informational search, participants were much less likely to view lower ranking listings than for navigational search
On average, users highly agreed with the statement “I expect the information I’m looking for to be in the top five results”
For informational tasks, as snippet length increased the number of listings reviewed decreased. For navigational tasks, the number of listings increased
For navigational tasks, users spent longer on tasks where the long snippets were shown. For informational tasks this actually decreased significantly
For informational tasks, accuracy was improved by providing longer snippets. For navigational tasks, the accuracy actually dropped as snippet length increased

In summary, users performed navigational tasks best when the snippets were short and on informational tasks when the snippets were long. This may be because for navigational tasks, the URL plays an important role in determining which listing is the “best”. By increasing the length of the snippet the URL becomes lost in the sea of text. The study poses the question of what would happen if the URL was placed above the snippet, below the listing title.

Heat map visualization of the number of fixations
across 3 users on a page of search results for an informational task with long contextual snippets

We presented a study using eye tracking techniques to
investigate user strategies for Web search. We looked at
how people respond to search results when the target is
systematically manipulated to be displayed at different
positions on two kinds of search tasks and found that users
seem to exhibit an implicit trust for the rank generated by
the search engine, particularly for informational tasks. In
addition, we looked at how varying the amount of
information in Web search results affected user
performance on the same tasks. We found that as we
increased the length of the query-dependent contextual
snippet in search results, performance improved for
informational queries, while it degraded for navigational
queries. Our eye tracking results suggest this difference in
performance was due to the fact that as the snippet length
increased, users paid more attention to the snippet and less
attention to the URL located at the bottom of the search
result.