Weekly user tests: Searching

During the Week 5 “explain the library website” test, one participant had a lot to say about Summon. He wasn’t happy about the change, although he did say that “it’s good, but it’s different and I’m an old man” (he was in his mid-20s at most). I wondered if he would prefer a newer style of search results page — the bento box (see this Musings about librarianship blog post for an overview with screen shots). I showed him a site with bento box results and asked him if he found this better than our Summon results. His reaction was immediate: “No, not in the slightest; this is more annoying… Maybe it’s just because it’s different, but I don’t like it.”

I found this interesting because a number of libraries have moved to bento box results (e.g. UCalgary, Simon Fraser, Duke, Stanford, Princeton, NCSU) and I find the concept quite compelling. Was this guy, who admittedly did not like change, an anomaly?

So for Week 6, not only did we look at how students were using our Summon search, but we asked them to do the same search on one or two of the bento box sites to see if bento worked better. Chocolate? Check. Off to the lobby!

Searching Summon

Overall, students had very little difficulty with searching Summon. Although in our “explain the library website” tests, students mentioned Boolean a lot, only one of our four participants used a Boolean search in Summon. Two used phrases of four or more words. All of them saw the filtering options on the left side and appreciated  that  they could narrow their searches in this way.

A few problems we noted:

• No clear ability to limit to eBooks
• Confusing to have Hold/Reserve button appear for Reserves items
• SFX link resolver interface very problematic (moving to 360Link should help)

Bento Box Results

Of the five students who saw the bento box results (4 this week, 1 the previous week), three disliked the whole concept instantly. Their reaction was immediate and visceral: “Oh, oh, this is confusing, I don’t like this.” “Uh, um, I don’t really understand what it’s telling me.” “Ohhhh… Not a big fan.”  I edited the audio of these reactions together and it’s a symphony of disgust.

Those who quite liked the initial results screen were then unimpressed when they clicked “more results” or the Articles heading and were brought to a discovery-style screen. They both said they would have preferred going to the discovery screen directly, one continuing “because it already has all the filters and everything here.” They did not see any value in separating out Articles from Books when Summon allowed them to do that with the facets on the left. And they weren’t interested in any of the other bento options beyond Articles and Books.

I was surprised that none of the students liked the bento box approach. I still think there are use cases where bento would be an improvement, but I don’t think it balances the negative experience for the majority of our users who really do just want articles and/or books.

One thing that makes me really happy about all of this was that we saved a lot of time and resources by testing the implementation of the concept at other libraries instead of starting to develop it ourselves. It’s certainly possible that our five random students were anomalies and that bento would be a big improvement for our users. But five out of five negative reactions is enough to let us decide to put our focus on other projects. We may well revisit bento again, but only after I hear cries of delight drown out the cries of dismay.

Advertisement