Website Refresh: First Round of Iterative Testing

As I mentioned in my last post, we’re doing a design refresh of our library website, with a goal to make it “beautiful.” As such, we’re not touching much of the organization. But of course we have to pay attention to not just how the information is categorized but also where it appears on the page. We learned that a few years back when we tried adding a “Spotlight” feature near our Library Hours (tl;dr: people stopped being able to see the Hours when other content shared the space). So we are firm believers that user testing and iterative design is vital in making sure we don’t make parts of our site invisible by moving elements around.

After the results of our user research earlier in the fall, we came up with a design drawn from the sites that our users liked most that also worked within our current site structure. The layout was essentially the same, with three major changes:

  • We pulled “Quick Links” out of the menu and put it in a box on the front page
  • Hours moved from a box on the side to a banner under the search box
  • Our Help and Chat button also moved to this banner

We wanted to do user testing to make sure that users could:

  • find today’s hours
  • get to the full set of hours
  • figure out how to access help or chat.

We also asked them if there was anything they hated about the draft design. Just to flag anything that could cause problems but that we weren’t specifically asking about.

Since we were doing this testing early in the process, we didn’t have a live site to show. Our Web Developer, the fabulous Kevin Bowrin, built the mockup in Drupal since he’s more comfortable in Drupal than in PhotoShop, but it wasn’t on a public server. So we used a printed screenshot for this round of testing.

The first version of the design had a grey banner and small text and it was clear after talking to a few users that visibility was a problem. We only talked to 4 people, but only 2 saw the Hours and they were really squinting to make it out. Finding when the library is open should be really really easy. We decided to increase the text size and remove the grey background.

Hours and chat button in grey
Version 1

This time, even fewer people saw the hours: 1 out 6. Since people didn’t see today’s hours, we couldn’t even get to the part where we tested whether they knew how to access the full set of hours. We decided to see if adding an “All Hours →” link would help; perhaps by echoing the convention of the “View More →” links in other parts of the page, it would be clearer that this section was part of the content.

Nope.

Hours and chat button in white banner below Search box
Version 3

Again, quite quickly we saw that this section remained invisible. Only 1 person in 5 saw it. One user noticed it later on and said that he’d thought that part of the website was just a heading so he ignored it. Clearly, something was making people’s eyes just skip over this part of the website. We needed another approach.

Kevin and I talked about a few options. We decided to try making the section more visible by having Library Hours, Help and Chat, and Quick Links all there. Kevin tweeted at me after I’d left for the day: “Just dropped the latest iteration on your desk. I kinda hate it, but we’ll see what the patrons have to say!” I had a look the next morning. I also hated it. No point in even testing that one!

Hours, chat button, and Quick Links all part of Search box area
A blurry photo of the hated, not-tested version 4

We decided to put Hours where the Quick Links box was, to see if that would be more visible. We moved chat down, trying to mimic the chat call-out button on the McMaster Library website. Quick Links were removed completely. We have some ideas, but they were never a vital part of the site so we can play with them later.

Success! Most of the people we talked to saw the Hours and almost all of them could get from there to the full set of hours. (I did this round of testing without a note-taker, thinking I could keep good enough track. “Good enough?” Yes. Actual numbers? No.) The downside was that most people didn’t notice the Help and Chat link (not pictured here). However, I think we’ll really need to test that when we can show the site on a screen that people can interact with. The “always visible” nature of that button is hard to replicate with a print-out. I feel like we’re in a good enough place that we can start building this as more than just a mock-up.

Oh, and no one we talked to hated anything about the design. A low bar perhaps, but I’m happy that we cleared it.

Hours beside Search box
Version 5

We did all of this in one week, over 4 afternoons. For version 3, Kevin just added text to the screenshot so we could get it in front of people faster. Quick iterating and testing is such a great process if you can make it work.

Next steps: menu interactions and interior pages.

Advertisements

Redesigning our Subject Guides: Student-First and Staff-Friendly

I presented about our Web Committee’s redesign project at Access 2016 in Fredericton, NB on October 5, 2016. We started doing user research for the project in October 2015 and launched the new guides in June 2016 so it took a while, but I’m really proud of the process we followed. Below is a reasonable facsimile of what I said at Access. (UPDATE: here’s the video of the session)

Our existing subject guides were built in 2011 as a custom content type in Drupal and they were based on the tabbed approach of LibGuides. Unlike LibGuides, tab labels were hard-coded; you didn’t have to use all of them but you could only choose from this specific set of tabs. And requests for more tabs kept coming. It felt a bit arbitrary to say no to tab 16 after agreeing to tab 15.

desktop-unfriendly

We knew the guides weren’t very mobile-friendly but they really were no longer desktop-friendly either. So we decided we needed a redesign.

Rather than figure out how to shoe-horn this existing content into a new design, we decided we’d take a step back and do some user research to see what the user needs were for subject guides. We do user testing fairly regularly, but this ended up being the biggest user research project we’ve done.

  • Student user research:
    • We did some guerrilla-style user research in the library lobby with 11 students: we showed them our existing guide and a model used at another library and asked a couple of quick questions to give us a sense of what we needed to explore further
    • I did 10 in-depth interviews with undergraduate students and 7 in-depth interviews with grad students. There were some questions related to subject guides, but also general questions about their research process: how they got started, what they do when they get stuck. When I talked to the grad students, I asked if they were TAs and if they were, I asked some extra questions about their perspectives on their students’ research and needs around things like subject guides.
    • One of the big takeaways from the research with students is likely what you would expect: they want to be able to find what they need quickly. Below is all of the content from a single subject guide and the highlighted bits are what students are mostly looking for in a guide: databases, citation information, and contact information for a librarian or subject specialist. It’s a tiny amount in a sea of content.guide-overload

I assumed that staff made guides like this for students; they put all that information in, even though there’s no way students are going to read it all. That assumption comes with a bit of an obnoxious eye roll: staff clearly don’t understand users like I understand users or they wouldn’t create all this content.  Well, we did some user research with our staff, and turns out I didn’t really understand staff as a user group.

  • Staff user research
    • We did a survey of staff to get a sense of how they use guides, what’s important to them, target audience, pain points – all at a high level
    • Then we did focus groups to probe some of these things more deeply
    • Biggest takeaway from the research with staff is that guides are most important for their teaching and for helping their colleagues on the reference desk when students have questions. Students themselves are not the primary target audience. I found this surprising.

We analyzed all of the user research, looked at our web analytics and came up with a set of design criteria based on everything we’d learned. But we still had this issue that staff wanted all the things, preferably on one page and students wanted quick access to a small number of resources. We were definitely tempted to focus exclusively on students but about 14% of subject guide use comes from staff computers, so they’re a significant user group. We felt it was important to come up with a design that would also be useful for them. In Web Committee, we try to make things “intuitive for students and learn-able for staff.” Student-first but staff-friendly.

Since the guides seemed to have these two distinct user groups, we thought maybe we need two versions of subject guides. And that’s what we did; we made a quick guide primarily for students, and a detailed guide primarily for staff.

We created mockups of two kinds of guides based on our design criteria. Then we did user tests of the mockups with students, iterating the designs a few times as we saw things that didn’t work. We ended up testing with a total of 17 students.

Once we felt confident that the guides worked well for students, we presented the designs to staff and again met with them in small groups to discuss. Reaction was quite positive. We had included a lot of direct quotations from students in our presentation and staff seemed to appreciate that we’d based our design decisions on what students had told us. No design changes came out of our consultations with staff; they had a lot of questions about how they would fit their content into the design, but they didn’t have any issues with the design itself. So we built the new guide content types in Drupal and created documentation with how-tos and best practices based on our research. We opened the new guides for editing on June 13, which was great because it gave staff most of the summer to work on their new guides.

Quick Guide

quick-guide

The first of the two guides is the Quick Guide, aimed at students. I described it to staff as the guide that would help a student who has a paper due tomorrow and is starting after the reference desk has closed for the day.

  • Hard limit of 5 Key Resources
  • Can have fewer than 5, but you can’t have more.
  • One of the students we talked to said: “When you have less information you focus more on something that you want to find; when you have a lot of information you start to panic: “Which one should I do? This one? Oh wait.” And then you start to forget what you’re looking for.” She’s describing basic information overload, but it’s nice to hear it in a student’s own words.
  • Some students still found this overwhelming, so we put a 160-character limit on annotations.
  • We recommend that databases feature prominently on this list, based on what students told us and our web analytics: Databases are selected 3x more than any other resource in subject guides
  • We also recommend not linking to encyclopedias and dictionaries. Encyclopedias and Dictionaries were very prominent on the tabbed Subject Guides but they really aren’t big draws for students (student quotations from user research: “If someone was to give this to me, I’d be like, yeah, I see encyclopedias, I see dictionaries… I’m not really interested in doing any of these, or looking through this, uh, I’m outta here.”)
  • Related Subject Guides and General Research Help Guides
  • Link to Detailed Guide if people want more information on the same subject. THERE DOES NOT HAVE TO BE A DETAILED GUIDE.
  • Added benefit of the 2-version approach is that staff can use existing tabbed guides as the “Detailed Guides” until they are removed in Sept.2017. I think part of the reason we didn’t feel much pushback was that people didn’t have to redo all of their guides right away; there was this transition time.

Detailed Guide

detailed-guide

  • From a design point of view, the Detailed Guide is simpler than the Quick Guide. Accordions instead of tabs
    • Mobile-friendly
    • Students all saw all the accordions. Not all students saw the tabs (that’s a problem people have found in usability testing of LibGuides too)
  • Default of 5 accordions for the same reasons that Key Resources were limited to 5 – trying to avoid information overload – but because target audience is staff and not students, they can ask for additional accordions. We wanted there to be a small barrier to filling up the page, so here’s someone adding the 5th accordion, and once they add that 5th section the “Add another item” button is disabled and they have to ask us to create additional accordions. add-accordion
  • There’s now flexibility in both the labels and the content. Staff can put as much content as they want within the accordion – text, images, video, whatever – but we do ask them to be concise and keep in mind that students have limited time. I really like this student’s take and made sure to include this quotation in our presentation to staff as well as in our documentation:
    • When I come across something… I’ll skim through it and if I don’t see anything there that’s immediately helpful to me, it’s a waste of my time and I need to go do something else that is actually going to be helpful to me .

And speaking of time, thank you for yours.

Adventures in Information Architecture Part 1: Test what we have

For a while now, Web Committee has been discussing revamping the information architecture on our library website. There are some good reasons:

  • more than half our of visitors are arriving at the site through a web search and so only have the menu — not the home page — to orient them to what our site is and does
  • the current architecture does not have an obvious home for our growing scholarly communications content
  • the current architecture is rather weak on the connection with the library building, which is a problem because:
    • people are searching the site for content about the building
    • there are more visits to the building than visits to the website

However, we also know that changing the IA is hard. Our students have already told us that they don’t like it when the website changes, so we really want to make sure that any change is a positive one. But that takes time.

And we have a pressing need to do something soon. The Library will be opening a new Ottawa Resource Room in the fall that has related web content and we can’t decide where it fits. So: user testing! Maybe our users can see something we don’t in our current IA. (Spoiler: they can’t)

We did guerrilla-style testing with a tablet, asking people to show us how they would find:

  • information relating to Ottawa (we asked what program they were in to try to make it relevant; for example we asked the Child Studies major about finding information related to child welfare in specific Ottawa neighbourhoods)
  • information about the Ottawa Room
  • (for another issue) how they would get help with downloading ebooks

As an aside: We’re not so naive to think that students use the library website for all of their information needs. We made a point of asking them where on the library website they would go because we needed to put the information somewhere on the website. For the ebooks question, we also asked what they would really do if they had problems with ebooks. 6/8 people said they would ask someone at the library. Yup. They’d talk to real person. Anyway, back to IA…

We talked to 8 different students. For information relating to Ottawa, the majority would do a Summon search. Makes sense. For information about the Ottawa Room itself, the answers were all over the place and nothing was repeated more than twice. So our users weren’t any better than we were at finding a place in our current IA for this information. (Hey, it was worth a try!)

So… we either need to shove the Ottawa Room somewhere, anywhere, in the structure we have or we need to tweak the IA sooner rather than later. So on to Web Committee for discussion and (I hope!) decisions.

Weekly user tests: Finding games

Our library has a pretty fantastic game collection of over 100 board games and almost 700 video games. But finding them? Well, it’s pretty easy if you know the exact title of what you want. But a lot of people just want to browse a list. And to get a list of all the video games you can borrow, you have two options:

  • Do a search for “computer games” in Summon and then go to Content Type facet on the left, click “more” and then limit by “Computer File”
  • Go to the library catalogue, select “Call & Other Numbers” and then under “Other Call Number” enter GVD if you want video games, but GVC if you want to see titles available through our Steam account. After that, you get a really useful results screen to browse:
    Search results GVD001, GVD002, GVD003, etc.

And if you want board games, the content type in Summon is “Realia.”

Sigh.

Obviously, this is ripe for improvement, but how best to improve? User testing!

We set up in the lobby (mostly – see postscript) and asked passing students if they had 2 minutes to answer a question and get a chocolate. We told them that we wanted to improve access to our game collection (alternating “video game” and “board game”) and so wanted to know what they would do to find out what games the library had. We had a laptop with the library website up, ready for them to use.

No one clicked anywhere on the page. No one mentioned the catalogue. They all would search Summon or Google or else ask someone in the library.

We asked them to tell us what search terms they would use, so now we can make sure that those Google searches and Summon searches will bring them to a page that will give them what they want. For Summon, that likely means using Best Bets, but everyone was consistent with the search terms they’d use, so Best Bets should work out okay.

Once we have all that ready, we can test again to see if this will work smoothly for our users. Or if we really do have to tell them about “computer file” and “realia.” [shudder]

Postscript

When we did testing last December, we set up in our Discovery Centre, a really cool and noisy space where students do a lot of collaborative work. We didn’t have to hustle too much to get participants; students would see our chocolate, come over to find out how to get some, do the test and that was that.

During our tests in the lobby this term, it’s been pretty much all hustle, and even after all these weeks I still don’t really like approaching people (I feel like the credit card lady at the airport that everyone tries to avoid). I kept thinking that we should head up to the Discovery Centre again for that gentler “display the chocolate and they will come” approach.

Well, we tried it today and got exactly one person in 20 minutes, despite lots of traffic. So we went back down to the lobby and got to the “we’re not learning anything new” mark in 15 minutes.

I’ll just have to learn to love the hustle.

Weekly user tests: Finding subject guides

This week we did a guerrilla-style test to see how (or if) people find our subject guides, particularly if they are not in our main listing. We asked “Pretend that someone has told you there is a really great subject guide on the library website about [subject]. What would you do to find it?” We cycled through three different subjects not listed on our main subject guide page: Canadian History, Ottawa, and Homelessness.

Some Context

Our subject guides use a template created in-house (not LibGuides) and we use Drupal Views and Taxonomy to create our lists. The main subject guide page has  an A-Z list, an autocomplete search box, a list of broad subjects (e.g. Arts and Social Sciences) and a list of narrower subjects (e.g. Sociology). The list of every subject guide is on another page. Subject specialists were not sure if users would find guides that didn’t correspond to the narrower subjects (e.g. Sociology of Sport).

Results

The 21 students we saw did all kinds of things to find subject guides. We purposely used the same vocabulary as what is on the site because it wasn’t supposed to be a test about the label “subject guide.” However, less than 30% clicked on the Subject Guides link; the majority used some sort of search.

Here you can see the places people went to on our home page most (highlighted in red), next frequently (in orange) and just once (yellow).subject_test

When people used our site search, they had little problem finding the guide (although a typo stymied one person). However, a lot of participants used our Summon search. I think there are a couple of reasons for this:

  • Students didn’t know what a subject guide was and so looked for guides the way they look for articles, books, etc.
  • Students think the Summon search box is for everything

Of the 6 students who did click on the Subject Guides link:

  • 2 used broad subjects (and neither was successful with this strategy)
  • 2 used narrow subjects (both were successful)
  • 1 used the A-Z list (with success)
  • 1 used the autocomplete search (with success)

One person thought that she couldn’t possibly find the Ottawa guide under “Subject Guides” because she thought those were only for courses. I found this very interesting because a number of our subject guides do not map directly to courses.

The poor performance of the broad subjects on the subject guide page is an issue and Web Committee will look at how we might address that. Making our site search more forgiving of typos is also going to move up the to-do list. But I think the biggest takeaway is that we really have to figure out how to get our guides indexed in Summon.

Weekly user tests: Login revisited

Back in Week 3, we did a test to see if people knew to use our Login button to get into their Library account to renew books, see fines, etc. It was not a great success in terms of recognizing that “Login” got you where you needed to go.

The Fabulous Kevin Bowrin and I thought that maybe people would respond better to a button labeled “My Account,” since some of the comments we received during the test used the word “account.” I altered the button, took a screenshot and headed out to the quad with our first staff volunteer (yay Jane!).

We limited our question to one: “Imagine you’ve taken books out of the library and you want to renew them. Where would you go to do that?” We consistently heard “Services” and “Borrowing” (which is under “Services”) and one person said they’d go to the catalogue. No one mentioned “My Account.” Sigh.

But, every section they mentioned would (eventually) get them to where they needed to be. So do we have a problem? Web Committee has decided we don’t. Login does get used, our analytics tell us that. The button was always meant to be a shortcut, and if not everyone goes to the shortcut but manages to find what they need, then maybe “My Account” is a solution in search of a problem. And I hate those. So Login will stay.

For now.

Weekly user tests: Searching

During the Week 5 “explain the library website” test, one participant had a lot to say about Summon. He wasn’t happy about the change, although he did say that “it’s good, but it’s different and I’m an old man” (he was in his mid-20s at most). I wondered if he would prefer a newer style of search results page — the bento box (see this Musings about librarianship blog post for an overview with screen shots). I showed him a site with bento box results and asked him if he found this better than our Summon results. His reaction was immediate: “No, not in the slightest; this is more annoying… Maybe it’s just because it’s different, but I don’t like it.”

I found this interesting because a number of libraries have moved to bento box results (e.g. UCalgary, Simon Fraser, Duke, Stanford, Princeton, NCSU) and I find the concept quite compelling. Was this guy, who admittedly did not like change, an anomaly?

So for Week 6, not only did we look at how students were using our Summon search, but we asked them to do the same search on one or two of the bento box sites to see if bento worked better. Chocolate? Check. Off to the lobby!

Searching Summon

Overall, students had very little difficulty with searching Summon. Although in our “explain the library website” tests, students mentioned Boolean a lot, only one of our four participants used a Boolean search in Summon. Two used phrases of four or more words. All of them saw the filtering options on the left side and appreciated  that  they could narrow their searches in this way.

A few problems we noted:

  • No clear ability to limit to eBooks
  • Confusing to have Hold/Reserve button appear for Reserves items
  • SFX link resolver interface very problematic (moving to 360Link should help)

Bento Box Results

Of the five students who saw the bento box results (4 this week, 1 the previous week), three disliked the whole concept instantly. Their reaction was immediate and visceral: “Oh, oh, this is confusing, I don’t like this.” “Uh, um, I don’t really understand what it’s telling me.” “Ohhhh… Not a big fan.”  I edited the audio of these reactions together and it’s a symphony of disgust.

Those who quite liked the initial results screen were then unimpressed when they clicked “more results” or the Articles heading and were brought to a discovery-style screen. They both said they would have preferred going to the discovery screen directly, one continuing “because it already has all the filters and everything here.” They did not see any value in separating out Articles from Books when Summon allowed them to do that with the facets on the left. And they weren’t interested in any of the other bento options beyond Articles and Books.

I was surprised that none of the students liked the bento box approach. I still think there are use cases where bento would be an improvement, but I don’t think it balances the negative experience for the majority of our users who really do just want articles and/or books.

One thing that makes me really happy about all of this was that we saved a lot of time and resources by testing the implementation of the concept at other libraries instead of starting to develop it ourselves. It’s certainly possible that our five random students were anomalies and that bento would be a big improvement for our users. But five out of five negative reactions is enough to let us decide to put our focus on other projects. We may well revisit bento again, but only after I hear cries of delight drown out the cries of dismay.