Redesigning our Subject Guides: Student-First and Staff-Friendly

I presented about our Web Committee’s redesign project at Access 2016 in Fredericton, NB on October 5, 2016. We started doing user research for the project in October 2015 and launched the new guides in June 2016 so it took a while, but I’m really proud of the process we followed. Below is a reasonable facsimile of what I said at Access. (UPDATE: here’s the video of the session)

Our existing subject guides were built in 2011 as a custom content type in Drupal and they were based on the tabbed approach of LibGuides. Unlike LibGuides, tab labels were hard-coded; you didn’t have to use all of them but you could only choose from this specific set of tabs. And requests for more tabs kept coming. It felt a bit arbitrary to say no to tab 16 after agreeing to tab 15.

desktop-unfriendly

We knew the guides weren’t very mobile-friendly but they really were no longer desktop-friendly either. So we decided we needed a redesign.

Rather than figure out how to shoe-horn this existing content into a new design, we decided we’d take a step back and do some user research to see what the user needs were for subject guides. We do user testing fairly regularly, but this ended up being the biggest user research project we’ve done.

  • Student user research:
    • We did some guerrilla-style user research in the library lobby with 11 students: we showed them our existing guide and a model used at another library and asked a couple of quick questions to give us a sense of what we needed to explore further
    • I did 10 in-depth interviews with undergraduate students and 7 in-depth interviews with grad students. There were some questions related to subject guides, but also general questions about their research process: how they got started, what they do when they get stuck. When I talked to the grad students, I asked if they were TAs and if they were, I asked some extra questions about their perspectives on their students’ research and needs around things like subject guides.
    • One of the big takeaways from the research with students is likely what you would expect: they want to be able to find what they need quickly. Below is all of the content from a single subject guide and the highlighted bits are what students are mostly looking for in a guide: databases, citation information, and contact information for a librarian or subject specialist. It’s a tiny amount in a sea of content.guide-overload

I assumed that staff made guides like this for students; they put all that information in, even though there’s no way students are going to read it all. That assumption comes with a bit of an obnoxious eye roll: staff clearly don’t understand users like I understand users or they wouldn’t create all this content.  Well, we did some user research with our staff, and turns out I didn’t really understand staff as a user group.

  • Staff user research
    • We did a survey of staff to get a sense of how they use guides, what’s important to them, target audience, pain points – all at a high level
    • Then we did focus groups to probe some of these things more deeply
    • Biggest takeaway from the research with staff is that guides are most important for their teaching and for helping their colleagues on the reference desk when students have questions. Students themselves are not the primary target audience. I found this surprising.

We analyzed all of the user research, looked at our web analytics and came up with a set of design criteria based on everything we’d learned. But we still had this issue that staff wanted all the things, preferably on one page and students wanted quick access to a small number of resources. We were definitely tempted to focus exclusively on students but about 14% of subject guide use comes from staff computers, so they’re a significant user group. We felt it was important to come up with a design that would also be useful for them. In Web Committee, we try to make things “intuitive for students and learn-able for staff.” Student-first but staff-friendly.

Since the guides seemed to have these two distinct user groups, we thought maybe we need two versions of subject guides. And that’s what we did; we made a quick guide primarily for students, and a detailed guide primarily for staff.

We created mockups of two kinds of guides based on our design criteria. Then we did user tests of the mockups with students, iterating the designs a few times as we saw things that didn’t work. We ended up testing with a total of 17 students.

Once we felt confident that the guides worked well for students, we presented the designs to staff and again met with them in small groups to discuss. Reaction was quite positive. We had included a lot of direct quotations from students in our presentation and staff seemed to appreciate that we’d based our design decisions on what students had told us. No design changes came out of our consultations with staff; they had a lot of questions about how they would fit their content into the design, but they didn’t have any issues with the design itself. So we built the new guide content types in Drupal and created documentation with how-tos and best practices based on our research. We opened the new guides for editing on June 13, which was great because it gave staff most of the summer to work on their new guides.

Quick Guide

quick-guide

The first of the two guides is the Quick Guide, aimed at students. I described it to staff as the guide that would help a student who has a paper due tomorrow and is starting after the reference desk has closed for the day.

  • Hard limit of 5 Key Resources
  • Can have fewer than 5, but you can’t have more.
  • One of the students we talked to said: “When you have less information you focus more on something that you want to find; when you have a lot of information you start to panic: “Which one should I do? This one? Oh wait.” And then you start to forget what you’re looking for.” She’s describing basic information overload, but it’s nice to hear it in a student’s own words.
  • Some students still found this overwhelming, so we put a 160-character limit on annotations.
  • We recommend that databases feature prominently on this list, based on what students told us and our web analytics: Databases are selected 3x more than any other resource in subject guides
  • We also recommend not linking to encyclopedias and dictionaries. Encyclopedias and Dictionaries were very prominent on the tabbed Subject Guides but they really aren’t big draws for students (student quotations from user research: “If someone was to give this to me, I’d be like, yeah, I see encyclopedias, I see dictionaries… I’m not really interested in doing any of these, or looking through this, uh, I’m outta here.”)
  • Related Subject Guides and General Research Help Guides
  • Link to Detailed Guide if people want more information on the same subject. THERE DOES NOT HAVE TO BE A DETAILED GUIDE.
  • Added benefit of the 2-version approach is that staff can use existing tabbed guides as the “Detailed Guides” until they are removed in Sept.2017. I think part of the reason we didn’t feel much pushback was that people didn’t have to redo all of their guides right away; there was this transition time.

Detailed Guide

detailed-guide

  • From a design point of view, the Detailed Guide is simpler than the Quick Guide. Accordions instead of tabs
    • Mobile-friendly
    • Students all saw all the accordions. Not all students saw the tabs (that’s a problem people have found in usability testing of LibGuides too)
  • Default of 5 accordions for the same reasons that Key Resources were limited to 5 – trying to avoid information overload – but because target audience is staff and not students, they can ask for additional accordions. We wanted there to be a small barrier to filling up the page, so here’s someone adding the 5th accordion, and once they add that 5th section the “Add another item” button is disabled and they have to ask us to create additional accordions. add-accordion
  • There’s now flexibility in both the labels and the content. Staff can put as much content as they want within the accordion – text, images, video, whatever – but we do ask them to be concise and keep in mind that students have limited time. I really like this student’s take and made sure to include this quotation in our presentation to staff as well as in our documentation:
    • When I come across something… I’ll skim through it and if I don’t see anything there that’s immediately helpful to me, it’s a waste of my time and I need to go do something else that is actually going to be helpful to me .

And speaking of time, thank you for yours.

Adventures in Information Architecture Part 1: Test what we have

For a while now, Web Committee has been discussing revamping the information architecture on our library website. There are some good reasons:

  • more than half our of visitors are arriving at the site through a web search and so only have the menu — not the home page — to orient them to what our site is and does
  • the current architecture does not have an obvious home for our growing scholarly communications content
  • the current architecture is rather weak on the connection with the library building, which is a problem because:
    • people are searching the site for content about the building
    • there are more visits to the building than visits to the website

However, we also know that changing the IA is hard. Our students have already told us that they don’t like it when the website changes, so we really want to make sure that any change is a positive one. But that takes time.

And we have a pressing need to do something soon. The Library will be opening a new Ottawa Resource Room in the fall that has related web content and we can’t decide where it fits. So: user testing! Maybe our users can see something we don’t in our current IA. (Spoiler: they can’t)

We did guerrilla-style testing with a tablet, asking people to show us how they would find:

  • information relating to Ottawa (we asked what program they were in to try to make it relevant; for example we asked the Child Studies major about finding information related to child welfare in specific Ottawa neighbourhoods)
  • information about the Ottawa Room
  • (for another issue) how they would get help with downloading ebooks

As an aside: We’re not so naive to think that students use the library website for all of their information needs. We made a point of asking them where on the library website they would go because we needed to put the information somewhere on the website. For the ebooks question, we also asked what they would really do if they had problems with ebooks. 6/8 people said they would ask someone at the library. Yup. They’d talk to real person. Anyway, back to IA…

We talked to 8 different students. For information relating to Ottawa, the majority would do a Summon search. Makes sense. For information about the Ottawa Room itself, the answers were all over the place and nothing was repeated more than twice. So our users weren’t any better than we were at finding a place in our current IA for this information. (Hey, it was worth a try!)

So… we either need to shove the Ottawa Room somewhere, anywhere, in the structure we have or we need to tweak the IA sooner rather than later. So on to Web Committee for discussion and (I hope!) decisions.

Weekly user tests: Finding games

Our library has a pretty fantastic game collection of over 100 board games and almost 700 video games. But finding them? Well, it’s pretty easy if you know the exact title of what you want. But a lot of people just want to browse a list. And to get a list of all the video games you can borrow, you have two options:

  • Do a search for “computer games” in Summon and then go to Content Type facet on the left, click “more” and then limit by “Computer File”
  • Go to the library catalogue, select “Call & Other Numbers” and then under “Other Call Number” enter GVD if you want video games, but GVC if you want to see titles available through our Steam account. After that, you get a really useful results screen to browse:
    Search results GVD001, GVD002, GVD003, etc.

And if you want board games, the content type in Summon is “Realia.”

Sigh.

Obviously, this is ripe for improvement, but how best to improve? User testing!

We set up in the lobby (mostly – see postscript) and asked passing students if they had 2 minutes to answer a question and get a chocolate. We told them that we wanted to improve access to our game collection (alternating “video game” and “board game”) and so wanted to know what they would do to find out what games the library had. We had a laptop with the library website up, ready for them to use.

No one clicked anywhere on the page. No one mentioned the catalogue. They all would search Summon or Google or else ask someone in the library.

We asked them to tell us what search terms they would use, so now we can make sure that those Google searches and Summon searches will bring them to a page that will give them what they want. For Summon, that likely means using Best Bets, but everyone was consistent with the search terms they’d use, so Best Bets should work out okay.

Once we have all that ready, we can test again to see if this will work smoothly for our users. Or if we really do have to tell them about “computer file” and “realia.” [shudder]

Postscript

When we did testing last December, we set up in our Discovery Centre, a really cool and noisy space where students do a lot of collaborative work. We didn’t have to hustle too much to get participants; students would see our chocolate, come over to find out how to get some, do the test and that was that.

During our tests in the lobby this term, it’s been pretty much all hustle, and even after all these weeks I still don’t really like approaching people (I feel like the credit card lady at the airport that everyone tries to avoid). I kept thinking that we should head up to the Discovery Centre again for that gentler “display the chocolate and they will come” approach.

Well, we tried it today and got exactly one person in 20 minutes, despite lots of traffic. So we went back down to the lobby and got to the “we’re not learning anything new” mark in 15 minutes.

I’ll just have to learn to love the hustle.

Weekly user tests: Finding subject guides

This week we did a guerrilla-style test to see how (or if) people find our subject guides, particularly if they are not in our main listing. We asked “Pretend that someone has told you there is a really great subject guide on the library website about [subject]. What would you do to find it?” We cycled through three different subjects not listed on our main subject guide page: Canadian History, Ottawa, and Homelessness.

Some Context

Our subject guides use a template created in-house (not LibGuides) and we use Drupal Views and Taxonomy to create our lists. The main subject guide page has  an A-Z list, an autocomplete search box, a list of broad subjects (e.g. Arts and Social Sciences) and a list of narrower subjects (e.g. Sociology). The list of every subject guide is on another page. Subject specialists were not sure if users would find guides that didn’t correspond to the narrower subjects (e.g. Sociology of Sport).

Results

The 21 students we saw did all kinds of things to find subject guides. We purposely used the same vocabulary as what is on the site because it wasn’t supposed to be a test about the label “subject guide.” However, less than 30% clicked on the Subject Guides link; the majority used some sort of search.

Here you can see the places people went to on our home page most (highlighted in red), next frequently (in orange) and just once (yellow).subject_test

When people used our site search, they had little problem finding the guide (although a typo stymied one person). However, a lot of participants used our Summon search. I think there are a couple of reasons for this:

  • Students didn’t know what a subject guide was and so looked for guides the way they look for articles, books, etc.
  • Students think the Summon search box is for everything

Of the 6 students who did click on the Subject Guides link:

  • 2 used broad subjects (and neither was successful with this strategy)
  • 2 used narrow subjects (both were successful)
  • 1 used the A-Z list (with success)
  • 1 used the autocomplete search (with success)

One person thought that she couldn’t possibly find the Ottawa guide under “Subject Guides” because she thought those were only for courses. I found this very interesting because a number of our subject guides do not map directly to courses.

The poor performance of the broad subjects on the subject guide page is an issue and Web Committee will look at how we might address that. Making our site search more forgiving of typos is also going to move up the to-do list. But I think the biggest takeaway is that we really have to figure out how to get our guides indexed in Summon.

Weekly user tests: Login revisited

Back in Week 3, we did a test to see if people knew to use our Login button to get into their Library account to renew books, see fines, etc. It was not a great success in terms of recognizing that “Login” got you where you needed to go.

The Fabulous Kevin Bowrin and I thought that maybe people would respond better to a button labeled “My Account,” since some of the comments we received during the test used the word “account.” I altered the button, took a screenshot and headed out to the quad with our first staff volunteer (yay Jane!).

We limited our question to one: “Imagine you’ve taken books out of the library and you want to renew them. Where would you go to do that?” We consistently heard “Services” and “Borrowing” (which is under “Services”) and one person said they’d go to the catalogue. No one mentioned “My Account.” Sigh.

But, every section they mentioned would (eventually) get them to where they needed to be. So do we have a problem? Web Committee has decided we don’t. Login does get used, our analytics tell us that. The button was always meant to be a shortcut, and if not everyone goes to the shortcut but manages to find what they need, then maybe “My Account” is a solution in search of a problem. And I hate those. So Login will stay.

For now.

Weekly user tests: Searching

During the Week 5 “explain the library website” test, one participant had a lot to say about Summon. He wasn’t happy about the change, although he did say that “it’s good, but it’s different and I’m an old man” (he was in his mid-20s at most). I wondered if he would prefer a newer style of search results page — the bento box (see this Musings about librarianship blog post for an overview with screen shots). I showed him a site with bento box results and asked him if he found this better than our Summon results. His reaction was immediate: “No, not in the slightest; this is more annoying… Maybe it’s just because it’s different, but I don’t like it.”

I found this interesting because a number of libraries have moved to bento box results (e.g. UCalgary, Simon Fraser, Duke, Stanford, Princeton, NCSU) and I find the concept quite compelling. Was this guy, who admittedly did not like change, an anomaly?

So for Week 6, not only did we look at how students were using our Summon search, but we asked them to do the same search on one or two of the bento box sites to see if bento worked better. Chocolate? Check. Off to the lobby!

Searching Summon

Overall, students had very little difficulty with searching Summon. Although in our “explain the library website” tests, students mentioned Boolean a lot, only one of our four participants used a Boolean search in Summon. Two used phrases of four or more words. All of them saw the filtering options on the left side and appreciated  that  they could narrow their searches in this way.

A few problems we noted:

  • No clear ability to limit to eBooks
  • Confusing to have Hold/Reserve button appear for Reserves items
  • SFX link resolver interface very problematic (moving to 360Link should help)

Bento Box Results

Of the five students who saw the bento box results (4 this week, 1 the previous week), three disliked the whole concept instantly. Their reaction was immediate and visceral: “Oh, oh, this is confusing, I don’t like this.” “Uh, um, I don’t really understand what it’s telling me.” “Ohhhh… Not a big fan.”  I edited the audio of these reactions together and it’s a symphony of disgust.

Those who quite liked the initial results screen were then unimpressed when they clicked “more results” or the Articles heading and were brought to a discovery-style screen. They both said they would have preferred going to the discovery screen directly, one continuing “because it already has all the filters and everything here.” They did not see any value in separating out Articles from Books when Summon allowed them to do that with the facets on the left. And they weren’t interested in any of the other bento options beyond Articles and Books.

I was surprised that none of the students liked the bento box approach. I still think there are use cases where bento would be an improvement, but I don’t think it balances the negative experience for the majority of our users who really do just want articles and/or books.

One thing that makes me really happy about all of this was that we saved a lot of time and resources by testing the implementation of the concept at other libraries instead of starting to develop it ourselves. It’s certainly possible that our five random students were anomalies and that bento would be a big improvement for our users. But five out of five negative reactions is enough to let us decide to put our focus on other projects. We may well revisit bento again, but only after I hear cries of delight drown out the cries of dismay.

Weekly user tests: Explain our website

It seems to me that many user tests (mostly usability tests) have students doing very specific tasks that they may never do in real life. Of course user testing is artificial, and usability testing generally needs to be task-based, but is it useful to evaluate our sites with people who don’t care about the task they’re performing?

I thought one way to elicit information about tasks that meant something to students would be to ask them to explain the site to a peer. This way, maybe we could get a sense of what sections of the site they saw as particularly useful and which were not even on the radar.

Armed with chocolate, we hit the lobby of the library to ask students for about 5 minutes of their time. We asked them to pretend I was a first year student and explain the library website to me — what was useful, what wasn’t, what I could safely ignore. The plan was then to ask what task they do most often on the library website and walk us through that task, and then to ask what task they found most confusing and walk us through that.

I’m going to pause here and say that I pretested our script and it worked really well in the pretest.

We did this test in Weeks 4 & 5, with 10 participants total, and what we found was that the students really only used the website to search. They searched the catalogue, Summon and databases, but that was about it. Some signed into Ares for their course reserves. A few used subject guides, but exclusively as a source for which databases to search. Did they mention any actual content that a library staff member wrote? Just once: booking a group study room.

So, the question about the task they do most often? Well, they had already answered it: they search. Most confusing task? When search doesn’t work.

There were a few interesting things that did come up (*list below). But I had been hoping to find some big pain points, areas of friction that we could attack. I guess it’s good news that students generally expressed satisfaction. But maybe we just asked the wrong questions. “Explain” is probably the wrong word. It probably worked well in the pretest because I pretested with a student employee. Who explains the website as part of his job. Ah, hindsight.

Now that I think about it more, I’m not surprised that search was all that came up. A huge percentage of use of the library website is search. Asking random students what task they perform on the site, is more than likely going to bring up search. So the next test: How well is search working? Could we do better?

 

*A few interesting things that did come up in these two tests:

  • Many students told us things they had obviously learned through instruction, but sometimes they got things a little wrong (use the catalogue for journal articles and Summon for books; in a Boolean search you put AND in brackets; you have to use Boolean to combine words in Summon; you have to use the Summon advanced search to limit by date; )
  • Some students find eBooks very inconvenient to use (“It’s like a scanned, older-than-pdf kind of thing”). A few prefer print books to eBooks.
  • Moving to MyCarletonOne login made login less confusing
  • Adding the action button to the group study room page made that page less confusing