Adventures in Information Architecture Part 1: Test what we have

For a while now, Web Committee has been discussing revamping the information architecture on our library website. There are some good reasons:

  • more than half our of visitors are arriving at the site through a web search and so only have the menu — not the home page — to orient them to what our site is and does
  • the current architecture does not have an obvious home for our growing scholarly communications content
  • the current architecture is rather weak on the connection with the library building, which is a problem because:
    • people are searching the site for content about the building
    • there are more visits to the building than visits to the website

However, we also know that changing the IA is hard. Our students have already told us that they don’t like it when the website changes, so we really want to make sure that any change is a positive one. But that takes time.

And we have a pressing need to do something soon. The Library will be opening a new Ottawa Resource Room in the fall that has related web content and we can’t decide where it fits. So: user testing! Maybe our users can see something we don’t in our current IA. (Spoiler: they can’t)

We did guerrilla-style testing with a tablet, asking people to show us how they would find:

  • information relating to Ottawa (we asked what program they were in to try to make it relevant; for example we asked the Child Studies major about finding information related to child welfare in specific Ottawa neighbourhoods)
  • information about the Ottawa Room
  • (for another issue) how they would get help with downloading ebooks

As an aside: We’re not so naive to think that students use the library website for all of their information needs. We made a point of asking them where on the library website they would go because we needed to put the information somewhere on the website. For the ebooks question, we also asked what they would really do if they had problems with ebooks. 6/8 people said they would ask someone at the library. Yup. They’d talk to real person. Anyway, back to IA…

We talked to 8 different students. For information relating to Ottawa, the majority would do a Summon search. Makes sense. For information about the Ottawa Room itself, the answers were all over the place and nothing was repeated more than twice. So our users weren’t any better than we were at finding a place in our current IA for this information. (Hey, it was worth a try!)

So… we either need to shove the Ottawa Room somewhere, anywhere, in the structure we have or we need to tweak the IA sooner rather than later. So on to Web Committee for discussion and (I hope!) decisions.

My CAIS poster about library space: background

I’m presenting a poster at the CAIS Conference on Wednesday, June 3 on some of the work I’ve been doing on student use of library space. I tried to limit the wordiness of the poster, so am including the background information here. I still have to check with the organizers about whether I can post a copy of the poster here after the conference, but if not I’ll give some high-level findings. [Update: Here’s the poster “Student use of library space: Where are they when, and why?“]

Our newly renovated library includes a great new space for undergraduate research (the Discovery Centre), which is administered outside of the library. Work is being done to evaluate the use of that space and I wanted to make sure that the rest of the space in the building was also being evaluated. Reading the literature about library space, I saw that many evaluations of space happen at a single point of time or, if they recur, a few times over a single day, or the same time of day over a single week. I knew that the use of our building changed throughout the day and throughout the term. I wanted to look at how the various spaces in our building were being used, and see how that use changed over time.

I started with pretty basic seat sweeps, using a floor plan and tracking where people were sitting and if any group work was being done. Happily, this was quickly taken over by the library’s Stacks staff as part of their regular routine. They did sweeps of all five floors of the library morning, afternoon, and evening, Monday to Friday from the beginning of November until the end of April. I analyzed that data, looking particularly at any trends that emerged around time of day or over the course of the term.

(The sweeps involved stupidly labour-intensive data collection, data entry, and data analysis. There is definitely a better way to do this. Libraries at NCSU and GVSU have some great models, but I knew that taking the time to investigate and adapt these to my own library would push the data collection even later in the school year, so I chose more difficult data gathering sooner rather than more efficient data gathering later. I find it easy to postpone projects when I know I can’t do them “the right way” but in this case I decided to Just Fucking Do It.)

At the same time, I was part of another project to evaluate how students were using the space in the Discovery Centre and the rest of the library. Part of that project included a questionnaire, and the poster includes selected results from that – mostly around group work. I wanted to include results from photo elicitation in the poster but wasn’t able to get enough participation this spring. I hope to get that part done this fall.

UXLibs conference: thoughts

My first post on UXLibs was bits taken from my conference notes. This is what shook out when I reread all my notes and reflected a bit.

Matthew Reidsma (who was somehow even more inspiring in person than online, and I’m not sure how that’s even possible) spoke in his keynote about Heidegger, including his concept of being-in-the-world, and the question “How does the world reveal itself to us through our encounters with it?” In my notes, I continued “How does the library reveal itself through our encounters with it?” and – more pertinent to my work – “How does the library website reveal itself through our encounters with it?” Matt went on to explain that by interacting with things, we are making meaning. So, by interacting with the library website, what meaning are we helping our students make?

This made me think of the great workshop I’d had with Andrew Asher on the first day. One of the many things we did was watch videos of students trying to find information. A second year student needed to find peer reviewed articles but clearly had no idea what this meant. A fourth year student came upon an article on her topic from the Wall Street Journal and thought it could be useful in her paper because it sounded like it was on her topic and came from a credible source (not seeming to realize that a credible source is not the same as a scholarly source).  I found it striking that neither of these students seemed to understand what scholarship looked like; what it meant for a thing to be a scholarly source.

So, taking those two points together, is there a way we can help students make meaning of scholarship through interacting with our website? And I don’t just mean, how can we help them understand how to find various scholarly materials (you find books in this way, you find journal articles in that way), but can we help them understand how to interact with a journal article in a scholarly context? Can we help them use that article to first create understanding and then create their own scholarly work?

This in turn circles back to Donna Lanclos’ keynote on the first day where she challenged us to move beyond helping our users with wayfinding, and engage with them in the act of creation. She challenged us to move beyond the model of the bodiless scholar whose chair is hard, who can’t leave the library to eat, and who has to endure horrible searching on crappy library websites to find what they need. The finding part doesn’t have to be so hard. The hard part should be thinking about what you’ve found and then making something new out of it.

So, to grab a phrase from Paul-Jervis Heath’s keynote, “how might we” design a library website that helps students make meaning out of the scholarship they are finding? How might we design a library website that helps students focus less on finding and more on thinking and creating?

Since reading Emma Coonan’s great piece in UKSG News, “The ‘F’ word,” about moving away from a focus on finding in the context of information literacy, I’ve been wondering how we could do this in the context of the library website. UXLibs has prodded me further, and – even better – given me some tools, techniques, and a giant mound of inspiration to get out and try to start working on it.

UXLibs conference: notes

I’ve been quiet of late, as we’ve not been doing any user testing this term; instead we’ve been taking a step back and thinking bigger about our website. But after attending the User Experience in Libraries conference (UXLibs) last week, I’m excited to move forward with user testing/research and thinking big.

St. Catharine's College, Cambridge, site of UXLibs
St. Catharine’s College, Cambridge, site of UXLibs

UXLibs was amazing amazing. Don’t believe me? Check out the #UXLibs Twitter stream during the week of the conference. I’m not going to try to capture the essence of the conference (see these posts by Ned Potter, and conference organizer Andy Priestner for that). Rather, I’m just pulling out particular bits from my notes that resonate most strongly with me. Many of these may not make sense out of context, but I’m happy to provide context if you ask.

From the keynote by Donna Lanclos:

  • What happens if we decenter staff expertise?
  • Find out what users understand not what they want
  • Not helping with wayfinding but engagement with creation
  • If an activity has intrinsic value, does it need to be assessed?
  • We want people to “revel in independent thought” (Revel!)
  • If we’re going to do ethnography, we have to be okay with feeling uncomfortable, and with feeling comfortable with ambiguity. We need institutional support for uncertainty.
  • A pedagogy of questions involves “a voracious not-knowing” (from @jessifer)
  • Do a small proof-of-concept project and use ethnography to see if it’s working

From a workshop with Andrew Asher:

[we explored a couple of ethnographic techniques: cognitive mapping (e.g. asking people to draw a map of the library from memory, or mapping out where they went when and what they did there), and respective process interviews (asking people to draw each step of a step-by-step process as you ask them about that process)]

  • The location of mapping exercises (i.e. in the library or away from it) doesn’t seem to influence the content of the maps created
  • Mapping can demonstrate where prime real estate is being used for low-impact things
  • Commuter campuses [and so probably commuter students] are very different from residential, when looking at mapping journals
  • Drawing can help with specificity but don’t get too hung up on the drawing

From the keynote by Paul-Jervis Heath:

  • People are fundamentally unable to tell you what will help them (they don’t know or don’t notice)
  • Should vs want creates an interesting tension -> how do you help people be the better version of themselves?
  • Books are sharks!
  • Rules of improv are good rules for ideation
  • I really have to read Gamestorming one of these days

From a workshop with Matt Borg and Matthew Reidsma:

[we were introduced to the wonderful world of grouping post-its with affinity mapping (by voice, pain points and then categories) and empathy mapping (by what people say, what they think, what they do, and what they feel)]

  • Maybe we should add “games” to our “search books, articles and more” Summon box
  • We need to have empathy with our colleagues, as well as with our users
  • Add the demographic, etc. metadata to post-its to make it easier to find patterns

From the keynote by Matthew Reidsma:

  • All those links on the website – people put them there
  • Interacting with things = making meaning
  • Usability is beyond functional, it’s making sure people have meaningful interactions with the world
  • It’s easy to recover from breakdowns [errors, confusion] when you understand how the thing you’re using/doing works
  • Usability could be helping people better understand our tools/services so they can better recover
  • Test to learn, not just perfect; learn how people cope

There was so so so much more than this. I have a follow-up post on some bigger picture stuff. But there’s so much more than that too. I’m going to be processing this conference for a while.

Web Rewriting Sprint

At the end of October, I was watching tweets coming out of a UX webinar and saw this:

"Idea: rewrite your web sprint"

I thought it sounded great, so ran it by Web Committee that same week and we scheduled a sprint for the end of term. Boom. I love it when an idea turns into a plan so quickly!

We agreed that we needed common guidelines for editing the pages. I planned to point to an existing writing guide, but decided to draft one using examples from our own site.

I put together a spreadsheet of all the pages linked directly from the home page or navigation menus, plus all the pages owned by admin or by me. Subject guides and course guides were left out. The committee decided to start with content owned by committee members, rather than asking permission to edit other staff members’ content. We prioritized the resulting list of 57 pages (well, 57 chunks of content – some of those were Drupal “books” with multiple pages).Spreadsheet categories: title, type, owner, linked from, notes, editing, checked

Seven of us got together on an early December afternoon (six in the room, one online from the East Coast). Armed with snacks, we spent 90 minutes editing and got through most of our top and mid-priority pages.

It was a very positive experience. We got a second set of eyes on content that may have only ever been looked at by one person. We were able to talk to each other to get feedback on clear and concise wording. And we saw pages that were already pretty good, which was a nice feeling too.

We’ve organized another sprint for reading week in February. We’re going to look at the top priority pages again, to see if we can make them even clearer and more concise.

 

Weekly user tests: Finding games

Our library has a pretty fantastic game collection of over 100 board games and almost 700 video games. But finding them? Well, it’s pretty easy if you know the exact title of what you want. But a lot of people just want to browse a list. And to get a list of all the video games you can borrow, you have two options:

  • Do a search for “computer games” in Summon and then go to Content Type facet on the left, click “more” and then limit by “Computer File”
  • Go to the library catalogue, select “Call & Other Numbers” and then under “Other Call Number” enter GVD if you want video games, but GVC if you want to see titles available through our Steam account. After that, you get a really useful results screen to browse:
    Search results GVD001, GVD002, GVD003, etc.

And if you want board games, the content type in Summon is “Realia.”

Sigh.

Obviously, this is ripe for improvement, but how best to improve? User testing!

We set up in the lobby (mostly – see postscript) and asked passing students if they had 2 minutes to answer a question and get a chocolate. We told them that we wanted to improve access to our game collection (alternating “video game” and “board game”) and so wanted to know what they would do to find out what games the library had. We had a laptop with the library website up, ready for them to use.

No one clicked anywhere on the page. No one mentioned the catalogue. They all would search Summon or Google or else ask someone in the library.

We asked them to tell us what search terms they would use, so now we can make sure that those Google searches and Summon searches will bring them to a page that will give them what they want. For Summon, that likely means using Best Bets, but everyone was consistent with the search terms they’d use, so Best Bets should work out okay.

Once we have all that ready, we can test again to see if this will work smoothly for our users. Or if we really do have to tell them about “computer file” and “realia.” [shudder]

Postscript

When we did testing last December, we set up in our Discovery Centre, a really cool and noisy space where students do a lot of collaborative work. We didn’t have to hustle too much to get participants; students would see our chocolate, come over to find out how to get some, do the test and that was that.

During our tests in the lobby this term, it’s been pretty much all hustle, and even after all these weeks I still don’t really like approaching people (I feel like the credit card lady at the airport that everyone tries to avoid). I kept thinking that we should head up to the Discovery Centre again for that gentler “display the chocolate and they will come” approach.

Well, we tried it today and got exactly one person in 20 minutes, despite lots of traffic. So we went back down to the lobby and got to the “we’re not learning anything new” mark in 15 minutes.

I’ll just have to learn to love the hustle.

Weekly user tests: Finding subject guides

This week we did a guerrilla-style test to see how (or if) people find our subject guides, particularly if they are not in our main listing. We asked “Pretend that someone has told you there is a really great subject guide on the library website about [subject]. What would you do to find it?” We cycled through three different subjects not listed on our main subject guide page: Canadian History, Ottawa, and Homelessness.

Some Context

Our subject guides use a template created in-house (not LibGuides) and we use Drupal Views and Taxonomy to create our lists. The main subject guide page has  an A-Z list, an autocomplete search box, a list of broad subjects (e.g. Arts and Social Sciences) and a list of narrower subjects (e.g. Sociology). The list of every subject guide is on another page. Subject specialists were not sure if users would find guides that didn’t correspond to the narrower subjects (e.g. Sociology of Sport).

Results

The 21 students we saw did all kinds of things to find subject guides. We purposely used the same vocabulary as what is on the site because it wasn’t supposed to be a test about the label “subject guide.” However, less than 30% clicked on the Subject Guides link; the majority used some sort of search.

Here you can see the places people went to on our home page most (highlighted in red), next frequently (in orange) and just once (yellow).subject_test

When people used our site search, they had little problem finding the guide (although a typo stymied one person). However, a lot of participants used our Summon search. I think there are a couple of reasons for this:

  • Students didn’t know what a subject guide was and so looked for guides the way they look for articles, books, etc.
  • Students think the Summon search box is for everything

Of the 6 students who did click on the Subject Guides link:

  • 2 used broad subjects (and neither was successful with this strategy)
  • 2 used narrow subjects (both were successful)
  • 1 used the A-Z list (with success)
  • 1 used the autocomplete search (with success)

One person thought that she couldn’t possibly find the Ottawa guide under “Subject Guides” because she thought those were only for courses. I found this very interesting because a number of our subject guides do not map directly to courses.

The poor performance of the broad subjects on the subject guide page is an issue and Web Committee will look at how we might address that. Making our site search more forgiving of typos is also going to move up the to-do list. But I think the biggest takeaway is that we really have to figure out how to get our guides indexed in Summon.