So, the thing about talking about this at Code4lib is that the Code4lib community is professionally pretty diverse. And when we're talking about bento box interfaces,

we have a distinct bias toward academic libraries.

I mean that *is* where the term originated and, as far as I know, that's mainly the context where it's used.

And—especially in academic libraries, we have sort of a collective history of implementing what we might call

“hacks,” especially when trying to facilitate a particular user experience on our websites. That's not to put anyone down, by the way, as we're all pretty much in this boat. We often have to get

different systems from different vendors to play nicely together.

Sometimes we just need to make things work together that weren't meant to work together.

Granted, things are better than they used to be. Open source software in libraries is thriving, and it's become more or less the norm for vendors to offer APIs or other methods of getting at our data.

And so we build better and more sophisticated hacks—like this lovely treadmill bike. Or bento box discovery interfaces.

But, you know, as I look at resource discovery solutions in libraries and outside of libraries,

So Bento box UI design is a workaround for this problem. It's mainly an attempt by libraries to deliver a single search that works better across a wider variety of user scenarios, given the realities of library systems and data. There are at least three things that it tries to do.

It gives more equal weight to resources from multiple systems that might otherwise drown each other out in a single results list. This helps bring more variety to a user's immediate attention.

It funnels people to the resources they need in the particular systems that natively store and provide services over those resources *after* they've searched.

And it attempts to accommodate different uses by separating results visually by resource type. People don't have to futz around with filters and facets that they tend not to use anyway.

From a strategic POV, bento box design also has benefits: mainly that libraries maintain more control over their primary search interface,

that these UIs actually solve them.

The truth is that the evidence isn't clear, especially when just looking at the user studies that are available.

On the one hand you have many studies showing pretty high user satisfaction with combined-results systems, primarily from undergraduates.

And on the other hand are studies suggesting that most users come to the library interface knowing at a very broad level what type of resource they need to get (like articles or books), and so results that blend them together are confusing.

First I want to mention a few things that are useful in interpreting the stats.

During my time at UNT, part of my job has been to

help guide improvements to our users' resource discovery experience, including:

a redesign of our library catalog,

Implementation of Summon, which ran as a beta in Spring 2012 and was released as a live service in the Fall

along with a website redesign. In the redesign, we featured

We ran Google Analytics on both systems to collect stats.

So by September 2014

we'd accumulated 3 years of data on our catalog and over 2 years of data on Summon. At that point we were already in the planning stages on a bento box design,

and I wanted to see what I could glean about how people used our existing discovery systems. So that's when I did my analysis.

My primary question was:

do our users actually seem to use articles differently than they use books and other catalog materials? Are those useful distinctions to make in our interfaces? And to answer that it seemed I also needed to prove two other things: one, that, in our environment,

people *are* mainly using Summon as an articles search and

they *are* mainly using the catalog to find books, media, etc. And two,

pull data from each of the two Google Analytics accounts—Summon and the catalog—using the GA API, for the period of

August 2011 to August 2014. I focused mainly on two different types of stats:

pageviews and search queries.

I used pageviews as an approximation of searches. I filtered by URL to remove anything that was not a hit on the first page of a search results screen, to ensure that I was comparing apples to apples in each system.

Then I used additional GA filters to look at different aspects of usage in each system, including

Aggregate patterns.

Distance usage.

Usage coming from the Find Box on the homepage compared to direct usage.

And usage of search options, including use of the options on the Find Box as well as usage of search options within each system.

With search queries I attempted to do some very rough query analysis.

I mainly looked at

the top 100 queries in each system—

top 100 queries overall and

top 100 entered via each Find Box tab. I also attempted

to normalize queries (where I did case and punctuation normalization and removed stop words), and then I also looked at the top 100

individual words,

top 100 bigrams,

and top 100 trigrams in each data set. I also looked at

a couple of overall characteristics of the data set that I won't have time to go into.

Now let's look at distance usage.

This is actually showing local usage, using two different metrics for “local.” It compares all searches (the red line) with searches coming from Denton, TX (where UNT is located, the blue line) and searches coming from the unt.edu domain (on-campus, the green line). We clearly see here that the catalog gets substantially more use from local users than Summon.

In the catalog, 78% of use comes from Denton, and 57% of use comes from on-campus,

while in Summon, 50% of use comes comes from Denton and only 25% of use comes from on-campus. Different patterns of use from different user groups.

Another angle: usage of these two systems by where users start their search sessions.

For the period of time after our website redesign, here's what we see. The red area in each graph shows the proportion of searches where the user's search session actually started from the Find Box on the homepage.

In Summon that proportion is about 75%,

while in the catalog, it's only 40%. We see that nearly as many catalog search sessions start

on the catalog home page, at 32%.

So, maybe there's a picture materializing now that people do use these two systems differently. To help us start characterizing this use,

I want to zero in on Summon.

I've mentioned that we primarily use Summon as a “find articles” search in our discovery environment. And that search actually limits to the “journal article” content type by default. But people can clear the default options by starting a new basic or advanced search, and of course they can change content type limits via facets, which of course people rarely do.

But we see some interesting things emerge if we dig down into the numbers.

Looking at Summon stats, we see that people change facets and filters only about 26% of the time (which falls in line with what other research has found, by the way).

About 8% of searches are unfaceted basic searches, and

About 6% of searches are unfaceted advanced searches.

That leaves about 59% of searches where the default filters set on the Find Box are not touched.

If we drill down into the 26% of searches where people *did* change facets, we see

changing Content Type accounts for only 9% of all changes.

And drilling down further into that category,

only about 9% remove the “limit to Journal Articles” filter,

and only about half a percent use the “exclude journal articles” filter.

What actually accounts for the most Content Type activity is

*adding* the “limit to Journal Articles” filter, at almost 22%. And if we look at *only* the activity where people added a content type limit (as opposed to removing a limit or adding or removing an exclusion), we see that jump up to

41%.

So here are the conclusions I draw from these numbers.

As we'd expect, people mostly don't mess with changing the defaults.

Even when people do change facets/filters, they rarely change content type.

Even when people do use facets/filters, they rarely explicitly remove the Journal Article limit.

People almost never actually exclude Journal Articles.

When people do change content type, adding the Journal Article content type limit back into their search is what we see most often. In context of all searches this is still a relatively small number, however you have to keep in mind that, since that limit is present on most searches by default, the opportunity to add the limit is a lot rarer than the opportunity to remove it or add other content type limits. And yet it's still by far the most added content type.

What about the catalog?

The best way to talk about this might be to look at our query analysis. The problem is that it's hard to show you the data in a presentation format, because it's mostly long lists of terms. But I'm going to try to show you enough to get the point across.

In the catalog,

looking at overall catalog searching, we see that the top queries tend to be searches related to video games, films, television, and books. We also see some searches in the catalog for particular resources—usually reference sources. This is just the top 20, but it's really similar to the rest of the top 100.

If we look just at searches conducted via the Find Box, we see similar sorts of things, but heavier on actual books (again, mostly reference books) than media. We also see a couple of topical searches pop up.

If I showed you bigrams and trigrams, we'd see more topics emerge. Music, history, and literature (in addition to the pop culture terms we've seen) become more prominent.

Let's compare that to what we see in Summon. In Summon,

we mostly see queries related to academic topics or current events. We also see quite a few searches for databases or other types of aggregate resources,

especially when we look just at the queries conducted via the Find Box, which otherwise are very similar to the overall queries.

If we looked at bigrams and trigrams in Summon, then we'd see more of the same, and a lot of social science, political science, technology, education, and library science related topics.

So—I know this just gives you a small taste, but the takeaway I think is that,

by and large, we are seeing different kinds of searches for different kinds of things in each system,

Okay, I'm about out of time, so let's wrap this up. When I proposed this presentation, I mentioned that the stats I wanted to show you

weren't exactly the “smoking gun” I was hoping for, and I think you see what I mean. In using this information to justify a bento box approach, I fully admit

that there may be a healthy dose of this going on here,

And of course there's always this. But I mentioned at the beginning of my presentation

that I think bento box UI design is interesting for more reasons than just what's pertinent to our immediate academic library context, and in fact I think the problems BB UI is ultimately trying to address are universal.

In the usage stats I've shown you, I don't think it's a stretch to say we see evidence

that people do search for and use different types of resources differently. And while we want to