AEAChicago2007 – “Search Analytics for Fun and Profit” by Lou Rosenfeld

Back to main AEAChicago2007 post

internal search logs are a missing tool
Jakob Nielson says 50% of users are search-dominated
Zipf curve - long tail distribution - for search results
in this case, try to optimize the short head
look for seasonal patterns
cluster types of queries to look for patterns
how to capture search queries: search logs, local database, commercial
	search solution
most frequent unique queries? do they retrieve quality results? which
	retrieve zero results?
click-through rates per frequent query? most frequently clicked result?
what are the referrer pages for frequent queries?
which queries retrieve popular documents?
then you generate specific questions
i.e. Netflix which most searched and clicked titles are least frequently
	added to the queue?
analytics won't tell you the answer to a problem, but they'll tell you
	the problem is there.
User Research
	type SKUs into catalog site that they looked up in the printed catalog
	BBC has reports for "people who searched on X also searched on..."
		using session data
	segment needs by security clearance, IP address, job function,
		account information
	or extrapolate segments directly from the data
	associate real queries with a persona - now you really know what
		they care about
Content Development
	start from failed queries - does content exist?
	are there titling, wording, metadata, or indexing problems
	"best bets" results defined manually
	identify points with no or way too many results where you could
		add help
	query syntax helps select search features to expose
	if people are using queries with boolean operators, make them
		more visible
	if get zero results, could show options to broaden search
	if get too many (200 or whatever), could show options to narrow
Interface Design:  search entry interface, search results
	consider what elements to include in search results - i.e. author
		name for books
	get more clickthroughs on result 10 than 6-9 on a page with
		10 results
	Financial Times saw people entering dates; so let them sort
		results by date
Retrieval Algorithm Modification
	Deloitte, Barnes & Noble, Vanguard show basic improvements
		(i.e. best bets) aren't enough
	needed to go into more complicated and expensive customizations
	add spell checking
	weight company names in metadata highly
Navigation Design
	if created "best bets" to show at top of query results, can also
		use to generate index
	Michigan State University builds A-Z index automatically based
		on frequent queries
	cuts across organizational sils
	from what pages are searches initiated? those pages are failing
		and people are stuck.
	what are the queries from those points?
Metadata Development
	classify queries as types of metadata, then mark documents with
		that information
	Netflix had movies, people, and genres
	get possible values for those categories - natural language, jargon,
		localization (lorry)
	most common queries are known-item - there's one correct answer
	long tail is often research queries, more open-ended
	do some sampling in long tail to check if it's very different from
		short head
organizational impact
	bad search results demonstrate what happens when content
		authors don't follow guidelines
	look at common queries and make sure good documents aren't
		falling in results
	Google Analytics and others make it easy to email reports - viral
		spread of information
	Financial Times looks for spikes in queries to find breaking stories
complements qualitative methods that can tell you *why* people do
need better tools for parsing logs, generating reports - thinks will get
	good this year
Hitwise and Comscore can help you benchmark against other sites,
	but are expensive
Google Trends may also be helpful
having a hard time writing book because can't get data from people
middle area of the tail may have fast-rising or slowly falling items
has free template for analyzing queries

About Jennifer Berk

I'm an analytics and data leader with a marketing and product mindset. I like online newspapers, science fiction and fantasy, and ugly fish.
This entry was posted in Conference, Information Architecture. Bookmark the permalink.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.