DESIGNING *for humans

Top Posts of 2008: DfH Year in Review

With just a few weeks left in the year (and even less productive blogging time), I thought I would put together this brief "best of 2008" on Designing for Humans.  If you're new to the blog, this is a good way to catch-up on some of the highlights you may have missed over the past year:

• User Research Technology - Always a core theme on DfH, this year I took a hands-on look at applying many emerging tools to support design research.  This included the Livescribe Pulse Pen, Casio's High Speed Camera and the IDEA award winning Size China anthropometric data set.  Looking to the future, I unveiled the FieldCREW workstation concept (pictured), and discussed the state of user research technology at the Design Research Conference.  Still waiting on the FitBit tracking device (hopefully in the approaching weeks).
• Reading - I recommend the best book on user research, a forthcoming book on product design, design research-related articles in the Harvard Business Review, human factors in mechanical engineering and an international standard on product usability that's not worth the money. 
• Awards - I blogged on the unique experience of judging both the IDEA awards and the ID Magazine Annual Design Review in the same year.  Also, read about the IDEA award winners in the research category.
• Ergonomics- I put together the FoRCePS guidelines for observing ergonomic problems during contextual observation.

...okay, my link pen ran out.

Interactive Video Object Manipulation

Interactive Video Object Manipulation from Dan Goldman on Vimeo.

Back from Thanksgiving break and excited to share an interesting video tool demo.  Easier to see than explain, these not-yet-available functions allow what I would describe as direct video manipulation. 

For design research and human factors analysis, relevant capabilities include annotating objects and people and tracking motion.  These features are surprisingly similar to some of the user interface tools in the FieldCREW research workstation concept.

Thanks to the IXDA discussion board for the tip.

The Best Damn Book on User Research.

"Using prose to create a data array is like using the edge of a fork to cut meat.  It can be done, but not well, unless the meat is tender indeed (corresponding to very simple data arrays)."
- Edward F. McQuarrie

Colorful language is not what makes Edward McQuarrie's Customer Visits the best book I've read on the topic of user research.  In fact, the book is overwhelmingly direct, cutting like a knife through the ambiguity and vagueness that surrounds the corporate customer research process. 

As a hardcore human-centered researcher,I approached the book with a skeptical manner: its sub-title is Building a Better Market Focus, and I learned of it indirectly, through a brochure for a seminar that McQuarrie was giving.  But I was quickly impressed by the organization and specificity of the content.

The book is structured around the processes for planning, conducting and analysis for site visits.  While it's focused on researching business customers, such as medical equipment and software developers, the methods and advice can be applied to other cases (e.g. consumer research). 

McQuarrie's to-the-point style is supported by case studies and referenced sources, providing both pragmatic and conceptual guidance.  What I found most useful was that Customer Visits specifically addresses the key questions that many organizations face when planning and conducting user research, such as:

• What types of customer are appropriate for field research - "Customer visits are most applicable when there are some hundreds or thousands of customers in the market, the product is technically complex, its application is highly contextualized, and the underlying technology allows for differentiated product offerings."
• Defining the right level and wording of qualitative research objectives - "Note that objectives that rest on words such as 'identify', 'explore', 'describe' and 'generate' properly come early in the decision process....Specific verbs that do not match the capabilities of customer visits would include 'test', 'select', 'evaluate', 'rank order', 'measure', 'forecast' and 'track'."
• Number of participants to include in a sample and creating a sample frame  - "a sample of thirty customers could be expected to identify 90% of all the needs that might exist in the total population of customers...a sample of twelve might uncover 70 to 75 percent of needs."

Obviously I can't articulate the full context and insight around a a topic in a quote, but I hope this gives you a sense of the level of detail that McQuarrie delivers. 

Of course, the book is not perfect - much of the information on writing appropriate interview questions and conducting observations should be known to professionals, and McQuarrie does not get into any deep domain knowledge or tools for conducting more effective observations (e.g. observing ergonomic issues).  But the strength of the book is in what comes before and after the observations.  The section on analysis procedures is excellent, providing a clear analysis framework ("partition, cluster, connect, and array") and addressing how to handle quantitative expectations in a qualitative context.

Finally, McQuarrie updated the book this year and thoughtfully covers trends in corporate user research such as data visualization, research data management, and the explosion of the the term 'ethnography' -

"...there is no inherent opposition between customer visits and ethnographic approaches.  A customer visit program can be made as ethnographic as you like...there exists a variety of business and market situations that require a broad range of approaches to information gathering.  Let a thousand flowers bloom."

Perhaps, a little more colorful language than I had initially stated, but in the case of this book, the author has earned the right to use it.

Event: User Research Friday - San Francisco

A half-day conference on user research (registration required) that includes alcohol:

• Friday, November 7th, 1-5pm
• @Mighty Gallery & Bar, 119 Utah Street, San Francisco
• Theme: What does user research accomplish?
• Speakers: Indi Young, Dan Saffer, Steve Portigal, Nate Bolt & Maya Duiker 

Video - An Ethnography and Interviewing Primer

A couple of IIT graduate students (Gabriel Biller & Kristy Scovel) have put together an entertaining video primer on field interview techniques.  You'll appreciate it if you've ever done street intercept interviews.  The video runs about 30 minutes.  I think this pairs well with Sam Ladner's (non-video) design research primer. 

Getting People to Talk: An Ethnography & Interviewing Primer

"User Anthropologist" Article in NY Times

There's already a lot of buzz about the April 13th New York Times Magazine article that prominently features the role of field research to inform design.  Can the Cellphone Help End Global Poverty? follows Nokia researcher Jan Chipchase in Ghana.  While the article is great for introducing the benefits of corporate user research to a mass audience, it is also tactially valuable in describing some of the field research and design techniques employed by Nokia.

For example, even when apparently traveling light, Chipchase relies on high-end reliable equipment:  "Pretty much wherever he goes, he lugs a big-bodied digital Nikon camera with a couple of soup-can-size lenses so that he can take pictures of things that might be even remotely instructive back in Finland or at any of Nokia’s nine design studios around the world."

Nokia not only conducts field research for gathering information, but a form of user-driven field design:

"Nokia’s temporary design studio sat in a rented two-room concrete hut at the intersection of two busy dirt lanes, across from a woman selling chunks of watermelon and peeled lemons and next to a large water tank labeled “Church of God.” There was a sheet of fabric strung up in front, with neat painted lettering that read: “Your Dream Phone. Share it with the world.” It went on to describe how the community was invited to come share ideas and drawings for the ideal mobile phone. Prizes were offered. So far, 140 people had shown up to sketch their dream phone."

And on-the-fly feedback on product concepts: "Each time the group stopped to chat with someone, Burns pulled out several prototypes — or “physical sketches,” as he called them — for potential phones, handing them over one by one for examination."

Overall, the article validates that common user research techniques done in context, can provide revealing information to drive design.

Also, see Chipchase's blog, Future Perfect.

Qualitative vs. Quantitative Ethnography

Exploring Ethnography for Design Research in the February issue of MD&DI is more than a follow-up to the classic article Ethnographic Methods for New Product Development. 

According to author Stephen Wilcox (Chair of IDSA Human Factors section), ethnographic research is now common in medical device development.  The majority of this research is of course, qualitative, and primarily focused on identifying opportunities:

"Much so-called ethnographic research—perhaps most of it—is designed simply to generate ideas, that is, to stimulate creativity. Inevitably, when members of device-design teams go into the field and see directly how their devices and other devices are used, it generates insight and stimulates new ideas."

But there is another type of ethnographic research that is as much about the validity of findings as it is about generating ideas (Sidebar: simply put, in research, validity refers to the degree that you are actually measuring what you are intending to measure).  Typically validity is associated with quantitative measurement based methods such as performance testing.  But Wilcox suggests several ways to increase validity in ethnographic research including careful sample selection, quantitative measurement and objective data recording.  This more robust approach to ethnographic research comes with a price:

"conducting such research is difficult, time-consuming, and, frankly, expensive, in comparison with the idea-generation type of ethnographic research."

It's unlikely that most organizations will be able to accommodate all of the steps necessary to conduct highly valid ethnographic research - especially since many are just getting into the practice of doing any field research regularly.  But Wilcox's recommendations should really be taken as best practices for conducting any type of user research effort (whether validity is an explicit intention or not).  For example, making sure that the  "sample accurately reflects the population of interest" is a fundamental research planning step.  The deeper challenge when addressing validity is knowing what you know - for example determining whether your sample is truly representative.

The February MD&DI issue also contains an article on considerations for designing medical devices for home use, and another article on integrating human factors into the medical device development process.

Finally, for a less technical, down to basics overview of ethnography, see Design Meets Research, from GAIN, AIGA's journal of business and design.

AnthroDesign Online Discussion Group

One of the best, but perhaps lesser known online discussion groups amongst designers is AnthroDesign, as in Anthropology & Design.  The group, which was originally made up solely of anthropologists, has grown to include a range of interests from corporate, academic, non-profit and consulting design/research worlds:

"This group consists of individuals interested in anthropology and design. We are interested in the role of applied anthropology in the corporate, public sector, and medical contexts. Not all list members are anthropologists, but group members share the common interest of applying ethnographic techniques and social sciences theory to industrial, software, and other types of product design."

Membership is permission-based.

LiveScribe - Paradigm Shifting User Research Technology?

I don't believe in "mission statements", but if Designing for Humans had one, it might be "to realize the application of emerging technologies in support of design research".  While I discuss various technologies, many are specialized or several years away from general applicability.  But I recently learned of the LiveScribe smartpen, and I can honestly say this is a technology that can have a valuable, near term impact for virtually all researchers.

The device is an electronic pen set to launch in the first quarter of 2008.  It has several features, but the one that stands out is called Paper Replay.  This feature:

"allows total recall... by simply tapping on your notes. When used to take notes during a discussion or lecture, the smartpen records the conversation and digitizes the handwriting, automatically synchronizing the ink and audio. By later tapping the ink, the user can replay the conversation from the exact moment the note was written. Notes and audio can also be uploaded to a PC where they can be replayed, saved, searched or sent."

In other words, the pen records audio in synch with your writing ,and indexes the audio with your writing.  As a result, subsequently tapping on a particular written note will play back the audio segment corresponding to the time when the note was written.  Still not clear? - then watch the comic-book style demo.

So what does this mean for design research?  Well, note-taking is a challenging skill, and typically we rely on a combination of hastily written notes and audio (or audio-video)recordings to document research.  The LiveScribe brings these two approaches together in an integrated way, potentially reducing equipment and streamlining workflow.

While the LiveScribe was not designed for user research applications, consider how it might be applied:

• During user interviews, the researcher can reference what he or she writes or sketches directly back to the interviewees words for clarity and idea expansion.
• In usability testing the facilitator can reference recorded comments directly back to a discussion guide document to quickly drill-down to supporting quotes.
• For ethnographic observations informant conversations and environmental sounds can be unobtrusively recorded while taking notes. 

The potential paradigm shift is moving from using handwritten notes and recordings as separate, complimentary tools to truly integrated ones.  Well, perhaps not paradigm shifting, but damn convenient.

Reality Check: I should caveat that I have not used this product yet directly and am basing my assumptions on what I have read, but expect a full review as soon as it becomes available.   Also the pen requires special gridded paper, for tracking purposes.  I also wouldn't expect the audio quality recorded on the pen to be of high caliber, which is pretty important.

With that all said, I wouldn't be surprised if the LiveScribe (or a similar product), become a part of the user researcher's tool belt, along with the camcorder, notepad, and granola bar.

Eight Design Research Themes for 2008: Technologies and Methodologies

2007 has witnessed the continued maturity of user research practices in product design/development organizations. As this continues, we look to 2008 and key areas of growth and change in user research technologies and methodologies. What many of these themes have in common reflects a shift from how to conduct research, to how to manage all of the research findings and results – clearly a positive trend and a nice problem to have.   Stay tuned into 2008 as these themes are tracked in further detail.

Technologies

Even a casual reader of this web log will have observed the ever-growing options in data gathering technologies available for a variety of research applications. For 2008, the themes in technology are diverse – from high definition video to a new resource of anthropometric head measurements of the Chinese population. But the more compelling tools address needs in organizing and analyzing qualitative data:

1. High Definition (HD) Video- HD video cameras are rising in popularity while falling in price. Higher resolution video means larger file sizes and typically more time for video editing and file management. On the other hand, greater visual clarity can be extremely valuable for studying fine motor control tasks, small control/interface element usage and visually-rich environments. Surgical observation and consumer electronics usability are two applicable areas for HD video.

2. International Anthropometric References – Much of the reference anthropometric data used to guide designs is based on the body dimensions European and North American populations, limiting applicability and, ultimately fit, to a broader user population. The availability of three-dimensional scanning technology, while still time-consuming and expensive, is driving the inclusion of additional populations. Size China is a program to create the first-ever digital database of Chinese head and face shapes for helmets, sunglasses and surgical masks. Such resources will provide a richer starting point for guiding form and size in product designs, but of course are not a replacement for fit testing with real participants.

3. Qualitative Data Management Software – As research capabilities mature, organizations will deal with a new set of challenges around handling larger volumes of research data. Research teams will struggle with organizing, presenting and efficiently re-using findings across projects. With that “embarrassment of riches, there is a need for techniques and tools that support research data management. For example, QSR Internationals’s forthcoming NVivo 8 provides a structure for entering, tagging and querying various forms of multimedia, qualitative data across multiple projects. These types of tools will enable more effective collaboration amongst both localized and geographically distributed researchers, and can provide a centralized repository for observational data.
4. Qualitative Data Analysis Software – The value of well-conducted research is extremely limited if it is not easily organized for effective communication. It is especially challenging to organize, analyze and interpret qualitative data such as user interview transcripts and observational field notes. Following many years of adapting general purpose software and technology, we now have access to a variety of software and hardware tools to support planning, collection, analysis and sharing of research data. Several new technologies can support unstructured data analysis in various ways including searching speech via text and syntactically mapping information. For example, IBM’s Many Eyes application visualizes text in a tree-like branching structure to enable more efficient analysis and data mining.

 

Methodologies

Design research methods will continue to adapt for studying the wider range of user experiences, beyond the primary product. Frameworks and techniques for mapping out user touch points will assist research planning, which will become specialized to particular domains (e.g. medical vs. consumer). Threading across all of this is the need for guidelines for effective research communication and presentation:

5. Comparative Ethnography - While many organizations are using ethnographic observation to understand end-user perspectives and stimulate innovative thinking, such research is frequently focused on a limited set of tasks and users. But a growing trend is to use ethnographic methodology to identify differences between contexts. For example, in a study of automobile driving behavior, Bresslergroup’s research plan not only focused on the in-car driving experience, but studied related, non-driving activities. Observing how comparable tasks (e.g. planning a route, choosing music to enjoy) are conducted in disparate contexts (in this case, in car vs. in home) provides unique insights to inform creative solutions.

6. Service Design - Beyond the “total product lifecycle” approach, organizations will need to understand where they fit within the range of loosely tied user experiences beyond the product itself. For example, medical implant designers should expand user research beyond surgery to understand the touch points that potential patients, caregivers and healthcare providers utilize to make treatment decisions, prepare for surgery, and deal with recovery and beyond. The emerging discipline of service design provides a framework for understanding how multiple types of providers and users interact across the various products, interfaces and environments where interactions and decisions occur.

7. Domain-Specific Research Methods – Although research practices can vary among domains (e.g. medical, consumer, industrial), core methods remain consistent. But as research teams mature, there is a movement towards industry-specific user research and design techniques. For example, in appliance design, usability testing with high-fidelity simulations is frequently necessary to elicit reliable performance feedback from consumers. By contrast, healthcare professionals are typically more capable of responding to lower-fidelity prototypes, partly attributable to their professional problem-solving processes.

8. Presenting Design Research – Typically, product development organizations can effectively present and communicate their work and capabilities in design and engineering. But even when products are backed by quality user research, teams may struggle with effectively communicating its influence on product design. Similarly, organizations have difficulty evaluating the research capabilities of potential employees. The Industrial Designers Society of Americas (IDSA) is leading the way in developing guidelines for design research presentations, starting with the organization of design research portfolio workshop & review at the Northeast District conference this April in Philadelphia.

Bang & Olufsen: Great Designers vs. Great Process

This past week's Businessweek contains an interesting article about the design process at Bang & Olufsen.  Bang & Olufsen: Design Reigns Supreme describes what might be called the "great man" theory of design, where design vision takes precedence over engineering and business, and does not include design research. This "model is a throwback to an earlier time when CEOs worked closely with gifted designers to differentiate their products in the marketplace".

While B&O is clearly a leader in aesthetic design of consumer electronics, they have been failing at making the transition to the digital world (e.g. from CDs to MP3s).  And it's evident from reading the article that this is partly attributable to the lack of a research process:

"They don't, for example, do even the basic market research ethnography common among consumer-oriented companies. Sorensen says consumers often don't really know what they want. Instead, B&O designers intuit the products that will fly."

Read between the lines and it's apparent that there's a lack of understanding about design research.  Ethnography is not about asking consumers what they want, it's about identifying their unmet needs, a very creative process in itself.  Later in the article there's a likely example of what happens when you rely on designer intuition:

"Take the digital music player. Even the company's loyal cognoscenti prefer Apple's iPod, with its elegant design and easy interface, to the $460 BeoSound 2--conceived by none other than Lewis in 2002. BeoSound 2 has been, by all accounts, a dud. Lewis says his mistake was not appreciating how quickly digital memory would grow. He figured with 50 songs on a device, the amount the original memory card would hold, consumers wouldn't need a screen to navigate through their music."

Perhaps if Lewis and his team had better understand their user audiences - typically high-net worth music lovers - it would have been discovered that 50 songs was not going to meet their needs long terms and a different design approach would have been more successful.

Of course hindsight is 20/20 and B&O is shifting its focus to bringing in designers who understand the digital world. Unfortunately, it appears that they're still missing the point - they don't so much need new designers, as a new design process.

. Rob Tannen

2007 EPIC Conference Preview

A quick note: A partial program for EPIC, the " third international Ethnographic Praxis in Industry Conference, to be held October 3-6, 2007 in Keystone, Colorado, USA", is available online.  Unfortunately the site requires a lot of back-and-forth clicking to drill down to individual detail sessions.

An Ethnography Primer

For those new to ethnography, or looking for a simple way to describe it to others, the AIGA and Cheskin have put together an ethnography primer.  The visual appealing PDF has limited content, but focuses on the value, basic steps, applications and terminology of ethnographic research to support design.  Perhaps most useful is the brief descriptions of the respective activities of the ethnographer and the designer during each step of the research process.

Check it out, and while you're there, explore AIGA's redesigned site.

User Research at Microsoft

MIT's Technology Review (a HIGHLY recommend newsletter) has an interview with Rick Rashid, Director of Microsoft Research.  Rashid gives some thoughtful answers to some rather confrontational questions regarding the state of innovation at Microsoft, and singles out interface adaptation as a key feature for improving usability.  He also refers to Microsoft's SenseCam project, a wearable camera for capturing everyday life events, as a memory-aid or "black box". It has clear value to ethnographic research, although it's current design seems to be a bit too bulky and obtrusive.

Design Research Quarterly - First Issue

The Design Research Society has published its first quarterly newsletter.  Of particular interest is the cover article by Liz Sanders, titled "Design Research 2006".  The article discussed the current research space in industry and academics, with a focus on the differences between the user-centered and the participatory research/design models.

Download design_research_quarterly_1.1 .pdf

ZIBA's Ethnographic Design Research in China

Businessweek's Innovation quarterly recently featured an article describing the ethnographic methods used by ZIBA to understand the Chinese laptop market for Lenovo.  The process included various techniques including deep cultural immersion, interviews, profiling and method acting.

Portigal on Design Research

Interesting online interview between Luke Wroblewski and Steve Portigal on the pragmatics of conducting ethnography for online product research.  Specifically - does observing someone using a web site in their normal environment provide advantages over conducting research in a lab?

"there are lots of “hidden” subtleties within digital social systems that govern how people behave. There are contexts of when and where that alter behavior. As an example, during a home visit a buyer on eBay may tell you: “I leave positive feedback when I get an item in good condition.” Their actual behavior, however, differs."

This is a two part interview.

Part 1 - http://www.lukew.com/ff/entry.asp?368

Part 2 - http://www.portigal.com/blog/design-research-a-conversation-with-steve-portigal-pt-2

Focusing on Ethnographic Methods

After focusing primarily on anthropometric data and methods for the last 18 months, we are shifting to spotlight ethnographic methods.  This is to reach a broader spectrum of design researchers who work on a range of product and design issues.  We'll continue to cover anthropometric data and sources in an ongoing manner.

Applied ethnography may be defined as ethnographic field work done to bring the consumer's or customer's point of view to the design and development of a new product. Applied ethnography can also be used to improve existing products. - Liz Sanders, provides an excellent overview of ethnography in product design research that defines and differentiates it from other methods and provides some basic tools and guidelines.

To continue learning on your own, here is some excellent background reading/references on ethnography for design.   Otherwise, stay tuned tune to the IDSA Human Factors site for forthcoming recommended content, case studies and methods.