Usability Testing for Founders Online


Founders Online home page design

Founders Online (founders.archives.gov) is an initiative to provide free public access to a massive digital compilation of the historical documents of the Founders of the United States of America: letters and other documents written and received by George Washington, Benjamin Franklin, John Adams, Thomas Jefferson, Alexander Hamilton, and James Madison.

The Ivy Group was contracted to perform research on best practices for navigation, layout, functionality, and accessibility, as well as the site design and front-end programming.

Update: founders.archives.gov launched on June 12, 2013.

Founders Online Document page template

Founders Online About page template

Background

In 2010 The National Archives, through its National Historical Publications and Records Commission (NHPRC), entered into a cooperative agreement with The University of Virginia Press (UVaP) to create Founders Online. UVaP sought to make the America Founding Era (AFE) available to a wider audience by means of an intuitively navigable web interface.  AFE, sold in sets and by individual volumes, has been produced by UVaP as a close digital analog of the print editions for a scholarly audience.

Some of the challenges in creating this web interface included:

  • identifying the optimum organization of material
  • determining search parameters
  • grouping and differentiating document and data types
  • deciding on how site content overall can be viewed (as in a table of contents)
  • integrating discovery tools, prompts, and research assistance
  • decisions surrounding translations
  • ADA compliance
  • planning a site architecture that allows for future enhancements and additions
  • designing a templating system that presents content consistently and appropriately
  • transmuting the presentation of scholarly material into digital formats appealing to history enthusiasts, genealogists, elementary and high school students and teachers, and the general public
  • integrating search engine optimization best practices

Identifying Best Practices

To come to a shared understanding of site attributes facilitating or impeding access, The Ivy Group reviewed 10 sites with database search functionality whose purpose was similar to that proposed for the UVaP Founding Father's Collection website. This exercise and related research provided the means to determine best practices in searchable database interface design and the initial site architecture of the UVaP Founding Father's Collection website.

Interviews with Research Professionals

To ascertain intuitive pathways and assess user behavior against interface design, the team developed a series of tasks to be performed on five sites, including the UVaP Founders Online proof-of-concept site. We assigned the tasks in seven in-depth interviews with testers, ranging from history enthusiasts to librarians to academics.

Interviews with the General Public

Incorporating what was learned from the interviews with research professionals, site design and front-end programming were refined to the point where navigability of the Founding Fathers user interface could be tested with less experienced users. The Ivy Group solicited responses from a variety of ages and demographics--61 respondents in total.

Test questions were based on unresolved issues that emerged during internal UVaP and Ivy Group meetings. During the course of testing, the test administrator asked follow-up questions if more input was needed. If the respondent's answer indicated that they did not understand the projected function of the item being tested, the administrators often explained the function in order to not skew responses to later questions.

The first test session took place in the café at James Madison's Montpelier. Visitors were encouraged to test the site in return for a 10% discount in the gift shop. Most respondents were over 60 years of age and not completely comfortable using computers; however, they did express interest in viewing and using the final product. Visitors were often tested in groups of 2-3, and there was a significant degree of consensus within each group and from group to group around answers to the test questions.

The second session took place in Piedmont Virginia Community College's Bolick Student Center. Respondents were generally students in the 18-25-year-old age range, although there were also some older participants, including café employees. A $5 ("for 5 minutes of your time") incentive was provided.

Final Round Testing

The research professionals were also invited back to submit additional feedback on the revised interface. In addition, experienced Ivy Group web testers who had no prior experience with the project assessed the interface. No incentives were provided to these groups.

The summary report included an overview of responses to test sites and provided recommendations and general takeaways for Founders Online site layout and navigation as well as recommendations for maximizing visual impact.  The Ivy Group then proposed site architecture and navigational structure, along with associated design options.