Usability testing is a critical component of healthy and accurate documentation.
By conducting user scenarios with testers that are unfamiliar with the project, you can identify knowledge gaps that may not have appeared in internal reviews.
Below, you'll find guidelines and best practices for holding usability tests.
Briefly cover who's responsible for tests and what you want them to accomplish. These can be general goals, such as ensuring documentation is accurate and easy to follow.
Include all setup steps and links a tester might need to complete testing tasks. Link to your documentation and the project you're testing (such as an API reference). Ideally, include a testing sandbox where testers can play around and make mistakes. If there's additional content that a tester may need but isn't strictly necessary, you may want to create another section for resources.
Identify what you want to accomplish by holding usability tests. You want testers to achieve basic developer tasks by reading your documentation. Representatives from the related documentation and API teams (if applicable) should also observe the test and note any gaps.
These are the scenarios your users will test. Approach these from an external developer's perspective: how would I accomplish a task if I'd never seen this documentation before? Try not to lead the reader into seeing what you want them to see. For example:
You're on a new team working on integrated credit card APIs, but you don't know much about credit cards themselves. Can you find relevant information in these docs that explain what a credit card is and how it works?
Make your testing schedule available and tag your testers and administrators in Confluence. This keeps everyone on the same page and helps you organize feedback.
When hosting your usability test, encourage your tester to vocalize their thought process as much as possible. Capture as many of these observations in your notes as possible, so you can understand how an outsider to your project digests your documentation. Throughout multiple tests, you'll likely find recurring trends or observations that you can act on and improve your docs.
After your initial round of tests, review your test results and collect feedback with your documentation team and other clients. Refine your documentation and test cases as much as possible, then hold another round of tests. Hopefully, you'll find that your testers have less critical observations and more positive feedback this time. Be sure to thank your testers for participating!