-
Notifications
You must be signed in to change notification settings - Fork 18
Summary
During phase 2 we interviewed 11 users about their experience with the PRA. We found that users struggled with PRA terminology and desired training/guidance around navigating the PRA process. More specifically, users wanted help with calculating burden, understanding when the PRA applies to a request and to have access to examples of PRA packages who had successfully cleared the process.
The challenges users face around the PRA from 11 user interviews
After discovering that users struggled to navigate the PRA clearance process, we came up with some potential solutions. They included; creating general guidance for desk officers to use, redesigning the ROCIS interface, and training PRA users on time saving hacks. Phase 2 proposed solutions:
Problems we observed and some solutions we brainstormed
During an in person workshop with 18F and OIRA the team determined that novice PRA users at small agencies were in the most the pain around the process, since this group of users didn’t have support at their agencies in putting together PRA packages. We decided to develop and test a prototype that would aim to answer the most basic PRA questions; What is the PRA? Why is it important? Does it apply to my work? If the PRA applies, which clearance, do I need?
In order to gut check our assumption that answering basic questions around the PRA would solve the problem for users at small agencies, we tested the prototype with users to get initial reactions. We hoped to learn the following; Do users understand the content? Are users able to navigate the tool? Does the tool provide them the information they need for deciding whether or not to get a clearance and what kind? Does the tool change the user's perception of the PRA?
PRA Prototype User Testing Iteration 1
User Feedback from the Prototype from 5 novice (1 year or less PRA users)
We surveyed 23 OIRA Desk Officers to understand what types of questions they receive from PRA users. The survey found that the most common questions asked of OIRA desk officers were: Do I need clearance for a certain activity? How long will it take? What type of clearance do I need? Do I need to submit a change request or revision? Desk officers were also spending about 2 hours per week responding to basic PRA questions and 2 hours per week providing PRA training to agency employees.
What types of basic questions do agency employees ask about the PRA? Select all that apply:
| Question | % Officers who receive this question |
|---|---|
| Do I need clearance for a certain activity? | 86.96% |
| Where do I start? | 21.74% |
| How long will it take? | 69.57% |
| What type of clearance to I need? (fast track, generic, etc.) | 69.57% |
| Do I need to submit a change request or a revision? | 65.22% |
| How do I calculate burden? | 47.83% |
| Can I have an example? | 60.87% |
| Where can I find guidance from OMB? | 56.52% |
| Who can help me at my agency? | 30.43% |
| Do I need clearance for a certain activity? | 86.96% |
| Other questions | 17.39% |
| Activity | Time spent |
|---|---|
| Responding to basic PRA questions that could be avoided if agency employees had a better understanding of the PRA? | 1.7 hours per week |
| How often (trainings per year) do you provide training on basic PRA issues for agency staff? | 2.2 times per year |
We took common scenarios users encountered when working with the PRA and used them to establish what some likely goals would be for users. Then, we determined what success in accomplishing those goals would look like for users. Based on what users needed to succeed, we determined what content had to be included on the site.
Having agreed to what content would help users succeed in their PRA related tasks, OIRA desk officers, members of CAPRA (The PRA Community of Practice), and novice users participated in a card sort to organize the site’s content. Participants worked in groups of 2-3 and categorized what we had determined to be the content that should live on the site. They also added content and categories as they saw fit. Based on the results of the card sort, we organized the content into the following categories: About the PRA, Do I need clearance, PRA approval process, Clearance types, Estimating burden, Additional resources, Get Help, and Glossary.
We conducted research with users in order to get initial thoughts on the site’s content, navigation, contact page, process diagram, homepage, and the glossary. Based on the feedback from users we iterated on all areas of the site we tested. Here’s a detailed look at what we heard:
We conducted research to understand user’s perception of the site including, how the site feels, who users think the site is for and who users believe authored the site. We also used the task based scenarios we created in Phase 3 to see if the site would help users achieve their tasks. Users were able to use the site to get through the scenarios and trusted the site overall.