Why do we do UAT?

User Acceptance Testing, or UAT, is a key stage in any project and is critical to a successful implementation. UAT is intended to achieve two main goals:

1. Confirm the workflow functions as designed
2. Confirm the design that has been implemented is fit for purpose for actual end-users who will be using the workflow following go-live

What is covered in UAT?

A successful testing cycle is largely dependent on having a clear test plan ahead of time. The test plan should include what will be tested, who will be responsible for testing, and when these tests will occur. On the topic of what will be tested, it is the client’s responsibility to prepare this plan but generally, there are two main approaches clients tend to use. At a minimum, it is recommended that a series of test scenarios are drafted that would outline high level use cases covering all functionality included in the design. Other clients may choose to draft formal test scripts that would be a more step-by-step set of instructions that testers will follow when participating in UAT. Which approach to use depends on a variety of factors including the complexity of the design, the variety of scenarios that are covered, and the availability of team members to prepare such testing materials.

Whether using the test scenario approach or formal test scripts, below are a few items to consider to ensure you have full test coverage of the functionality.

• Permissions –If your workflow includes role-based permission requirements you will need to make sure you plan to test the workflow by users in each distinct role. It is important to test that any roles/departments that have been configured are only able to access the appropriate data and perform the specific actions in TAP that are aligned with the original requirements.
• Notifications –When testing notifications associated with your workflows, here are a few points to review:

» Recipients – Are all necessary recipients included in my email?

» Content – Is all necessary data and instruction conveyed in subject/body of the email so the recipient knows how to action the request?

» Delivery – Were all my recipients able to receive the email?

Note: If your workflow will include external participants make sure you include some external parties in testing to verify that notifications can be received outside of your organization

• Approval Routing–Are all approvals associated with your workflow occurring correctly? Are they being routed to the correct parties? Are the approvals occurring in the proper order (or simultaneously if leveraging a parallel workflow)? Are any necessary escalations or exceptions being covered properly in what has been implemented?
• Integrations–Any other applications that will have a touchpoint with TAP will need to be tested as well to ensure that the correct functions are performed and performed at the correct timing. Be sure to include any necessary error handling scenarios that have been built into the design.
• Dashboard configurations–If you are creating any dashboards as part of your deployment make sure to review the following as part of your testing:

» All necessary information is made available in the dashboard (any required columns are included)

» The correct information is made available in the dashboard (all information is accurate to what was entered and is pulling from the correct location)

Please Note: A general point to remember when testing is that it is often necessary to test both a positive and a negative test case. In other words make sure everything happens when it should, but doesn’t happen when it shouldn’t. As a basic example, if you are testing whether a certain field is required you would need to both make sure the form can be submitted when the field is populated and can not be submitted when the field is not populated. Far too often testers focus on just the positive, or “happy path”, test cases.

Who is involved in UAT?

The participants of UAT often fall into three main groups. Care is needed to determine the appropriate testers to use as a representative for these groups. A good UAT participant needs to have a full understanding of the process followed by everybody they are representing to ensure they can give accurate feedback on whether what has been implemented is fit for purpose. Plan to include as many as necessary in order to have full coverage of this knowledge. In addition and perhaps most critically, a good UAT participant is one who has the time necessary to devote to testing. Empower your testers to set aside time to participate as this will be a burden that can otherwise take a backseat to tester’s day-to-day responsibilities.

• End Users –These will be the main UAT testers. These should be people who will eventually be your normal users of the workflow. Be sure to think through all aspects of the workflow functionality including approvers. It may be necessary to modify your workflows to use stand-ins for more senior approvers if they will not be able to participate in UAT directly.
• SMEs –These will be the people supporting the UAT testers and are often members of the main project team or participants of the design phase of the project. SMEs need to understand both how to use the TAP workflow and the users’ daily processes so they can help wed the two concepts as the testers have questions.
• External Users –These are the end-users who will be using the workflow but are not a member of your organization. It is important to use actual external users when possible rather than just using internal testers to perform that function. It will help to make sure that everything works correctly when accessing externally and can introduce useful feedback that may otherwise be overlooked.

Please Note: It is also recommended to designate an individual or group responsible for coordinating issues reported during UAT, especially if there is a large base of test users. Having one person responsible for logging issues on the client side can be extremely helpful to speed up issue resolution. In addition, it is necessary to prioritize issues that are raised and that can only be done by someone who has an overview of what all has been reported.

 

When do we conduct UAT?

UAT should be scheduled as the last major milestone prior to go-live. There are three checkpoints that must be reached prior to conducting UAT.

1. Design is Locked.

We want to make sure the functionality being tested is representative of the functionality that will be in place following go-live. If significant aspects of the design are still being updated then we are not yet ready for UAT.

2. Design is Tested.

UAT is not going to be successful if a previously untested workflow is released to a large group of testers to for use. It is important to include some basic testing before releasing to the wider UAT group to ensure that there are no show-stopping issues that completely disrupt their ability to test. That does not mean to say that every issue must be resolved before starting UAT; just the items that would be considered blockers from making progress. As such it is often best to start with a smaller testing group who can verify the functionality before opening to a wider UAT group for their feedback on the workflow process as a whole.

3. Participants are Trained.

Please note, unless otherwise stated in the SOWit is the client’s responsibility to complete the training of their users including those that will be participating in UAT. Often it is best to do a small training first with the UAT participants and then a training with the full user base closer to go-live. Waiting to train the main user base helps to ensure that any changes required coming out of UAT are in place so when the full user based is trained they see exactly what will be going into production. In addition, it reduces the gap between training and go-live during which users are susceptible to forgetting their training.

 

How do we manage a successful UAT? 

UAT can have a lot of moving parts so it is important for the client and Mitratech project teams to be aligned on how we will be managing that process.  Below are our recommendations for handling some of the logistics for UAT. 

• Issue Tracking – Mitratech can provide a shared Google doc that can be used to report issues that are found during UAT.  Doing so can help keep track of all open issues as well as facilitate communication regarding retesting issues once they have been addressed.  It also helps when reviewing priority so that we ensure the Mitratech team is focusing on the issues that may have the greatest impact to the end users. 

• Meeting Cadence – Status meetings are critical to discuss any questions or concerns coming out of UAT.  A weekly cadence will often suffice but may be needed more frequently if there are a high number of issues to review or if it is an especially complex workflow/large UAT group. 

• Timeline – Everybody must be aligned on the key dates associated with UAT.  Below is a list of the most critical with elaboration where appropriate. 

» Training Date(s) 

» UAT Start Date (or dates if doing a phase approach) 

» Testing Completion Date – A date with lead time before the close of UAT by which all testing must be completed.  We cannot have testing go right up until the final day as it would not leave any time to resolve issues that may be found.  At least 3 days is recommended but maybe longer depending on design workflow complexity. 

» Final Retest Date – Following the testing completion date a final round of issue resolution will occur.  All fixes must then be retested to ensure nothing goes into PROD without being tested first.  This is the date by which those retests will be completed.  It is especially critical to have testers ready to respond ASAP as fixes are made available between the Testing Completion Date and the Final Retest Date. 

» Sign-Off Date – A date on which sign off of UAT will be expected so that final arrangements for go-live can begin. 

UAT Exit Criteria – At the end of UAT, sign off will be expected to confirm that the requirements for the workflow have been satisfied and we are ready to go-live.  Before UAT begins it is good to discuss what will be required for sign off.  In particular, it is important to get aligned on issue resolution.  Mitratech’s goal will certainly be to resolve all issues reported in UAT prior to go-live, but that is not always possible for a variety of reasons.  It may be necessary to move into Production with some known issues.  As previously mentioned, it will be important to prioritize issues as they are raised.  Before UAT begins the teams should agree what defect priority levels must be resolved to sign off. 

3www.mitratech.comWhen do we conduct UAT?UAT should be scheduled as the last major milestone prior to go-live. There are three checkpoints that must be reached prior to conducting UAT.-Design is LockedoWe want to make sure the functionality being tested is representative of the functionality that will be in place following go-live. If significant aspects of the design are still being updated then we are not yet ready for UAT.-Design is TestedoUAT is not going to be successful if a previously untested workflow is released to a large group of testers to for use. It is important to include some basic testing before releasing to the wider UAT group to ensure that there are no show-stopping issues that completely disrupt their ability to test. That does not mean to say that every issue must be resolved before starting UAT; just the items that would be considered blockers from making progress. As such it is often best to start with a smaller testing group who can verify the functionality before opening to a wider UAT group for their feedback on the workflow process as a whole.-Participants are TrainedoPlease note, unless otherwise stated in the SOWit is the client’s responsibility to complete the training of their users including those that will be participating in UAT. Often it is best to do a small training first with the UAT participants and then a training with the full user base closer to go-live. Waiting to train the main user base helps to ensure that any changes required coming out of UAT are in place so when the full user based is trained they see exactly what will be going into production. In addition, it reduces the gap between training and go-live during which users are susceptible to forgetting their training.How do we manage a successful UAT?UAT can have a lot of moving parts so it is important for the client and Mitratech project teams to be aligned on how we will be managing thatprocess. Below are our recommendations for handling some of the logistics for UAT.-Issue Tracking–Mitratech can provide a shared Google doc that can be used to report issues that are found during UAT. Doing so can help keep track of all open issues aswell as facilitate communication regarding retesting issues once they have been addressed. It also helps when reviewing priority so that we ensure the Mitratech team is focusing on the issues that may have the greatest impact to the end users.-Meeting Cadence–Status meetings are critical to discuss any questions or concerns coming out of UAT. A weekly cadence will often suffice but may be needed more frequently if there are a high number of issues to review or if it is an especially complex workflow/large UAT group.-Timeline–Everybody must be aligned on the key dates associated with UAT. Below is a list of the most critical with elaboration where appropriate.oTraining Date(s)oUAT Start Date(or dates if doing a phase approach)oTesting Completion Date–A date with lead time before the close of UAT by which all

4www.mitratech.comtesting must be completed. We cannot have testing go right up until the final day as it would not leave any time to resolve issues that may be found. At least 3 days is recommended but maybe longer depending on design workflow complexity.oFinal Retest Date–Following the testing completion date a final round of issue resolution will occur. All fixes must then be retested to ensure nothing goes into PROD without being tested first. This is the date by which those retests will be completed. It is especially critical to have testers ready to respond ASAP as fixes are made available between the Testing Completion Date and the Final Retest Date.oSign-Off Date–A date on which sign off of UAT will be expected so that final arrangements for go-live can begin.-UAT Exit Criteria–At the end of UAT, sign off will be expected to confirm that the requirements for the workflow have been satisfied and we are ready to go-live. Before UAT begins it is good to discuss what will be required for sign off. In particular, it is important to get aligned on issue resolution. Mitratech’s goal will certainly be to resolve all issues reported in UAT prior to go-live, but that is not always possible for a variety of reasons. It may be necessary to move into Production with some known issues. As previously mentioned, it will be important to prioritize issues as they are raised. Before UAT begins the teams should agree what defect priority levels must be resolved to sign off.