Chris Nguyen: UX

Context

Copilot in Viva Engage had been released to a subset of users. The functionality we were looking at included:

Get ideas for a post
Draft post
Get feedback on post
Rewrite post

The purpose of this research project was to get some early qualitative feedback and identify how people are using it (or why they're not) as well as any issues.

Preparation

Developed research plan with feature team
Worked with another researcher to come up with research objectives
Spoke to data science colleague to ask what analytics were available and if he had observed any patterns he thought were interesting (for additional hypotheses)
Aligned with internal AI benchmarking team to do qualitatively what they would be doing quanitatively
DIY recruitment including standard tools (e.g. Respondent.io) as well as reaching out to our client-facing sales team/account managers and reaching out to our internal recruitment networks.

Execution

Conducted a round of semi-structured interviews with early adopters (and observers on the call)
There was a range of usage levels
Our original plan systematically went through each feature and rated them on multiple factors
In practice, this was not feasible within the time
I pivoted and eliminated redundancy by combining some sections, making the ratings less systematic and asking them in a more succinct/conversational way

Key Findings/Themes

Users were satisfied by time/effort saving on grunt work and specificity of content
Users/organizations are still transitioning which proved to be a barrier to Copilot use
Usability could be improved with more specific guidance and discoverability

Outcome

Identified use cases where it's more/less useful
Features did the job but there were some improvement areas identified in terms of change management and usability
Validated some of our in progress work
Presented findings to a number of different audiences including product team, UXR team, senior leadership and technical team
Informed design work to improve the onboarding experience, increasing discoverability and providing more guidance as to how and when to use Copilot
Further research (round 2 of this study as well as validation of the new design concepts, identifying ESL as an additional use case to dig into/recruit for)

"Your work is consistently an example of top quality user experience research. You bring valuable insight from users to our product team, and you present that insight in ways that spark empathy and understanding and are also very actionable (e.g. your summaries are incredibly valuable). Also you bring such positivity, clarity and proactiveness to the team which is so appreciated and enjoyed!"
Feedback from Product Owner

Viva Engage Copilot

Context

Role

Preparation

Execution

Key Findings/Themes

Outcome