My team was asked to design and develop a more nuanced method for measuring employee sentiment towards one of Microsoft's products, using methods intended to surface and display progress towards key sentiment aspirations and detect key sentiment risk areas. Strong employee sentiment towards the product is a critical driver of advocacy, leading to increased revenue and a strong indicator of product impact on company effectiveness and efficiency.
Being a technology company, the team relied on a variety of data sources to assess how employees feel about their experience with this product. These data sources include analysis of periodic surveys, detailed in-product monitoring and instrumentation, periodic qualitative interviews, and executive feedback from business units. While investments in the product and underlying network showed measurable improvements in product performance, expected increases in employee perception have been difficult to detect.
As a member of the core design team of three, we started with qualitative interviews with the Microsoft team to surface key sentiment aspirations and risks. From there we generated a taxonomy of words through brainstorming the potential emotional responses to the product, factual observations about the product, and personal judgments of the product. We clustered the words, identified antonyms, and began to reduce the number of possible selections by evaluating the relative usefulness of the words in diagnosing specific issues in sentiment and their potential use decision making when it comes to product investments. We assigned an underlying score to each word to indicate the relative positive or negative strength as determined by the goals of the project team.
Each respondent is asked first to scan through the words and select the ones that reflect their experience with this product. Then, we ask each respondent to select up to 5 words that best characterize their experience with this product. We calculate a sentiment score for each respondent based on their selections.
I participated in all aspects of the design from information architecture, instrument design, data visualizations.
Note: All data listed here is purely illustrative.
NPS and Sentiment Index:
Net Promoter Score Composition:
Sentiment Index Composition:
Sentiment Aspiration / Risk Performance:
The instrument was administered using Survey Monkey (an online survey tool). Respondents reported that the survey was fast, easy to use, and judged that it allowed them to accurately reflect their true sentiments about the product. Qualitative assessments suggested that the data analysis we presented accurately reflected the tone and sentiment of respondents.
The scope of our work focused exclusively on developing a method for capturing, calculating, and reporting on sentiment regarding this product. We did not produce statistically valid results and instead tested the instrument with a small population to illustrate the visualizations and conclusions that could be drawn from a larger sample size.