ATS Automation Survey
Research Objective
Validate the user pain points we identified during our ATS Automation Opportunity discovery study. After analyzing the survey results, we were able to validate the productivity and monetary impacts of the top pains identified during the discovery initiative across the business.
Team
-
Senior User Researcher
-
User Researcher (me)
-
Director, Product Management
Tools
-
Qualtrics
-
Miro
-
Google Docs
Context
This survey was the direct result of a discovery initiative we conducted to understand where automation could solve pains for our field Colleagues. Specifically our stakeholders wanted to understand what are the most manual and tedious tasks that Colleagues are spending time on that also get in the way of their primary job - recruit candidates to fill open jobs.
During that initiative we were able to conduct twenty 1:1 interviews with field Colleagues and identified nine areas of opportunity where automation could make an impact on productivity. However, with over 400 branches and hundreds of field Colleagues, the business wanted more data to help prioritize where these opportunities fit in the product roadmap.
The problem
The discovery interviews conducted during the qualitative half of this study resulted in nine opportunities for automation to solve pain points in colleague workflows. The findings provided items for a roadmap of future product improvements.
Q.
How do we identify which automation opportunities to prioritize on the roadmap?
A.
Validate discovery initiative findings with data (a.k.a. survey time!)
Designing the Survey
Most Crucial Question(s)
-
How much time does documenting interactions actually take?
-
What communications take up the most time in a colleague’s day to respond to and/or troubleshoot?
-
How much time does cleaning candidate funnel data take and what data points do they use most often?
Screenshots (left to right)
Opportunities resulting from qualitative half of study, stakeholder prioritization workshop, survey question brainstorm and alignment
Building the Survey
Organization
Five sections
-
Intro
-
Documenting interactions
-
Managing communications
-
Cleaning candidate funnels
-
Closing
Logic
We then applied logic to the sections and even the questions within each section so that users were only served the questions that were most applicable to them based on their self-reported title and job responsibilities.
Why?
We wanted to prevent
-
total survey error
-
survey fatigue
-
data inaccuracy
Total surveys sent
1142
Survey open
2 weeks
Finished responses
209
Response rate
18.3%
Margin of Error
6%
Confidence interval
95%
Sending the Survey
Turning Data Into Insights
Uncovering ways to save
After analyzing the survey results, the team was able to breakdown how much the business was spending on in operational costs on the three tasks we explored further in this survey. Based on participant reported data here's approximately what each of these opportunity areas was costing the company.
-
Documenting interactions
-
$489,325 per week
-
$25,444,900 per year
-
-
Managing communications
-
$491,280 per week
-
$25,546,560 per year
-
-
Cleaning candidate funnels
-
$74,497 per week
-
$3,873,844 per year
-
How did we get there?
Key findings: Documenting interactions
Almost 40% of Colleagues document
less than half of interactions
When Colleagues document 75% or more of interactions, it takes several hours out of their day
Key findings: Managing communications
Pay and attendance related topics
come up most often with Associates
Managing communications can often take up the majority of a Colleague’s day
Key findings: Cleaning candidate funnels
Candidates get archived after a combination of contact attempts and time without activity
Candidate profiles usually get rejected if there are issues with a background check or they don’t meet job qualifications
Time spent on these functions
Documenting interactions
~21,360
hours per week
across all branch colleagues
Managing communications
~21,275
hours per week
across all branch colleagues
Cleaning candidate funnels
~3,239
hours per week
across all branch colleagues
Recommendations & Impact
Final Recommendations
Prioritize automation & process solutions to make documentation in ATS easier and faster for Colleagues
-
46% of Colleagues report this task keeps them from their key function;
-
SPMs (Staffing Performance Managers. Typically onsite with clients and associates) are more severely impacted
-
It’s very challenging, and probably unrealistic, for Colleagues to document everything with current tools and processes
-
Approximately 30% of interactions are not documented based on reported data. This impacts
-
Data integrity and reporting accuracy
-
Ability for branch colleagues & support teams to stay informed
-
Associate experience
-
Client experience
-
Recommendations for further research
-
Payroll processes
-
Associate time entry
-
Colleague time tracking
Impact
Based on these findings, the Product team was able to build an automation inside the new ATS to address candidate statuses and alleviate the need for manual documentation of candidate statuses.
This will have the biggest impact on the time spent managing candidate funnels.
Professional Learnings
Coding write-in's takes more time than you think
There were a surprising number of write-in responses which really helped add color to the data. However, they took quite a while to sift through the responses, align on how to code them, actually code them, and then add to our research repository.
Next time
Strike more of a balance between when to give write-in opportunities and bake in a few more hours for coding in the overall research timeline.
