Relevance beats frequency
Relevance was ranked the number one factor in whether a message feels noisy; intrusiveness ranked last. There's no magic number of messages — people want the right message at the right moment.
Survey + interviews
When does a helpful bot become noise?
The team wanted to build an alerting system with a user-set noise threshold. First we had to answer a harder question: what do employees actually count as bot noise, and where does the line sit?
33
survey responses
7
in-depth interviews
58%
want work bot messages 'as needed'
#1
relevance ranked top noise factor
What we asked
Define the noise before you build the filter.
- How do users define bot noise, and what drives it — frequency, relevance, timing?
- Which bot interactions feel disruptive or unwanted?
- Is bot noise measurable, and where is the threshold between acceptable and disruptive?
How we ran it
- Method
- CSAT survey followed by in-depth interviews
- Survey
- 33 employee responses on usefulness and frequency
- Interviews
- 7 participants: 5 software engineers, 1 business systems analyst, 1 program manager
- Bots in scope
- Butterfly automation, Google Drive, interview, task, IT, crash, thermal stress, enterprise authentication
Which bot actions do employees find useful?
- Communication88%
- Workflow automation52%
- Meeting management39%
- FAQ / employee support24%
How often should work-related bot messages arrive?
- As needed58%
- Max 3× a week24%
- Max once a day12%
- Every two weeks or less6%
There is no preferred number. People want important information exactly when it's needed — and non-work announcements no more than once a day.
Key takeaways
Engineers feel it hardest
Engineers interact with bots far more than other roles, and carry correspondingly more frustration with them.
Actionable beats informational
Messages carrying a link or a decision drove engagement. Repeats of information already in email were dismissed outright.
No way out is the core pain
People could not unsubscribe or tune frequency for many bots, so unwanted notifications simply kept arriving.
Time and location blindness reads as noise
Building emergency alerts sent to people who aren't on site, or weekend pings that ignore delay rules, were consistently rated negative.
Critical work earns more messages
Approvals, task escalations, and issue resolution can come often. Company-wide announcements should not exceed roughly once a day.
Where the noise actually came from
- Thermal Stress Heat bot firing on every data hall entry, with no opt-out.
- Google Drive bot duplicating notifications already delivered by email.
- Diff comment bot announcing a comment without showing it, forcing a context switch.
- Task notifications fanned out beyond the actual owners.
- IT bot re-prompting after a clear 'no', with no way to explain.
- Crash bot and other reliability signals repeating daily.
In their words
“Interview bot sends me the same information that I already have in my email. I feel it's not required.”
“A lot of surveys come through as bot messages, and they're perceived as noise too.”
“My manager wouldn't like me to spend time taking surveys — I just don't have the bandwidth.”
“The Enterprise Authentication bot reminding me to update my password keeps me up to date.”
What I recommended
People won't get past their preconceptions about bots unless we build with their frustrations in mind.
- 01Give people an easy, obvious way to unsubscribe from bots that add no value.
- 02Require opt-in approval before a bot starts messaging employees.
- 03Design messages around user preference — channel, redundancy, and content — not bot convenience.
- 04Offer daily digest summaries instead of a stream of individual pings.
- 05Let users set their own notification frequency and volume thresholds.
- 06Review every bot before it goes live, and honor delay, silent, and location rules.
Next steps
Design an alerting system built around user-set preferences and community guidelines for bot creation.
Validate the mechanism with employee feedback on efficiency and noise reduction.
Run a follow-up survey to benchmark sentiment toward the improved alerting system.