Most organisations have a separation of duties as far as in-app chat is concerned. The in-app messenger belongs to support, and product discovery has not much to do with those chats since it happens in the scheduled interviews with customers. That makes the continuous product discovery process unnecessarily slower and thinner.
Teresa Torres defines continuous discovery as "at a minimum weekly touchpoints with customers by the team that's building the product where they conduct small research activities in pursuit of a desired product outcome."
We’ll admit, weekly is super hard. In Pragmatic Institute's 2021 annual survey, "69% PMs reported spending zero hours a month interviewing potential customers," and 39% PMs reported zero hours interviewing customers.
Meanwhile, customers are starting conversations in your product every day & the Product Managers stay away from them.
The mistake: treating messenger conversations as support work
Traditionally conversations with customers through the in-app messenger are judged on metrics like - first response time, time to resolution, conversations closed etc. So naturally the conversations in that channel are expected to be closed quickly, rather than learning from them.
The moment that chat is resolved - all the invaluable info the customer provided (unknowingly) in their own words (e.g. their workarounds, their context etc) - got pushed into an archive. Never to be looked at again.
A SaaS team we work with recently saw the costs when their customer used the in-app messenger to report a small bug. One question led to another, and the Product team realised the bug mattered a lot to that customer because of the unique way they used the product.
If the team had focused only on support metrics, they'd have resolved the chat as fast as possible and moved on. The learning would never have happened.
A chat is a customer-initiated touchpoint. Continuous discovery asks you to create weekly touchpoints; in chat, the customer already reached out to you.
Advantages of in-app chats for Product Managers
In-app chat has three advantages over scheduled interviews:
- It's in context. The customer is in your product, at the moment of friction - so naturally your product team has better awareness of the customer situation.
- It's proactive. The customer picked the topic, and wants to share with you some questions or suggestions. Instead of the PM researching the topics, the customer has already done the ‘research’.
- It's already happening. The PMs are not trying to identify relevant audiences by digging through usage/segmentation data etc.
But such a conversation definitely has its limits. Torres points out that "talking to a customer" often means "getting pulled into a sales call or being consulted on a support ticket," and that these "aren't a replacement for good customer interviews." Chats are short and skew toward problems.
Where in-app chats can be misleading
The bigger trap is sampling. These in-product chat conversations can never represent the entire user base. These are the customers who are self-selected. This is volunteer bias at play: volunteers "intrinsically have different characteristics from the general population of interest." So most likely these proactive customers have different traits as compared to the majority of your customers. And on top of that they also show response bias in action through what they say.
In a 2024 Academy of Management conference paper, Allen, Bremner and McDonald argue that self-selected user communities "attract members with atypical preferences," and find that PC-game developers "highly responsive to feedback from (unrepresentative) users tend to release less commercially successful games."
Here comes the recency bias. Tversky and Kahneman's availability heuristic explains at length why last week’s chat feels like a trend. We humans are wired to judge frequency by how easily examples come to mind.
Six discovery signals to listen for in in-app
PMs don't need to analyse every chat with customers. They just need to listen for these:
Treat an incoming request as a clue, instead of a requirement. e.g. "add a button" is the start of a conversation about the problem behind it, discover the pain point then prioritise it properly.
Asking discovery questions in a live chat: read the room
There's no rigid rule. Whether PMs should ask their questions depends on the mood of the conversation and how open the customer is. You will get shouted on if you approach a frustrated customer who is still waiting for the fix. Yet a relaxed one who's just received their help is often ok to talk.
Fix the problem first. Then, if the door is open, try a story-based, past-tense prompt like Torres's "Tell me about the last time this happened. What were you trying to do?"
Ask one thing at a time, and avoid leading or double-barreled questions. If the thread gets rich, offer a call.
Conclusion
In-app messenger conversations offer product growth opportunities that support teams often miss.
If your product doesn't have an in-product messenger/chat box, add one. And then staff it with Product Managers and support customers there directly. That is a better strategy than assigning these chats to support staff who are incentivised to close them asap.Product Discovery is already happening in that chat window. Your Product Managers just need to be in the conversation.
FAQs
What is the continuous discovery process?
Per Torres, it's at least weekly customer touchpoints by the team building the product, running small research activities toward a desired outcome.
Can chat replace customer interviews?
No. Chat gives you context in the moment; interviews give you depth (see customer feedback vs user research). Use chat to just scratch the surface & build a relationship, hopefully leading to an interview down the line.
Should PMs answer support chats themselves?
Yes, we think this should happen regularly. Your PMs would hear problems in customers' own words and can ask follow-ups almost instantaneously.
