AI Follow-Up Questions: Turn Surveys into a Conversation
You, as a Product Manager, are in a meeting with a customer whose team has used your product for over a year. You are trying to figure out their pain points to improve your product further. And at one point the customer says - “we just need faster exports”.
As an experienced Product Manager, you know it would be incompetent to add a ‘faster export’ card to the roadmap & move on. So you probe the customer further in the hope that your product can solve the underlying problem, not just address the symptom. You ask questions like - what do you do with the exported file once you have it? And the story starts to unravel from there.
The customer tells you that every Monday, someone from her team exports the data from your product & feeds it into a spreadsheet to build pivot tables & charts. And these visuals eventually appear into a deck for the leadership meeting. Now you’ve got something to work with. You have already sensed that faster export might save a few seconds for the customer, but that task shouldn’t exist at all. Your product should be capable of providing the exportable dashboard charts, in the first place.
Notice how that follow up question is where the useful answer lived. The customer’s first answer was a solution she'd already imagined.
Where the survey version of this story goes wrong
But can you expect a similar outcome through your in-product feedback survey? Usually, the customer sees the survey. Rates reporting a 2 out of 5, types "faster export" in the comment box. The survey thanks her and disappears for a few weeks. A week later this customer’s comment is one of ten other feedback messages tagged ‘export’ in a spreadsheet. And so the product team decides that they should be spending engineering time on improving the export feature.
The survey did nothing wrong, in fact it delivered on what was expected of it. But the problem is it is not intelligent enough to ask the follow up question. That has always been a problem with surveys - they are good at telling you how many customers feel something but fails at highlighting their why. That's why Product Managers request calls from customers to learn what the survey couldn't. (For when a survey is enough and when you need proper research, see our guide to customer feedback vs user research.)
What changes when the survey asks next question intelligently
But with AI, in-product surveys can become somewhat intelligent. After the customer answers, AI decides whether it should ask one more question to the customer based on her responses to survey questions. And if useful, AI adds a single follow up question to the survey for this customer.
For the customer above, that AI follow up survey question might be: "What do you do with the export once you have it?" It's the same question the PM asked in the meeting. But this time around, the customer is being asked the question inside the product without getting into a 30 minute meeting. Now if they answer just as they did during the meeting, your feedback export sheet looks different.
But not every survey response deserves a follow up question. That would be annoying to your customers. Truly intelligent AI knows when to skip the follow up.
Why the surveys should ask only one AI follow up question
It's tempting to probe every answer the way a freshly recruited Product Manager would. But restraint helps. It's good to recall that the customer got into your product to get something done, not to be interviewed unplanned. So, it's only sensible to get out of their way. Get their consent for a follow up conversation, if necessary. But more than one AI follow up question in in-product surveys is usually a bad idea.
Try AI follow up Survey questions in Olvy
Olvy in-product surveys now support AI follow-up question to any survey type. And that question is asked only when Olvy AI decides more context would help. The question is clearly labelled as AI-generated & customers can skip it if they want. The answer to that question appears along with this customer’s responses to other survey questions. Try it on your next survey and find out which "faster export" requests are really about something else.