Live mode is a visualizer for the Tone feature which analyzes your speech and predicts how you might sound to those around you. It’s the first experience the customer is taken to after OOBE.
As the lead designer, I was responsible for designing an experience that accurately translated the live pacing of the ML predictions on screen. The challenge turned out to be trying to remain true to the pace and frenetic energy of the prediction engine while still remaining readable on screen. To do this, I worked through the timing and easing in after effects with our animator. Once approved, I worked with engineering to implement.
As the lead designer, I was responsible for designing an experience that accurately translated the live pacing of the ML predictions on screen. The challenge turned out to be trying to remain true to the pace and frenetic energy of the prediction engine while still remaining readable on screen. To do this, I worked through the timing and easing in after effects with our animator. Once approved, I worked with engineering to implement.
After OOBE, customers are dropped into Live mode for their first experience with the product. Unfortunately, early trials feedback revealed that customers didn’t know what to do once there. To mitigate, I used learnings from the Body scan experience to design a lightweight FTUE (first time user experience). FTUE allows us to prompt and guide users through their first use of Live mode. Once implemented, trials data improved almost immediately.
The happy path depicts the best experience for most users. The microphone check the pops the interstitial for customers who don’t have microphone enabled. The interstitial screen allows us to be transparent and tell the customer up front why we need mic permission and what they’ll be missing out on if they decline. The conversation summary gives the customer a snapshot of the conversation without having to go to the conversation details.
There are 3 states that I defined for the experience, start, idle, and live. The start state is before the customer has started the conversation. The idle state is for when silence is detected, latency, and processing. The live state is during conversation capture.
The states proved challenging when providing feedback to customers since the API only returned a success or fail. I went through rounds of negotiation to get engineering to agree to the idle state.
This feature required a super immersive deep dive into the scientific realms of the unknown. I loved every minute of it. Defining the ball and descriptor behavior and color mapping required understanding the intricacies of the ML and framework under the hood.
