This article outlines the evaluation criteria involved in selecting a real-time Speech-to-Text or ASR  for LLM-powered AI Copilots and Real-time agent assist applications in the contact center. This article is intended for Product Managers and Engineering leads in Contact Center AI SaaS companies and CIO/CDO organizations in enterprises that are looking to build such AI co-pilots.

The buzz around Gen AI-powered Co-Pilot & Realtime Agent Assist  

A very popular use-case for Generative AI & LLMs is the AI Co-pilot or Realtime Agent Assist in contact centers. By transcribing an agent-customer conversation in real-time and feeding the transcript to modern LLMs like Open AI's GPT, Facebook's LLAMA2 or Google's Gemini, contact centers can guide their agents to handle their calls more effectively and efficiently.

An AI Co-pilot can deliver great business benefits. It can improve CSAT and NPS as the AI can quickly search and present relevant knowledge-base to the agent, enabling them to be more knowledgeable and productive. It can also save Agent FTE costs by reducing AHT and eliminating wrap time.

In addition by building a library of "gold-standard" calls across various key call types, LLM can also deliver personalized coaching to agents in an automated way using Generative AI.Companies are finding that while Gen AI-powered Co-Pilots are especially beneficial to new hires, they also deliver benefits to agents with tenure too.

Building an AI-powered Co-Pilot requires three main components - a) A real-time ASR/Speech-to-Text engine for transcription 2) An LLM to understand the transcript and 3) Agent and Supervisor/Manager facing web applications. The focus of this blog post is on the first component - the real-time ASR/Speech-to-Text engine.

Now here are the four key factors that you should look at while evaluating the real-time ASR/Speech-to-Text engine.

1. Ease of Integration with Audio Source

The first step for any AI Co-Pilot is to stream the agent and customer real-time media to an ASR that supports streaming Speech-to-Text. This is easily the most involved engineering design decision in this process.

There are two main approaches  - 1) Streaming audio from the server-side. In an enterprise contact center, that would mean forking the media from either an enterprise Session Border Controller or the Contact Center Platform (which is the IP-PBX). 2) Streaming audio from the client side - i.e from the Agent Desktop. An Agent desktop can be a OS based thick client or a browser-based thin client - this depends on the actual CCaaS/Contact-Center platform being used.

Selecting the method of integration is an involved decision. While there are advantages and disadvantages to both approaches, server-side approaches have been the preferred option. This is because you would avoid the need to install client software and plan for compute resources at the agent desktop level.

However if you have an on-premise contact center like an Avaya, Cisco or Genesys, the integration can become more involved. This is because each platform has its own mechanism to fork these media streams and you also need to install the ASR/STT behind the corporate firewall (or open it up to access a Cloud-based ASR/STT).

Net-net, there is a case to be made for client-side streaming too - because not all companies may have the expertise available within the company.

There are modern CCaaS platforms like Amazon Connect, Twilio Flex, Genesys Cloud and Five9 that offer APIs/programmable access to the media streams. You are in luck if you have one of these platforms. Also if the PSTN access is through a programmable CPaaS platform - like Twilio, Signalwire, Telnyx etc, then it is quite a

2. Protocol support from the ASR/STT 

Once you finalize a method to fork the audio, you would need to consider the standard protocols supported by the ASR/Speech-to-text engine. Ideally, the ASR/STT engine should be flexible and support multiple options. One of the most common approaches today to stream audio over websockets. It is important to confirm that the ASR/Speech-to-Text vendor supports two-channel/stereo audio submission over websockets. There are other approaches - sharing audio over gRPC and over raw RTP.

3. Speed/ Latency of ASR/Speech-to-Text model

The next big consideration is the latency of real-time ASR/Speech-to-Text model - which in turn depends on the underlying neural network architecture of the model. In order to provide timely recommendations to the Agent, it is important to target ASRs that can deliver word-by-word transcript in less than one second and ideally in about 500 milliseconds. This is because there is additional latency associated with collecting and submitting the transcript to LLMs and then delivering the insights onto the Agent Desktop. 

4. Affordability

Last but not the least, it is really important that the price for real-time transcription is affordable in order to build a strong business case for the AI Co-Pilot. It is important to confirm that the agent and caller channel are not priced independently as that very often kills the business case.

If you are building an LLM-powered AI Co-pilot and would like to engage in a deeper discussion, please give us a shout! You can reach us at sales@voicegain.ai.

‍

‍

This blog post is intended for anyone responsible for upgrading/migrating an MRCP-based Nuance ASR nearing EOL (End of Life). They can explore how Voicegain ASR simplifies and economically extends the life of existing speech-IVR platforms. It serves as a 'drop-in' replacement for grammar-based Nuance ASR.

Nuance ASR reaching End of Life

There are several hundred (if not thousands) telephony-based speech-enabled IVRs that act as the 'front-door' for all customer service phone calls for enterprises of all sizes. These speech-enabled IVRs are built on platforms like Genesys Voice Portal (GVP), Genesys Engage, Avaya Aura Experience Portal(AAEP)/Avaya Voice Portal , Cisco Voice Portal (CVP), Aspect or Voxeo ProphecyVoiceXML platform and several other such VoiceXML based IVR solutions. The systems predominantly use Nuance ASR as the speech recognition engine.

Unlike contemporary large vocabulary neural-network-based ASR/STT engines, the traditional Nuance ASR is a grammar-based ASR. It uses the MRCP protocol to talk to VoiceXML based IVR platforms. Most of these systems were purchased in the last two decades (2000s and 2010s). Customers typically paid a port-based perpetual license fee (the IVR platforms were also licensed similarly). Most enterprises have a software maintenance/AMC contracts for the Nuance ASR and this is usually bundled along with the IVR platform. The Nuance Recognizer versions in the market vary between 9.0 and 11.0. As of June 2022, Nuance had announced end of support for Nuance 10.0. It is our understanding in speaking with customers that the last version of Nuance sold – Nuance 11.0 Recognizer will approach either end-of-life or end-of-Orderability sometime in 2025^*.

Nuance upgrade path is challenging 

Also in speaking with customers, we have understood that customers who currently license the MRCP grammar-based Nuance ASR would have to upgrade to Nuance’s Krypton engine, the new deep-learning based ASR in 2025. Nuance Krypton can only be accessed using the modern gRPC based API and not over MRCP, which makes this upgrade expensive and time-consuming. Because of this, Customers would need to upgrade not just their the ASR but also the entire IVR platform. This is because most legacy IVR platforms - especially would do not support gRPC. This might also entail migrating the existing call flow logic –which is likely written in a VoiceXML app studio or written in a build tool and generated as VoiceXML pages – would also need to be ported.

All of the above steps makes the upgrade process very challenging. While there is a strong case to be made for the merits of upgrading to a deep-learning based ASR to support conversational interactions (better automation rates and more natural user-experience), it is critical for customers that this upgrade/migration is done on the customer’s timelines and not under the gun on the vendor’s clock.

Voicegain as a future-proof drop-in replacement for Nuance ASR

Voicegain offers a drop-in replacement for the Nuance grammar-based ASR. We are the only modern deep-learning/AI (neural-network-based)ASR in the market that natively supports both traditional speech grammars (grxml, SRGS) and large-vocabulary conversational interactions. We are also one of the very few ASR vendors that can be accessed both over a traditional telephony-based protocol like MRCP and a modern web-based method like web-sockets (or gRPC). So the same neural-network model supports both the old and the new protocols. This allows you a future-proof method of replacing Nuance ASR with minimal effort while safeguarding this investment for the long term.

Net-net, by just "pointing" the ASR resource on the VoiceXML platform to the IP-address of the Voicegain MRCP ASR in your network, you can replace the entire Nuance ASR with the Voicegain ASR. Customers would not need to even change or modify a single line of code of the speech-IVR application logic.

In other words, a client can retain the existing telephony/IVR setup and just perform a "drop-in replacement" of Nuance MRCP ASR with Voicegain MRCP ASR.

Longer-term the same Voicegain ASR can perform large vocabulary transcription because it is a neural-network based ASR; so when the customer is ready to replace the directed-dialog Speech IVR with a conversational interaction, the Voicegain platform will already support it.

Get Started for free today

To discuss your upgrade situation in more detail, please contact us over email at sales@voicegain.ai.We can answer any questions that you have. You could also get started with a free developer account by following these instructions. There is no credit card required and we offer 1500 hours of usage for free. Here is a link to the instructions; after you sign up, please contact us at support@voicegain.aiand request MRCP access.

‍

* Nuance ASR and Nuance Krypton are trademarks of Nuance, Inc which is now part of Microsoft. Please confirm the End of Life announcement and the protocol capability directly with the company. Our information in this blog post is anecdotal and has not been verified with Nuance.