Skip to main content

Voice Communications with Ionic

Build voice and AI-powered conversational experiences using drop-in predictions and translation features for Ionic Multi-Experience apps.

Voice Input Button

Getting Started#

Follow the AWS Amplify Predictions guide to configure the Predictions API in your Ionic app.

Adding Voice Input UI#

To create a typical voice-input UI with a persistent microphone record button, add an Ionic Fab control to the desired page:

  <ion-fab horizontal="center" vertical="bottom" slot="fixed">    <ion-fab-button (pointerdown)="startVoiceCapture()" (pointerup)="endVoiceCapture()">      <ion-icon name="mic"></ion-icon>    </ion-fab-button>  </ion-fab>

Capturing Voice Input#

Next, add the methods to your page class to start and stop voice capture:

  async startVoiceCapture() {    const constraints = { video: false, audio: true };
    try {      this.stream = await navigator.mediaDevices.getUserMedia(constraints);    } catch (error) {      throw new Error(`      MediaDevices.getUserMedia() threw an error.       Stream did not open.      ${error.name} -       ${error.message}    `);    }
    this.recorder = new window.MediaRecorder(this.stream);
    this.recorder.addEventListener('dataavailable', async ({ data }) => {
      try {        const fullText = await Predictions.convert({          transcription: {            source: {              data            },            // language: "en-US", // other options are "en-GB", "fr-FR", "fr-CA", "es-US"          },        }).then(({ transcription: { fullText } }) => setResponse(fullText))
        // Process the text      } catch (e) {        console.error('Unable to convert speec', e);        throw e;      }    });
    this.recorder.start();    return this.recorder;  }
  async endVoiceCapture() {    this.recorder?.stop();    this.stream.getTracks().forEach((track) => track.stop());  }

Convert Audio to Text#

To Convert audio to text, use the Predictions.convert API as shown above:

const fullText = await Predictions.convert({  transcription: {    source: {      bytes    },    // language: "en-US", // other options are "en-GB", "fr-FR", "fr-CA", "es-US"  },});

Resources#

See the Ionic Multi-experience Demo for an example voice-driven UI experience.