Listening to intents over MQTT using Python
In this tutorial, you will learn how to create a standalone Python application that listens to various events coming from the Snips Platform, in the form of MQTT messages.
The Snips Hermes Protocol is the set of messages and exchange of messages between the various components of the Snips Platform. The messages are MQTT messages. The figure below describes the flow of messages during a simple voice interaction session.
You will most likely be interested in the following messages:
hermes/hotword/<hotword_id>detected: when a wakeword is detected, this message will be sent, and Snips will listening to your voice command. You can for instance use this message to play a little tone, or start a led animation, indicating the start of a listening sessionhermes/asr/textCaptured: when listening, Snips will transcribe your query into text, and after a small period of silence, it will stop listening, and send this message. You may use this to play another tone, or stop a led animation, indicating end of listeninghermes/intent/<intent_name>: after text has been captured, the NLU module will process the text and transform it into an intent, which is the final representation of your query that you can use as an actionable item. This is where you want to put your intent handler code
When launched, Snips starts an MQTT broker that it uses to broadcast the above MQTT messages. We can connect to this broker using the paho-mqtt library:
$ pip install paho-mqtt
The connection code looks as follows, replacing MQTT_BROKER_HOSTNAME with your broker hostname or IP:
import paho.mqtt.client as mqttHOST = 'raspi3-mika2.local'PORT = 1883def on_connect(client, userdata, flags, rc):print("Connected to {0} with result code {1}".format(HOST, rc))def on_message(client, userdata, msg):print("Message received on topic {0}: {1}"\.format(msg.topic, msg.payload))client = mqtt.Client()client.on_connect = on_connectclient.on_message = on_messageclient.connect(HOST, PORT, 60)client.loop_forever()
When a wakeword is detected, a message is sent on the hermes/hotword/<hotword_id>/detected topic, where hotword_id is the ID of your wakeword. The default value is default, which we will use here. We extend the above example to detect a wakeword and trigger an action as follows:
def on_connect(client, userdata, flags, rc):print("Connected to {0} with result code {1}".format(HOST, rc))# Subscribe to the hotword detected topicclient.subscribe("hermes/hotword/default/detected")def on_message(client, userdata, msg):if msg.topic == 'hermes/hotword/default/detected':print("Wakeword detected!")
We now extend our example to also trigger and action when an intent is detected by the NLU component. In this case, a message is sent on the hermes/intent/INTENT_NAMEtopic, where INTENT_NAME is the name of our intent.
def on_connect(client, userdata, flags, rc):print("Connected to {0} with result code {1}".format(HOST, rc))# Subscribe to the hotword detected topicclient.subscribe("hermes/hotword/default/detected")# Subscribe to intent topicclient.subscribe('hermes/intent/INTENT_NAME')def on_message(client, userdata, msg):if msg.topic == 'hermes/hotword/default/detected':print("Hotword detected!")elif msg.topic == 'hermes/intent/INTENT_NAME':print("Intent detected!")
We don’t have to manually subscribe to the topic for each intent. If we want to trigger an action when receiving any intent, we can simply subscribe to the wildcard topic hermes/intent/#:
def on_connect(client, userdata, flags, rc):# Subscribe to any topic starting with 'hermes/intent/'client.subscribe('hermes/intent/#')
In the following example, when receiving an intent, we parse the payload to extract the slots associated with our intent:
import jsondef on_message(client, userdata, msg):if msg.topic == 'hermes/intent/lightsTurnOnSet':payload = json.loads(msg.payload)name = payload["intent"]["intentName"]slots = payload["slots"]print("Intent {0} detected with slots {1}"\.format(name, slots))
This completes the basic tutorial on how to write an MQTT client in Python that triggers actions upon receiving wakewords and intents from the Snips platform.
