IoT Home Automation with Hindi Commands using Google

In this project, we will be connecting and controlling our ES32 Wi-Fi based module from our mobile phone’s Google Assistant Application. The charisma here is that your google assistant will take voice commands in Hindi and perform actions on ESP32. It will then perform actions based on our voice commands. This project is amazing and helpful for those who want to make a Voice operated Home Automation for their parents who are not able to speak English. To make everything work, we will also require Adafruit IO and IFTTT. We will add up everything to work.

Fig. 1: Connect them all

Material to get started with this IoT Project

  1. ESP32 (Check the datasheet, if you are using a different version)

    Fig. 2: ESP32

  2. USB Type C cable to program ESP32 from a laptop or PC (most Android phones use this type of cable).

    Fig. 3: USB Cable

  3. Relay Module – Relay is a switching Module. In a relay, we can control switching AC or DC appliances digitally by providing input to relay input pins. Here we have used a 2-channel relay operating on a 5 Volts battery.
    Fig. 4: 2 channel Relay

    To connect your AC appliance, use the following connection with the Relay Module use the following connection. The Signal pin is IN1/IN2.

    Fig. 5: Relay connection

    with some wires and breadboard.

Back End Set-up

Set-up with Google Assistant

We must Set up Hindi in Languages option of Google Assistant account. To do this go to Google Assistant App > Settings > Phone > Voice and Speech – Other Voice Settings> Offline Speech Recognition. Now download English(India) and Hindi(Bharat).

Fig. 6: Google account Set-up

You can download other languages too as per your preference. For further reference, you can check here.

Set-up with Adafruit IO

Go to Adafruit IO and Sign Up or Sign In to your Adafruit IO account.

Fig. 7: Adafruit account set-up

After creating Adafruit, you should be taken to Adafruit Home Screen. Now Select Feeds from hand menu> Click Action and Create a New Feed from the drop-down menu.

Fig. 8: Create a new Feed

Give this any name you wish, I created “onoff”. Now, Go to Dashboards > Actions > Create a New Dashboard. Similarly name this one also, I named it “LightAutomation”.

Fig. 9: Create a new Dashboard

And click on Blue “+” on the Top right side of your Dashboard to create a new block. Select “Toggle” Block.

Fig. 10: Create a new block

Now, choose your feed and click “Next Step”.

Fig. 11: Choose Feed

Fill the block settings and “Create the Block”.

Fig. 12: Block Settings

Set-up with IFTTT

Create an account on IFTTT by Signing Up with your same Google Account you are using in your phone. Now go to My Applets > New Applet > Click “ + this “ In Search services tab type Google Assistant and select it. Click “Say a simple phrase” trigger and fill this data with English and Hindi phrases and “Create Trigger”.

Fig. 13: Create Trigger

Now, select “+ That” to create actions in the Adafruit IO so that Google Assistant can trigger Adafruit IO.

Fig. 14: Select That

Here we will select Adafruit as our Action Service.

Fig. 15: Choose Action Service

For the very first time, this will ask your Adafruit Login Credentials. Fill your Adafruit IO account details there.

Choose “Send data to Adafruit IO” action and fill the Action field by choosing you “Feed name” and “Data to Save” and “Create Action” and click “Finish”. This data will be the same as filled in Block Settings in Adafruit IO (Fig. 12).

Fig. 16: Action Field

We need to create two such blocks for ON and OFF for the same feed.

Fig. 17: On and Off Triggers

Steps for the ESP32 support in Arduino IDE

Ignore this step if you already have the setup of ESP boards in Arduino IDE.

Fig. 18: ESP32 support in Arduino IDE

Here to code the ESP32 we need an Integrated Development Environment and we will use Arduino IDE software. Arduino IDE is a cross-platform application. It is written in Java and coded in C/C++ with some special rules. To download the latest Arduino IDE from here. Arduino IDE does not contain support of ESP32 family so to install the ESP-32 Board in Arduino IDE, you can refer here.