Labels

How to detect car engine anomaly by analyzing engine noise?

Following to my article on "Starting your exciting journey of Connected Field Service and Azure IoT Hub", I started working on a practical scenario about measuring noise in your surrounding and generating alerts in #PowerPlatform. In this article I want to summarize all resources required to implement such a scenario and my learning. I hope this will add on the resources available in the community so you can use it as a walkthrough to implement a practical scenario.

In this article, you will see what architectural components are required to implement this simple scenario using Azure Iot and Connected Field Service. This article focuses on what is happening underhood of the platform. Luckily with Connected Field Service Application, you have everything managed behind the scene and you don't need to worry much but this walkthrough enables to you understand what options you have in such scenarios. A big shout out to Sahan Wijayasekera for providing me a new MX Chip DevKit.

Scenario

The scenario is about connecting MXChip IoT DevKit to your car or any place with noise and analyze the noise level by recording and sending the noise in form of Wave stream to an Azure IoT Hub. The Azure IoT Hub sends the data to an #Azurefunction which calculates the noise level using a simple formula and the function calls a #MicrosoftFlow to create alerts in #PowerPlatform. This can lead to number of endless scenarios.

Considerations

  1. The function for calculating the noise level from a wave file is extremely simple as well. There are so many scientific information which you can read here, here, here and here.
  2. Calculating the noise level is not an easy task. There are many considerations involved and if you want to have the real working model, you will need to work on analyzing audio files which is beyond the scope of this demo.
  3. It is possible and desirable to calculate the noise level in the device and send only the alerts to Azure IoT. This will reduce the traffic and the load on your Azure. However, for the sake of experiment I am sending all the noise data to Azure and calculate the noise level in Azure function.
  4. In this demo, I am not listening to the noise all the time. I start recording on press of button A. I send the noise data to Azure on press of button B. I made this change to the scenario to demonstrate working with buttons in MX Chip and also reduce the traffic to Azure.

Architecture

The architecture of this sample is very simple. I am using an IoT Hub and Azure function to calculate and propagate the IoT events to the #PowerPlatform. On the device side, there is an Arduino application running which listens to noises and sends the noise to the Azure function.

A very comprehensive architecture of a connected field service is created in the below diagram which can simply be implemented using the #ConnectedFieldService application. However, I just wanted to implement it in a simpler way. Full details of the #ConnectedFieldService architecture can be seen in this documentation.

Components

The logical diagram of components is demonstrated below:

Ardiuno App

This component is a very program which reads the input from Audio, ButtonA and ButtonB of the device and does the following:
  1. On startup, it initializes the device and gets ready to listen to surrounding noise. It also checks the connectivity to Azure. 
  2. On press of ButtonA , it records and surrounding noise and stores the stream in a buffer.
  3. On press of ButtonB, it sends the stream in the buffer to Azure.
To implement this part of the application, you will need to take following actions:
  1. Setup your device MXChip device. Please refer to this link to start.
  2. Setup your Visual Studio environment. Please refer to this link.
  3. You will need to learn how to deploy your code to the MXChip device. The simple way to upload your code your code to the device is to bring your MXChip device to Configuration mode. This means everytime you want to upload your updated code, Press A (and keep pressing) and then press reset (while still pressing A). Then release reset (While still pressing A) and then release A. Now you are ready to upload your code.
  4. If you want to debug your code in the device, you can refer to this link. 
Here is the sample Ardiuno code:

#include "AZ3166WiFi.h"
#include "DevKitMQTTClient.h"
#include "AudioClassV2.h"
#include "stm32412g_discovery_audio.h"
#include <math.h>

 #define MFCC_WRAPPER_DEFINED
#define MODEL_WRAPPER_DEFINED

 //Constants and variables- Start//
enum AppState
{
  APPSTATE_Init,
  APPSTATE_Error,
  APPSTATE_Recording,
  APPSTATE_ButtonAPressed,
  APPSTATE_ButtonBPressed
};
// variables will change:

 static AppState appstate;
static int buttonStateA = 0;
static int buttonStateB = 0;
static bool hasWifi = false;
static bool hasIoTHub = false;

 AudioClass &Audio = AudioClass::getInstance();
const int AUDIO_SIZE = 32000 * 3 + 45;

 char *audioBuffer;
int totalSize;
int monoSize;

 static char emptyAudio[AUDIO_CHUNK_SIZE];

 RingBuffer ringBuffer(AUDIO_SIZE);
char readBuffer[AUDIO_CHUNK_SIZE];
bool startPlay = false;

 void SendMessage(char *message)
{
  // Send message to Azure
  if (hasIoTHub && hasWifi)
  {
    char buff[512];

     // replace the following line with your data sent to Azure IoTHub
    snprintf(buff, 512, message);
    if (DevKitMQTTClient_SendEvent(buff))
    {
      Screen.print(1, "Sent...");
    }
    else
    {
      Screen.print(1, "Failure...");
    }
    delay(2000);
  }
  else
  {
    // turn LED on-off after 2 seconds wait:
    Screen.print("NO BUTTON DETECTED");
    delay(1000);
    Screen.clean();
  }
}

 void setup()
{
  // put your setup code here, to run once:
  memset(emptyAudio, 0x0, AUDIO_CHUNK_SIZE);

   if (WiFi.begin() == WL_CONNECTED)
  {
    hasWifi = true;
    Screen.print(1, "Running!!!");

     if (!DevKitMQTTClient_Init(false, true))
    {
      hasIoTHub = false;
      return;
    }
    hasIoTHub = true;

     // initialize the pushbutton pin as an input:
    pinMode(USER_BUTTON_A, INPUT);
    pinMode(USER_BUTTON_B, INPUT);
    appstate = APPSTATE_Init;
  }
  else
  {
    hasWifi = false;
    Screen.print(1, "No Wi-Fi");
  }
}

 void loop()
{
  // put your main code here, to run repeatedly:
  Screen.clean();
  // while(1)
  {
    // read the state of the pushbutton value:
    buttonStateA = digitalRead(USER_BUTTON_A);
    buttonStateB = digitalRead(USER_BUTTON_B);
    if (buttonStateA == LOW && buttonStateB == LOW)
    {
      //SendMessage("A + B");
    }
    else if (buttonStateA == LOW && buttonStateB == HIGH)
    {
      // WAVE FORMAT
      Screen.clean();
      Screen.print(0, "start recordig");
      record();
      while (digitalRead(USER_BUTTON_A) == LOW && ringBuffer.available() > 0)
      {
        delay(10);
      }
      if (Audio.getAudioState() == AUDIO_STATE_RECORDING)
      {
        Audio.stop();
      }
      startPlay = true;
    }
    else if (buttonStateA == HIGH && buttonStateB == LOW)
    {
      // WAVE FORMAT
      if (startPlay == true)
      {
        Screen.clean();
        Screen.print(0, "start playing");
        play();
        while (ringBuffer.use() >= AUDIO_CHUNK_SIZE)
        {
          delay(10);
        }
        Audio.stop();
        startPlay = false;
        SendMessage(readBuffer);
      }
      else if (buttonStateA == HIGH && buttonStateB == HIGH)
      {
        Screen.clean();
      }
    }
    delay(100);
  }
}
void record()
{
  Serial.println("start recording");
  ringBuffer.clear();
  Audio.format(8000, 16);
  Audio.startRecord(recordCallback);
}

 void play()
{
  Serial.println("start playing");
  Audio.format(8000, 16);
  Audio.setVolume(80);
  Audio.startPlay(playCallback);
}

 void playCallback(void)
{
  if (ringBuffer.use() < AUDIO_CHUNK_SIZE)
  {
    Audio.writeToPlayBuffer(emptyAudio, AUDIO_CHUNK_SIZE);
    return;
  }
  int length = ringBuffer.get((uint8_t *)readBuffer, AUDIO_CHUNK_SIZE);
  Audio.writeToPlayBuffer(readBuffer, length);
}

 void recordCallback(void)
{
  int length = Audio.readFromRecordBuffer(readBuffer, AUDIO_CHUNK_SIZE);
  ringBuffer.put((uint8_t *)readBuffer, length);

}

Azure function

This is the simplest of all. All you have to do is to receive the stream and calculate the noise level. This can be very sophisticated but it is out of scope of this article. Here is the code:

using IoTHubTrigger = Microsoft.Azure.WebJobs.EventHubTriggerAttribute;

 using Microsoft.Azure.WebJobs;
using Microsoft.Azure.EventHubs;
using System.Text;
using System.Net.Http;
using Microsoft.Extensions.Logging;
using System;

 namespace IoTWorkbench
{
    public static class IoTHubTrigger1
    {
        private static HttpClient client = new HttpClient();

         [FunctionName("IoTHubTrigger1")]
        public static void Run([IoTHubTrigger("%eventHubConnectionPath%", Connection = "eventHubConnectionString")]EventData message, ILogger log)
        {

             log.LogInformation($"C# IoT Hub trigger function processed a message: {Encoding.UTF8.GetString(message.Body.Array)}: " + System.Text.Encoding.Default.GetString(message.Body));
            byte[] buffer = message.Body.ToArray();
            short sample16Bit = BitConverter.ToInt16(buffer, 0);
            double volume = Math.Abs(sample16Bit / 32768.0);
            double decibels = 20 * Math.Log10(volume);
            log.LogInformation(decibels.ToString());
        }
    }
}

Handshaking

In order the device to send messages to the Azure function, the device must know the endpoint in which it should send the data. You can take steps in the this link to register your device with Azure function. It is all about using Azure IoT Workbench.



References

https://docs.microsoft.com/en-us/dynamics365/field-service/developer/connected-field-service-architecture 
Photo of the car in the flow diagrams by Steinar Engeland on Unsplash
Labels:

Comments

Post a Comment

Popular Posts