Author: eshrawan

This week, I worked on setting up the Azure IoT hub instance with the configurations and adding the ESP8266 to the devices list for communicating with the cloud. I faced a number of issues in setting this up, as the ESP8266 Wi-Fi module we ordered does not have much documentation or listed steps for connecting to Azure. I used a modified Azure SDK for the NodeMCU version of the chip through the Arduino IDE, but there are additional requirements like flashing the firmware since we’re relaying it through the Arduino Uno. Flashing the firmware is very OS-dependent, so I am thinking about how we’re going to integrate all of this together down the line. I fell behind schedule this week, not being able to work with Caroline on the sensor system assembly, but we will begin this immediately since there is still a sensor that has not arrived yet. Connecting this hardware to the cloud was harder than I had imagined, previously coming from building just software solutions on the cloud. I feel like my shortfall in not being able to contribute this week has given me some anxiety about playing catch-up, but I will work on this immediately and make it a priority for me to complete it.

This week I worked on preparing our decided cloud platform: Azure IoT Hub and starting the software integration of Aditti’s wavefront segmentation computer vision program with the USB camera. The camera integration proved to be a difficult task, as with closer distance and smaller arenas (I used Letter sized paper as the “arena”), the camera shadow would be interpreted as an additional object. I hence had to use an artificial light source from my phone camera to make sure objects were illuminated correctly. I suspect we will need further testing on making sure the camera is stable and can click frames without a shadow appearing from its overhead nature. I have attached a few pictures below illustrating my testing of the camera feed. We have created a Github repository for our code and CAD files so far.

For the Azure application, there are currently two steps I must perform. One is to link an Arduino with the Wifi module, and the next step would be to link this Wifi Module to the cloud. I have found the following resources to help me investigate the same. (https://blog.avotrix.com/azure-iot-hub-with-esp8266/ for connecting the ESP8266 to the cloud and https://www.instructables.com/Get-Started-With-ESP8266-Using-AT-Commands-Via-Ard/ for controlling the ESP8266 from the Arduino). Next week, after we have all the robot CAD files printed out, Caroline and I will start the next step on sensor array assembling and cloud data collection.

The courses that helped me understand this course are 10-301 Introduction to Machine Learning for computer vision segmentation, 18-220: Electronic Devices and Analog Circuits for understanding basic commands and time delays on the Arduino. We spent some additional time researching multitasking on the Arduino for parallel processes. I have no formal experience in Cloud Computing, but I am certified as a Machine Learning Engineer and Cloud Architect for Google Cloud Platform, which has helped me immensely in investigating implementation on the cloud this week.

This week I focused on researching cloud implementations and alternatives for our sensor data collection and for hosting our classification algorithm. I also focused on the proposal presentation with Aditti and Caroline, where we met several times to go over various design details. There are several alternatives to consider: we could collect the data directly onto the Arduino serial monitor for training purposes, and send telemetry data for classification once our model is implemented using payloads with Azure. We could also send the same payloads for both collecting the training data and for the actual classification. The Machine Learning model for classification would be imported into a Jupyter/Python instance. I also found a project which utilizes Python notebooks and libraries along with AWS IoT to read data from sensors, and am spending more time doing trade studies between the two. I am more drawn towards AWS because of my prior experience with it, but doing research on AWS and Azure shows advantages for both. They are economical solutions that offer a lot of message-sending abilities from various IoT devices to their dashboards. The main tradeoff I envision currently is the difference in ML capabilities between Azure and AWS. While AWS is less friendly to beginners and more costly, Azure’s ML capabilities might be harder to implement with their IoT hub. My goal for week 5 is to create an instance with an Arduino board I have lying around and see if I can send some basic data about an LED light being on/off on Azure, on schedule with the Gantt chart presented during our proposal.