Category: Minji’s Status Reports

This week, we filmed for our final project video on campus. Jiamin had to bring parts of the project back home, so we wanted to make sure we have all the video clips before she left.

We also worked on our final presentation slides together. We collected some feedback from the TA, so we will work on finalizing the slides and preparing the script. Iris and I are also currently working on fully integrating the MySQL database to our web app, so it is capable of storing the log data we collect. We are also aiming to finish the details page before the final presentation if time permits.

This week, I worked on the IoT application to safely establish a full gateway to from the Nano to the backend of our web application. We used a demo application that used a Node.js backend (provided by Cindy!). We chose to deploy it on Azure App Service because it allows us to use the existing SDK to easily write programs to move data between the our IoT device and the web application.

Now that the demo app is safely deployed, next week I will work on actually deploying our own web application. I already have designed the wifeframes for the pages so I have work on coding up the interface and connect it to the backend now.

This week we had a demo to show our project’s progress so far. We were able to showcase the basic functionalities of the RFID scanner, face/mask detection, temperature collection working separately. Moving along, we are going to finish up integrating the systems and switching the gear to work on the IoT solution.

We received some guidance from Cindy regarding the IoT side. We are using an existing IoT application (from popular tutorials) to first have a demo app up and running. This allows us to first configure the pipeline between the Nano and our application’s backend, so we can work on the actual application interface afterward. Right now, we are successfully sending the temperature and RFID data to the hub and is able to use Azure’s SDK’s to write a script to forward the data to a specific endpoint. We’re still in the process of configuring the application back-end’s endpoint to receive the data. We’ll continue to work on that and hopefully finish up setting up the pipeline by the middle of the next week.

We also had a meeting with TechSpark to discuss possible encasement designs. We weren’t able to have an actual meeting to discuss the specifics since TechSpark was closed for a bit due to a positive Covid case. 🙁

We have brainstormed some ideas and also ordered an existing enclosure for the RPi camera. We are planning on setting up another meeting to see if we can have a prototype ready in a week or two.

This week, Iris and I worked together to debug the face/mask detection program and add the IR sensor module to the Jetson Nano. The original program that we were trying to run for face/mask detection uses OpenCV for the analysis. However, we found out that the Jetson Nano doesn’t support the OpenCV libraries that are required for the program. As a result, the program wasn’t being optimized when run on a Nano and the frame rate was extremely slow.

We had to look for a different library to use for our face/mask detection and decided to try out YOLOV3. This is a more lightweight library that can often be used to run computer vision programs on a Nano with a decent frame rate (10~13 fps). We downloaded all the necessary packages and are currently waiting for the training step to finish (it usually takes ~90 minutes).

We ran into many problems when using the MLX90614 IR sensor. It requires an I2C connection with the Nano, so we hooked up the I2C SCL and SDA pins to the Nano’s corresponding GPIO I2C pins. However, we found out that the Nano wasn’t able to detect the I2C connection to the sensor when we tried running the i2cdetect command. We decided to go through several steps of debugging: (1) check if the sensor works with other boards (Arduino Uno, RPi), (2) check if the sensor is getting powered properly by the Nano’s 3.3V output pin, (3) check if the sensor has enough current flow. We went through all of the steps above and concluded that there were no issues with the sensor or the sensor’s power connection/current.

After some research, we found out that it might be an issue with the i2cdetect program not recognizing our specific sensor’s I2C connection. We decided to write up a quick script that tries to read bytes from the I2C bus at a specific address. This was possible because we knew that the sensor’s I2C address is at 0x5A. After executing the script, we discovered that we were actually reading valid data from the sensor! So we were able to produce valid temperature measurements from the MLX90614 sensor and output it from the Nano. The demo video of this short script can be found in this week’s team status report.

This week, I was able to open up the cloud gateway to allow cloud communication between the Jetson and the Azure IoT system. I used the IoT Hub’s built-in endpoints to send sample strings from the Nano and checked if those messages were being received by the hub. I successfully ran the test script and checked that all of the messages were being delivered in real-time. I also worked on configuring the Jetson and have it registered on the Hub so I can remotely monitor the status of the device.

Next week, I am going to finish building the skeleton of the app and draw out some sketches of the wireframes. This will allow me to get a headstart in the app development process.

 

This week, I focused on thinking about the database and designing the model for storing the temperature data. I will basically need two core tables — one for a community/organization member’s profile and one for recorded temperature. These two tables will have a one-to-many relationship. The profile record will consist of the RFID tag identification code and extracted profile information such as their first and last name. The temperature record will have the temperature, time stamp, and the location of the temperature measurement. The location is necessary for our core use case, so multiple devices can be installed throughout the building to keep track of who was in which room/area of the building. I started building out the model on the back-end side of the application using Django.

I am also preparing to start opening up the cloud gateway for receiving messages from the Nano. Next week, I’m going to work with Iris to connect the Nano to the internet using the wifi module that we already installed. We’ll test out the message transmission by sending simple test messages and see if we actually receive them on the IoT hub’s end. We aim to be able to send and receive messages between the Nano and the cloud by the end of next week.

This week we spent most of our times working on the design review document. We received many feedbacks after the design review presentation and took them into consideration when constructing our design review document. It helped us to think more about the metrics and validation, especially regarding the accuracy of the facial and mask detection.

This week, I worked on installing Azure IoT Edge on our nano device and configuring it to be registered on our IoT Hub. For now, it seems like we don’t need extra credit for Azure IoT for testing and validating. However, if we need more quota for future testing purposes, we should ask Vyas or Cindy for credit. I also started going through tutorials for building a full-stack web application using React.js and Django. Next week, I will start working on the skeleton of the application and also design the database model for storing the temperature records.

This week, Iris and I got together to install the WiFi module in the Jetson Nano. We also noticed that we’re missing the microSD card, which is required to install Linux on the Nano, so we placed a new order. Once the card arrives, we can continue to install necessary modules in the Nano for IoT and face/mask detection.

I did more research on Azure IoT Edge and IoT Hub products. Installing Edge on Nano will be pretty simple, and I’ll be able to configure it to be used as an Edge device. For IoT Hub, I will register the Nano as a connected Edge device. All of this can be easily done once the SD card arrives.

We spent most of this week working on the Design Review. We had to make more detailed design choices for the software stack, and it was helpful to have a detailed implementation before moving forward in the project. We also figured out the logic flow of the Nano once it receives inputs from the RFID scanner, the IR sensor, and the RGB camera. Next week, Iris and I will work to configure the Nano to receive inputs from all three sources and also register as an edge device on IoT Hub to open the Cloud Gateway.

I started to do research on the general structure of an IoT application. Below is a summary of some of the things I learned this week.

Major components of an IoT project:

• IoT Hardware Device — a physical device that interacts with the environment
• Connectivity — the link between your device and the cloud
• Product Cloud — servers that take in and process data

We can reflect this structure onto the scope of our project:

• IoT Hardware Device — Jetson Nano that is connected to the RFID scanner and IR/RGB cameras
• Connectivity — WiFi, by attaching the WiFi module to Jetson Nano to enable internet connection
• Product Cloud — Microsoft Azure IoT Suite

The decision for the product cloud wasn’t very obvious. There are many cloud solutions available, but I found out that most of the community IoT projects that use Jetson Nano utilize the Microsoft Azure platform.

Next week, I am planning on looking more into the language to use for the software platform. I am currently leaning towards using full-stack Java for software development.

Helpful Resources I Found:

https://dev.to/azure/getting-started-with-iot-edge-development-on-nvidia-jetson-devices-2dfl