So far I worked more on the Unity-Web application side of the project. I wanted to test calling Unity functions would work with streaming data. This is to mimic the data that would be streamed over from the Jetson Nano to the Web Application. I decided to use the MQTT protocol to stream over data from a Python script. I used the MQTT Eclipse Mosquitto broker. The data will be sent from the Python script to the Django Web application in the format of a string. For now, I made it so that it sends over fake dummy data that contains the gesture name, and the x and y coordinates for testing.
I wrote a script on the Django application to call a Unity function I wrote called “Rotate” every time a new data string was published onto the broker. When Rotate is called, it will parse the data that was just sent in. When I tested with dummy data, at first I ran into some issues with the model getting stuck after a certain degree change. However I was able to fix that by getting rid of a rotation limit that I incorrectly wrote initially. As of now, the model is able to rotate in the right direction (up-left, down-right, up-right, up-left) based on the data. I am still working on the algorithm to translate the data that is streamed in, into rotating at more precise angles.
At the same time, I am also looking into how to perform zoom operations with another set of dummy data. I have gotten zooming for a single view of the game object done through script, now I need to see how to access all four cameras (which is needed to get all four perspectives of the 3d model), and zoom in on all four cameras. I also am planning this week to see how to correlate our data that would be streamed in to how fast the zoom will occur.
This week, I hope to finish up a basic working of the zoom and rotate functionalities based on dummy data that is being streamed in. This way we can start trying to integrate in the later part of the week.