This week I worked on with my group more design related portions of our project. We finished up the design review slides and started thinking about the design review paper. We also got some sensors in this week and have been testing them out.
I continued to work on the Unity – Web application portion of the project. Last week I worked solely on Unity to see if I can take in dummy data and make changes to the model (i.e. rotation, moving). These changes would be reflective of what gesture the user will be making on our trackpad portion of our project. We decided that the flow of our project would be that the Jetson Nano will send over data to our web application, then the web application will communicate via Javascript to our Unity application that is embedded inside our web app.
I created the web application in Django which will host our web application portion of our project. Then I embedded the Unity application from last week onto the Django web app. I wanted to figure out how to send serialized data from the web application to Unity. This is so that when we are able to get data from the sensors to the web app, we are able to send that information to Unity to reflect the changes. I researched into how Unity communicates with a web application (specifically django). So far, if i press a button which represent either moving up, down, left, or right, that will call the appropriate Unity functions I wrote to move the 3d model left, up, right, or down. Thus we are able to get basic serial data from a html page to the Unity application.
This week, I realized that the way I implemented rotation last week, requires the use of certain Unity functions that track the mouse deltas for you. The only problem is that the function takes in input whenever the user makes a mouse drag and calculates the mouse delta, however we need to be able to make rotations with data sets (x,y coordinates of sensors). I started to look into how we can replicate the effect of this function using just x, y positions that we get from dummy data (which will later be sensor x,y coordinates).
For the upcoming week, I am planning to write a script that will send dummy data via wifi to the web application and perform translations to the 3d model. I am not 100% sure, but I am thinking about also using the MQTT protocol much like Edward and Anushka did for the sending of data from sensors to their visual application. I will also work on looking into rotations algorithm more this week.