Author: rdeng1

Richard’s Status Report for 4-26

This week, we did our demo to the professor and ta. This went pretty well, and while we were happy that it wasn’t a disaster, we were also aware of what we had left to do. We made a list of the things we still had to complete and got to work.

Since we weren’t planning on changing the webapp from what we showcased in the demo, I didn’t work on it. Instead, I helped Niko on timing the interaction layer stuff, so that we could have solid numbers for the presentation, and so we could back up our answers to the questions our peers and fellow students will probably have.

We discussed how to do this for an hour or two and decided that perhaps the best way would be to ensure the clocks on the nodes were synced, and log everything that happened. I’ll use timing an interaction as an example. We logged the start of the interaction, which is a value on a sensor changing. We also logged the end of the interaction, which is a value on a target device changing. Subtracting the log times of these events gives us a latency figure, which we intend to present tomorrow.

There are some caveats to this method: this latency is much lower on the virtualized aws network than it would be on a home wifi network. The network on aws is state of the art, probably 10 gb/s, while really good wifi networks can reach almost 1 gb/s. AWS machines themselves are also pretty fast, while the devices our system would have run on, raspberry pi’s, are significantly slower.

We feel that even though AWS technology is much better and faster than a home wifi network would be, by our estimates our technology would still fall well under the goal threshold we set at our design presentation.

Richard’s Status Report for 4-19

This week, we focused heavily on integration. Although Niko and I were already pretty integrated, we realized that there was actually a lot left that we had to do. There were around seven or six places that my code sent Niko’s library a string but he expected an integer, or something along those lines. We spent a few hours going through the whole system, debugging every single error. This was much more time consuming than I thought it would be.

Another thing that took a while for me was to add additional functionality. Niko’s library included code to delete and add interactions, which I didn’t have in my webapp. To ensure that Niko hadn’t wasted his time to write this code, I added in these features to the web application, so that his code is actually called.

Because I didn’t have Niko’s code running on my computer, I actually had to write test-dummy implementations of Niko’s library to test my code. This was very important and I think that it saved us hours on integration. It ensured that all the functions were getting passed exactly what they were expecting, and there were as few surprises as possible.

I think overall, we did a pretty good job with integration. I communicated with Niko very clearly, and even then still ran into multiple misunderstandings/miscommunications. This is a lesson to show how important communication actually is in group projects.

Richard’s Status Report for 4-12

This past week, we demo’d our project. We gained valuable feedback from the instructors afterward, and we will use this feedback to fuel our last sprint.

In the big picture, we have three parts to the project. My webapp is made so that Niko can implement whatever he needs to very easily. Basically, I wrote the backend in Python so that to integrate with Niko’s code, all the code has to do is import Niko’s library and call his functions.

As you may have seen during the demo, the three parts of the project are still quite a ways from becoming totally integrated. This past week, I focused on reading Niko’s interaction layer code and figuring out how what functions he wrote should be invoked in the Flask backend. This was tedious, because a bunch of the code was dispersed among quite a few files to look through. From here, all I have left to do is style up the frontend a little and test the webapp to see if there are any bugs. I will help Niko if he gets confused or Rip if he needs any help with his hardware web application.

For the next sprint, we will all have to work pretty hard. We want to have a successful project, and we’ll try our best to achieve that. To stay on schedule, I have to monitor Niko’s progress on filling out the backend with his function invocations and jump in if he has too much on his plate.

Richard’s Status Report for 4-5

This week, I set up the backend for the webapp. Originally, I had wanted to use ExpressJs as the backend framework because of how lightweight it was. We also all have a little bit of experience with this. However, this past week, we decided it would be better to switch this to a Python Flask backend.

This decision was made with ease of integration in mind. Niko already wrote a significant portion of the interaction layer code in Python. Porting this over to Javascript would have been a ton of extra work with no extra benefits. At the time of this decision the backend wasn’t written or connected to the frontend yet, so switching the framework for it wouldn’t have resulted in any loss of work or time.

This week, I set up the Flask backend and connected it to the frontend so that Niko could implement the API using the library he wrote. I tested the backend/frontend connection by having the API return “dummy” or “fake” data. I knew the whole webapp was working by deploying the webapp locally and checking to see if every page showed correctly, displaying the dummy data.

I’m on schedule, and to stay this way, I’ll have to keep working diligently. I’ll have to help Rip a little bit with his hardware web application, because I have a bit more background with this kind of stuff than him.

Team Status Report for 3-29

This past week, we had to redirect our project because of the novel coronavirus. As a team, we discussed and decided how our project needed to change, now that we can not meet in person.

The web application will not need to change much, if at all. The front-end remains essentially unchanged. The back-end now needs to make remote calls to the smart devices over the internet, instead of over the local network like we had planned in the beginning of the project.

Obviously, the hardware layer will have to change quite a bit. Essentially, we’ll emulate the hardware layer using software. We thought of a few ways to do this, including using QEMU software and another webapp. Ultimately, qemu introduced unnecessary complexity into the project, so we are going with the secondary web application. This was a difficult decision to make, but ultimately we don’t have many other options. Also, once this gets built, the software layer can be a good start to test the system if we want to build new features.

The Interaction Layer will also have to change significantly. The interaction layer, which acts like a translator between the hardware (now software emulated) and the webapp, will have to mounted on emulated hardware instead of real hardware. We don’t have experience with this, so we don’t know how difficult this could turn out to be. It can’t be stressed enough how important communication is for the development of the interaction layer. This talks to both the webapp and the hardware layer, so there are double the opportunities for misunderstandings and miscommunication.

Here is an updated GANTT chart for the rest of the semester:

Richard’s Status Report for 3-29

This week, we refocused our project. After meeting with the professor, we decided that the best way for us to go is to continue working on the project as per before COVID-19, pretending the internet is our local network.

My portion of the project, the webapp, was least affected by this. Therefore, I was able to write a significant amount of code this week. Specifically, I’ve added pages for individual interactions, forms for new devices, and a form for new interactions. I’ve also added links to the home page to a few other pages. This means I’ve just about wrapped up the frontend of the React webapp, and started planning out the backend with Niko.

I’m on schedule with regards to our updated schedule, and to stay on track I will have to keep working diligently.

Richard’s Status Report for 3-22

A lot has changed since in our project.

I was traveling to get home until Wednesday, March 18th. That morning, we had a meeting with the course staff to discuss changes in the course and guidelines on pivoting our project based on those changes. We do not know if we will get the hardware that we ordered before we left for spring break.

I did not make forward progress with the web application because we were trying to figure out how to rescope our project. I decided that it was more important to the project to put aside the web app for now and help redesign the project to our new constraints.

This puts me behind schedule, so I will have to use the extra time I have from practicing social distancing to work harder to get back on track.

Richard’s Status Report for 3-15

Up until Wednesday March 4, I made progress building out the React front-end. I added the interactions page and filled it with dummy data. This page shows the interactions between the devices. If an interaction is clicked, the user is redirected to a page that shows more details about the interaction. Here, the user can edit the interaction, adding/changing a device or condition. The site still looks pretty simple, but it’s starting to take shape.

The backend will be developed alongside the device systems. It will make calls to the SQL databases on the devices. Once the database table structure is laid out, the backend can be built out. Ideally, the device and backend will be built in parallel, minimizing confusion about models and api calls.

Teamwise, we ordered the parts we needed. I specifically do not need any physical parts currently to continue to develop the web application.

From March 4 to March 15, I traveled so I did not do any coding for this project.

Team Status Report for 2-29

This past week, we gave our design presentation and worked on the design proposal.

During this time, we shifted our design process quite a bit. In the beginning of this project, it seemed like we were coming up with an early design and then thinking about why it was good or bad. Now, our design process has turned into defining a problem, coming up with options to solve each part of the problem, then weighing the pros and cons for each of those options to come up with a design for the whole system. This has helped us a lot with justifying the design decisions we make.

After our design presentation, we received quite a bit of feedback from our peers and instructors:

  • Think more about the security of the system, since that was one of the system’s supposed pro’s stated earlier.
  • Think critically about the 3 second recovery spec in relation to a node in the system going down. Is this possible, given that the node killed was the “master node”?
  • Consider alternative master node selection techniques than “lowest serial number”
  • Must consider more granular testing of distributed system than “chaos testing”. Messages aren’t read, how will we know how things go wrong?
  • If the master node fails, how will the user get to access the web application? Another node would have spun up the web app, but how will the user know that IP address?
  • How will external devices gain access to our IoT devices, which may be behind a private network?
  • How will we achieve pub/sub protocols while using the low energy requirements imposed by IoT device constraints? Specifically, we will look into ESP8266 & ESP32

Jiaqi actually came to us after the class period and talked to us, suggesting we look into a new device that a lot of IOT projects use. It is called NodeMcu, and has WIFI connectivity, processing power, and can host a simple server, serving a small web application. It could be a much cheaper alternative to using a raspberry pi. We are currently evaluating this option, seeing if it fits our needs and if we will have access to these units given the recent corona-virus events.

Even though this was a design phase, the team has spent some time experimenting with and getting familiar with the technology to build the system. This is important so we can hit the ground running when the time comes to implement the design. All three team members must be on the same page, especially about communication between the different components. A big part of the design doc will be how the data is stored and sent between each device. Planning this out ahead of time will make life easier as we implement each part and make debugging the system easier as well.

In moving forward and formalizing our design, we have carefully considered the feedback given to us. Integrating this feedback will allow us to produce a higher quality project. Reflecting on our feedback, we have realized as a team the importance of finalizing and creating a solid design. Without carefully constructing and reviewing our design in this phase of the project, we will suffer the consequences throughout the rest of the term.

 

Richard’s Report for 2-29

This past week, I was focused on implementing more of the Web Application that will be used for controlling the system devices and interactions.

Specifically, I fleshed out the interactions page. Each interaction is represented by a condition, a device, and an action. When users visit the interaction manager page, they can view their current interactions as well as define new ones.

As a group, we decided that we would host the web application on one of the raspberry pi’s. Since all our simulation devices are on raspberry pi’s, we thought it didn’t really matter which device the webapp was hosted on, so our delegation for which one actually hosted was pretty shallow: the host would just be the device with the lowest serial number in the system. After the feedback from the presentation on Monday, we realized it might be beneficial to have a slightly more involved process for deciding which device gets to host the webapp, especially in the case of a node going down.

In terms of our schedule, I’m slightly behind, as I have not begun work on the backend yet. The backend depends highly on the models that the databases on the devices will employ, so while my team is working up to that point, I will continue work on the frontend.

In the next few days, I want to mount the web application on a raspberry pi through docker, timing how long it takes for it to become live. I also want to see how automated this process could be, because while it ideally should be fully automated, I suspect it will be a little harder than it seems.

During the design review, our TA Jiaqi told us about an alternative to the raspberry pi board: a nano board often used for IOT applications. This board can connect via WIFI, has a processor, can host a simple server, and is a lot cheaper than the pi’s. We are considering switching to use this board, and if we do decide this, I’ll have to figure out how to mount the web app on this as well.