Niko’s Status Report for 3-21
This past week, I tried to adjust to the shift to online classes. I got back to Pittsburgh on Sunday the 15th, but spent a majority of the week packing up all my belongings. I plan to drive back home tomorrow (Sunday the 22nd) so that I can be with my family during the pandemic.
That being said, I worked with my team to create a modified statement of work (see our content page). This document serves to show how we have changed our scope and requirements to better fit the necessary shift to an almost entirely software project. Based off of our discussions, it seems that both my interaction layer and Richard’s webapp will remain largely unchanged, while Rip’s hardware will need some level of change. Prior to this change, my interaction layer was to interact with the hardware through an API that allows me to read sensor data, and write commands to control the devices. Since we no longer have hardware, Rip will recreate a similar API so that my code can continue to function as before, but he will implement a software library behind the API to emulate the hardware.
Niko’s Status Report for 3-14
From 3-4 through now, I have been traveling, and my laptop has been in the apple store. Unfortunately all apple stores nationwide have been shut down, and my laptop is currently still in the shop. I am looking into how I can get a replacement device.
Update: As of Sunday 3-15, the specific store that has my laptop will reopen for 24 hours for people to get their repaired devices. I now have my laptop and can continue working.
Niko’s Status Report for 3-7
From 2-29 to 3-2, I continued developing the prototype as discussed in my previous post, and did research based off of our design review feedback. I primarily looked into the nano board as an alternative to the raspberry pi. While it seems like it could be a viable alternative, we already have raspberri pi’s on hand, and with the unpredictable shipping out of China I believe it’s the best option to just move forward with the pi’s.
On 3-2 I had to send my laptop to the apple store for repairs, and so did not do any coding from 3-2 to 3-7.
Richard’s Status Report for 3-22
A lot has changed since in our project.
I was traveling to get home until Wednesday, March 18th. That morning, we had a meeting with the course staff to discuss changes in the course and guidelines on pivoting our project based on those changes. We do not know if we will get the hardware that we ordered before we left for spring break.
I did not make forward progress with the web application because we were trying to figure out how to rescope our project. I decided that it was more important to the project to put aside the web app for now and help redesign the project to our new constraints.
This puts me behind schedule, so I will have to use the extra time I have from practicing social distancing to work harder to get back on track.
Richard’s Status Report for 3-15
Up until Wednesday March 4, I made progress building out the React front-end. I added the interactions page and filled it with dummy data. This page shows the interactions between the devices. If an interaction is clicked, the user is redirected to a page that shows more details about the interaction. Here, the user can edit the interaction, adding/changing a device or condition. The site still looks pretty simple, but it’s starting to take shape.
The backend will be developed alongside the device systems. It will make calls to the SQL databases on the devices. Once the database table structure is laid out, the backend can be built out. Ideally, the device and backend will be built in parallel, minimizing confusion about models and api calls.
Teamwise, we ordered the parts we needed. I specifically do not need any physical parts currently to continue to develop the web application.
From March 4 to March 15, I traveled so I did not do any coding for this project.
Rip’s Status Report for 2-29
This week I took notes in the design review and worked on the design presentation. I’ve been thinking about some of the comments we got back regarding privacy too.
In our presentation, we claim that having all your data locally helps with privacy, but we don’t mention privacy again. We forgot about privacy for the most part during our initial designs and I’ve been thinking of ways to incorporate that into our design. What I think I’d like to do is encrypt the consumers data that is produced in our system with a customer key. That way, only the consumer that has registered these devices can access that data and holds all permissions.
Team Status Report for 2-29
This past week, we gave our design presentation and worked on the design proposal.
During this time, we shifted our design process quite a bit. In the beginning of this project, it seemed like we were coming up with an early design and then thinking about why it was good or bad. Now, our design process has turned into defining a problem, coming up with options to solve each part of the problem, then weighing the pros and cons for each of those options to come up with a design for the whole system. This has helped us a lot with justifying the design decisions we make.
After our design presentation, we received quite a bit of feedback from our peers and instructors:
- Think more about the security of the system, since that was one of the system’s supposed pro’s stated earlier.
- Think critically about the 3 second recovery spec in relation to a node in the system going down. Is this possible, given that the node killed was the “master node”?
- Consider alternative master node selection techniques than “lowest serial number”
- Must consider more granular testing of distributed system than “chaos testing”. Messages aren’t read, how will we know how things go wrong?
- If the master node fails, how will the user get to access the web application? Another node would have spun up the web app, but how will the user know that IP address?
- How will external devices gain access to our IoT devices, which may be behind a private network?
- How will we achieve pub/sub protocols while using the low energy requirements imposed by IoT device constraints? Specifically, we will look into ESP8266 & ESP32
Jiaqi actually came to us after the class period and talked to us, suggesting we look into a new device that a lot of IOT projects use. It is called NodeMcu, and has WIFI connectivity, processing power, and can host a simple server, serving a small web application. It could be a much cheaper alternative to using a raspberry pi. We are currently evaluating this option, seeing if it fits our needs and if we will have access to these units given the recent corona-virus events.
Even though this was a design phase, the team has spent some time experimenting with and getting familiar with the technology to build the system. This is important so we can hit the ground running when the time comes to implement the design. All three team members must be on the same page, especially about communication between the different components. A big part of the design doc will be how the data is stored and sent between each device. Planning this out ahead of time will make life easier as we implement each part and make debugging the system easier as well.
In moving forward and formalizing our design, we have carefully considered the feedback given to us. Integrating this feedback will allow us to produce a higher quality project. Reflecting on our feedback, we have realized as a team the importance of finalizing and creating a solid design. Without carefully constructing and reviewing our design in this phase of the project, we will suffer the consequences throughout the rest of the term.
Richard’s Report for 2-29
This past week, I was focused on implementing more of the Web Application that will be used for controlling the system devices and interactions.
Specifically, I fleshed out the interactions page. Each interaction is represented by a condition, a device, and an action. When users visit the interaction manager page, they can view their current interactions as well as define new ones.
As a group, we decided that we would host the web application on one of the raspberry pi’s. Since all our simulation devices are on raspberry pi’s, we thought it didn’t really matter which device the webapp was hosted on, so our delegation for which one actually hosted was pretty shallow: the host would just be the device with the lowest serial number in the system. After the feedback from the presentation on Monday, we realized it might be beneficial to have a slightly more involved process for deciding which device gets to host the webapp, especially in the case of a node going down.
In terms of our schedule, I’m slightly behind, as I have not begun work on the backend yet. The backend depends highly on the models that the databases on the devices will employ, so while my team is working up to that point, I will continue work on the frontend.
In the next few days, I want to mount the web application on a raspberry pi through docker, timing how long it takes for it to become live. I also want to see how automated this process could be, because while it ideally should be fully automated, I suspect it will be a little harder than it seems.
During the design review, our TA Jiaqi told us about an alternative to the raspberry pi board: a nano board often used for IOT applications. This board can connect via WIFI, has a processor, can host a simple server, and is a lot cheaper than the pi’s. We are considering switching to use this board, and if we do decide this, I’ll have to figure out how to mount the web app on this as well.
Niko’s Status Report for 2-29
This past week, I decided to use sqlite as the database for storing the local data on each device. We are aiming to make our code footprint as light as possible while still keeping the system fast, and sqlite helps to accomplish both those goals.
I also set up 2 raspberry pi 4’s that I have lying around at home. I had never set one up before, so it took a lot longer than I had initially expected. After getting all the parts I needed, I loaded up the Raspbian OS. I then connected them to the local network and set up sqlite on each of them, and spent some time figuring out how to read from and write to the database. After that, I figured out how to designate one of the pis as an MQTT broker, and then set up processes on both pis and on my laptop to act as clients. I was able to get all of the processes to subscribe to channels and receive published messages.
In terms of our schedule, I am a bit behind where I had intended to be. Most of this week was spent on design, and so I’m not as far into prototyping as I would like. I also had a couple projects and an exam this week for my other classes, which took time away from capstone.
In order to get caught up, I plan to spend most of Tuesday and Wednesday working on prototyping, since all my other work is due by Tuesday morning, and I leave for spring break Wednesday night.
The main thing I want to accomplish in the next few days is to finish thinking through our design. Our design review brought up a couple potential issues with our proposed solution that as a group we need to address. We still have time to do so over the weekend before our final design is due Monday night. I also would like to work on prototyping the system, and to have some barebones system functional.
Here are the problems in our design that we have found over the past week / were brought up in our design review:
- Look into nano board – often used in IoT applications, potentially powerful enough for our use case, and far cheaper than a raspberry pi
- Think more about security of the data, maybe encrypt data when on the the wire and in the db
- We would like to achieve maximum 1 second delay between device interactions. However, our master failover rate is 3 seconds – so in the case of a master node going down, that 1 second requirement is violated.
- Consider other master selection techniques than “lowest serial number”
- Maybe host broker on different device than webapp to take load off master
- Need more granular failure testing than just “chaos testing”. Things can go wrong in interactions between devices too, not just on device uptime.
- Hardest part about distributed systems is resiliency – should really think more about it
- After a device fails and a new master is
selected, need to think more about how devices will be forwarded to the new webapp host, since it will be at a different IP address. - How will devices outside of the network access the node hosting the webapp, since it’s in a private network? Look into static IP addresses
- Low energy pub / sub: ESP8266 + ESP32