Category: Team status reports

Echo & Page:

• Finished testing our completed version of the project against our metrics.
• Filmed videos for final video
• Decided on division of labor on the final video

We have finished the project on time. Started working on the final video and report.

Yuhan:

• implemented a logging feature
• implemented user login, so users can now only view their own private alarms, but both public and their own private ringtones
• implemented alarm & ringtone deletion
• deployed website using pm2

I am currently on schedule.

The website is currently alive at http://ec2-3-129-61-132.us-east-2.compute.amazonaws.com:4000/

Echo & Page

Implemented a script on the Raspberry Pi and achieved communication between the back end of the web application and the Raspberry Pi. The Raspberry Pi now will receive a message from the back end at user specified alarm time. It will also receive and play the user specified ringtone after activation.

Implemented wall detection in our final script. The robot will halt and get ready for the next request from the server after detecting a wall based on our own algorithms.

A portable power supply is tested to be working and we are now able to put everything onto the robot and start final optimization and testing.

We are on schedule.

Yuhan

This week(week of 11/9 – 11/15):

• prepared for interim demo on Wednesday
  • cleared and prepared sample alarm and schedule data
• deployed current implementation to EC2(Ubuntu)
• tested code snippet on Pi
• supported Echo and Peizhi in the integration of the code snippet into the main program running on Pi
• set up access for Echo and Peizhi for easier debugging

Next week(week of 11/16 – 11/22):

• continue testing communication between Pi and web app
• modify communication code snippet on RPi so it writes to files instead of printing to stdout(to keep a record of messages & when they are received, to validate reliability of the alarm scheduling)
• implement alarm and ringtone deletion
• research on and implement database change

I am on schedule.

Problems: refer to personal update.

Echo & Peizhi

This Week:

• Installed heatsinks and a 3.5-inch screen that’s mounted in the shroud for our raspberry pi 3.
• Implemented wall-detection algorithm through sensor data and self-developed algorithm
• Optimized obstacle detection and avoiding algorithm

Problems:

• Purchased wrong product. Expected a power bank, turned out to be a power adaptor
• Displaying captured video on the 3.5-inch screen, won’t fit even with 100×150 display window

Next week:

• Integration for web application and hardware platform
• Further optimization & Mounting Pi and camera onto the base
• Purchase power bank

We are on schedule.

Yuhan

This week:

• implemented backend with MongoDB
  • web app data pulled from & stored to cloud
• implemented communication between web app and Pi with SQS(message queue)
  • send messages on web app backend(javascript, scheduled by node-cron), receive and delete messages on Pi(python)
  • did not test it on Pi, used local machine instead

Problems: Refer to my personal update.

Next week:

• prepare for interim demo on Wednesday
  • deploy current implementation to EC2
  • clear and prepare sample alarm and schedule data
• continue testing communication between Pi and web app
  • test code snippet on Pi
  • handle connection errors gracefully, especially for receiving messages
• support Echo and Peizhi in the integration of the code snippet into the main program running on Pi
  • figure out a way to host the web app with minimum down time, and maximum secure access to EC2, MongoDB and SQS for Echo and Peizhi for easier development and debugging

I am on schedule. For the plans for the next few weeks, refer to my personal update.

Peizhi & Echo:

we almost finished all hardware portion of our project. Right now, we are on schedule.

This week:

• finished integrating human avoidance algorithm with obstacle avoidance algorithm
• finished tuning parameters for the robot’s moving speed. (might adjust later)
• started looking into sockets and how raspberry pi communicates with webapp

Next week:

• mount hardware on top of the robot base
• finish the very last “alarm goes off” algorithm when the robot hits the wall
• integrate with Yuhan’s webapp
• optimize our algorithm
• find solutions to the case when sensor data received from robot blows up

Yuhan:

This week:

• researched possible ways of real time communication from web app to Pi
  • tried to work with just MongoDB and failed
  • looked up azure Iot hub, RabbitMQ, etc
  • settled on AWS SQS
• wrote pseudo code hosted on both Pi and the web app/remote server

Next week:

• borrow a Pi
• Turn pseudo code into actual code
  • test MongoDB works on remote server (for permanent storage)
  • test AWS SQS works from remote server to Pi (for real time communication)

Peizhi & Echo

We are about 1 week ahead of our schedule.

This week:

• Obstacle detection & reaction
• Rotation when idle/lost track

Next week:

• Spot issue with Pi camera (likely burnt), purchase new one if necessary
• Purchase Raspberry Pi tool kit and mount the Pi + camera on the robotic base
• Integrate the obstacle detection script with our main Computer Vision script

Yuhan

My schedule is unchanged and posted in my individual status update.

There is no significant change to the web app system design either.

Refer to my individual update for screenshots of the web app.

The current priority/risk is to integrate Pi with web app. I will be focused on it next week onwards. To mitigate, My plan is to go over the following test cases:

1. send data for one alarm to Pi, and see:
   1. if the robot base can be activated within a reasonable latency/in real time;
   2. if the robot base can play the ringtone sequence correctly.
2. schedule one alarm, and see if the robot base can be activated at scheduled time;
3. schedule 3 alarms in one row;
4. schedule one alarm 3 times;
5. schedule one alarm with a specified interval, and see if the robot can be  repeatedly activated reliably.

Peizhi & Echo:

We are ahead of our schedule.

This week:

• the robot now is able to perform “run-away-from-person-action” correctly while keeping our frame rate still at a minimum of 10 fps.
• wrote design report
• made the robot keep playing song while moving

Next week:

• figure out how robot should move when obstacles are encountered
• when in idle stage (when no person is detected), the robot self rotates until person appears in view
•  how to physically mount raspberry pi, robot base, usb acceleration and all other parts together as one piece of product

Yuhan:

The most significant risk in my project is how communication between Pi and web app should be scheduled. The most straightforward approach it seems is to use weekly “schedule” command from the Open Interface in the control while loop of Pi. This way the web app only has to communicate to the Pi once(e.g. “alarms at 8am every Monday”) and iRobot will repeat the alarm routine every week. However, I am not very sure if pycreate2 supports the schedule command. I am also not sure how customized the weekly alarm routine can be and if it fits our requirement(e.g. we need the routine to include actuator commands as well as song commands).

A contingency plan is to leave it to the web server to schedule the alarms, i.e. “alarms at 8am every Monday” = web server sends signal to Pi at 7:55am on 10/19, then again on 10/26, etc. I think it requires a more complicated logic that might get buggy more easily and harder to maintain. Right now I am mostly focused on frontend development and then connecting it to the database and will deal with it as it comes later on.

My updated timeline(same as my individual status report):

• week 7(here):
  • web app build and deployment set up
  • spike on communication between web app and Pi
  • create alarm UI
  • alarm schedule UI
• week 8:
  • create ringtone UI
  • pass alarm data from create alarm page to alarm schedule page
  • pass ringtone data form create ringtone page to create alarm page
• week 9:
  • backend set up
  • connect UI with backend
  • connect web app with Pi
• week 10(demo)

Peizhi & Echo:

We are ahead of our schedule. we originally scheduled to move on to hardware portion starting next week, yet we already made significant progress on this part during this week.

This week:

• finished optimizing TensorflowLite on raspberry pi and successfully reached 10 fps
• made raspberry pi control the movement of the robot base. Now it can perform simple instructions such as move in any direction and play song
• robot base was able to move according to the person that is detected through pi camera and CV algorithm on pi. Algorithm needs to be optimized next week

Next week:

• program the robot to perform “run-away-from-person-action” correctly while keeping our frame rate still at a minimum of 10 fps.
• program the robot to move in a way to avoid all obstacles
• try figuring out the hardware part, that is, how to physically mount raspberry pi, robot base, usb acceleration and all other parts together as one piece of product

Echo& Page

We are on schedule.

This week:

• finished researching and testing out different CV algorithms on PC
• installed OS onto raspberry pi 3 and successfully ran Tensorflow lite

Next week:

• optimize Tensorflow lite on raspberry pi to reach our intended detection speed and accuracy
• finish up all CV part and start looking into the hardware

 

Yuhan