This week we spent most of our time on our design review presentation, finalizing parts and design decisions regarding interface communications, etc.  We met a couple times last weekend and once during the week to get these done and help Neeti rehearse.  Yesterday we met again to chalk out our design review report and get some of the things we could do before receiving parts done.  I setup a git repo and made sure we could all push and commit from both Pis.  Then I did some research into datasets available for gesture detection and found these: https://lttm.dei.unipd.it/downloads/gesture/ and http://www.nlpr.ia.ac.cn/iva/yfzhang/datasets/egogesture.html.  Because our gestures aren’t readily available, we might need to manufacture our own (this might not be enough data to train a neural net, but it also might be since we only basically need two gestures).  In our peer feedback, some people brought up the point that CNN might not be the most accurate model to do gesture detection and so I emailed Professor Savvides who taught us Pattern Recognition Theory last semester and would be the best person to advise us on this.  Then, I worked with Shrutika on getting a TCP connection between the Pis working and processing keystrokes using Pygame.

I think we are a little behind schedule because we had aimed to get our manual mode working by spring break, but spring break is next week.  We thought that since we’d ordered our parts last Monday, we’d have them by Friday, but there seems to have been a slight delay in ordering.  We will probably be able to get our manual mode (without gesture detection) working by this Friday if our parts arrive by Wednesday.  If not, I’m hoping we get the parts by the end of the week and then we might be able to realistically get manual mode working by the end of the first week after spring break.

I have two onsites though and will be flying out this Wednesday.  I will try to work on our project (specifically the ML models) a bit during spring break since I’m back on March 10th.  I’m not too worried about us catching up on manual mode, but I do think we have a significant amount of experimentation to do with the microphones and that we should get started on the in parallel in order to save time.

This week I mainly worked with Shrutika to try to figure out how to ssh into the Pi’s.  Initially, our plan had been to use the RPi GUI by connecting it through an HDMI cable to our laptops and then easily setup everything (in the way that the official RPi tutorials explained).  However, we weren’t able to find the right adapter/cable since the RPi 4 needs a micro-HDMI.  We didn’t want to waste any more time ordering parts and so I started researching how we could begin using the Pi without this cable.  I remembered that we’d ssh’ed directly into the Pi in embedded and began researching how to do that for our new Pis.

This turned out to be way more complicated than any of us expected.  The process differed from embedded because in that, we had already setup and tested UART before trying to setup wifi in the last lab and here we didn’t have that.  It was pretty easy to connect to raspberrypi.local with the Pi connected to my laptop through an ethernet cable, but then when we wanted to connect it to a wifi network the steps differed.  Once Shrutika figured out that it was possible to connect to her home wifi network, we assumed it would be easy to repeat the process on CMU networks.  That didn’t work.  I found out that there is apparently a small difference between the ethernet MAC address and the wifi MAC address on every device (and we’d been registering with the ethernet MAC addr instead of the wifi one).  Once we changed this, it took a day to register and then worked.

Apart from this, I spent some time researching the way we’d use the camera and OpenCV to do gesture detection (building an ML model, etc) and also briefly looked into the available GPIO libraries.  Since we took basically an entire week to get the Pis setup, we definitely underestimated the effort some things might take.  We are still not really behind since our main goal is to get manual mode working by spring break and I think we can still accomplish at least most of that.

Over the next week I hope to get our motor control working with Neeti while Shrutika simultaneously works on Pi-Pi communication.  We will also be spending some time on the design review report and presentation prep.