Posted on

Weekly Status Report #2: 9/22-9/29

Celine:

The first half of this week I spent researching cameras that can interface with a raspberry pi, and ordering the camera I found to be sufficient for our initial experimentation, other parts we as a team found to be necessary for an initial implementation, and the connecting cable for the camera. I found 8MP and 14MP Arducams, both of which can connect to raspberry pis. However, the 14MP camera requires a USB interface board in order to be able to connect to the pi. As the most recent USB version implemented for this camera is a USB3, and no raspberry pi currently supports USB3, I decided that it would be better to stick with the 8MP camera for now. After we buy the raspberry pi and gauge our budget, we can determine if a camera upgrade is worth it. Additionally this week I have been setting up OpenCV on my personal laptop. I will be testing the setup this weekend.

Next week, I hope to have tested out using OpenCV with some pre-trained Tesseract net. I also hope to have laser cut a duplicate gear to complement the one Indu ordered this week. We will be aiming to have the chain belt mechanism implemented this week, as we have the necessary parts ordered for it already.

Effie:

This week I looked further into some hats and began designing out how the pin connections will need to be laid out to properly connect the pi to some encoder motors and servos and the camera and a speaker and mic, and I looked up details of how to install drivers. We ordered some more parts and this week plan to order the remainder to get to testing initial ideas.

Indu:

This week I continued researching the methods used in page turning devices and worked on deciding what parts would be used for testing purposes. I also ordered some of these parts, and have an initial idea of the next set of parts I am planning to order next week, such as the motor that will drive the device.

I have only initial calculations for the page-turning device, such as for the conveyor chain being around 3 feet long in the final device. I need to further work on these calculations, to better design a schematic for the project. That is what I plan on doing next week. I also want to actually start building a test device next week with the parts that Celine and I have ordered.

Posted on

Introduction and Project Summary

Project Presentation Immediate Alert Cystem (Team 9) NarrAUTOr Proposal

NarrAUTOr is a reading robot assistant able to help in a variety of situations, whether it is used by a child, a student, or a person with a disability. It incorporates mechatronics, hardware, image processing, and computer vision in order to turn the pages of and read aloud the book you would like to hear.

The setup of the device itself will include multiple components controlled by a portable computing unit; two page scrunching mechanisms that uses friction to create a loop at the base of the page, where the page connects to the spine of the book,  corresponding to the direction (forward or backward) the user would like to flip; a page flipping mechanism that flips the scrunched page; a camera sitting atop a stand that points down at the book and snaps a picture of it; and a computer vision program that takes the picture of the open pages, translates the image to text, and reads out the text.

While there are existing devices with similar function to the one we are proposing, our implementation takes the best features of the ones we have seen and combines them into one.  The feature that sets our project apart is how we will include an algorithm that determines when after scanning a set of pages to turn to the next set of pages, so that there is no lag in the reading of the book to the user. An additional good feature is that we will be able to flip pages backwards, so that users can review what was previously read; to prevent the need to re-scan pages as the user goes backwards, we will enable Narrautor to save the last few audio files produced from previous pages.

In addition to this post, we have uploaded our initial project report.  It is important to note, that this report’s content is actually different from the project description given in the previous paragraph.  This is due to our team deciding to pivot from that project idea to this new one last Friday. Our initial project idea that we proposed to the course staff, was a real-time bus detection system that would use computer vision from camera data along Forbes Avenue to find a Pittsburgh Port Authority (PA) bus (specifically any of the 61s) coming to a nearby bus stop and notify users that it was coming within a few minutes.  Some explanation of why we pivoted from this idea is noted in the following bullet points:

   – learned that there was currently data like this being observed by CMU’s Traffic21 group, so contacted them to see their data, which they told us we could use for this project

   – spoke with one of the researchers, saw their data, and went to see the cameras at their location on Friday, September 14th

   – The camera data was not at the resolution we needed (was too low) as while we were able to detect a PA bus, we were not able to distinguish what type it was (e.g. a 61A).

   – Apparently they were only allowed to be on the open WiFi, but that sometimes lost connection, and they had to consistently make sure that the camera had power.

   – While we would have been able to use Traffic21’s data, after seeing it and their cameras, we determined that we would need different cameras, and more of them since we wanted to be able to track buses in advance.

   – The cameras they used and the setup of it, was around $200 for one camera (we believe), so if we wanted 3 extra cameras we would have already exceeded our budget in that area with nothing left to spare for other items.

   – Overall, while the idea seemed cool, and we were glad to have learned of a group on campus that is doing something similar, we decided that it was not a good idea to implement this semester.

 

Due to this pivot, we have also uploaded our Project Proposal Presentation, which was given on Wednesday, September 19th to the 18-500 class, and details the idea for project idea NarrAUTOr.

Posted on

Weekly Status Report #1: 9/15-9/22

Celine:

This first week, I mostly worked on the project presentation that we had to give on Wednesday. I spent most of my time investigating how the components that go into our project fit together and the details of what the computer vision and image processing components of our project will look like. I researched into if we would need to create our own dataset, resources for helping us determine the extent to which we would be able to use OpenCV and Tesseract to perform the text recognition, and setting up OpenCV.

In this coming week, I will be installing all the necessary components required for running OpenCV on my laptop, then beginning to write the image processing framework that our page images would be passed to from the camera. I would also like to try testing out use of Tesseract.

 

Effie:

After pivoting our idea last weekend I wrote up the new project proposal and spent most of this week working on parts of and editing our presentation slides, and researching what components we’d need and how we might go about integrating them. After meeting with Sam earlier this week and discussing Raspberry Pi hats I spent time compiling a preliminary parts list including a raspberry pi, motors + chain (for conveyor belt + lifting pages mechanism), camera, camera extension cord, speaker, mic, breadboard, cables, and some hats/breakout boards to be able to get this all going!

In the coming week, I plan to finalize the parts list and put in some orders  and begin research on how I would load drivers for the components so I might be able to already have that ready when they arrive.

 

Indu:

This week I researched implementations of the page-turning device (my links of this research are some of the ones in slide 5 of the Project Proposal Presentation).  I spent most of the week working on the Project Proposal Presentation that we gave on Wednesday’s class. I have also looked into potential parts to buy for the construction of this device, and how people have constructed projector stands, as that is the design idea I want to use in building the stand for the camera.

Link showing projector stand setup (specifically looking at for how pivot works in stand):

https://www.youtube.com/watch?v=PViAHD1t5zg

We are a little behind in terms of ordering parts, which we are attempting to remedy on Monday, by deciding to purchase some initial smaller parts so we can test them out before we construct the actual page-turning device.

By next week I hope to have a partially completed technical design of the entire page turning device and camera stand.