Posted on

Team Status Report for 9/30/23

One of the most significant risks in our project lies within the accuracy of our computer vision algorithm. We plan to combat this by using a suggestion box after analyzing a circuit – this box will display the five circuits that our algorithm has the highest confidence in. From these circuits, the user can choose which circuit is the one that they drew. By doing this, we are able to achieve higher levels of accuracy without needing to perfectly detect every component. We also are moving the labeling of each component value to the application instead of in the drawing. Rather than having users write the value associated with a component right next to it, we will have them enter in the relevant information once the circuit has been loaded into the applications. This way our computer vision algorithm will not have a hard time trying to figure out which value is associated with each component.

Changes were made in regards to our initial idea of creating a web application. After discussion, our group came to a conclusion that using a mobile application would make more sense given our specified use case of accessibility for middle schoolers. Due to this, all the work done on the web application will be discarded and next week Jaden will be focusing on learning and developing the mobile application instead. As around a week of work has been voided, more hours must be put in to make up the time that was lost over this past week.

We have also been spending lots of time on creating our design presentation – we made many changes since our proposal presentation that had to be added to the design presentation. While creating the presentation, we did lots of research on other computer vision algorithms that were used in similar applications to figure out how to justify our accuracy expectations. Between the team over 20 research papers were looked into in regards to this issue. Papers included sentence recognition, electronic component recognition, and other computer vision models. We ultimately decided that for computer vision we will utilize the Hough circle algorithm for image segmentation, ORB transform for feature detection, and a Brute Force matcher for feature matching.

We ended up making many changes to our project in the past couple of weeks that we believe will help mitigate risks that will occur with our computer vision algorithm and with having a complex backend. Even though we wasted a week on the web application, since we do not have to setup a complex backend we are still on track with our schedule and are making good progress with our project.

Posted on

Team Status Report for 9/23/23

One of the features that could jeopardize the progress of our project is our CV algorithm. Majority of our group’s time spent this week was researching and testing different types of image, feature, and edge detection algorithms on hand drawn circuit components. While we have made progress honing in on the type of algorithm we want to use, there is still a lot of uncertainty that needs to be solved. Since we have just been researching this week no changes have been made to our initial plan. The block diagram needs to be modified slightly. What we did not account for was our CV algorithm needing to interact with the database in order to pull training and validation data. This isn’t going to incur any extra costs or difficult refactoring because we knew we would be using a database anyways. Besides this, there are no changes to what we discussed in our proposal presentation.

 Next week we would like to start testing algorithms that can separate electrical components given a whole circuit and keep experimenting with individual component detection. When researching this past week, we realized that trying to extract sub-images of the individual components from a picture of the entire circuit is harder than we thought, as per Devan’s work this week. We will likely require dots/circles at the end of each terminal of a component to mark each individual component. At this moment we have not made any scheduling changes. Prior to making the schedule, we understood that since none of us have any CV experience that deciding on an algorithm would be paramount to this project’s success and so we have dedicated time to research. Given our current progression and the deadlines that are approaching, we are confident that we can stay on track to our initial plan and finish everything we planned to finish for the next week. 

We considered both public health + safety and economic factors as we developed our proposal. Firstly, our project reduces the safety risk that electricity poses to children trying to construct circuits. This is very important because parents can have peace of mind that their children will not be hurting themselves when learning about electricity. We also are aiming to reduce the cost that students and their families will incur by studying electricity and circuits. People will no longer have to pay 75+ dollars for a power supply, breadboard, wires, and resistors – instead they can just use our product for free.

Here is what comparing two drawn resistors looks like to the computer:

Posted on

Stephen Dai’s Status Report for 9/23/23

This week I researched different types of algorithms to use for feature matching in order to detect and identify individual electrical components. The two primary feature detection algorithms I considered were SIFT and ORB. From testing what I noticed was that ORB significantly handled variation in size and orientation much better than SIFT, and just had more features matching in general.

In the above photo you can see the results of using SIFT and ORB when comparing two images of resistors. ORB is clearly better with 132 / 381 matches, while SIFT only has 8 / ~48. Here is the visualization for the matches with ORB:

Note that the drawings of the resistors were first pre-processed with a contour detector that Devan found, which is why the background is entirely black and the contour of the resistors are in white.

When I tried comparing a resistor that was aligned vertically and one that was aligned horizontally, this was the result:

SIFT could not even generate any matches while ORB still could.

At this point I am confident that ORB is the algorithm we will use to generate FAST keypoints and BRIEF descriptors.

The one problem I have run into is that when using a resistor and voltage source drawing, the results of the matchings aren’t as polar as I expect them to be.

This is the result of comparing a resistor and voltage source drawing. If we compare to the matches of two similar resistors (first screenshot), we can see that ORB produces slightly less matches for the one with a resistor and voltage source. But, if we compare these results with two resistors of different orientation (second screenshot), they are pretty similar (~100 matches). While this may seem worrisome, I think that this is not too problematic because I believe the solution to this is to include various orientations of components in our training and validation datasets. This way, the matching of the same components in different orientations will be stronger and should contrast more with the matching results of two different components.

I am on schedule with my progress. I have established a good foundation with the algorithms to use for individual piece detection, and I want to continue exploring and tweaking to see if I can better improve the matching contrast with different components. Another thing I want to specifically try is using a different matching algorithm, as all of the above experiments used a brute force matcher. My steps for next week will be to create a reasonable feature vector with metrics that will be used to determine classification.