This week our group spent most of our team researching and testing CV algorithms for image, feature, and edge detection. I spent time testing different algorithms with two hand drawn resistors, a horizontal and vertical wire, and a voltage/current source. My research included implementing mean squared error, Haris Corner detection, and template matching algorithms. Mean square error seemed to have the lowest error rate when comparing two individual components, however implementing with complex circuits would be too inefficient to run with the WebApp. My testing included taking and comparing each hand drawn electrical component previously listed and comparing the error rate with one another to see which pairings produced the smallest error. The algorithm did work well, however is not scalable with the size of the project. The other two algorithms were not as effective with single component detection and I therefore did not pursue any further research. From there I created the WebApp for our website and started working on the upload page. The upload page is planned to be completed by early this week and I plan for it to be completed by Monday evening. Currently I am on schedule as I need to complete the upload page by Monday and continue to help with research. Next week I want to look at number detection to help with other CV components outside the circuit itself, reading the values written next to each electrical component.
Devan Grover’s Status Report for 09/23
This week I explored computer vision algorithms to separate the components from the rest of the circuit. In order to properly do this, I came up with the idea to have circular nodes at the end of every component’s two terminals. Then, I will use an algorithm to detect circles such as the Hough Circle’s algorithm in order to find these nodes. Next, each node that is found will have an algorithm run upon it in which it will find the closest nodes in the same x and y coordinates. Then, a bounding box will be drawn between the sets of two dots and this box will be looked at to find a component.
I tried to implement this with a simple picture of a resistor but was unable to get it to work. The circles were not being detected in the image. I tried to run a simple filter on the image to detect the components being drawn. I made every pixel that had a black value of over a threshold value completely black, and every pixel with a lower black value completely white. This also did not work well in differing light conditions like the picture below.
Furthermore, the circles on the end of the resistor were not even detected. I believe I need to look further into the problem because we need to make sure we can detect components separately from the whole circuit. I tried a contour plot which detected edges relatively well, but the nodes on each component were not filled in because the contour plot only detects edges. This is problematic because we will have other circular components like voltage and current sources that we will need to ensure By individually identifying components and wires with this technique, we can easily build a netlist for our circuit simulator. I also briefly explored the concept of netlists and how they work so that we can implement it for our circuit simulator.
I am still on track with the schedule because I am still researching CV algorithms. I want to have a picture of a resistor with the circles on its ends being detected properly by our circle detection algorithm. This will prove that we can go on with the technique I discussed above to detect individual components.
Team Status Report for 9/23/23
One of the features that could jeopardize the progress of our project is our CV algorithm. Majority of our group’s time spent this week was researching and testing different types of image, feature, and edge detection algorithms on hand drawn circuit components. While we have made progress honing in on the type of algorithm we want to use, there is still a lot of uncertainty that needs to be solved. Since we have just been researching this week no changes have been made to our initial plan. The block diagram needs to be modified slightly. What we did not account for was our CV algorithm needing to interact with the database in order to pull training and validation data. This isn’t going to incur any extra costs or difficult refactoring because we knew we would be using a database anyways. Besides this, there are no changes to what we discussed in our proposal presentation.
Next week we would like to start testing algorithms that can separate electrical components given a whole circuit and keep experimenting with individual component detection. When researching this past week, we realized that trying to extract sub-images of the individual components from a picture of the entire circuit is harder than we thought, as per Devan’s work this week. We will likely require dots/circles at the end of each terminal of a component to mark each individual component. At this moment we have not made any scheduling changes. Prior to making the schedule, we understood that since none of us have any CV experience that deciding on an algorithm would be paramount to this project’s success and so we have dedicated time to research. Given our current progression and the deadlines that are approaching, we are confident that we can stay on track to our initial plan and finish everything we planned to finish for the next week.
We considered both public health + safety and economic factors as we developed our proposal. Firstly, our project reduces the safety risk that electricity poses to children trying to construct circuits. This is very important because parents can have peace of mind that their children will not be hurting themselves when learning about electricity. We also are aiming to reduce the cost that students and their families will incur by studying electricity and circuits. People will no longer have to pay 75+ dollars for a power supply, breadboard, wires, and resistors – instead they can just use our product for free.
Here is what comparing two drawn resistors looks like to the computer:
Stephen Dai’s Status Report for 9/23/23
This week I researched different types of algorithms to use for feature matching in order to detect and identify individual electrical components. The two primary feature detection algorithms I considered were SIFT and ORB. From testing what I noticed was that ORB significantly handled variation in size and orientation much better than SIFT, and just had more features matching in general.
In the above photo you can see the results of using SIFT and ORB when comparing two images of resistors. ORB is clearly better with 132 / 381 matches, while SIFT only has 8 / ~48. Here is the visualization for the matches with ORB:
Note that the drawings of the resistors were first pre-processed with a contour detector that Devan found, which is why the background is entirely black and the contour of the resistors are in white.
When I tried comparing a resistor that was aligned vertically and one that was aligned horizontally, this was the result:
SIFT could not even generate any matches while ORB still could.
At this point I am confident that ORB is the algorithm we will use to generate FAST keypoints and BRIEF descriptors.
The one problem I have run into is that when using a resistor and voltage source drawing, the results of the matchings aren’t as polar as I expect them to be.
This is the result of comparing a resistor and voltage source drawing. If we compare to the matches of two similar resistors (first screenshot), we can see that ORB produces slightly less matches for the one with a resistor and voltage source. But, if we compare these results with two resistors of different orientation (second screenshot), they are pretty similar (~100 matches). While this may seem worrisome, I think that this is not too problematic because I believe the solution to this is to include various orientations of components in our training and validation datasets. This way, the matching of the same components in different orientations will be stronger and should contrast more with the matching results of two different components.
I am on schedule with my progress. I have established a good foundation with the algorithms to use for individual piece detection, and I want to continue exploring and tweaking to see if I can better improve the matching contrast with different components. Another thing I want to specifically try is using a different matching algorithm, as all of the above experiments used a brute force matcher. My steps for next week will be to create a reasonable feature vector with metrics that will be used to determine classification.