Weekly Status Reports

Unfortunately we ended up making the decision to not implement diodes in our project. Devan was having too much difficulty with the implementation. The positive side to this is that our circuit classification accuracies have gone up, because previously there were some cases where switches would be classified as diodes. The component classification accuracy has stayed the same at around 83%, and the circuit classification accuracy is around 86%. Because we removed diodes, we have to redraw test images that were used that contained diodes. We plan on continuing testing a lot tomorrow and next week, just to get plenty of data to use in our report.

 

Computer vision unit testing:

• Individual component classification testing
  • Tested different image preprocessing algorithms and parameters
  • Tested how classification performed with various implementations
    • With special casing on wires
      • Determined that this was highly effective in classifying wires, which allows us to remove wires from the dataset so we don’t even have false positives of wires
    • With varying numbers of matches considered (ex: only considering the top 20 matches of features between the component image and a dataset image)
      • Determined that considering all the matches and not leaving any out had the highest classification accuracy
    • With different feature matching algorithms
      • Determined ORB feature vectors + BRIEF descriptors was the best combination
    • With circle detection to separate voltage+current sources and lightbulbs from other components
      • Determined that the circle detection was beneficial and worked well in identifying voltage+current sources and lightbulbs
      • Interestingly resistors would also get detected to have a circle, but this ended up being fine because in the feature matching there would be a clear difference between the resistors and circular components
    • If a component’s best matching score doesn’t reach some sort of satisfactory threshold, rerun the component matching with all component types in consideration (redo without circle detection)
      • Determined that a consistent threshold could not be determined and that the circle detection was more accurate
  • Tested with various sizes of the dataset
    • Notably as long as there is at least one image of a component in each of its possible orientations, the classification accuracy was similar to when having multiple images.
    • Need to test more with this
• Node detection testing
  • Tested different image preprocessing algorithms and parameters
  • Tested images with different lightings and shadows
  • Determined that even with improper lighting, the node detection works well as long as the nodes are drawn large enough and are properly filled in
• Full circuit classification testing
  • Tested complete computer vision code with circuit images of various number of components (4 to 10)
    • Determined that all incorrectly classified circuits were because of poor component classification, not because of failure of node detection

Circuit Simulator Testing

• Generate netlist with maximum of 8 components
• Run netlist through existing simulator tool
  • CircuitLab
• Compare results of simulation 
  • Voltage at every node
  • Current through each component 
• Tested 25 circuits
  • Voltage Sources, current sources, resistors, light bulbs
• Current and voltage were correct on every circuit
• 100% simulator accuracy given a netlist

Usability Testing

• Surveyed 7 individuals ages 12-14
• Gave them circuits to draw
• Answered questions on a scale of 1-10
• “How easy was it to upload your circuits?”
  • Average score 10/10
• “Was it clear how to input values?”
  • Average score 6/10
  • Found confusing which component they were inputting values for
• ““How useful were the tips on the home screen when drawing your circuit?”
  • Average score 7/10
  • Found example drawing of circuit most helpful
•  “Were the headers of the page useful when asked to do complete some task”
  • Average score 9/10
• “Do you think adding more tips and headers would make it more clear”
  • Average score 7/10
•  “How clear were the schematics of all the circuits displayed?”
  • Average score 9/10
•  “How easy was it to recognize what you needed to redraw with your circuit if it wasn’t an option?” 
  • Average score 7/10
• Average score of survey 7.85
• Working on implementing a more clear way to input values for components 

Stephen managed to get the individual component classification accuracy up to around ~85% in testing, and is still in the process of measuring the full circuit accuracy. The size of the dataset has increased to encompass 56 component images, but in terms of file size it is actually still 3 MB, which is what we predicted two weeks ago. In addition to this, because Stephen switched to using BRIEF descriptors, which are only available with opencv-contrib extra modules, this is an added step for the build process of the integration. This week we were able to completely finish integrating the entire project on one codebase. The CV and the phone app are correctly linked to display the circuit that the CV algorithm recommends and the circuit simulator is able to run with the inputs the user provided. All that is left to be done is to parse the output of the circuit simulator and display it on a final page, however this will be less than a day’s work as we have already parsed the output of the CV code which was far more challenging. In addition to this, the display of the circuit has a couple bugs that need to be worked on, but this is also expected to take around a day. Jaden is planning on finishing both these parts by Tuesday. 

Now that most of the integration is finished, Devan is finishing up working on the model for the diode. Once these parts are finished, the project will be completed.

One change that has an associated cost is the memory usage of the dataset file. Stephen created a YML file with feature vectors representing 27 component images that had a size of 1.4 MB. Our ideal dataset size would comprise 54 component images, which would correspond to a YML file size of ~3 MB. This is bigger than our previously anticipated 500 KB size for 50 component images, but because our maximum application size is set as 100 MB, we don’t expect this 2.5 MB difference to be a problem. In addition, there were some changes regarding how the user will input the component values. When showing progress there were concerns of the UI and it being tailored to a younger audience. Now rather than the user clicking on components to input values, they are presented with text of each component name to click on to input values. We expect this will be more user friendly and will cause less confusion.

We also worked a bit on integrating the computer vision and circuit simulator components of our project. Our circuit simulator is now directly able to simulate a circuit from an image generated by the computer vision algorithm. We are still unable to get the computer vision library to build for XCode though, which may mean we will have to use the Intel Mac lab.

Overall we are more behind schedule than we would like, but hope to make this up over next week’s break. Our goal is to have the full integration complete by the time we come back from break, which gives us time to work on final documentation during the first week back. 

We have all developed lots of skills that we did not previously have by working on our application. Stephen had never used computer vision before, Jaden had never coded an iOS app before, and Devan had limited c++ experience. By working on this project, we were able to develop our skills and become better engineers. From a logistical standpoint, we have learned to work well together by giving each other status updates often to mark our progress and make sure we are relatively on track. This also allows us to pass ideas between each other even though our work is very compartmentalized. Planning the gantt chart also helped a lot because we are able to manage our time better and not spend too much time thinking about what to do next after completing a task.