Category: Jaden D’Abreo’s Status Reports

This week I finished the iOS application, conducted usability testing, and tested the full pipeline of the code. I was able to finish the iOS application early and the week which allowed me to schedule interviews throughout the week to conduct usability testing. The surveys went well, however there is a clear problem with the way we ask users to input values for the circuit. The overwhelming response was that they found it confusing which component that were inputting a value for. Therefore, I plan to try and incorporate adding a number below or next to the components that correspond to the label in the input field. Hopefully by doing so this would cause less confusion. I have asked a couple individuals from the test group to meet again early next week and have them answer the question regarding inputting values once more. The usability results were slightly lower than we wanted, however making the changes to the inputting page should get us to our goal comfortably. The pipeline is fully functional, there are some minor changes that need to be implemented. When the user uploads a bad picture, instead of not doing anything I plan to add some text saying the picture was not readable. Apart from this there is not much else that needs to be done. I plan on adding these minor changes tomorrow and begin working on the design report. 

This week I made final touches to the phone application and started integrating all the subsystems together. Integrating was a bit challenging, however it was completed! Dev and I were able to integrate each subsystem together and now the project can be completely run within one codebase. As this was our first time working with Swift, the process took a lot of time to debug. Dev and I were first able to complete integrating the phone application and the circuit simulator, therefore I was able to model his header and bridge file to model my integration with the computer vision. Thus, once completing the first integration, the second integration was a bit more straightforward, but still challenging. There were a lot of additional things needed to be added to the codebase to make the CV and the phone application work together. Bridging these two systems together required saving the state of file paths, adding extra parameters, and modifying both existing codebases, the phone application and the CV, to function properly. However, there is still one bug that needs to be fixed. The circuit does not display as it did with the hard coded coordinates due to a slight flaw in logic. However, this is not a very pressing bug and should be resolved within a day’s work. The correct components were placed on the screen, just not in the correct position, and I have located the flaw in the code. In addition, I have to display the contents from the circuit simulator, however this will also be even less work than the other fixing the bug. I am slightly behind on my work, I expected to be completely integrated and finished with the phone application by this week, however with the final presentation tomorrow, as I will be presenting, I will turn my focus tomorrow to that. Even though I am behind on my work, I believe the status of the project is in a great place and the entire system will be completed early this coming week!

This week I spent time debugging the display of the circuits. When using the coordinates from the computer vision the display was not working correctly. As the majority of the phone application that is left to complete is integrated with the display of the circuit, I was not able to make progress on what is left to be done. I was able to make good progress with this and should not have to spend more time to get it completely working. I was not able to allocate as much time this week to the project as I had other commitments that took up a lot of my time. While I am behind on my work I plan to spend a good amount of time during Thanksgiving to get back on track. Regarding the timeline, I did not set Thanksgiving as work days, however working throughout the week will allow me to get back on track. I expect to be done with the phone application completely by the end of next week and will start testing the following week. I plan to integrate the computer vision with the phone application the following week. This includes testing to make sure the computer vision receives the uploaded picture correctly as well as receiving the coordinates to display the circuit. As the phone application code is tailored to send and receive these items I plan to get that done early in the week.  

 

This week I was able to complete the user being able to tap the circuit they want to analyze as well as tapping components to input values. However, there is still work to be done with the tapping component functionality. Firstly, I need to display the value the user inputted somewhere on the page so they can keep track. In addition, I need to display a next button once all the components have values so it can get sent to the circuit simulator. There is one bug that needs to be fixed with the tapping functionality. If the user taps a component on the previous page, the page for selecting a circuit, the box will appear to input a value. This bug is marginal and does not cause any functionality problems, however it is something worth fixing if time permits. I am on track with my work and plan to have the input functionality fully completed by Tuesday. I am planning on testing my subsystem with the other subsystems sometime in the next week. I will analyze results by making sure the application is sending and receiving data correctly and the application still works the same as with the hardcoded values it is currently using. 

This week I was able to complete my expected work. The phone application is able to display circuit components with coordinates correctly. The code is ready to be demoed, but as it is not implemented with the computer vision code, it uses hard coded coordinates at this point. I have created five pages, one for each recommended circuit, and will create a sixth page in case the user needs to reupload a circuit. I plan to focus on integrating the computer vision code with the current state of the phone application in the next week. This includes correctly sending the image the user uploaded to the phone application and parsing the coordinates the program creates to display the circuit. As I have never done this before I do not know how much time this will take, but expect it to take less than a week, hence allocating the whole week for it. I am on track with my work and plan to spend all of Sunday prepping for the interim demo. In addition, if we complete integrating the computer vision and phone application early in the week, I expect to finish the page allowing users to input the values for each of their components.

This week I was attempting to create the page that appears after the user uploads a picture. However, I ran into a lot of bugs when trying to get a page to appear. A lot of the time when a new page was created images that I intended to appear on the page did not and when the new page did not appear the images would appear on the upload page. I was not able to fix this bug before today due to the ethics assignment, but I plan to spend all day tomorrow making progress. I also spent time researching how to create an image in a certain location given coordinates. I spent time testing dummy code to understand how I should try and display the picture when receiving the coordinates from the computer vision algorithm. I was able to create the images of each component we plan to identify, in all orientations. I am behind on work, however I have allocated time in this coming week to make progress and plan to fix this bug I am currently stuck on tomorrow. By next week I plan to display circuits given the images I have stored in the assets folders after the user inputs their picture.

I was able to get the upload functionality of the iOS application working. Now the user can upload a picture from their photo library on the app after leaving the home page. The user also has the ability to go back to the home page and leave the image upload page. In addition to this, there have been some frontend changes to the home page as there were previously some bugs when swiping across the pages.The next step would be to get the iOS application to read the coordinates created by the computer vision algorithm and be able to display the circuits appropriately. 

 

As this was during fall break not that much was expected to get down, however since I was behind on the iOS application I wanted to get the upload functionality working before coming back to school. I am on track with my work as this got completed, but now will have to allocate more time to working on the rest of the project as all that is remaining are much more difficult to implement. 

 

A tool that I plan to implement is unit testing. Specifically, unit testing the upload feature of the application and the production of the circuit UI on the phone. This will be the main focus over the coming weeks to help integrate with the computer vision algorithm.

 

This week I spent my time working on the phone application. I have made some progress, however not as much as I would have liked, due to Swift development being much different to other forms of app development I have done prior. The early stages of the week I spent researching Swift as I have no iOS app development experience. I spent a lot of time watching videos and looking at StackOverflow to learn the basic functionalities. Prior to working,  I planned to finish the home and upload page of the app by the end of this week and I was able to do the home page, however I did not complete the upload page. I implemented a sliding home page with three pages of images, titles, and labels. The page includes the functionality of the app and some tips in regards to uploading a circuit. In addition, I have implemented a skip button that leads to a new page. The functionality of the page has not been implemented, however it will lead to the upload page, where the user will upload their hand-drawn circuit. I plan to work on the phone application during fall break and make progress to being able to run the simulator after uploading the image. Currently I am a little bit behind on work. As the report is due this week as well as other class commitments, I will not be able to work on the project until after Friday. However, I plan to spend time during the break to catch back up on work and make the phone application more functional as the computer vision portion of the project has made great advancements. This means I must research how to allow the phone application to interact with the circuit simulator and the computer vision model. Thus, my goals before returning back from fall break include finishing the upload page and start implementing the backend functionality of the app.

 

This week I spent most of my time finishing all the frontend portions of the initial web application. I finished the upload page earlier in the week, however I extended the scope of the website after discussing with my teammates to help people that are using the tool for the first time. Hence, I added a FAQ page that explained basic necessities that the simulator will need, such as darkening nodes for detection and attempting to draw the connections straight. All the work I have done has been purely software work and research. Last semester I took a class on web development, Web Application Development (17-437), to write the frontend code of the web application. However, our group came to a consensus that it would be better to use a mobile application instead of a web application due to the specifications of our use cases. Because of this, my work for this week will be discarded and I will have to put in more hours next week to get started on the development of the mobile application. I also looked through computer vision research papers in the hopes of understanding how to justify our accuracy expectations. After our presentation, one piece of feedback we received was that our accuracy expectations were not justified. We were told the best course of action would be to look through computer vision research papers and understand on what basis they quantified their accuracy expectations. Through my research I was not able to find anything of substantial help. Most papers did not mention any accuracy expectations and the papers that did were models building upon previous algorithms that already had a score, thus the expected accuracy expectations were greater than the previous algorithms accuracy score. Currently, I am behind on work. Because of the group’s decision to move to a mobile application, I will have to put back the work I had planned for this week and focus on researching mobile development. I plan to put in an extra hour a day this week, potentially more, in order to get back on track.

 

(Header of home page)

This week our group spent most of our team researching and testing CV algorithms for image, feature, and edge detection. I spent time testing different algorithms with two hand drawn resistors, a horizontal and vertical wire, and a voltage/current source. My research included implementing mean squared error, Haris Corner detection, and template matching algorithms. Mean square error seemed to have the lowest error rate when comparing two individual components, however implementing with complex circuits would be too inefficient to run with the WebApp. My testing included taking and comparing each hand drawn electrical component previously listed and comparing the error rate with one another to see which pairings produced the smallest error. The algorithm did work well, however is not scalable with the size of the project. The other two algorithms were not as effective with single component detection and I therefore did not pursue any further research. From there I created the WebApp for our website and started working on the upload page. The upload page is planned to be completed by early this week and I plan for it to be completed by Monday evening. Currently I am on schedule as I need to complete the upload page by Monday and continue to help with research. Next week I want to look at number detection to help with other CV components outside the circuit itself, reading the values written next to each electrical component.