{"id":168,"date":"2024-04-07T03:11:01","date_gmt":"2024-04-07T03:11:01","guid":{"rendered":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/s24-teamb0\/?p=168"},"modified":"2024-04-07T03:11:01","modified_gmt":"2024-04-07T03:11:01","slug":"lynns-status-report-for-04-06","status":"publish","type":"post","link":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/s24-teamb0\/2024\/04\/07\/lynns-status-report-for-04-06\/","title":{"rendered":"Lynn&#8217;s Status Report for 04\/06"},"content":{"rendered":"<p><span style=\"font-weight: 400\">04\/06<\/span><\/p>\n<p><strong>Progress<\/strong><\/p>\n<p><span style=\"font-weight: 400\">I spent the first half of the week figuring out a valid virtual keyboard that could allow standard GUI item name and price input. The initial plan was to use the built-in virtual keyboard in RPI, but the current packages available are not compatible with 64-bit RPI OS. Therefore I turned to a virtual keyboard component within the web application. The current keyboard is attached.<\/span><\/p>\n<p><span style=\"font-weight: 400\">As for the VUI side, the web app could now provide financial report and entry list based on the audio input \u201cgenerate report\/get entries from [start date] to [end date]\u201d. Since the date information is \u201cYear-Month\u201d, the filtering date is set as the 1st day of the start month and the last day of the end month. The web app could also provide audio output corresponding to user voice input. After all response text files are finalized, the app should be able to work for visually-impaired groups to use.<\/span><\/p>\n<p><img decoding=\"async\" loading=\"lazy\" class=\"alignnone size-medium wp-image-169\" src=\"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/s24-teamb0\/wp-content\/uploads\/sites\/276\/2024\/04\/Screenshot-2024-04-06-at-21.25.03-300x157.png\" alt=\"\" width=\"300\" height=\"157\" srcset=\"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/s24-teamb0\/wp-content\/uploads\/sites\/276\/2024\/04\/Screenshot-2024-04-06-at-21.25.03-300x157.png 300w, http:\/\/course.ece.cmu.edu\/~ece500\/projects\/s24-teamb0\/wp-content\/uploads\/sites\/276\/2024\/04\/Screenshot-2024-04-06-at-21.25.03-768x402.png 768w, http:\/\/course.ece.cmu.edu\/~ece500\/projects\/s24-teamb0\/wp-content\/uploads\/sites\/276\/2024\/04\/Screenshot-2024-04-06-at-21.25.03-1024x536.png 1024w\" sizes=\"(max-width: 300px) 100vw, 300px\" \/><\/p>\n<p><span style=\"font-weight: 400\">I also did unit tests on the audio input part and rendering functionalities for all web pages.<\/span><\/p>\n<p><strong>Schedule<\/strong><\/p>\n<p><span style=\"font-weight: 400\">I am on schedule now.<\/span><\/p>\n<p><strong>Next step<\/strong><\/p>\n<p><span style=\"font-weight: 400\">I am planning to implement the VUI of the entry-entering function and apply tests for it. I would also work with our team to conduct larger-scale tests that involve more subsystems.<\/span><\/p>\n<p><strong>Verification<\/strong><\/p>\n<ul>\n<li><span style=\"font-weight: 400\">Tests for speech recognition accuracy:\u00a0<\/span><\/li>\n<\/ul>\n<p><span style=\"font-weight: 400\">To test the speech recognition pipeline, I wrote a script that accepted keyboard input as a session of recording and translating. To perform unit tests, I pressed the corresponding keyboard and gave voice commands. The script should be able to output converted text that is identical to the audio input.<\/span><\/p>\n<p><span style=\"font-weight: 400\">Since the expected user audio inputs are short sentences starting with standard verb keywords, I first conducted unit tests on the verb keywords in both quiet environments and crowd environments. Words tested include \u201cEnter\u201d, \u201cGet\u201d, \u201cGenerate\u201d and similar verbs. The accuracy was about 95% for these verbs, with a relatively low accuracy when the program starts first and a relatively high accuracy after the 1st input.\u00a0<\/span><\/p>\n<p><span style=\"font-weight: 400\">Then I tested whether the script could catch numbers including both price numbers and dates accurately. With the keyword \u201cdollar\u201d attached to the price number, the recognition accuracy would reach 99% with the current test cases. For example, \u201cfive point three dollars\u201d could be translated to \u201c$5.3\u201d directly. As for dates, the \u201cYear-Month\u201d pattern could be successfully converted to text without difficulty in most cases. However, the accuracy for the month \u201cMay\u201d is lower than all other months, so it would be necessary for us to provide some backup for the inaccurate translation.\u00a0\u00a0<\/span><\/p>\n<p><span style=\"font-weight: 400\">As for item names, the accuracy drops to 95% again due to the large number of possible nouns. The pipeline could catch items that have relatively unique and complex pronunciations, whereas words with simpler pronunciations may be converted to similar words that are incorrect.<\/span><\/p>\n<ul>\n<li><span style=\"font-weight: 400\">Tests for web application VUI:<\/span><\/li>\n<\/ul>\n<p><span style=\"font-weight: 400\">After the speech recognition pipeline was implemented within the web application, I conducted the same unit tests listed in the last category in the app. The performance was generally the same, which means that the web application framework would not affect the speech recognition process.<\/span><\/p>\n","protected":false},"excerpt":{"rendered":"<p>04\/06 Progress I spent the first half of the week figuring out a valid virtual keyboard that could allow standard GUI item name and price input. The initial plan was to use the built-in virtual keyboard in RPI, but the current packages available are not compatible with 64-bit RPI OS. Therefore I turned to a &hellip; <\/p>\n<p class=\"link-more\"><a href=\"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/s24-teamb0\/2024\/04\/07\/lynns-status-report-for-04-06\/\" class=\"more-link\">Continue reading<span class=\"screen-reader-text\"> &#8220;Lynn&#8217;s Status Report for 04\/06&#8221;<\/span><\/a><\/p>\n","protected":false},"author":926,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[11],"tags":[],"_links":{"self":[{"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/s24-teamb0\/wp-json\/wp\/v2\/posts\/168"}],"collection":[{"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/s24-teamb0\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/s24-teamb0\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/s24-teamb0\/wp-json\/wp\/v2\/users\/926"}],"replies":[{"embeddable":true,"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/s24-teamb0\/wp-json\/wp\/v2\/comments?post=168"}],"version-history":[{"count":1,"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/s24-teamb0\/wp-json\/wp\/v2\/posts\/168\/revisions"}],"predecessor-version":[{"id":170,"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/s24-teamb0\/wp-json\/wp\/v2\/posts\/168\/revisions\/170"}],"wp:attachment":[{"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/s24-teamb0\/wp-json\/wp\/v2\/media?parent=168"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/s24-teamb0\/wp-json\/wp\/v2\/categories?post=168"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/s24-teamb0\/wp-json\/wp\/v2\/tags?post=168"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}