{"id":195,"date":"2024-04-28T02:34:10","date_gmt":"2024-04-28T02:34:10","guid":{"rendered":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/s24-teamb0\/?p=195"},"modified":"2024-04-28T02:34:10","modified_gmt":"2024-04-28T02:34:10","slug":"team-status-report-for-4-27","status":"publish","type":"post","link":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/s24-teamb0\/2024\/04\/28\/team-status-report-for-4-27\/","title":{"rendered":"Team Status Report for 4\/27"},"content":{"rendered":"

Risk Mitigation<\/strong><\/p>\n

The main risk of our system is the possible failure in parsing the voice commands. To resolve this issue, we listed out the supported commands that are guaranteed to work and integrated these instructions in the \u201chelp\u201d command. The user can simply say \u201chelp\u201d to get sample commands and start exploring more from there.<\/p>\n

 <\/p>\n

Design Changes<\/strong><\/p>\n

There is no change in our design.<\/p>\n

 <\/p>\n

Updated Schedule<\/strong><\/p>\n

The updated schedule is as reflected in the final presentation.<\/p>\n

\"\"<\/p>\n

 <\/p>\n

List of all tests<\/strong><\/p>\n

Unit tests<\/p>\n\n\n\n\n\n\n\n\n
Test<\/span><\/td>\nExpected Performance<\/span><\/td>\nActual Performance<\/span><\/td>\n<\/tr>\n
Latency (voice command)<\/span><\/td>\n4 seconds to render the page<\/span><\/td>\nAverage 4.52 seconds to render the page<\/span><\/td>\n<\/tr>\n
Battery life<\/span><\/td>\nConsumes less than 50% of power with monitor on for 1 hour<\/span><\/td>\nPower drops from 100% to 77%, consuming 23% of total power<\/span><\/td>\n<\/tr>\n
Portability<\/span><\/td>\n500 grams<\/span><\/td>\n500 grams<\/span><\/td>\n<\/tr>\n
Accessibility<\/span><\/td>\n32% of right half of screen<\/span><\/td>\n35.45% of right half\u00a0 of screen<\/span><\/td>\n<\/tr>\n
Noise reduction<\/span><\/td>\n90% accuracy of integral test under 70dB environment<\/span><\/td>\n9 out of 10 commands work as expected, 90% accuracy<\/span><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

 <\/p>\n\n\n\n\n\n\n\n
Test<\/span><\/td>\nExpected Performance<\/span><\/td>\nActual Performance<\/span><\/td>\n<\/tr>\n
Audio to text<\/span><\/td>\nLess than 20% of word error rate (WER)<\/span><\/td>\n98.3% accuracy, edge case exists<\/td>\n<\/tr>\n
Text to command (NLP)<\/span><\/td>\nIdentify verb with 100% accuracy;<\/span><\/p>\n

Identify item name, money, date, and number with 95% accuracy;<\/span><\/p>\n

NLP process takes less than 3s<\/span><\/td>\n

Identify <\/span>verb, money, date, and number with <\/span>100% accuracy;<\/span><\/p>\n

Identify <\/span>item name with <\/span>96% accuracy<\/span><\/p>\n

NLP process takes about <\/span>2.5s<\/td>\n<\/tr>\n

Item classification (Glove)<\/span><\/td>\n90% of item names correctly classified<\/span><\/td>\n18 out of 20 item names are correctly classified, 90% accuracy<\/span><\/td>\n<\/tr>\n
Voice response<\/span><\/td>\nAll audio requests should be assisted with a voice response<\/span><\/td>\nThe app will read out the content of the response (entries, report, etc.)<\/span><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

System tests<\/p>\n

User Experience Test:\u00a0Each of the 5 volunteers would have half of the day to interact with our product without our interference.\u00a0Volunteers are expected to give feedback about what they like and dislike about the product.<\/p>\n

Overall Latency:\u00a0Test different commands (e.g. record\/change\/delete entry, view entry, generate report, etc.)\u00a0Expect <15s from pressing the button to rendering the page\u00a0for all commands.<\/p>\n

Overall Accuracy:\u00a0Expect >95% accuracy for the whole system.<\/p>\n

Test Findings<\/strong><\/p>\n

There are some improvements that could be made about the user interface. The font could be larger for normal users. We support “remove” command but not “delete” command, while there are delete buttons which could be misleading. The audio instruction is too long for some user to remember what commands are supported. It might be necessary to tell the user to strictly follow the instructions, or the accuracy of commands will be very low. We will make corresponding improvements before the final demo.<\/p>\n","protected":false},"excerpt":{"rendered":"

Risk Mitigation The main risk of our system is the possible failure in parsing the voice commands. To resolve this issue, we listed out the supported commands that are guaranteed to work and integrated these instructions in the \u201chelp\u201d command. The user can simply say \u201chelp\u201d to get sample commands and start exploring more from … <\/p>\n

Continue reading “Team Status Report for 4\/27”<\/span><\/a><\/p>\n","protected":false},"author":881,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[12],"tags":[4],"_links":{"self":[{"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/s24-teamb0\/wp-json\/wp\/v2\/posts\/195"}],"collection":[{"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/s24-teamb0\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/s24-teamb0\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/s24-teamb0\/wp-json\/wp\/v2\/users\/881"}],"replies":[{"embeddable":true,"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/s24-teamb0\/wp-json\/wp\/v2\/comments?post=195"}],"version-history":[{"count":1,"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/s24-teamb0\/wp-json\/wp\/v2\/posts\/195\/revisions"}],"predecessor-version":[{"id":199,"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/s24-teamb0\/wp-json\/wp\/v2\/posts\/195\/revisions\/199"}],"wp:attachment":[{"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/s24-teamb0\/wp-json\/wp\/v2\/media?parent=195"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/s24-teamb0\/wp-json\/wp\/v2\/categories?post=195"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/s24-teamb0\/wp-json\/wp\/v2\/tags?post=195"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}