Author: sgehlot

This week, we continued working on the various components of iRecruit. Jessica worked on integrating the eye contact and screen alignment parts together for option 1 of the behavioral interview options, and figured out how to store summary information for each video recording. She was able to combine the eye contact and screen alignment parts to alert the user of both subpar eye contact and screen alignment. There are setup phases for the eye detection and facial landmark detection parts, where the frame of reference coordinates for the center of the eyes, nose, and mouth are calculated in the first 5 seconds. If a user’s eyes, nose, or mouth are off-center, they are alerted with the appropriate category (eye contact or screen alignment). She also implemented the video summaries for the profile part of the behavioral interviews. There is a common text file that keeps track of all the video summaries. For each video recording, the interview number, timestamp, number of times the user had subpar eye contact, and number of times the user had subpar screen alignment are appended to a new line in the file. Next week, she plans on integrating this text file with Django, so that the video summaries show up on the profile page. She also plans on continuing to test the eye contact only option, and beginning to test the screen alignment only option and the integrated option. 

Mohini worked on a couple different things this week. Since she finished most of the implementation last week, she spent her time this week testing and refining. She refined the backend database model that kept track of each user and their corresponding questions and answers. Then, she tested the speech recognition algorithm using both automated and manual testing. Automated testing consisted of having a fixed testing data set of approximately 50 samples whereas manual testing consisted of retrieving the testing input through the signal processing algorithm. She will continue this testing next week to determine the true accuracy of the algorithm. Lastly, Mohini started working on the final report and incorporating some of the design changes that we made throughout the project. Next week, she will start recording her parts for the final video demo. 

Shilika worked on the web application and added finishing touches such as the Completed Technical Interview page and minor css changes. She also worked on testing the speech recognition algorithm. She ran the automated testing and collected data. Additionally, she ran the in-built gnb methods in python to compare the results of that against the neural network. Next week, she will continue testing the performance of the speech recognition model, and work on the final video and report. 

This week, we continued working on our respective parts of the project. We decided that instead of storing the behavioral interview videos, we will be storing a summary for each video with the amount of times the user had subpar eye contact and/or screen alignment. This is because we found out that it is difficult to display videos embedded in HTML from a local directory (for privacy and security reasons in HTML5), and it did not make sense to store all the videos in a public space. We also realized that it may be more helpful to the user if we provide them with a summary of how many times they had subpar eye contact and/or screen alignment for each behavioral interview practice they do. This way, they have concrete data to base their improvement off of.

Jessica worked on integrating the behavioral interview questions that Shilika wrote into the rest of the facial detection code and the behavioral portion of the tips page on the web application. She used the questions that Shilika researched at the beginning of the semester, and selected a handful of them to place into a global array. Using the Python random library, one question is selected at random and displayed at the top of the video screen for the user to refer to as they wish. She also implemented the behavioral section of the tips page to provide the user information about behavioral interviews and how to prepare for them. Mohini and Shilika will be working on the technical section of the page. Next week, she plans on combining the eye contact and screen alignment portions together for Option 1 of the facial detection part. This will provide users with alerts for both subbar eye contact and screen alignment. She also plans on figuring out how to keep track of the amount of times that the user had subpar eye contact and/or screen alignment during each video recording. This will be the summary available to users in their profile.

Mohini worked on improving the accuracy of the speech recognition algorithm through the addition of another hidden layer in the neural network and adding more training data. While both these attempts were not successful, she plans to continue trying to improve the accuracy by tweaking some of the other parameters of the neural network. Additionally, she continued working on the technical page. The chosen category, question, user’s answer, and correct answer are all displayed on the screen. She then stores all this information into a database so that a list of the logged in user’s questions and answers can be displayed on the completed technical interview practices page, which can be accessed through the profile page. Next week, Mohini will continue trying to fine tune the parameters of the neural network and start the testing of the speech recognition algorithm. 

Shilika worked on the neural network portion of the speech recognition algorithm. She was able to successfully add another layer and update all parameters and variables accordingly. This required changes to Mohini’s working neural network algorithm which had one hidden layer. The accuracy of the updated algorithm did not reach the expectations that we have set (60% accuracy). Next steps for Shilika include continuing the improve the accuracy of the speech recognition by changing variable parameters such as the number of hidden units and the amount of training data. She will also work on the profile page of the web application. 

This week, we continued working on the various portions of iRecruit. We made a decision to change the help page into a tips page, where we give users interviewing suggestions. This was because we thought that our dashboard page covered the purpose and navigation of iRecruit, and it would be helpful to give users information about good interviewing techniques, how to practice/dress for interviews, and more. We also decided to change the initial setup phase back to 5 seconds after more testing, because we realized if a user is set up and ready to go (sitting at a desk against a bare background), 5 seconds proves a sufficient amount of time.

Jessica worked on implementing the initial setup phase and off-center screen alignment detection for the mouth, and updating the home, dashboard, and technical interview pages on the web application. The initial setup phase and off-center screen alignment detection was similar to how she did them for the nose last week, where the X and Y coordinates of the mouth are stored into arrays for the first 5 seconds. Then, the average of the coordinates is taken, which provides the frame of reference coordinates of what is “center.” While recording, if the user is not within range of the frame of reference, we alert them with a pop-up message box. The home, dashboard, and technical interview pages were updated to match the behavioral interview page. They are now complete with detail and navigation functionality. Next week, Jessica is planning on integrating the behavioral interview questions that Shilika wrote into the facial detection code, so users know what question to answer during their recording. She is also planning on implementing the tips page on the web application.

Mohini worked on the web application component of the project. She created a user database that will be used to store summary information about the completed interview practices for each individual user. A change that the team decided to make is to store the counts of subpar eye contact and screen alignment as a part of this summary information rather than the videos themselves. We believed that this would be the most informative to the user. Additionally, Mohini fixed a few minor bugs regarding the login process and made sure that the workflow of the entire web application is as smooth as possible. She made more progress on the technical interview page as she displayed the user’s chosen category and a possible question to ask the user. Next steps include completing this workflow and retrieving the user’s submitted answer to check for accuracy purposes and to store in the user database. 

Shilika continued her work on adding an additional hidden layer to the neural network. She realized flaws in her algorithm from last week and had to re-do and re-write the approach. She is in the process of coding the new algorithm and will continue this process throughout the week. Additionally, she will help improve the accuracy of the neural network by adding more training data.

This week, we continued working on implementing our respective three portions of the project. We made a design decision for the facial detection portion to give users three options to account for different levels of experience with behavioral interviews. The first option is for users who are of beginner-level, and allows for them to practice with both eye contact and screen alignment. We thought this would be good for users who are unfamiliar with behavioral interviewing or the iRecruit behavior interviewing platform, to give them maximum feedback. The second and third options are for users who are of intermediate-level to advanced-level, and allows for them to practice with either only eye contact or only screen alignment. We thought this would be good for users who know their strengths and weaknesses for behavioral interviewing, and only wish to receive feedback on one technique. 

Jessica worked on implementing the initial setup phase and off-center screen alignment detection for the nose, and updating the behavioral interview page on the web application. She was able to store the X and Y coordinates of the nose into arrays for the first 10 seconds, and then take the average of those coordinates to calculate the frame of reference coordinates. If the current coordinates of the nose for the video frame are not within range of the frame of reference coordinates, the user is alerted with a pop-up message box. She updated the behavioral interview page to give the user an overview and provide them with the three different options. Next week, she is planning to work on the initial setup phase and off-center screen alignment detection for the mouth, and updating the dashboard and technical interview pages.

Mohini worked on integrating the signal processing and machine learning components together with the Django webapp. The output from the signal processing is saved to a text file which is the input to the machine learning algorithm which then outputs the predicted category into a separate text file. Then, Django reads from this text file to display the predicted category on the webpage. When the user records the category of questions they are interested in receiving, the webpage displays the category name through the speech recognition algorithm. The accuracy of this algorithm is quite low, so next steps would be fine tuning the model to increase the accuracy. 

Shilika worked on the css and html for the web application portion, and saving videos to the profile page of the web app. She also worked on the neural network portion of the speech processing aspect. She is researching on improving the accuracy of the current neural network and implementing one more hidden layer in the neural network. Next week, she will continue improving the accuracy of the neural network and in turn the speech recognition.

 

This week, we continued working on implementing our respective portions of the project, making progress in the three main parts. Jessica worked on implementing a center of frame reference, facial landmark detection, and testing the eye detection portion. She thought it would be helpful to have a centered guideline for users to position themselves accordingly during the initial setup phase, so that they have a reference for the center of the video screen. She continued working on the facial landmark detection, and was able to get the coordinates of the center of the nose and mouth. The eye detection portion was also tested more, and the results seem to align with the accuracy goal. Next week, she will work on completing the initial setup phase for the facial landmark detection, and would like to complete the off-center screen alignment portion for the nose as well. She will also continue testing both the eye detection and screen alignment parts. 

Mohini finalized the signal processing algorithm and started making the training data set for the neural network algorithm. This week concluded the signal processing portion of our project, so I will be focusing on the machine learning portion as well as integrating the different components of our project together for the rest of the semester. Next week, I will be working on testing the neural network after finishing generating the rest of the training data set. 

Shilika began reviewing the neural network concepts, as this is the next technical aspect of the technical interview portion that she will help tackle. She also continued to work on the web application to improve the css and features that appear in the front-end to make the app more user friendly. Next week, she will continue working on the web application and the neural network.