Month: November 2020

This week, we continued working on our respective parts of the project. We decided that instead of storing the behavioral interview videos, we will be storing a summary for each video with the amount of times the user had subpar eye contact and/or screen alignment. This is because we found out that it is difficult to display videos embedded in HTML from a local directory (for privacy and security reasons in HTML5), and it did not make sense to store all the videos in a public space. We also realized that it may be more helpful to the user if we provide them with a summary of how many times they had subpar eye contact and/or screen alignment for each behavioral interview practice they do. This way, they have concrete data to base their improvement off of.

Jessica worked on integrating the behavioral interview questions that Shilika wrote into the rest of the facial detection code and the behavioral portion of the tips page on the web application. She used the questions that Shilika researched at the beginning of the semester, and selected a handful of them to place into a global array. Using the Python random library, one question is selected at random and displayed at the top of the video screen for the user to refer to as they wish. She also implemented the behavioral section of the tips page to provide the user information about behavioral interviews and how to prepare for them. Mohini and Shilika will be working on the technical section of the page. Next week, she plans on combining the eye contact and screen alignment portions together for Option 1 of the facial detection part. This will provide users with alerts for both subbar eye contact and screen alignment. She also plans on figuring out how to keep track of the amount of times that the user had subpar eye contact and/or screen alignment during each video recording. This will be the summary available to users in their profile.

Mohini worked on improving the accuracy of the speech recognition algorithm through the addition of another hidden layer in the neural network and adding more training data. While both these attempts were not successful, she plans to continue trying to improve the accuracy by tweaking some of the other parameters of the neural network. Additionally, she continued working on the technical page. The chosen category, question, user’s answer, and correct answer are all displayed on the screen. She then stores all this information into a database so that a list of the logged in user’s questions and answers can be displayed on the completed technical interview practices page, which can be accessed through the profile page. Next week, Mohini will continue trying to fine tune the parameters of the neural network and start the testing of the speech recognition algorithm. 

Shilika worked on the neural network portion of the speech recognition algorithm. She was able to successfully add another layer and update all parameters and variables accordingly. This required changes to Mohini’s working neural network algorithm which had one hidden layer. The accuracy of the updated algorithm did not reach the expectations that we have set (60% accuracy). Next steps for Shilika include continuing the improve the accuracy of the speech recognition by changing variable parameters such as the number of hidden units and the amount of training data. She will also work on the profile page of the web application. 

This week, we continued working on the various portions of iRecruit. We made a decision to change the help page into a tips page, where we give users interviewing suggestions. This was because we thought that our dashboard page covered the purpose and navigation of iRecruit, and it would be helpful to give users information about good interviewing techniques, how to practice/dress for interviews, and more. We also decided to change the initial setup phase back to 5 seconds after more testing, because we realized if a user is set up and ready to go (sitting at a desk against a bare background), 5 seconds proves a sufficient amount of time.

Jessica worked on implementing the initial setup phase and off-center screen alignment detection for the mouth, and updating the home, dashboard, and technical interview pages on the web application. The initial setup phase and off-center screen alignment detection was similar to how she did them for the nose last week, where the X and Y coordinates of the mouth are stored into arrays for the first 5 seconds. Then, the average of the coordinates is taken, which provides the frame of reference coordinates of what is “center.” While recording, if the user is not within range of the frame of reference, we alert them with a pop-up message box. The home, dashboard, and technical interview pages were updated to match the behavioral interview page. They are now complete with detail and navigation functionality. Next week, Jessica is planning on integrating the behavioral interview questions that Shilika wrote into the facial detection code, so users know what question to answer during their recording. She is also planning on implementing the tips page on the web application.

Mohini worked on the web application component of the project. She created a user database that will be used to store summary information about the completed interview practices for each individual user. A change that the team decided to make is to store the counts of subpar eye contact and screen alignment as a part of this summary information rather than the videos themselves. We believed that this would be the most informative to the user. Additionally, Mohini fixed a few minor bugs regarding the login process and made sure that the workflow of the entire web application is as smooth as possible. She made more progress on the technical interview page as she displayed the user’s chosen category and a possible question to ask the user. Next steps include completing this workflow and retrieving the user’s submitted answer to check for accuracy purposes and to store in the user database. 

Shilika continued her work on adding an additional hidden layer to the neural network. She realized flaws in her algorithm from last week and had to re-do and re-write the approach. She is in the process of coding the new algorithm and will continue this process throughout the week. Additionally, she will help improve the accuracy of the neural network by adding more training data.

This week, we continued working on implementing our respective three portions of the project. We made a design decision for the facial detection portion to give users three options to account for different levels of experience with behavioral interviews. The first option is for users who are of beginner-level, and allows for them to practice with both eye contact and screen alignment. We thought this would be good for users who are unfamiliar with behavioral interviewing or the iRecruit behavior interviewing platform, to give them maximum feedback. The second and third options are for users who are of intermediate-level to advanced-level, and allows for them to practice with either only eye contact or only screen alignment. We thought this would be good for users who know their strengths and weaknesses for behavioral interviewing, and only wish to receive feedback on one technique. 

Jessica worked on implementing the initial setup phase and off-center screen alignment detection for the nose, and updating the behavioral interview page on the web application. She was able to store the X and Y coordinates of the nose into arrays for the first 10 seconds, and then take the average of those coordinates to calculate the frame of reference coordinates. If the current coordinates of the nose for the video frame are not within range of the frame of reference coordinates, the user is alerted with a pop-up message box. She updated the behavioral interview page to give the user an overview and provide them with the three different options. Next week, she is planning to work on the initial setup phase and off-center screen alignment detection for the mouth, and updating the dashboard and technical interview pages.

Mohini worked on integrating the signal processing and machine learning components together with the Django webapp. The output from the signal processing is saved to a text file which is the input to the machine learning algorithm which then outputs the predicted category into a separate text file. Then, Django reads from this text file to display the predicted category on the webpage. When the user records the category of questions they are interested in receiving, the webpage displays the category name through the speech recognition algorithm. The accuracy of this algorithm is quite low, so next steps would be fine tuning the model to increase the accuracy. 

Shilika worked on the css and html for the web application portion, and saving videos to the profile page of the web app. She also worked on the neural network portion of the speech processing aspect. She is researching on improving the accuracy of the current neural network and implementing one more hidden layer in the neural network. Next week, she will continue improving the accuracy of the neural network and in turn the speech recognition.