This week, I worked on continuing to broaden the data set and train the model. Unfortunately, it was difficult to find other dynamic ASL datasets readily available. I tried to download the how2sign dataset, but there was an incompatibility issue with the script to download it. I tried to debug this for a bit and even reached out to the creator of the script, but haven’t gotten to a solution yet. I tried the MS-ASL dataset from Microsoft, but the data linked to YouTube videos that were all set to private. I requested permission to access the Purdue RVL-SLLL dataset, but I haven’t gotten a response yet. I also looked at ASL-LEX, but it is a network of 1 video corresponding to each sign, which is not very helpful. At this point, since it’s difficult to find datasets, I’ve just been continuing to create my own videos, following the details of the DSL-10 dataset videos I currently have trained, such as the same number of frames, and amount of videos per class. I have added 32 classes of the most common phrases used in conversation for our use case: “good”, “morning”, “afternoon”, “evening”, “bye”, “what”, “when”, “where”, “why”, “who”, “how”, “eat”, “drink”, “sleep”, “run”, “walk”, “sit”, “stand”, “book”, “pen”, “table”, “chair”, “phone”, “computer”, “happy”, “sad”, “angry”, “excited”, “confused”, “I’m hungry”, “I’m tired”, “I’m thirsty”. Because there are a lot of videos and there will be more, I am running into storage issues on my device. I am wondering if there is a method or separate server that allows quicker processing of large datasets like this.
My progress is still slightly behind schedule because I am still working on word translation. I plan to catch up this week as we prepare for the interim demo.
Next week, I will continue to train my custom dataset to allow for more variety in translated gestures. Also, I will work on continuous translation since right now I am working on word translation, but we need to eventually work on continuous sentences. I will also be working with my teammates to integrate our parts right now on the IOS app for deployment of our current product state.