Following the discussion with the professor earlier this week, I went on to explore some of the options that he had mentioned. In particular, I started looking into a speaker-dependent speech recognition system instead of a speaker-independent one. While the scope or difficulty of the task drastically reduces, it is still interesting to experiment with the capabilities of a system that can perform recognition on code-switching speech.
According to a paper published in 1993, speaker-dependent ASR can reach very low error rates within 600-3000 sentences of speech in English. However, there have been no similar attempts in code-switching speech between Mandarin and English.
So far, I collected around 600 sentences of myself speaking in purely English, purely Mandarin, and mixed. For next week, I plan on training a model using the data I’ve gathered together with existing English and Mandarin corpora.