This week I was able to successfully migrate my architecture and data to my personal AFS ECE space and make significant progress on getting the architecture to run. Previously, I struggled to find a place to run my machine learning architecture, because this course doesn’t provide AWS credits and I was only able to successfully SCP to the AFS Andrew space, which truncated my data at 2 GB according to the quota of that space. By changing my default home directory to the AFS ECE space and working with Samuel to successfully SCP my data directly to the AFS ECE space, I am now able to run my architecture, which has been a huge relief. Since then, I have generated training and validation datasets. That was somewhat challenging, because the data is so high dimensional, it not only took time, but I also had to do it piecemeal, because doing it all at once allocated too much memory and threw a CPU error. Then, I worked on the architecture itself. By more closely examining the 3D architecture I had coded locally inspired by https://keras.io/examples/vision/3D_image_classification/, I found a helpful research paper that discussed that 4 iterations of a 3D convolutional layer, max pooling, and batch normalization was effective at extracting features on 3D data. Therefore, I sought out to code this. I went back and forth between using TensorFlow and PyTorch. I feel like TensorFlow is easier to work with, but during data generation, it required converting between NumPy arrays and tensors, which took up too much memory. Therefore, I have settled on creating my architecture in PyTorch, which I am not as familiar with but am working through the different syntax. Currently, I am working out the dimensions by hand and verifying them in code to make sure they match input shapes.
Due to the workflow of my weekend, I will be spending significant amount of time on the machine learning architecture tomorrow, not today. Taking that into account, I am right on schedule to have a functioning architecture that is outputting some metrics by the interim demo.
In the next week, I will demo a machine learning architecture that is able to perform inference on one of the pieces of training data. I will also have corresponding loss, accuracy, and f1 score metrics.
0 Comments