Weekly Report #1
Stella, Sue Jeong (Software)
- Successfully connected webcam using OpenCV and Python
- Background subtraction
- In progress of making neural network
- Parallely studying HLS to move over the neural network onto the FPGA
Michael (Hardware)
- Studying HLS to be able to integrate neural network onto FPGA
- Studying Xilinx zedboard and usage with HLS
- Connecting FPGA to laptop with UART (in progress)
- Researched Convolutional Neural Network parralizability for FPGA implementation
Weekly Report #2
Stella (Software)
- In progress of modifying the return value (training + testing accuracy -> classified output + probability)
- In progress of designing layers of the neural network and running tests to figure out which combinations will give back a higher correction rate
- Studying HLS to move over the neural network onto the FPGA
Sue Jeong (Software)
- In progress of preprocessing the images captured by webcam using MATLAB to fit the requirements of our classifying layers (a matrix that represents the pixel values of the image in 28*28 size)
- In progress of designing layers of the neural network and running tests to figure out which combinations will give back a higher correction rate
- Studying HLS to move over the neural network onto the FPGA
Michael (Hardware)
- Completed working on the tutorial for vivado HLS
- In progress of setting up the UART interface and will be able to test it this weekend and next week
- Planning to put CNN in vivado hls and start optimizing that when it is ready
Weekly Report #3
Stella (Software)
- Collected 300 samples per gesture (0-9)
- In progress of training neural network on PC before trying to move the neural network onto the FPGA
- Finished preprocessing static images to pass into classifying layers (a matrix that represents the pixel values of the image in 28*28 size)
- Studying HLS to move over the neural network onto the FPGA
Sue Jeong (Software)
- Collected 300 samples per gesture (0-9)
- In progress of training neural network on PC before trying to move the neural network onto the FPGA
- In progress of modifying the return value (training + testing accuracy -> classified output + probability)
- Studying HLS to move over the neural network onto the FPGA
Michael (Hardware)
- Working on connecting FPGA to computer via UART communication
- Made a few basic HLS programs to practice optimizing parallel C code with respect to size and speed
- Looked into CNN code to help get it ready to move to the FPGA
Weekly Report #4
Stella (Software)
- Collected 500 samples per gesture (0-4)
- Trained neural network on PC (gestures 0-4) with an accuracy of > 99%
- In progress of resizing images (200x200 -> 28x28) to pass into classifying layers in real-time
- In progress of restructuring NN to be more optimizable on hardware
Sue Jeong (Software)
- Collected 500 samples per gesture (0-4)
- Modified the return value to print the classified output onto a terminal
- In progress of preprocessing the images to not require a black background in order to execute at the reported accuracy
Michael (Hardware)
- Successfully ran a convolutional layer via HLS with the estimated weights on a 20x20 image - optimized the loops (unrolled them) to increase performance 100 fold, although this also increased space used on the FPGA
- In progress of calculating how much space a 200x200 image will be needed on a zed-board based on how much parallelization we want
- In progress of connecting zed-board to computer via UART
Weekly Report #5
Stella (Software)
- Successfully resized images (200x200 -> 28x28) to pass into classifying layers in real-time
- Done preprocessing the images to not require a black background (with some noise)
- In progress of restructuring NN to be more optimizable on hardware
Sue Jeong (Software)
- Studied the kernel on zed-board
- In progress of setting up communication to the zed-board via UART
- In progress of restructuring NN to be more optimizable on hardware
Michael (Hardware)
- Calculated timing with respect to the zed-board
- Setting up UART communication to zed-board
- Running the conv net on the FPGA
- Optimizing the conv net for the FPGA
Weekly Report #6
Stella (Software)
- In progress of collecting 2000 samples per gesture (0-9) and training for 28x28 scale NN
- In progress of making a new UI for displaying the output
- In progress of restructuring NN to be more optimizable on hardware
Sue Jeong (Software)
- In progress of restructuring NN to be more optimizable on hardware
- In progress of setting up communication
Michael (Hardware)
- This week I kept working on the Uart connection : in process of getting the connection with our module on the FPGA
- Continuing to work on receiving signal from computer via pyserial
- Continuing to optimize cnn on fpga
Weekly Report #7
Stella (Software)
- Collected 1000 samples per gesture (0-4) and trained for 28x28 scale NN
- Wrote static image classification code and successfully ran it on PC
- Separated training and classifying part so that we can put just the classification part on the Pi
- In progress of linking a PiCamera to our system to make real-time classification on the Pi possible
Sue Jeong (Software)
- Separated training and classifying part so that we can put just the classification part on the Pi
- In progress of linking a PiCamera to our system to make real-time classification on the Pi possible
- In progress of setting up the environment on Pi to support our OpenCV and NN system
Michael (Hardware)
- Got Uart to send info from board to computer
- In progress of integrating Vivado with FPGA
- In progress of sending data from the computer to the FPGA with pyserial
Weekly Report #8
Stella (Software)
- Downloaded all libraries needed for static and real-time classification
- Successfully ran static classification on the Pi with images that had been captured and moved onto the SD card with a USB connector
- Successfully connected static classification network on the Pi with OpenCV on the PC - whole system can be run by just two terminal commands on the PC
- Calculated time and energy consumption of neural network on the PC and Pi
- In progress of setting up real-time classification
Sue Jeong (Software)
- Downloaded all libraries needed for static and real-time classification
- Successfully ran static classification on the Pi with images that had been captured and moved onto the SD card with a USB connector
- Successfully connected static classification network on the Pi with OpenCV on the PC - whole system can be run by just two terminal commands on the PC
- Calculated time and energy consumption of neural network on the PC and Pi
- In progress of setting up real-time classification
Michael (Hardware)
- Discovered Issue with matrix size while transfering data to fabric
- Finished UART connection from computer to FPGA core
- Implemented pyserial to interface with keras modules
- Working on using AXI-LITE to allow connection from core to fabric