This week has been the final stretch in implementing the final software subsystem of our project. We were able to move all of the hardware from the ECE lab testbench onto a cart. We strapped down everything so that the wires and components don’t move around and disturb the signals traveling. In addition to this, we decided to do the process of data collection and overlaying on the laptop as we have been doing. However, in the past we were using 2 laptops, my laptop to collect the SDR data and send it over UDP and Vrishab’s laptop to receive the data to do the beamforming algorithm. This was all done for testing and so would be inconvenient for the actual final product and so Txanton and I worked on integrating the two software scripts into one so that it could be configured to work solely on mine. Throughout the week we worked on doing test runs to get lobe width, we tested to see that the box in the middle could respond to different signal strengths at different distances.
Txanton’s Status Report for 05/01/2021
This week we had the discussion with the Eberly center which I felt was pretty productive. We were able to give some feedback which seemed to be well received and we were also able to hear some of the experiences of the other teams.
Regarding the project, this week I helped to interface the overlay code with the beamforming code by receiving the data over UDP. I also helped to tweak some of the specific values as we were testing in order to make the overlay code better correspond with the constraints of our system. Luckily from our testing it seems to be corresponding with the location of the transmitting device pretty well so I am hopeful that we won’t need to make any significant changes for the final video.
Next week I will continue to help the team test the accuracy and make any future adjustments as we prepare for our final video and report.
Team Status Report for 04/24/2021
This week we had our ethics discussion where we were able to get some valuable feedback about some ethical considerations for our project. This will be useful for our final report and gave us some additional perspective for our future work.
Regarding our project, this week we worked on getting the beamforming array to work with 4 SDR’s and antennas. After updating our beamforming pipeline to receive data from all 4 antennas and adjusting the algorithm to use 4 inputs instead of 2, we were able to get an angle of arrival for the signal picked up by the antenna array. We also made progress on the heatmap overlay code and are able to now draw boxes over of the webcam input based on the calculated angle of arrival. This means we have the two subsystems of our project mostly complete and will be able to start integrating them next week.
Next week we will continue working on integrating the two subsystem’s so that the vision pipeline is actually receiving data from the beamforming pipeline instead of the dummy test data. After that all we will have to do is more testing and minor adjustments and then we should be on track to finish our project.
Enock’s Status Report 04/24/2021
This week and the last week during carnival Vrishab and I worked on collecting a lot of data from our antenna configuration in order to see if the two antennas we had were properly synced and picking up proper signals. We found out that in our test environment in the ECE lab there were WiFi signals on our channel that were interfering with our calculations and so we were able to remove them from the network when testing which fixed all of our issues. After removing these devices we found that our signals were synced properly and correlation worked well and we could achieve and angle of arrival from the data. From here we integrated two more antennas for the full 4 antenna design which was a very smooth transition. Upon adding the new antennas all we had to do was account for N=4 antennas in our code rather than N=2 and everything worked perfectly.
We then spent time collecting data with 4 antennas and performing beamforming with two devices in the room and calculating the angle of arrival from these devices which worked for us. During this time of testing Txanton was able to send up the heatmap overlay code which we were able integrate into the system with our webcam fairly easily.
The objective for next week will be to integrate the webcam such that it interfaces with our angle of arrival data since currently all we interfaced currently is dummy data onto the webcam input. We will also take measurements to figure out our resolution to see if it matches our initial goals.
Vrishab’s Status Report for 04/24/2021
Integration
This week I worked on integrating our vision pipeline with our signal processing pipeline. The vision pipeline consists of software written by Txanton that draws boxes over camera input from OpenCV. The signal processing pipeline comprises all of the beamforming architechture and outputs an angle of arrival (AOA) at which a WiFi-transmitting device is located. The integration is performed by sending the AOA over UDP to a separate computer which receives the webcam input.
Progress Since Last Week
We have finished our signal processing pipeline and have now started on the integration phase per our project schedule.
Tasks for Next Week
We need to finish our integration between the camera pipeline and our signal processing pipeline. We will also measure the true resolution of our system and compare the results with our theoretical calculations.
Txanton’s Status Report for 04/24/2021
This week we had our ethics discussion. I found the meeting to be pretty interesting as before I read the papers assigned I had a very different view of engineering ethics. Typically I thought of issues such as computer vision with biased/non-diverse data sets leading to discrimination, but after reading the papers it opened my eyes to more intentional ethical decisions such the parkways in New York.
Working on our project, I spent this week working on expanding the image overlay code. I started making it so that we can overlay a box after receiving an angle from the beamforming array. What I am doing to attempt this is by dividing the horizontal resolution by the horizontal field of view of the camera. This has been a little difficult to find as most manufacturers only provide the diagonal field of view. After performing this calculation I have an approximation for how many pixels correspond to 1 degree which I can use to then draw a box around that angle that was provided by the beamforming algorithm. Currently this is only in one dimension, however the process should be pretty easy to adjust to make it two dimensional once the antenna array has been formed that way. Since we are testing and I am remote I am not 100% up to date on the accuracy of the beamforming array so I currently have it placing a larger box around the calculated angle that corresponds to +/- 5 degrees in order to account for inaccuracy of the algorithm. The number of degrees for this box is also very easy to adjust so once I fully know the general accuracy of our array I can update that to correspond.
For next week I am going to continue to work with the rest of my team to update these values to make them more actually correspond with our measurements. I will also be doing some I/O in order to receive the angle from our beamforming algorithm as currently they are two separate things and I have been testing my angle based boxes using hard coded values and loops not actual data points. I will also update the code to use 2 angles to affect the box’s vertical size as well if we decide to make our array in that orientation.
Updated Schedule
Enock’s Status Report for 04/10/2021
This week Vrishab and I spent time on the real time signal processing in time for demo. Through this we were able to start programming and finishing the antenna capture design,
For real time processing we were able to write python classes and functions that interfaced with GNURadio such that we were able to capture real-time daata and send it through UDP so that this data could be processed. We were able to set it up such that the laptop (Macbook) acted as the client and the antennas (surface tablet/SDRs) acted as the server such that the server would send the collected data from the servers to the client which would do all the processing for the correlation and angle of arrival data.
However, we faced issues with the hardware as we had two SDR’s running but during real-time analysis of the QT data we noticed that one signal was always more noisy and less strong than the other which ended up having to do with our downmixer. This was discovered by switching out the SMA cable, SMA-RPSMA adapters, and even VCO’s and antennas. We found out that through this process that one of the downmixer outputs transmitted data less strongly than the other downmixwer we ordered which seems to be a factory default.
Upon figuring this out and changing downmixers, we were able to see similar if not almost identical signals on each antenna. We were able to see strong and accurate signals and were able to collect data that would be able to used for correlation and phase detection.
The next step for this week is to make sure the outcome of this data is suitable for beamforming and that beamforming works come demo time. We are sure that our data is correct, but we must check that the beamfoming output is also correct for the heatmap overlay. As we work on this, we will provide dummy data to Txanton such that he can create a dummy heatmap overlay in time for demo
Txanton’s Status Report for 04/10/2021
This week I worked on reading the OpenCV documentation to get a better understanding of how to capture webcam input in python. I was able to find a Python script that allows us to get the webcam input and display it on screen and am trying to figure out the best way to overlay an image onto it. This will allow us to overlay a heatmap onto the camera input so that we can show where the signals are coming from. I also helped to debug the hardware that we were using. For some reason one of the outputs from the downconverter was quite a bit lower powered than the other and we figured out that when we swapped to the other downconverter we did not have this issue, it leads us to believe that something may be faulty with that board and it took a while to figure that out. Once we switched this board we did additional testing on the correlation/synchronization of the SDRs so to help with this I wrote an additional binary for the STM32 to tell the VCO to output a 150 MHz signal so that we could skip the downconverter in order to work with just the SDR’s.
This next week I will help the team integrate the remaining two SDRs and continue working on overlaying images to the camera feed in Python so that we can start getting to the final stages of our project.
Team Status Report for 04/10/2021
Signal Power Calculations
This week, we worked on computing reliable estimates of signal power from the received WiFi signals. Vrishab carried out some theoretical calculations from which we verified that undersampling the captured signals would not affect the received power. The implementation of these calculations in software was carried out by Enock.
Downconverter Debugging
In computing correlations between our captured WiFi signals, we found that one of the signals appeared to be corrupted by a high degree of noise. To debug this, we first determined that the source of the noise was not one of the SDRs. Txanton wrote a script to set the VCO output to a frequency that could be captured by the SDRs, which we were able to use to confirm that the SDRs were not the source of the problem. We found that one of the outputs from the downconverter board we were using was faulty, so we replaced our downconverter with the additional one that we had.
Server Implementation and Correlation Analysis
We implemented a simple UDP server to facilitate real-time data transfer from our data capture apparatus to a computer for processing. The server implements a correlation script that can ascertain the sample delay between signals from each antenna.
Progress Since Last Week
We have greatly improved our signal capture pipeline for two antennas, which was quite slow last week and suffered from incorrect correlations induced by a faulty downconverter.
Tasks for Next Week
We will add two additional RTL SDRs to our system. For now, we will use the faulty downconverter since one of its two lines still works. We will seek a replacement on monday if it cannot be fixed. We will also finish implementing angle of arrival estimation, which is a simple equation applied to the delay between the two signals which we have already obtained