Posted on

Michael’s Status Report 4/6

What did you personally accomplish this week on the project? 

 

For this week, I wrote a python script that would simulate the central node’s function so I could test the Wi-Fi link for stability. The script will listen for an incoming packet and then perform the needed decompression steps before it displays it as a live video feed. The script also allowed me to explore the limits of the current Wi-Fi systems by adjusting the stream quality to see at what level we start seeing dropped frames. I found that the Wi-Fi is robust enough to support the encoder running at nearly the highest quality without significant frame drops. 

In addition, we also switched the antenna jumpers on the ESP over to the external IPEX  connector. This is so that I can use an external patch antenna for better transmission quality. Previously, I was just using the built-in PCB antenna which is suboptimal since it is so small and limited by the PCB area. With this new external antenna, I was able to get about 30 meters of range while maintaining a good video feed. The test was performed indoors with a noisy RF environment and in a non-line of sight situation with two brick walls between the receiver and the transmitter. I’d expect this number to increase significantly when I run this test outdoors and in a line-of-sight environment. The current patch antenna that we have on it is a 3dBi omnidirectional one. This can always be switched out to a antenna with higher gain if we need better performance

The final thing that I was able to get done this week was verify that I could connect to the receiving ESP32 and have it read valid data. There were a few things that had to be fixed on the receiving side which was mainly to enable IPv4 fragment reconstruction. Fragment reconstruction is needed since the packet size exceeds that of Wi-Fi MTU of around 1550 bytes. 

Link to video of streaming python script: https://drive.google.com/file/d/1OHjypw3lSpivNJLjFea4oVtCaHKOl_wL/view?usp=sharing
 

Is your progress on schedule or behind? If you are behind, what actions will be

taken to catch up to the project schedule?

 

Currently on schedule

 

What deliverables do you hope to complete in the next week?

 

For next week, I hope to run the range test outdoors to verify my assumptions

Posted on

Neelansh’s Status Report for 6th April, 2024

What did you personally accomplish this week on the project? Give files or photos that demonstrate your progress. Prove to the reader that you put sufficient effort into the project over the course of the week (12+ hours). 

This week I worked on finalizing everything for the demo, since our demo was on Wednesday for us. I developed and tested the ESP32 receiver node and worked with my teammates to allow for transmission from not just one ESP32, but from multiple different ESP32s acting as STA’s to one single ESP32, acting as an AP. 

I then worked on the one part I was unable to display in the demo, the SPI interface. I developed the entire SPI interface and then tested it via Oscilloscopes and Arduinos to ensure that the entire transmission via the interface was working, and that we were sending the correct byte streams.

Is your progress on schedule or behind? If you are behind, what actions will be taken to catch up to the project schedule? 

I am on schedule.

What deliverables do you hope to complete in the next week?

Next week I plan on integrating these two parts : receiving data from multiple streams, and then sending it to the FPGA via the SPI interface. After that I would work with my teammates to start building a module and then start testing the entire system.

Now that you have some portions of your project built, and entering into the verification and validation phase of your project, provide a comprehensive update on what tests you have run or are planning to run. In particular, how will you analyze the anticipated measured results to verify your contribution to the project meets the engineering design requirements or the use case requirements?  

For the verification of my work, I plan on conducting exhaustive tests to measure the correctness and efficiency of my system.

I am currently able to send basic data from one ESP32 to the other and receive it correctly with 100% correctness (no packet drops). 

I plan on sending real time image data (to simulate real world use cases) from multiple ESP32s acting as STA (remote node), to one single ESP32 acting as the AP (receiver node), with multiple data access points. I would then test this with multiple ESP32s sending data at a single point of time to satisfy the use case requirements set in the earlier stages of having at least 6 different camera nodes being able to transmit data simultaneously. 

Apart from this, I would also test the SPI interface with oscilloscopes to ensure that all the 6 different camera streams are coming in correctly. This would also include testing in real life scenarios, with variations in the distance between the sender and receiver (10m, 20m, 30m, … until failure), and also with varying number of objects in between them to try and recreate the camping site. I will go to Schenley park and conduct my tests there to ensure that the trees or the foliage does not cause my system to fail. Lastly, I would run the entire system for hours at stretch (try to find the breaking point), and use python scripts to continuously test for correctness and ensure that the packet losses are within the limits set earlier.  

 

Posted on

Team Status Report For 3/30

What are the most significant risks that could jeopardize the success of theproject? How are these risks being managed? What contingency plans are ready?

We have mainly addressed main risk from last week. We were able to solve this through the setup of the WiFi on the ESP so that we were able to free up more bandwidth for the image encoding the remote node size.

Because of this being tested and is reliable, we currently do not have any further risks. We are happy with this solution as it did not require us to reduce the image quality being sent from the camera.

Were any changes made to the existing design of the system (requirements,block diagram, system spec, etc)? Why was this change necessary, what costs does the change incur, and how will these costs be mitigated going forward?

No changes

Posted on

Michael’s Status Report 3/30

What did you personally accomplish this week on the project? 

 

For this week, I got the performance of the JPEG pipeline working at a sufficient level to enable 10fps streaming and verified it using wireshark. Since integration has not yet happened, I used the hotspot of my phone to act as an access point and have the ESP32 join that access point. I also then joined my computer with wireshark. The ESP32 was then commanded to send data to my computer via the phone’s access point. When the packet reached the laptop, the time was noted down for measurement purposes. Since the maximal transmission unit is 1500 bytes, there is a need for packet fragmentation. Therefore, to measure the time correctly, those packet fragments must be discarded. The picture below is the measurement result showing that we are able to hit a frame interval of about 100ms which corresponds to 10fps. 

Is your progress on schedule or behind? If you are behind, what actions will be taken to catch up to the project schedule?

Currently on schedule

 

What deliverables do you hope to complete in the next week?

For next week, I will begin integration with Neelansh for the receiving node. The remote node is now functionally complete with the only thing that is missing being the detection of movement using the luminance channel data. 

Posted on

Varun’s Status Report for 3/30

What did you personally accomplish this week on the project? Give files or photos that demonstrate your progress. Prove to the reader that you put sufficient effort into the project over the course of the week (12+ hours).

This week was mostly spent into integrating the JPEG decoder into the pipeline and debugging issues that arose. Another big piece of the puzzle that I had missed was converting the decoded YCbCr output of the JPEG pipeline back into RGB.
This was fairly tricky to implement as this required a lot of floating point computation. I had to work on pipelining the design so that I could actually fit the design onto the FPGA. There are about 28 18×18 multipliers available on the FPGA, each pixel required about 4 of the 18×18 multipliers so I had to make sure to find a way to sufficiently parallelize/sequentially the decoding to use the multipliers the most appropriately.

Is your progress on schedule or behind? If you are behind, what actions will betaken to catch up to the project schedule?

As of right now, I am on schedule so I’m not worried about my progress. Though it will be a little tight to get everything in for the interim demo.

What deliverables do you hope to complete in the next week?

I plan on hopefully getting everything sorted out for the interim demo. 

Posted on

Neelansh’s Status Report for 30th March, 2024

What did you personally accomplish this week on the project? Give files or
photos that demonstrate your progress. Prove to the reader that you put sufficient
effort into the project over the course of the week (12+ hours).

I worked on the SPI interface and setting everything up, including data access points on the receiver ESP32 node, setting up the SPI interface and then transmitting data through it, ensuring that the data is not just garbled bytes but actual legitimate data, and then worked on ensuring the entire system works together as one module. This involved setting everything up together, using oscilloscopes to measure the data being transmitted via the SPI interface, and then also sending and receiving data from multiple ESP32s.

Is your progress on schedule or behind? If you are behind, what actions will be
taken to catch up to the project schedule?

It is on schedule.

What deliverables do you hope to complete in the next week?

Next week I plan on doing the interim demo, and working with my teammates on getting their components ready for a final test, and then start joining the individual components built by each of us together.

Posted on

Varun’s Status Report for 3/23

What did you personally accomplish this week on the project? Give files or photos that demonstrate your progress. Prove to the reader that you put sufficient effort into the project over the course of the week (12+ hours).

This week was mostly worked on improving the speed of the JPEG decoder so that it better meets timing. I’ve included a copy of the SystemVerilog file for this. Previously, the design ran at about 100MHz, but with a better pipeline (8 stage pipeline to process 8 pixels of the MCU), I’m able to increase the throughput by a factor of 16. I’m able to better utilize the resource of the FPGA to process more pixels per clock and also increase the clock speed up to around 200MHz. This should make it more possible to handle the effective 120FPS requirement from the input streams.

Is your progress on schedule or behind? If you are behind, what actions will betaken to catch up to the project schedule?

As of right now, I am on schedule so I’m not worried about my progress.

What deliverables do you hope to complete in the next week?

I plan on integrating this design more into the current pipeline. Right now the JPEG processor stands along but I need to incorporate the SPI interface to it as well as appropriately pass it to BRAM so that the display can view the image.

Posted on

Team Status Report For 3/23

What are the most significant risks that could jeopardize the success of the

project? How are these risks being managed? What contingency plans are ready?

The most significant risk right now is the encoder side. As mentioned in Michael’s status report, there are severe constraints on what we can do on the encoder side due to the need to use the PSRAM module on the ESP32. The PSRAM module’s 40 MB/s is a hard limit that is hard for us to work around. Being one of the first stages of the pipeline means that any changes in the encoder side will trickle down and cause issues for the central receiving node and the FPGA decoder. Current contingency plans in case this PSRAM issue does materialize is to set the encoder at a lower quality which will minimize the amount of data that needs to be processed by the LWIP and Wi-Fi stacks. The reduced workload will in turn alleviate the pressure on the PSRAM module.

 

Were any changes made to the existing design of the system (requirements,block diagram, system spec, etc)? Why was this change necessary, what costs does the change incur, and how will these costs be mitigated going forward?

No changes

Posted on

Michael’s Status Report for 3/23

What did you personally accomplish this week on the project? 

 

For this week, I finally got the compression code ported over to the EPS32. This means that we now have all the steps that are needed on the remote camera node coded up and working. The remote camera node is now able to take a picture, compress it and then send it over via Wi-Fi to a receiver. Porting this over was significantly harder than I thought since we kept running into issues with the camera driver co-existing with Wi-Fi code. In integrating these two, we kept running into issues where the Wi-Fi sending code would stop transmitting at random intervals which would then incur a packet fragment time to live exceedance error message on the receiver. After a lot of debugging and configuration changes, I was able to solve the issue by making some changes on the encoder side and pinning the Wi-Fi task to core 1, which leaves core 0 free to handle the camera. However, this performance is still on the lower side since we are not limited by the bandwidth to the PSRAM. The PSRAM lives on a QSPI bus that runs at 80MHz. Thus, we are limited to a maximum of 40 MB/s of memory bandwidth and unknown latency. The internal data DRAM is only 320KB of size thus it is not an option for us to store a complete frame buffer. Keep in mind that this PSRAM is shared between LWIP, camera, Wi-Fi, and compression. 

 

Image captured, encoded, and then transmitted from ESP32

 

Is your progress on schedule or behind? If you are behind, what actions will be

taken to catch up to the project schedule?

 

Currently on schedule

What deliverables do you hope to complete in the next week?

 

For next week, I hope to begin integration with the FPGA. This will mostly entail me providing a bunch of test data to Varun which he will then run through the FPGA to make sure that we are in agreement in regards to the data format and algorithm stages

Posted on

Neelansh’s Status Report for 23rd March, 2024

What did you personally accomplish this week on the project? Give files or photos that demonstrate your progress. Prove to the reader that you put sufficient effort into the project over the course of the week (12+ hours). 

This week was a long one. I worked on building the QuadSPI interface for transmitting data from the receiver ESP32 to the lattice ECP5 FPGA. This involved understanding the firmware of the FPGA to understand how to set up the quad data transfer points and send bits. I also had to write code to set up the SPI on the ESP32 after reading and understanding the entire manual. I had to experiment with the clock speed, configure the Quad SPI mode based on the clock polarity and clock phase that would match the FPGA (slave device). All these tasks involved reading, writing code, and extensive debugging. I also attended the ethics seminar and actively contributed to the discussion.

Is your progress on schedule or behind? If you are behind, what actions will be taken to catch up to the project schedule? 

The project is on schedule.

What deliverables do you hope to complete in the next week?

Next week, I plan on integrating the QuadSPI interface with the ESP32 and start receiving data on the ESP32, serializing it, and then transmitting it to the connected FPGA to simulate real world conditions.