Posted on

Team Status report for 24th February, 2024

What are the most significant risks that could jeopardize the success of the project? How are these risks being managed? What contingency plans are ready?

Currently the major risks we have are ensuring that enough frames are transmitted from the remote camera nodes to the central receiver node, ensuring that our threshold of less than 10% frames being dropped is not crossed. Another issue is the fast decompression of these 6 incoming frames on the FPGA so that the streaming on the monitor is seamless and without any glitches. 

As explained in our presentations, we plan on testing these things in the coming week, and if we face issues with transmission, we plan on adding more data access points on the receiver ESP32, and if there are compute complexities with the FPGA, then we plan on trying to work more on optimizing the system verilog code or try doing some computation on the EPS32 or perhaps switch to a larger FPGA as a worst case scenario. 

Were any changes made to the existing design of the system (requirements, block diagram, system spec, etc)? Why was this change necessary, what costs does the change incur, and how will these costs be mitigated going forward?

Currently no major changes have been made to the system and we are working as per our initial design plan.

Provide an updated schedule if changes have occurred. 

No changes have occurred on our schedule as of now. Things are on track.

This is also the place to put some photos of your progress or to brag about a component you got working.

Here is a video of the FPGA driving a display with the Arduino commanding pixel values: https://photos.app.goo.gl/LBfp1qN6J4SgLGJp9

 

Posted on

Varun’s Status Report for 2/24

What did you personally accomplish this week on the project? Give files or photos that demonstrate your progress. Prove to the reader that you put sufficient effort into the project over the course of the week (12+ hours).

I worked on quite a lot of the work required for the FPGA. My main accomplishments this week are I wrote some code to interface with the HDMI driver. I was able to rewrite the driver to work with our refresh rate and unique setup of compositing multiple images into one frame. I have attached a video showing an Arduino driving some precomputed pixels to the display over its SPI connection.

Is your progress on schedule or behind? If you are behind, what actions will betaken to catch up to the project schedule?

As of right now, I am on schedule so I’m not worried about my progress.

What deliverables do you hope to complete in the next week?

Next week I hope to start work on writing code for the actual JPEG decoder. This will require translating the C implementation that is currently written into hardware.

https://photos.app.goo.gl/LBfp1qN6J4SgLGJp9

Posted on

Michael’s Status Report 2/24/2024

What did you personally accomplish this week on the project? 

For this week, we have the proof of concept JPEG encoder and decoder written and tested end to end. The current code is able to take in an array of RGB values and then compress them using methods inspired by the JPEG specification. The decoder is also there to aid in Varun when he implements the JPEG decoder on the FPGA. The current code is able to achieve a compression ratio of about 5.75:1 which is in line with our design assumptions. 

It is possible to improve this further by truncating the lower bits of Cr and Cb values so that only the most significant 4 bits of each channel are left. With this implemented the compression ratio rises to only slightly above 6:1. My hypothesis is that the bit packing of Cr and Cb is significantly increasing the entropy compared to just encoding Cr and Cb separately. Therefore even though we save encoding an entire channel, the rise in entropy wipes out most of the gains. Since the human eye is only very sensitive to the luminance encoded in the Y channel, the loss of the lower bits shouldn’t significantly degrade the quality issue. 

The only big advantage of this bit packing is the run time. The one where the Cr and Cb values are truncated is almost 25% faster than the regular version. Therefore, I have saved a copy of the code in case we need a little bit more performance and are willing to sacrifice some quality

 

On the toolchain setup front, I was also able to verify my toolchain installation by flashing one of the hello world examples on the boards that arrive earlier in the week and verifying its output if what is expected. 

 

Image Before Compression
Image After Compression
Image Compressed Using Packed Cr & Cb

Is your progress on schedule or behind? If you are behind, what actions will be taken to catch up to the project schedule?

Currently on  schedule, the JPEG encoder and decoder didn’t present too many bugs in the process of writing it

What deliverables do you hope to complete in the next week?

For next week, I hope to run the JPEG encoder and decoder across a more rigorous set of test images to verify its functionality. Assuming all goes well, I plan to also start porting the JPEG encoder code on the ESP32 in anticipation of integration with the camera driver. 

Posted on

Neelansh’s Status Report for 24th February, 2024

What did you personally accomplish this week on the project? Give files or photos that demonstrate your progress. Prove to the reader that you put sufficient effort into the project over the course of the week (12+ hours). 

This past week I worked with my teammates on perfecting the design presentation and solidifying our use case. We then worked on doing more calculations and research on finalizing the hardware and software stack to be used.

I have started working on developing the code for the receiver node (the ESP32 on the receiver node which will then transmit the received data frames to the FPGA). I have finalized the tool chain setup and have figured out how the complete implementation will work. The ESP32 Wifi-Setup has also been completed and I am now working on actually transmitting the data and completing the initial physical hardware setup. I also started testing the code for the JPEG decompression algorithm in C, which will then be used by my teammates to convert into System Verilog and apply on the FPGA. Me and Michael have already tested on a few test images, and the code does seem to work well, giving us a compression ratio of about 5.75:1, which is good enough for our needs. 

Is your progress on schedule or behind? If you are behind, what actions will be taken to catch up to the project schedule? 

We are currently on schedule and all things are on track. If any problems occur, we plan on using the mitigation strategies discussed in our proposal and design presentations. 

What deliverables do you hope to complete in the next week?

In the next week, I will be working on writing the JPEG Buffer which will store the incoming data frames from all the remote camera nodes. I will also be working with my teammates on helping them with their parts and ensuring overall progress of the project. 



Posted on

Team Status Report for 2/17

What are the most significant risks that could jeopardize the success of the project? How are these risks being managed? What contingency plans are ready?

 

As of right now, the main concerns we have are related to the WiFi stack and how much throughput we are actually able to get. We have attempted to minimize these risks by ordering the ESP32s that we plan on using last week. We will implement them as soon as possible and determine if more changes will have to be made. The main contingency plan that we have currently is if the throughput isn’t sufficient, we will include a larger antenna on the ESP32 or setup multiple ESP32s on the central node so that we can use multiple WiFi Channels for the camera nodes. Other measure could include increasing the compression setting on the JPEG compression algorithm or downsamping Cr Cb color components

 

Were any changes made to the existing design of the system (requirements,block diagram, system spec, etc)? Why was this change necessary, what costsdoes the change incur, and how will these costs be mitigated going forward?

 

No changes have been made to the current design. Proof of concept work for the modified JPEG compression algorithm is underway. Once the proof of concept compression algorithm is completed, we will have more information to decide if any aspects of the systems need to be changed. Work on the FPGA is also ongoing and once a proof of concept is complete, we will be able to revisit the design to determine if any changes are needed

Part A was written by Neelansh Kaabra, B was written by Varun Rajesh and C was written by Michael Lang.

 

Part A: 

Our product has immense applications and considerations for public health, safety, and welfare. Our product is a security system, so we essentially care most about people’s safety and welfare, allowing them to monitor their surroundings when they are camping, and giving them an additional layer of protection and information about their campsite. 

 

Being able to monitor the surroundings from all 360 degrees brings a sense of safety and security to the user, thus helping with the psychological and physiological well being of the user too. Our product helps better the public’s psychological state by allowing them to have a higher sense of comfort, while providing them with security benefits.

 

Part B:

This product has some social implications. One of the major benefits are for people who are afraid of the wilderness or have concerns about their safety when they are outdoors. This could bring a more diverse group of people together to go enjoy the outdoors. Beyond that however, I do not believe there are more social impacts. 

 

Part C:

Our product uses commonly available components for all of the different modules. There shouldn’t be a significant issue in getting a system of production setup to mass produce these. The only somewhat specialized component is the FPGA. Even so that FPGA we are using is actively supported by a large company (Lattice). Distribution shouldn’t also be an issue since none of our parts require special handling or have an expiration date. Being cost-effective is one of our major design goals and thus owning the product should be too difficult financially

 

Posted on

Neelansh’s Status Report 2/17/2024

What did you personally accomplish this week on the project? Give files or
photos that demonstrate your progress. Prove to the reader that you put
sufficient effort into the project over the course of the week (12+ hours).

This week was not a lot of physical work, but more so about planning and ideating. We finalized the components we need, and started preparing for the design presentation
next week. I am working on the ESP32 on the receiver node, and consequently spent my time going over data sheets and doing mathematical calculations to finalize the hardware necessary. I also worked on researching the existing tool chains present and how they can be incorporated into our solution.
I then spent time making a basic timeline for my own part, and designing the initial structure of the receiver node, along with the coding components. I spent time writing basic code for the decoding part on the receiver node, and spent time researching online for existing solutions, so I could differentiate ours from them and get the best parts from the existing ones.
I and my teammates also met Professor Tamal and Jason, our TA this week and finalized our plans for the upcoming few weeks.
We also spent time figuring out what products to order for an initial order so that we can start testing, and ended up placing an order for a few items.

Is your progress on schedule or behind? If you are behind, what actions will be taken to catch up to the project schedule?

We are currently on schedule and are waiting for the products to come so that we can start testing.

What deliverables do you hope to complete in the next week?

For next week, we plan on giving the design presentation on either Monday or
Wednesday, and also will be doing basic testing on initial products, to be able to give a better representation of what our final product is going to look like. I will also be getting the Tool chain setup finalized and ready.

Posted on

Michael’s Status Report for 2/17

What did you personally accomplish this week on the project?

For this week I got the Espressif IDF for ESP32 setup on my laptop. The IDF is needed for us to compile and flash code to the ESP32 once we have the hardware in hand. The order for ESP32 was also put in on Monday along with the camera module. I hope to get that in hand soon so I can test out the toolchain that I have installed and debug any issues that may arise.
I also started writing the JPEG encoder code while I am waiting for the ESP32 to come in. My hope is that once the ESP32 comes in, I can immediately port the code over to the ESP32, saving us some time later on. Coding it up on my laptop also serves as a proof of concept on our modified JPEG algorithm and allows us to begin to make optimizations so that it can better run a low power ESP32. So far, I have the RGB to YCrCb color space conversion code done and verified along with an unoptimized version of the discrete fourier transform algorithm. For each one of the encoder components, I also have to write a decoder for it as well to verify functionality

Is your progress on schedule or behind? If you are behind, what actions will be
taken to catch up to the project schedule?

Currently slightly ahead of schedule but it is possible that unforeseen bugs on the compression pipeline can result in it taking longer than initially expected

What deliverables do you hope to complete in the next week?

For next week, I hope to verify my toolchain installation on an actual ESP32 when it arrives. On the compression front, I hope to have a fully working version of the code so that our initial assumptions can be verified.

Posted on

Varun’s Status Report for 2/17

What did you personally accomplish this week on the project? Give files or photos that demonstrate your progress. Prove to the reader that you put sufficient effort into the project over the course of the week (12+ hours).

I worked on quite a lot of the work required for the FPGA. My main accomplishments this week are I wrote some code to interface with the DRAM chip on my FPGA. This will be extremely important is this will serve as the backbone for how the frame buffer for the display will be stored. I also found some sample code online on how to setup HDMI on my FPGA and was able to display a static image. I also spent a lot of time working on the design presentation as well as helping my teammate practice for his presentation next week.

Is your progress on schedule or behind? If you are behind, what actions will betaken to catch up to the project schedule?

As of right now, I am on schedule so I’m not worried about my progress.

What deliverables do you hope to complete in the next week?

I hope to rewrite the HDMI interface  to work with our specific resolution needs as well as integrate it into the DRAM.

 

Posted on

Team Status Report 2/10/2024

What are the most significant risks that could jeopardize the success of the project? How are these risks being managed? What contingency plans are ready? 

 

As described in the project proposal, the main technical risks we have identified in the project are as follows:

 

1.We need to be able to compute frame compression fast enough: 

One of our primary requirements for the project was to ensure that the entire system is inexpensive and requires low power to run. In light of this, we decided to use ESP32 microcontrollers for compressing and transmitting the image frames from the camera. ESP32’s present a compute limitation to the system, and could probably cause the frame compression algorithm to not run as fast as expected. If need be, we plan on switching to difference compression algorithm such as delta compression

2.Stream enough data over the wireless connection:

Our prototype MVP uses 6 cameras, and a single receiver node. The receiver node’s  ESP32 will be receiving data frames from all the 6 camera nodes simultaneously, and hence there will be a high amount of data being transmitted over the wireless connection. We need to ensure that we drop no more than 10% of the data frames. Having less than 10% dropped frames means that there will only be a 100ms of video that will be lost when a frame is dropped. No animal will be able to cross the surveillance area in less than 100ms. In case we exceed that 10% threshold, we plan on increasing the data access points on the receiver node. Frame drop percentage will be computed by comparing the number of frames transmitted vs the number of frames received, and ensuring that the loss percentage does not go above 10%.

3.Decompress all the incoming frames fast enough:

As mentioned above, the receiver node will be collecting data from 6 camera nodes, decompressing them, and then driving the display. The decompression algorithm would have to be fast enough to ensure concurrent streaming from all the 6 camera nodes, without high latency and computation errors. We will be using a FPGA on the receiver node to perform the decompression, and if an issue arises due to the FPGA, we plan on opting for a larger FPGA with a higher compute capability and parallel processing techniques. 

4.Optimize performance to minimize power consumption: 

One of the major requirements from any portable security system is to ensure that we don’t need to charge it often. Keeping this in mind, we envision our system to be able to run for at least 24 hours on a single charge / battery. The entire system’s performance, including the camera’s feed capturing, compression, transmission, decompression and streaming to the portable monitor, all have to be optimized so that the setup works for at least 24 hours at a single charge / battery setting. Our contingency plan for this would be to increase the battery size if the system ends up taking too much power, even after final optimizations. 

 

Were any changes made to the existing design of the system (requirements, block diagram, system spec, etc)? Why was this change necessary, what costs does the change incur, and how will these costs be mitigated going forward? 

 

No major changes have been made yet, however we still need to decide on the final specs for the camera (240p or not), and other technical specs for the microcontroller and monitor, based on our use case. 

 

Provide an updated schedule if changes have occurred. 

No schedule changes as such.

Posted on

Neelansh’s Status Report 2/10/2024

This was the first week after initial team selection and project ideation. We finalized what we will be creating over the course of the semester, and divided up tasks based on each team member’s interests and proficiency. I will be managing more of the software stack, and handling the ESP32 on the receiver node. 

 

My major role will be to ensure that all the data being transmitted from the 6 camera nodes is received appropriately on the receiver, and is then transmitted in the correct format to the FPGA for further processing. 

 

A major task for this week was the Proposal Presentation. I was the speaker, and therefore had to rehearse and plan the presentation with my teammates. I believe the presentation went well, as per the feedback received from my peers and the instructors. There are no images that we took during the presentation, however the proposal presentation slides have been uploaded to the website for reference.

 

Me and my teammates met multiple times over the course of the previous week to finalize the slides and rehearse multiple times before going into the presentation. Apart from that, I timed myself and rehearsed solo by recording myself, talking into the mirror, and planning out exactly what I would be saying.  

 

Is your progress on schedule or behind? If you are behind, what actions will be taken to catch up to the project schedule? 

 

We are currently on schedule and there are no major delays as of yet.

 

What deliverables do you hope to complete in the next week?

 

In the coming week, we plan to do more research into finalizing our use case, and consequently finalizing the hardware specifications we require for fulfilling our goals. This will involve multiple iterations of ideas and selection of suitable hardware, based on both our budget and intended use. I and my teammates will also do mathematical calculations and run simulations (maybe try out different FPGAs, monitor displays, microcontrollers, etc.), over the course of the week to finalize the hardware specifications we need, and then curate a list of all the products we would require.