Posted on

Amelia’s Status Report for March 8

This past week I did not do anything since it was spring break. The week before last I focused on getting the design review done before the Friday deadline. I did not spend as much time getting comfortable with the Vitis EDA tools as I had planned, however, I’m sure over the course of the next few weeks, having hands on exposure will be enough for me to figure things out. I therefore accomplished one out of my two goals from two weeks ago.  I think I am still on schedule.

Given I have quite a busy upcoming week, I plan to focus on figuring out how to pull power data from our new FPGA and also look into multiuser authentication as an extension to our UI. I expect these to be manageable goals for the upcoming week. I would also like to spend some time organizing our website.

Posted on

anirudhp_status_report_03/08/25

This week, I focussed on two primary aspects of the project:

  1. Ethical considerations and how this will adjust the benchmark. For this system, I have made some minor improvements to the model so that it simply refuses to autocomplete certain types of text — eg: Medical, urgent action etc.
  2. Analysing the Microsoft Bitnet Paper in order to suggest performance improvements that we could target.

Overall, the aspects that I was able to achieve are:

  1. Reduced hallucination rate by over 6%, but this was naturally at the expense of the model simply refusing to provide an output.
  2. Identified the Look-up-table implementation and the indexing system as the major speedups which would provide 40% more throughput in the system.

My goals for next week are:

  1. I want to be able to connect with the FPGA wirelessly and transmit the query onto the board(this I can do simply after booting Linux on the core that the board has) so I’d probably do this before we start working on the synthesis flow.
  2. Prepare more on Vitis to see how I would synthesize a basic block that detects the query and pasts the exact same text as the query into the output(this can be seen as a prelim step, we would simply replace the short circuit with our model in order to complete the system)

I wanted to keep pretty conservative goals for this week given that we are finally going to start interfacing with hardware, and this will always come with a number of challenges relating to the setup and the use of the system. At the same time, I still think that the above goals that I have listed are reasonable.

We’re currently well ahead of schedule.(Approx 2 weeks)

Posted on

Team Status Report 03/08/2025

This week we had a couple of targets that were mostly achieved:

  1. We noticed that our setup script had some issues and was a bit unstable to run. But it was still reasonably fast, so we didn’t have any problems running it several times until it worked completely. So we wrapped the script in a loop with a try-except block so that it kept running until the full setup worked. We would have liked to have debugged the code but there are not that many gains that we could have made by doing this, and preferred to focus on the hardware segment.
  2. We analyzed the bitnet paper that Microsoft had published and came up with an overall block diagram that would accelerate the system, and did some preliminary calculations on how much of a speedup we would be able to attain over the classical form of the core that we were using. From the looks of it we would be able to save a number of cycles and shrink the overall size of the arithmetic blocks to meet the speed specificaitons that we had.
  3. We analyzed the ethical impacts of our project and completed the design review report.

 

Over the next week, our aims are:

  1. We should get the final FPGA and then synthesize our base core and model onto it. Using this we want to benchmark the following
    1. Total size footprint — See if we can fit a bigger model in.
    2. Tokens/Sec and latency to first token — This gives us an idea of how much of a speedup we would need over the existing hardware system. We would probably need to adjust the block diagram to meet this value.
    3. Power telemetry — This is a new FPGA so we would need to get an idea of how to pull power data from the new FPGA.
  2. Also, we would like to extend the UI script to interface with the FPGA and start thinking about the authentication and scheduling systems for multi-access. Mainly to see whether it is in fact feasible, not to see how we would implement it.