{"id":17,"date":"2025-01-31T20:39:59","date_gmt":"2025-01-31T20:39:59","guid":{"rendered":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/s25-teame1\/?p=17"},"modified":"2025-01-31T20:39:59","modified_gmt":"2025-01-31T20:39:59","slug":"amelias-status-report-for-feb-1st","status":"publish","type":"post","link":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/s25-teame1\/2025\/01\/31\/amelias-status-report-for-feb-1st\/","title":{"rendered":"Amelia’s Status Report for Feb 1st"},"content":{"rendered":"

This week I explored a number of trained BitNets that are supported by microsoft’s bitnet inference framework<\/a>. The goal of this was to find a model that would be small enough to fit on an FPGA, but worked well enough to be repurposed into a viable product.<\/p>\n

Initially, we wanted to work with a model<\/a> that had only 70M parameters, in the hopes that we could fit that model on any FPGA we wanted. However, after trying to chat with it, I found that the low number of parameters contributed to very poor performance as seen in my conversation with it below:<\/p>\n

\"\"<\/p>\n

I tested a few more models with larger parameters (up to 10B) from this family of models.<\/a>\u00a0While they perform significantly better,\u00a0 these models are too large to fit on any FPGA we can afford (the 10B parameter model is around 3GB after quantization). I ultimately settled on this model<\/a>, that has around 700 million parameters and is around 300 MB after quantization. This model is for text completion, as you can see below, so that is likely the direction we will take for our final project.<\/p>\n

\"\"
The prompt here was “what did I do today?” and it autocompleted the rest<\/figcaption><\/figure>\n","protected":false},"excerpt":{"rendered":"

This week I explored a number of trained BitNets that are supported by microsoft’s bitnet inference framework. The goal of this was to find a model that would be small enough to fit on an FPGA, but worked well enough to be repurposed into a viable product. Initially, we wanted to work with a model … <\/p>\n

Continue reading “Amelia’s Status Report for Feb 1st”<\/span><\/a><\/p>\n","protected":false},"author":1067,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[3],"tags":[],"_links":{"self":[{"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/s25-teame1\/wp-json\/wp\/v2\/posts\/17"}],"collection":[{"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/s25-teame1\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/s25-teame1\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/s25-teame1\/wp-json\/wp\/v2\/users\/1067"}],"replies":[{"embeddable":true,"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/s25-teame1\/wp-json\/wp\/v2\/comments?post=17"}],"version-history":[{"count":2,"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/s25-teame1\/wp-json\/wp\/v2\/posts\/17\/revisions"}],"predecessor-version":[{"id":28,"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/s25-teame1\/wp-json\/wp\/v2\/posts\/17\/revisions\/28"}],"wp:attachment":[{"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/s25-teame1\/wp-json\/wp\/v2\/media?parent=17"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/s25-teame1\/wp-json\/wp\/v2\/categories?post=17"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/s25-teame1\/wp-json\/wp\/v2\/tags?post=17"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}