Team D1: Sugar DB

This week I had to spend some time debugging issues with fully integrating the forecasting models into our web app. After working on those issues, I’ve started focusing on the web app design and implementation of certain demo features that we want ready for Friday. Currently, the web app lacks dynamic and interactive elements as well as good examples to show the extremes of our tool, so I’m working on creating some preset scenarios that users could choose from. These include scenarios like “hot, sunny day” and “cloudy, windy day” and “christmas” to show how loads, renewable energy generation and the grid optimization are affected by external features.

For next week, I plan to contribute to the poster, video and final report. I’ll also spend some more time working on the front end for the forecasting statistics tab to add any visualizations that might be interesting.

I spent this week verifying that our forecasting models are fully performing at the level we expect and are fully integrated into the overall system. In addition, I’ve spent the latter half of the week working on our final presentation, since I will be giving the presentation next week.

For my models, I ended up moving to a simpler architecture (random forest) for all three of my models, which has produced the best results so far. All three of my models are now achieving the goal of NRMSE <20%, including load which was the most difficult dataset to manage. After coding the pipelines for each model so that our web app can call one function to generate predictions for a given location, I also spent some time visualizing trends in our results through graphs with matplotlib, which are useful for verification purposes and to display in our final presentation. I attached an example below for reference, showing the similarity in hourly trends for our loads predictions and actual loads from our test dataset.

Next week, besides giving the final presentation, I will be focused on ensuring our web app interface is completely ready for demo.

Question: As you’ve designed, implemented and debugged your project, what new tools or new knowledge did you find it necessary to learn to be able to accomplish these tasks? What learning strategies did you use to acquire this new knowledge?

I had to learn a significant amount about energy management systems and specifically renewable energy in order to understand how to model them for our project. This mostly consisted of reading through online research papers about renewable energy forecasting to understand the concepts and methods used in the past. I also had to review and learn more about machine learning concepts I had learned in prior classes, which meant looking back at work I’ve done in those classes.

I spent this week working on fixing issues we encountered during the integration of our three subparts as well as trying new methods for training my models that show better results than my previous methods.

Focusing on the model improvements, I was able to use the rankings provided by the Auton Lab’s AutoML tool to select Random Forest Regression as a more effective model for our wind data than LSTMs were. I was also able to discern the most effective preprocessing steps to take as well as optimal hyper-parameters, all of which i used to write a new python script for training and predicting on a random forest regression pipeline. This method achieved impressive results in terms of testing metrics, with an R^2 score of 0.922 and a NRMSE of 9.8%, the best scores we’ve gotten yet and well under our goal for this project. Next week I plan on using the AutoML tool to analyze our load and solar data as well and see if a better model can be used for each of them.

In terms of verification for my portion of the project, I’ve been utilizing Sklearn’s evaluation modules (mean_absolute_error, mean_squared_error, r2_score, etc.) to record metrics for every algorithm I try in my modeling of power data. These metrics are outputted for test data, which is unseen by the model during training to avoid data leakage. In addition, these metrics are compared with the linear regression metrics I generated at the beginning of this project, to ensure improvements have been made. In the absence of a ground truth for predictions made using future weather forecasts, I ensure manually that my results follow a logical trend over time (i.e. solar increases in the day and decreases in the night).