Team D1: Sugar DB

This week is was able to initialize my codebase for experimenting with ML forecasting, and I helped my group with narrowing down our design plans for the presentation next week.

In order to inform our decisions about what training and testing methods will be most appropriate, I devoted the first half of this week to understanding the limitations of the data we have available to us. I figured out how to efficiently access wind, solar, and general weather data and then used the wind data to produce baseline results with a persistence algorithm (predicting the previous value as the next value). This allowed me to output an array of error metrics for a naive forecasting approach that I can use to revise our goals for our models and have as a benchmark for later results. After my experiments and further research into appropriate ML techniques, I spent the latter part of the week developing design plans for the forecasting portion of the project and updating our presentation slides accordingly.

I’d say we are more than on schedule with our progress so far and after we have our design fully nailed down, progress on the codebase should come quickly. Next week, I plan to generate baseline results for our solar data and load data before starting to experiment with some slightly more effective regression models (linear regression, random forest regression). We will also devote time to our design report, taking into account feedback from our design presentation.

This week I mainly focused on researching viable ML models for renewable energy forecasting as well as finding data sources. From my research, i was able to find multiple papers and surveys comparing methods for renewable forecasting. Based on these papers, I was able to narrow my list of algorithms that I’ll start experimenting with to SVM, RNN, ARIMA, and more. From my search for data sources I found global weather data for day-ahead forecasting and model training as well as solar radiation datasets for model validation from the National Solar Radiation Database. Data for wind turbine generation proved more difficult to find for multiple locations, but I was able to find some Kaggle datasets for turbines in Europe that could be used for validation purposes. These sources are linked at the bottom of this post.

My progress is on schedule so far. For next week I plan on setting up a remote repository for work on renewable forecasting and develop a detailed outline for our ML pipeline including plans for data scraping, preprocessing, training, validation, and testing. I intend on focusing first on SVMs as the favorite of the algorithms I found, and I’ll look to make a visual representation of our full ML pipeline.

Weather API:  https://openweathermap.org/api

NSRB Solar Datasets: https://nsrdb.nrel.gov/data-viewer

Kaggle Wind Dataset: https://www.kaggle.com/datasets/berkerisen/wind-turbine-scada-dataset