Posted on

Anya’s Status Report for 03/08/25

Work Accomplished :

  • This week, I focused on developing components of the system that interact with Home Assistant and Nordpool electricity pricing data. The primary tasks included designing and implementing the ESPHomeClient, HomeAssistantClient, and NordpoolClient while also working on the frontend and backend integration.
  • Completed the design report (System Architecture, Quantitative Design Requirements, System Implementation, Testing Methodology)
  • Started working on the frontend + backend functionality of the web app


Implemented an ESPHomeClient to enable communication between ESPHome-based IoT devices and Home Assistant. This module allows devices to send and receive sensor data while maintaining real-time connectivity with Home Assistant.

    • Designed a HomeAssistantClient to interact with Home Assistant’s API, enabling data retrieval and control over smart home devices.

    3. NordpoolClient

    • Built a NordpoolClient to fetch electricity prices from Nordpool via Home Assistant.
    • Integrated API calls to retrieve real-time electricity prices and structured the data for easy analysis.

    On the frontend side, I developed and integrated an energy flow chart that visually represents energy distribution and load optimization within the system.



    Progress:

  • On track with regards to the schedule and GANT chart
  • Currently waiting for integration with Raspberry Pi and  power sensors via ESPHome.
  • Once hardware integration is complete, live data from the sensors will replace simulated inputs in the ML and forecasting module

Tasks to complete next week

  • Conduct end-to-end testing of energy data collection, processing, and visualization.
  • Compare original vs. optimized energy loads to assess efficiency improvements.
  • Identify any bottlenecks in data flow between ESPHome, Home Assistant, and the dashboard.
  • Start integrating predictive analytics for energy consumption forecasting via some sort of inference API
Posted on

Team’s Status Report for 02/22/25

Risks

  • Running ML models on Raspberry Pi could cause performance limitations due to limited computing power. To address this, I will optimize models for efficiency and consider offloading computations to an external server.
  • Additionally, compatibility between ML frameworks and Home Assistant might pose challenges.I will validate API integrations using Postman to address this.

Changes

No major changes in implementation/design yet. After trying to deploy the ML model on RPi, if the RPi has limited storage/processing power, will deploy the ML forecasting on a computer.

Progress

  • Automation setup with Home Assistant.
  • Initial implementation of optimization models.
  • Model training and prediction framework.
  • CSV data storage and Docker integration.
  • Nordpool grid price retrieval setup.
Posted on

Anya’s Status Report for 02/22/25

Accomplishments this week

Home Assistant Automation Implementation [Integration Platform]

Defined shell commands and set up tasks in config yaml files for data publishing via the HA API

Grid Price Integration from Nordpool

  • Configured retrieval of real-time electricity prices from Nordpool.
  • Integrated Nordpool API into the optimization workflow to factor dynamic pricing.
  • Implemented logic to optimize energy consumption based on real-time grid prices.
  • Instantiated the grid price optimization problem using plp.LP
Challenges & Next Steps
  • Sensor Deployment: I am awaiting final integration of sensors to obtain real-time load consumption data
  • Performance Optimization:  I need to fine-tune the grid pricing model for higher accuracy.
  • Scalability: Need to expand the system to support multiple optimization strategies dynamically.
  • Progress and Schedule
  • Completed foundational tasks and progressing into real-time data collection and optimization refinements.
  • Minor delays in sensor delivery [due to snow storm] could impact the timeline for model training.
  • Action Items for Next Week
    • Deploy sensors and verify real-time data collection.
    • Train models using load consumption data
    • Conduct further tests on the dayahead optimization strategy.
    • Optimize automation workflows in Home Assistant.
Posted on

Team’s Status Report for 15/02/25

Risks

  1. Need to test how different LP solvers (CVX, PuLP etc) output different results for the optimization and choose which one is the most accurate at reducing energy consumption cost

Changes

Direct communication between the ESP32 and Django backend for storing load consumption data

Schedule & Progress

  • Optimization algorithm framework finalized
  • Design specifications (including components in the embedded layer, ML model, backend and mobile app and communication protocols between these finalized)
  • Design presentation done
  • external API libraries and datasets (and data collection mechanisms for ML models) identified
  • Sketched up a model of the demo house with dimensions and ordered materials accordingly.

    Additional Weekly Questions (A written by anyab, B written by Erika , C written by Maya)
  • Part A :
  • Health: SmartWatt contributes to community health by reducing reliance on non-renewable energy sources (increasing reliance on solar panel energy, which will help decrease air pollution in the long run through all of the fossil fuels that haven’t burnt). Additionally, SmartWatt has an optimization function that maximizes consumption of solar power, thereby reducing the need for drawing power off the grid
  • Welfare: By optimizing energy costs and enhancing efficiency, SmartWatt supports economic welfare, reducing household energy expenses and ensuring consistent power availability for essential needs.
  • Safety : Users can see what their power consumption for different appliances are, and can determine if an appliance is faulty based off the power profile.
  • B : SmartWatt, our household energy optimization and monitoring system, addresses the growing need for sustainable energy management within residential communities. Rising energy costs and increasing awareness of climate change have created a strong societal demand for solutions that reduce consumption, lower utility bills, and promote environmentally responsible behavior. SmartWatt empowers homeowners to monitor and optimize their energy use through real-time analytics, personalized recommendations, and automated control of appliances. By reducing energy waste, SmartWatt not only decreases household expenses but also contributes to broader efforts to lower carbon emissions and combat climate change—issues that affect society as a whole.Socially, SmartWatt fosters a culture of energy-conscious living within neighborhoods and communities. The platform can be extended to include social features that allow users to compare their energy savings with neighbors, participate in local energy-saving challenges, and share tips, encouraging collective action toward sustainability. Additionally, SmartWatt’s affordability and user-friendly interface ensure accessibility for diverse socioeconomic groups, helping low-income households reduce utility costs and alleviate energy poverty. By promoting sustainable habits and supporting energy equity, SmartWatt advances social and environmental responsibility, creating a positive impact on both individuals and their communities

C: Consideration of Economic Factors

SmartWatt is designed to optimize energy consumption in residential homes, directly impacting economic factors by reducing household electricity costs. By integrating real-time monitoring, predictive analytics, and automation, SmartWatt ensures that homeowners can minimize unnecessary energy expenditure, ultimately leading to lower utility bills. The optimization algorithms leverage data from smart sensors to shift power usage to off-peak hours, allowing users to take advantage of lower electricity rates where time-of-use pricing is available.

On a broader scale, SmartWatt contributes to economic efficiency by reducing strain on power grids, decreasing the demand for expensive energy production during peak times. The system also enables better load balancing, which can prevent overuse and extend the lifespan of household appliances, reducing replacement and maintenance costs over time. By making energy management more accessible and cost-effective, SmartWatt supports economic sustainability and affordability, ensuring long-term financial benefits for users while promoting responsible energy consumption at a larger scale.

However, the initial cost may be high as many appliances will need to be wired into Home Assistant for both control and monitoring. With stuff like the ESPHome it is a lot cheaper given some technical knowledge

Posted on

Anya’s Status Report for 15/02/25

[Code can be viewed at : https://github.com/erika-24/smart-watt]

  1. Defined system parameters for power optimization (including the decision variables, established constraints for battery and load management, and objective function which is a function of grid power and solar energy produced)
  2.  Started working on an optimization class that balances energy production, consumption, and storage.
  • Utilized linear programming to optimize deferrable loads.
  • Considered cost function variations: profit, cost minimization, and self-consumption.
  • Used PuLP LP solver to solve the problem.
  • Ran preliminary simulations on sample energy dataFinalized System Components
    • Selected SQLite as the database for storing load consumption and power data, as well as outputs from the ML model
    • Chose appropriate power sensors (INA226) and relay switches, as well as load profiles for the appliances for our model house
    • Defined the embedded layer for system control (including all interactions between RPi5, Django backend, ML model, circuit components)

Completed the block diagrams for the design presentation and went over it as I will be presenting this week

Schedule & Progress



Waiting on Maya to send me real-time power consumption data once the power sensors arrive, so while the load consumption forecast model is almost done, it needs to be trained on real time data. I would say I am on schedule.

Next Steps

  • Implement improvements to the LP solver configuration (including implementing the self consumption objective function)
  • Begin integration with IoT power monitoring system (research how HTTP requests between ESP32 and the backend occur)
  • Conduct further simulations to validate the algorithm’s robustness (on more real-time load consumption data)
  • Start working on generating a data frame for grid pricing (via API calls)

 

Posted on

Team Status Report for 02-08-2025

Presented the proposal to the class

Risks : 

  1. Right now, our load forecasting model is trained on synthetic data. Main risk is overfitting to synthetic data. Once we switch to real-time power consumption logs from ESP32 sensors, our model may struggle to adapt to real-world variations such as sudden spikes in energy usage (a heater turning on) or unusual load drops (someone turning off all appliances).

Mitigation : Collect real IoT data logs from ESPHome devices to improve model accuracy. Implement outlier detection to ignore unusual spikes or drops in power usage. Use XGBoost & RandomForest models, which handle nonlinear patterns better. Will plan to use multiple forecasting models (XGBoost, ARIMA, LSTMs) and use ensemble methods to combine predictions.

2. The reliability and quality of the optimization algorithms are directly dependent on the quality of the forecast data (for load forecasting + solar power + grid prices). Lower forecast errors lead to more accurate optimization results, whereas high forecast errors can lead to bad optimization outcomes

Changes : 

No significant design changes have been made to the system yet.

Progress:

  • Solcast API integration for access to real-time weather forecasts
  • Load forecasting module that predicts load for next 24h within household
  • Set up home assistant instance
  • Connect ESPHome to home assistant
  • We have researched and validated the necessary electronics components to execute our plan

 

Posted on

Anya’s Status Report for 2/8/25

Work Accomplished  (code available at https://github.com/erika-24/smart-watt)

  • Started working on a ML-based time-series forecasting model to predict load demand from training dataset (semi-synthetic for now). Used skforecast models (LinearRegression, ElasticNet, KNeighborsRegressor, RandomForestRegressor, XGBoost).
  • Some challenges while writing code : How to handle missing values in data. I used forward fill and backward fill for now, will try using other interpolation methods as well as see what the output looks like if rows with missing data are deleted.Final output: Generated a 48-step ahead forecast (30-minute intervals) for future load demand.

For the photovoltaic power output forecast,

  • Integrated Solcast API to fetch solar power forecast data for energy optimization.
  • Implemented API calls with rooftop ID and API key authentication.
  • Parsed JSON responses and converted them into Pandas DataFrames for easier analysis

I also started doing research on the cost function and objective functions as well as constraints to optimize.  The approach will likely be a combination of linear programming (objective fn is affine) and model predictive control.

Progress :

My progress is on track with regards to the Gantt Chart. I set up synthetic data pipeline for now, but will switch to live IoT device logs.

Things to do for next week:

Current Approach: Using synthetic training data for load forecasting.
Next Step:  Retrieve real-time power consumption logs from IoT/smart meter devices that we plan to use for the demo.
1.  Set up an IoT device integration (ESP32 with a smart plug for getting power consumption data).
2. Use MQTT or ESPHome API to stream real-time power consumption data.
3.  Store device logs in InfluxDB or a time-series database.

Current Approach: Integrated PV output from Solcast, but not yet synchronized with load forecasts.
Next Step: Merge solar and load forecasts to optimize energy storage & grid consumption.

Action Plan:
1. Align solar forecast timestamps with predicted load timestamps.
2. Start implementing the optimization algorithm (Linear Programming, MPC) to balance solar vs. grid power usage

3. Evaluate forecasting accuracy with r2_score and MAE/RMSE

4. Implement hyperparameter tuning for XGBoost to improve accuracy