Category: Anya’s Status Reports

Work Accomplished

• Implemented conversational AI interface allowing users to query the system about energy optimization at their home.
• Created suggestion system that guides users with prompt examples for better engagement with the AI assistant
• 
• The optimization system was enhanced with a user preferences framework that allows the user to select custom timeslots in which to run devices.
• The system incorporates earliest start times and latest end times for each scheduled device.
• Then designed a new constraint-based optimization that respects user preferences while maximizing energy savings
• A three-tier priority system was implemented:
  • Low Priority: Maximizes energy savings with flexible timing (priority weight: 0.3)
  • Medium Priority: Balances energy savings with user-preferred times (priority weight: 0.5)
  • High Priority: Strictly adheres to user time preferences (priority weight: 0.8)
• Priority settings directly influence how the optimization algorithm weighs time constraints against cost savings
• 
• 
• 

Progress

Frontend and backend up and ready.

Right now, I need to implement the API routes that will actually trigger device control — switching devices on or off based on user input or automated schedules. Slightly behind schedule with regards to the GANT chart.

These routes will act as the bridge between the UI actions and the actuation layer. Once the endpoints are set up and mapped to the appropriate device control logic, the system will be able to execute real actions, completing the loop from user interaction to physical outcome.

Next Steps

• Tie each route to the code that interacts with the device (ESP 32GPIO pins/ Home Assistant API).
• Then test with real devices to validate actual switching.
• Refine ML or linear programming algorithms that decide when to turn devices on/off.
• Incorporate feedback loops from usage data.

Work Accomplished:

• Did some testing of a LSTM neural network architecture for time-series energy consumption prediction
• Tracked self-consumption metrics for solar utilization, peak reduction and cost savings % according to a baseline.
• The original cost is based on the predicted load with original device schedule. The optimized cost uses the new schedules with shifted loads

Mock backend for schedules : (I need to parse the results of optimization algo and figure out whats a good way to put that into device schedules). This is what the frontend should look like

Current display of predicted vs optimzied power. Need to scale this up accordingly once the power sensors are connected to Home Assistant and download the data from there.

Progress: I would say I am a little behind because a) the power sensor data is not logged in, need to do the integration of the backend with Home Assistant to feed that data, look at transience to analyze average power consumption b) The grid pricing data is fed in through Nordpool which has an integration  with Home Assistant, but for now I am training the LSTM by downloading a csv from Home Assistant rather than having it dynamically via an API (API calls are expensive)

Challenges and Next Steps:

• Improve load shifting algorithm with better device priority handling [figure out a way to parse info about optimized loads from an aggregate to a device level]
• Also right now, all devices have the same priority.  Need to assign weights based on device priorities (like running a fridge vs fan, fridge is way more important)

• Use these weights in the objective function to favor high-priority loads when minimizing cost or peak demand.

• Add detailed logging for performance metrics to validate optimization results

• Integration between HA and backend would be just creating a docker container and adding that to HA vs API requests

Work Accomplished:

Spent a significant amount of time debugging the optimization constraints and attempting different techniques to resolve DCP rule violations in the optimization model. To resolve it, I made sure the SOC dynamics equation is now linear in the decision variables, added fallback options (within CVXPy library) to try different solvers if the primary ECOS solver fails.

The SGD and L-BFGS-B methods are somewhat working, for the linear programming solvers its thinking the constraints are of conic form so I need to debug that.

Here is the output for SGD

Started working on a recommendations analyzer that processes energy data to identify patterns and anomalies. I developed a classification system that categorizes insights by priority and potential impact. Added functionality to track user actions on recommendations to improve future suggestions

Worked on the recommendations frontend.

For the frontend, I implemented a responsive dashboard that visualizes energy usage, production, and optimization results across devices

Challenges

One of the main challenges in SGD and LGBFS optimization algorithms is that the forecasted energy demand is very close to the actual load.

• If the forecasted load closely matches the actual load, then the gradient of the cost function is near-zero.
• This results in very small updates to optimization variables, reducing the ability of the algorithm to adapt or improve scheduling decisions.
• Mitigation Technique

Progress

I am on track regards to the schedule. Still waiting on the hardware integration to be complete and polling data from the power sensors, which will be fed into a database and processed to be displayed on the frontend.

Next Steps

• Power sensor data arrives at high frequency (every few seconds or milliseconds). A proper database in SQLite is required to be created for  logging and retrieval.
• When calculating average power consumption, factor in transients when devices switch on/off

• Develop interactive dashboards that display key metrics from the optimization algorithm, such as grid usage, battery state of charge, device schedules, and cost calculations.

• Start working on the integration of the backend with Home Assistant and ESPHome firmware. I have created the clients and API requests (communication protocol library) but actually need to start debugging

Work Accomplished :

• This week, I focused on developing components of the system that interact with Home Assistant and Nordpool electricity pricing data. The primary tasks included designing and implementing the ESPHomeClient, HomeAssistantClient, and NordpoolClient while also working on the frontend and backend integration.
• Completed the design report (System Architecture, Quantitative Design Requirements, System Implementation, Testing Methodology)
• Started working on the frontend + backend functionality of the web app

Implemented an ESPHomeClient to enable communication between ESPHome-based IoT devices and Home Assistant. This module allows devices to send and receive sensor data while maintaining real-time connectivity with Home Assistant.

• • Designed a HomeAssistantClient to interact with Home Assistant’s API, enabling data retrieval and control over smart home devices.

  3. NordpoolClient

  • Built a NordpoolClient to fetch electricity prices from Nordpool via Home Assistant.
  • Integrated API calls to retrieve real-time electricity prices and structured the data for easy analysis.

  On the frontend side, I developed and integrated an energy flow chart that visually represents energy distribution and load optimization within the system.

  
  
  Progress:

• On track with regards to the schedule and GANT chart
• Currently waiting for integration with Raspberry Pi and  power sensors via ESPHome.
• Once hardware integration is complete, live data from the sensors will replace simulated inputs in the ML and forecasting module

Tasks to complete next week

• Conduct end-to-end testing of energy data collection, processing, and visualization.
• Compare original vs. optimized energy loads to assess efficiency improvements.
• Identify any bottlenecks in data flow between ESPHome, Home Assistant, and the dashboard.
• Start integrating predictive analytics for energy consumption forecasting via some sort of inference API

Accomplishments this week

Home Assistant Automation Implementation [Integration Platform]

Defined shell commands and set up tasks in config yaml files for data publishing via the HA API

Grid Price Integration from Nordpool

• Configured retrieval of real-time electricity prices from Nordpool.
• Integrated Nordpool API into the optimization workflow to factor dynamic pricing.
• Implemented logic to optimize energy consumption based on real-time grid prices.
• Instantiated the grid price optimization problem using plp.LP
• 

Challenges & Next Steps

• Sensor Deployment: I am awaiting final integration of sensors to obtain real-time load consumption data
• Performance Optimization:  I need to fine-tune the grid pricing model for higher accuracy.
• Scalability: Need to expand the system to support multiple optimization strategies dynamically.
• Progress and Schedule
• Completed foundational tasks and progressing into real-time data collection and optimization refinements.
• Minor delays in sensor delivery [due to snow storm] could impact the timeline for model training.
• Action Items for Next Week
  • Deploy sensors and verify real-time data collection.
  • Train models using load consumption data
  • Conduct further tests on the dayahead optimization strategy.
  • Optimize automation workflows in Home Assistant.

[Code can be viewed at : https://github.com/erika-24/smart-watt]

1. Defined system parameters for power optimization (including the decision variables, established constraints for battery and load management, and objective function which is a function of grid power and solar energy produced)
2.  Started working on an optimization class that balances energy production, consumption, and storage.

• Utilized linear programming to optimize deferrable loads.
• Considered cost function variations: profit, cost minimization, and self-consumption.
• Used PuLP LP solver to solve the problem.
• Ran preliminary simulations on sample energy dataFinalized System Components
  • Selected SQLite as the database for storing load consumption and power data, as well as outputs from the ML model
  • Chose appropriate power sensors (INA226) and relay switches, as well as load profiles for the appliances for our model house
  • Defined the embedded layer for system control (including all interactions between RPi5, Django backend, ML model, circuit components)

Completed the block diagrams for the design presentation and went over it as I will be presenting this week

Schedule & Progress

Waiting on Maya to send me real-time power consumption data once the power sensors arrive, so while the load consumption forecast model is almost done, it needs to be trained on real time data. I would say I am on schedule.

Next Steps

• Implement improvements to the LP solver configuration (including implementing the self consumption objective function)
• Begin integration with IoT power monitoring system (research how HTTP requests between ESP32 and the backend occur)
• Conduct further simulations to validate the algorithm’s robustness (on more real-time load consumption data)
• Start working on generating a data frame for grid pricing (via API calls)

 

Work Accomplished  (code available at https://github.com/erika-24/smart-watt)

• Started working on a ML-based time-series forecasting model to predict load demand from training dataset (semi-synthetic for now). Used skforecast models (LinearRegression, ElasticNet, KNeighborsRegressor, RandomForestRegressor, XGBoost).
• Some challenges while writing code : How to handle missing values in data. I used forward fill and backward fill for now, will try using other interpolation methods as well as see what the output looks like if rows with missing data are deleted.Final output: Generated a 48-step ahead forecast (30-minute intervals) for future load demand.

For the photovoltaic power output forecast,

• Integrated Solcast API to fetch solar power forecast data for energy optimization.
• Implemented API calls with rooftop ID and API key authentication.
• Parsed JSON responses and converted them into Pandas DataFrames for easier analysis

I also started doing research on the cost function and objective functions as well as constraints to optimize.  The approach will likely be a combination of linear programming (objective fn is affine) and model predictive control.

Progress :

My progress is on track with regards to the Gantt Chart. I set up synthetic data pipeline for now, but will switch to live IoT device logs.

Things to do for next week:

Current Approach: Using synthetic training data for load forecasting.
Next Step:  Retrieve real-time power consumption logs from IoT/smart meter devices that we plan to use for the demo.
1.  Set up an IoT device integration (ESP32 with a smart plug for getting power consumption data).
2. Use MQTT or ESPHome API to stream real-time power consumption data.
3.  Store device logs in InfluxDB or a time-series database.

Current Approach: Integrated PV output from Solcast, but not yet synchronized with load forecasts.
Next Step: Merge solar and load forecasts to optimize energy storage & grid consumption.

Action Plan:
1. Align solar forecast timestamps with predicted load timestamps.
2. Start implementing the optimization algorithm (Linear Programming, MPC) to balance solar vs. grid power usage

3. Evaluate forecasting accuracy with r2_score and MAE/RMSE

4. Implement hyperparameter tuning for XGBoost to improve accuracy

---