{"id":175,"date":"2022-07-07T14:53:37","date_gmt":"2022-07-07T18:53:37","guid":{"rendered":"http:\/\/jialiangxiang.com\/wordpress\/?page_id=175"},"modified":"2024-01-12T00:26:32","modified_gmt":"2024-01-12T05:26:32","slug":"work","status":"publish","type":"page","link":"https:\/\/jialiangxiang.com\/wordpress\/","title":{"rendered":"Work"},"content":{"rendered":"\n

Reinforcement Learning HVAC Control Testbed for EnergyPlus (Spring 2023) <\/h2>\n\n\n\n
\n
\"\"<\/a><\/figure>\n
Loss function converging for a simple agent showing the testbed functioning as expected (training done by me and Trent Tepool) <\/figcaption><\/figure>\n\n\n\n

<\/p>\n\n\n\n

\n
\"\"<\/a><\/figure>\n\n\n\n
\"\"<\/a><\/figure>\n
My teammate, Trent Tepool, processed grid emission data and we trained an agent to minimize operational carbon with my testbed<\/figcaption><\/figure>\n\n\n\n

I developed a simple reinforcement learning testbed to both support the development of HVAC control agents and potentially allow such agents to be part of architectural design workflow. The test bed builds upon EnergyPlus\u2019s official Python API 2.0 to perform simulations for lightly-modified IDF files exported from Design Builder. Upon that, the Gymnasium API is created to support many popular reinforcement learning libraries. Finally, it utilizes StableBaseline3 library on top of the API to initialize and train the agents. <\/p>\n\n\n\n

A few testbeds have been developed for EnergyPlus, such as IBM\u2019s Reinforcement Learning Testbed for Power Consumption Optimization and SinerGym, but they tend to be structured around specific aspects of performance such as power consumption and agent effectiveness instead of being general-purposed. They also lack support for user-provided IDF files. <\/p>\n\n\n\n

\n
\"\"<\/figure>\n
Coordination between the agent thread and EnergyPlus simulation thread<\/figcaption><\/figure>\n\n\n\n

This project, albeit much smaller in scale, is more light-weight and flexible: it\u2019s directly based on EnergyPlus\u2019 API so all actuators and sensors supported can be used; it simulates the IDF file directly with EnergyPlus without modifying it, instead relying on deep calling points within the simulation loops to overwrite actuator settings, so any valid IDF file (from DesignBuilder for example) should be compatible, making our testbed a potentially attractive option for design projects where the trained control agents become part of the energy simulation and thereby integral part of the design workflow. <\/p>\n\n\n\n

Source Code: https:\/\/github.com\/Jia053123\/CarbonControlRL<\/a> <\/p>\n\n\n\n

<\/div>\n\n\n\n

Shape Grammar Diagram Designer (2021-2023) <\/h2>\n\n\n\n
\n