Reinforcement Learning HVAC Control Testbed for EnergyPlus (Spring 2023)

Loss function converging for a simple agent showing the testbed functioning as expected (training done by me and Trent Tepool)

My teammate, Trent Tepool, processed grid emission data and we trained an agent to minimize operational carbon with my testbed

I developed a simple reinforcement learning testbed to both support the development of HVAC control agents and potentially allow such agents to be part of architectural design workflow. The test bed builds upon EnergyPlus’s official Python API 2.0 to perform simulations for lightly-modified IDF files exported from Design Builder. Upon that, the Gymnasium API is created to support many popular reinforcement learning libraries. Finally, it utilizes StableBaseline3 library on top of the API to initialize and train the agents.

A few testbeds have been developed for EnergyPlus, such as IBM’s Reinforcement Learning Testbed for Power Consumption Optimization and SinerGym, but they tend to be structured around specific aspects of performance such as power consumption and agent effectiveness instead of being general-purposed. They also lack support for user-provided IDF files.

Coordination between the agent thread and EnergyPlus simulation thread

This project, albeit much smaller in scale, is more light-weight and flexible: it’s directly based on EnergyPlus’ API so all actuators and sensors supported can be used; it simulates the IDF file directly with EnergyPlus without modifying it, instead relying on deep calling points within the simulation loops to overwrite actuator settings, so any valid IDF file (from DesignBuilder for example) should be compatible, making our testbed a potentially attractive option for design projects where the trained control agents become part of the energy simulation and thereby integral part of the design workflow. 

Source Code: https://github.com/Jia053123/CarbonControlRL

Shape Grammar Diagram Designer (2021-2023)

This is an ambitious project that aims to run layout optimization algorithms on shape grammars to produce emergent architecture program diagrams that both follow the aesthetic rules and satisfy the program requirements.

In spring 2023, a computer vision system is built to improve the generation of the right hand shape. A deep convolutional neural network is built from scratch and trained as the designer repeatedly sketches out his/her idea, generating training samples along the way. Compared to symbolic methods, this enables rule applications on imprecise hand drawings and thereby enabling the integration of defining and sketching.

For each of the two grammars, the three columns show:
left hand shape, human-generated right hand shape, AI-generated right hand shape

As a key inspiration for my future endeavors, this project aspires to make computational design more like sketching than math. By hinging the formal system on the looks of the drawing rather than the numerical values that encode the drawing through coordinate systems, this project illustrates a new paradigm of computational design where the structure of a drawing is embedded in the drawing process without explicit parameterization on the designer’s part.

Source Code: https://github.com/Jia053123/DiagramDesigner

Foliage Shading (Fall 2022)

While most architecture shadings consist of large pieces of vertical or horizontal fins, I yearn for a shading system akin to wavering summer leaves.

The algorithm is stochastic and agent based. For each iteration, each leaf turns, grows and survives to find the best configuration for itself.

The irregularity in the arrangement is caused by randomization in initialization and simultaneous adjustment of direction and size for each leaf. While better performance can be achieved by removing the noise in initialization and updating the directions for all leaves in specific order before updating their sizes, the result is visually bland for simple site conditions.

In evaluation, the optimized result is compared with a baseline case where all parameters are chosen randomly given the identical total shading area. Simulations show the optimized case consistently outperforming the baseline case in glare reduction.

Source Code for the Grasshopper plugin: https://github.com/Jia053123/FoliageShading

Moiré Editor (2020-2021)

This is an iPad and MacOS application for creating line moiré art, designed and developed by myself. It aspires to efficiently explore emergent properties arising from simple orders through computational means as a model for a new generation of computational design tools.

A moiré pattern consists of overlapping layers of alternating black and white strips that move against each other. This application has two rows of panels: the upper row contains panels of sliders that specify and control each layer of strips, whereas the lower row specify and control global properties of the moiré pattern that emerge from the layers chosen by the user.

Source Code: https://github.com/Jia053123/MoireStudies-iOS-

Arch Studio Projects (pre-2020)