Cardiff University | Prifysgol Caerdydd ORCA
Online Research @ Cardiff 
WelshClear Cookie - decide language by browser settings

Deep reinforcement learning for flexibility provision from industrial loads

Shah, Margi ORCID: https://orcid.org/0000-0003-2222-8412 2026. Deep reinforcement learning for flexibility provision from industrial loads. PhD Thesis, Cardiff University.
Item availability restricted.

[thumbnail of Margi Shah PhD Thesis.pdf]
Preview
PDF - Accepted Post-Print Version
Download (4MB) | Preview
[thumbnail of Cardiff University Electronic Publication Form] PDF (Cardiff University Electronic Publication Form) - Supplemental Material
Restricted to Repository staff only

Download (742kB)

Abstract

Industries such as steel and cement production are highly energy-intensive and together account for approximately 14% of global greenhouse gas (GHG) emissions from energy and industrial processes, making them pivotal to achieving climate change mitigation targets. Several decarbonisation pathways are being explored in the steel sector, including increased electrification through electric arc furnaces, hydrogen-based direct reduction, and carbon capture and storage. Similarly, decarbonising cement production relies on multiple pathways, including alternative heat supply technologies such as electrification, clinker substitution, and also carbon capture and storage. Electrification not only enables substantial reductions in energy-related CO2 emissions, but also but also unlocks operational flexibility, enabling industrial loads to provide demand response (DR) to support to support the integration of variable renewable generation into electricity systems. This thesis investigates the operational scheduling of scrap-based electric arc furnace (EAF) steel production and cement manufacturing to enable their participation in DR programmes. By exploring flexible operation and participation in electricity markets, the research contributes to a broader understanding of how future steel and cement plants can support the transition towards net-zero emissions while playing a more active role in maintaining power system stability. However, scheduling industrial plants is challenging due to the complexity and interdependencies of production processes, as well as uncertainties associated with electricity market participation. To address these challenges, this thesis develops data-driven scheduling approach based on Deep Reinforcement Learning (DRL). DRL is particularly well suited to this problem as it enables sequential decision-making in an uncertain process through learning closed-loop feedback control while accounting for complex process constraints. This capability makes it a promising tool for enabling flexible industrial operation and enhancing the participation of energy-intensive industries in DR programmes. The first contribution is a DRL methodology for optimising the operational scheduling of EAF based steel plant participating in electricity spot market. The proposed model is designed to handle varying problem sizes and heterogeneous processing times associated with steelmaking tasks. It explicitly considers exogenous uncertainties arising from volatile electricity prices and on-site renewable generation. Furthermore, the methodology demonstrates how critical production constraints are incorporated to ensure steel quality and maintain operational feasibility. The performance of the proposed approach is evaluated against a traditional mixed-integer linear programming (MILP) formulation and a policy gradient-based DRL method, proximal policy optimisation (PPO). The second contribution examines ancillary service market participation. A steel plant can reduce its load by controlling the transformers supplying power to its EAFs. Due to their high energy intensity and inherent consumption flexibility, such plants are well-positioned to participate in the ancillary service market, providing revenue opportunities while supporting grid stability. Building on the DRL methodology developed, this thesis investigates the optimisation of steel plant participation in the reserve market, simultaneously responding to time-varying electricity spot prices. The study examines the role of a steel plant as a Balancing Mechanism Provider (BMP) in the ancillary service market, analysing how short-term operating reserve (STOR) provision can be co-optimised with energy procurement. Specifically, it optimises plant scheduling when STOR is dispatched while ensuring compliance with market and operational constraints. In addition, the thesis derives the plant’s bidding strategy for the day-ahead STOR market, including simulations of worst-case scenarios in which the bided capacity is fully utilised throughout the reserve window, ensuring that production is not compromised. The third contribution extends the DRL methodology to cement production, which operates as a continuous, non-batch process unlike steelmaking. The study demonstrates how DRL can optimise operational scheduling under electricity spot market participation, explicitly considering price volatility and process-specific constraints. The performance of the method is also evaluated against standard MILP approach. In summary, this thesis develops a data-driven approach for operational scheduling in steel and cement production, leveraging inherent process flexibility to reduce electricity costs and provide grid-supporting services. These findings are particularly relevant to stakeholders in the United Kingdom (UK) and global steel and cement industries and the electricity sector, offering actionable guidance for infrastructure-coordinated decarbonisation. Ultimately, the research supports the net-zero ambitions and provides a transferable framework for other hard to-abate industrial sectors.

Item Type: Thesis (PhD)
Date Type: Completion
Status: Unpublished
Schools: Schools > Engineering
Uncontrolled Keywords: 1. Demand Response 2. Optimisation 3. Reinforcement Learning 4. Production process 5. Steel plant 6. Cement plant
Date of First Compliant Deposit: 10 November 2026
Last Modified: 10 Nov 2026 16:46
URI: https://orca.cardiff.ac.uk/id/eprint/190495

Actions (repository staff only)

Edit Item Edit Item

Downloads

Downloads per month over past year

View more statistics