FFA Working Papers 6:003 (2026)70

Natural Gas Storage Valuation Using Deep Reinforcement Learning

Masood Tadi, Milan Fičura, and Jiří Witzany
Prague University of Economics and Business, Faculty of Mathematics and Physics - Charles University

We study natural gas storage valuation under a stochastic futures term structure using deep reinforcement learning (DRL). The storage problem is formulated as a continuous-state, continuous-action Markov Decision Process and solved using the Deep Deterministic Policy Gradient (DDPG) algorithm with Prioritized Experience Replay (PER) buffer and a constraint-aware policy network. We benchmark the approach against intrinsic and rolling intrinsic strategies and find that DRL consistently outperforms intrinsic valuation and achieves competitive performance relative to rolling intrinsic in markets with jumps and seasonality. The results show that DRL provides a practical valuation framework that captures additional extrinsic value under realistic market dynamics and operational constraints.

Keywords: Natural Gas Storage, Rolling Intrinsic Valuation, Deep Reinforcement Learning

Received: June 12, 2026; Revised: June 12, 2026; Accepted: July 14, 2026; Published online: January 22, 2026  Show citation

ACS AIP APA ASA Harvard Chicago Chicago Notes IEEE ISO690 MLA NLM Turabian Vancouver
Tadi, M., Fičura, M., & Witzany, A.J. (2026). Natural Gas Storage Valuation Using Deep Reinforcement Learning. FFA Working Papers, 6, Article 2026.003. https://doi.org/10.XXXX/xxx.2026.004
Download citation
  • BibTeX (.bib)
  • Bookends (.ris)
  • EasyBib (.ris)
  • EndNote (.enw)
  • EndNote 8 (.xml)
  • ISI WoS (.isi)
  • Medlars (.medlars)
  • Mendeley (.ris)
  • MODS (.xml)
  • Papers (.ris)
  • RefWorks (.txt)
  • RefManager (.ris)
  • RIS (.ris)
  • MS Word (.xml)
  • Zotero (.ris)
    • Open full article

    References

    1. Bacaloni, Tiziano, Aldo Glielmo, and Marco Taboga (2025). "Natural-gas storage modelling by deep reinforcement learning". In: Proceedings of the 6th ACM International Conference on AI in Finance, pp. 829-837. Go to original source...
    2. Bachouch, Achref et al. (2022). "Deep neural networks algorithms for stochastic control problems on finite horizon: numerical applications". In: Methodology and Computing in Applied Probability 24.1, pp. 143-178. Go to original source...
    3. Bjerksund, Petter, Gunnar Stensland, and Frank Vagstad (2011). "Gas storage valuation: Price modelling v. optimization methods". In: The Energy Journal 32.1, pp. 203-228. Go to original source...
    4. Boogert, Alexander and Cyriel De Jong (2006). "Gas storage valuation using a Monte Carlo method". In. Breslin, John et al. (2009). "Gas storage: Rolling intrinsic valuation". In: Energy Risk, January 2009, pp. 61-65.
    5. Buehler, Hans et al. (2019). "Deep hedging". In: Quantitative Finance 19.8, pp. 1271-1291. Go to original source...
    6. Carmona, Ren´e and Michael Ludkovski (2010). "Valuation of energy storage: An optimal switching approach". In: Quantitative finance 10.4, pp. 359-374. Go to original source...
    7. Curin, Nicolas et al. (2021). "A deep learning model for gas storage optimization". In: Decisions in Economics and Finance 44.2, pp. 1021-1037. Go to original source...
    8. Daluiso, Roberto et al. (2020). "Pricing commodity swing options". In: arXiv preprint arXiv:2001.08906. Go to original source...
    9. De Jong, Cyriel (2015). "Gas storage valuation and optimization". In: Journal of Natural Gas Science and Engineering 24, pp. 365-378. Go to original source...
    10. Felix, Bastian (2012). "Gas Storage Valuation: A Comparative Simulation Study". In.
    11. Fischer, Thomas G (2018). Reinforcement learning in financial markets-a survey. Tech. rep. FAU discussion papers in economics.
    12. Geman, H´elyette (2005). Commodities and commodity derivatives: modeling and pricing for agriculturals, metals and energy. John Wiley & Sons.
    13. Giannelos, Spyros (2025). "Reinforcement Learning in Energy Finance: A Comprehensive Review." In: Energies (19961073) 18.11. Go to original source...
    14. Gibson, Rajna and Eduardo S Schwartz (1990). "Stochastic convenience yield and the pricing of oil contingent claims". In: The journal of finance 45.3, pp. 59-976. 11 Go to original source...
    15. Hinton, Geoffrey (2012). Lecture 6e rmsprop: Divide the gradient by a running average of its recent magnitude. Coursera lecture slides. url: https://www.cs.toronto.edu/~tijmen/csc321/slides/lecture_slides_lec6.pdf.
    16. Honor´e, Anouk (2011). "European natural gas demand, supply, and pricing: Cycles, seasons, and the impact of LNG price arbitrage". In: OUP Catalogue.
    17. Jaillet, Patrick, Ehud I Ronn, and Stathis Tompaidis (2004). "Valuation of commodity-based swing options". In: Management science 50.7, pp. 909-921. Go to original source...
    18. Jung, Sang-Woo et al. (2025). "Multi-Agent Deep Reinforcement Learning for Scheduling of Energy Storage System in Microgrids". In: Mathematics 13.12, p. 1999. Go to original source...
    19. Kingma, Diederik P (2014). "Adam: A method for stochastic optimization". In: arXiv preprint arXiv:1412.6980.
    20. Lai, Guoming, Fran¸cois Margot, and Nicola Secomandi (2010). "An approximate dynamic programming approach to benchmark practice-based heuristics for natural gas storage valuation". In: Operations research 58.3, pp. 564-582. Go to original source...
    21. Lillicrap, Timothy P et al. (2015). "Continuous control with deep reinforcement learning". In: arXiv preprint arXiv:1509.02971.
    22. L¨ohndorf, Nils and David Wozabal (2021). "Gas storage valuation in incomplete markets". In: European Journal of Operational Research 288.1, pp. 18-330. Go to original source...
    23. Longstaff, Francis A and Eduardo S Schwartz (2001). "Valuing American options by simulation: A simple least-squares approach". In: The review of Financial studies 14.1, pp. 113-147. Go to original source...
    24. Malyscheff, Alexander M and Theodore B Trafalis (2017). "Natural gas storage valuation via least squares Monte Carlo and support vector regression". In: Energy Systems 8.4, pp. 815-855. Go to original source...
    25. Peters, Gareth William et al. (2013). "Calibration and filtering for multi factor commodity models with seasonality: incorporating panel data from futures contracts". In: Methodology and Computing in Applied Probability 15.4, pp. 841-874. Go to original source...
    26. Polyak, Boris T (1964). "Some methods of speeding up the convergence of iteration methods". In: Ussr computational mathematics and mathematical physics 4.5, pp. 1-17. Go to original source...
    27. Sage, Manuel, Joshua Campbell, and Yaoyao Fiona Zhao (2024). "Enhancing battery storage energy arbitrage with deep reinforcement learning and time-series forecasting". In: Energy Sustainability. Vol. 87899. American Society of Mechanical Engineers, V001T01A007. Go to original source...
    28. Schaul, Tom (2015). "Prioritized Experience Replay". In: arXiv preprint arXiv:1511.05952.
    29. Schwartz, Eduardo and James E Smith (2000). "Short-term variations and long-term dynamics in commodity prices". In: Management Science 46.7, pp. 893-911. Go to original source...
    30. Schwartz, Eduardo S (1997). "The stochastic behavior of commodity prices: Implications for valuation and hedging". In: The Journal of finance 52.3, pp. 923-973. Go to original source...
    31. Secomandi, Nicola (2010). "Optimal commodity trading with a capacitated storage asset". In: Management Science 56.3, pp. 449-467. Go to original source...
    32. Silver, David et al. (2014). "Deterministic policy gradient algorithms". In: International conference on machine learning. Pmlr, pp. 387-395.
    33. Sutton, Richard S et al. (1999). "Policy gradient methods for reinforcement learning with function approximation". In: Advances in neural information processing systems 12.
    34. Thompson, Matt, Matt Davison, and Henning Rasmussen (2009). "Natural gas storage valuation and optimization: A real options application". In: Naval Research Logistics (NRL) 56.3, pp. 226-238. Go to original source...
    35. Warin, Xavier (2012). "Gas storage hedging". In: Numerical methods in finance: Bordeaux, June 2010. Springer, pp. 421-445. Go to original source...
    36. Yan, Xuemin (2002). "Valuation of commodity derivatives in a new multi-factor model". In: Review of Derivatives Research 5, pp. 251-271. Go to original source...
    37. Zhao, Xingyu et al. (2025). "Optimization of Start-up Strategies of Gas Injection Compressor in Underground Gas Storage Using Deep Reinforcement Learning". In: Simulation Modelling Practice and Theory, p. 103204. Go to original source...

    Return to the content