FFA Working Papers 6:004 (2026)254
Reinforcement Learning for Perpetual Futures Market Making
- Department of Banking and Insurance, University of Economics
This paper studies market making in BTC and ETH perpetual futures with a simple event-driven reinforcement learning setup. PPO chooses quote skew and quote width, while the environment tracks execution gains and inventory losses separately. A temporary reduction of the adverse-fill penalty makes early training more stable. Across simulated and historical data, PPO quotes more selectively, carries smaller inventory, and usually ends with better portfolio value than a zero-action baseline. BTC is tighter than ETH, so the same control logic requires market-specific calibration, yet the qualitative
policy behavior remains consistent across both markets. The results are therefore encouraging in a narrow sense: a compact and interpretable RL policy can improve inventory control and overall portfolio outcomes in two different crypto perpetual environments, even though robust profitability claims remain out of scope.
Keywords: reinforcement learning, market making
Received: June 15, 2026; Revised: June 15, 2026; Accepted: July 14, 2026; Prepublished online: July 14, 2026; Published online: January 22, 2026 Show citation
References
- Thomas Ho and Hans R. Stoll. Optimal dealer pricing under transactions and return uncertainty. Journal of Financial Economics, 9(1):47-73, 1981.
Go to original source... - Maureen O'Hara. Market Microstructure Theory. Blackwell, 1995.
- Marco Avellaneda and Sasha Stoikov. Highfrequency trading in a limit order book. Quantitative Finance, 8(3):217-224, 2008. Go to original source...
- Richard S. Sutton and Andrew G. Barto. Reinforcement Learning: An Introduction. MIT Press, second edition, 2018.
- Yuriy Nevmyvaka, Yi Feng, and Michael Kearns. Reinforcement learning for optimized trade execution. In Proceedings of the 23rd International Conference on Machine Learning, pages 673-680, 2006. Go to original source...
- John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017.
- Alvaro Cartea, Sebastian Jaimungal, and Jose Penalva. Algorithmic and High-Frequency Trading. Cambridge University Press, 2015.
- Thomas Spooner, John Fearnley, Rahul Savani, and Andreas Koukorinis. Market making via reinforcement learning. In Proceedings of the 17th International Conference on Autonomous Agents and MultiAgent Systems, pages 434-442, 2018. Go to original source...
- Olivier Guéant and Iuliia Manziuk. Deep reinforcement learning for market making in corporate bonds: Beating the curse of dimensionality. Applied Mathematical Finance, 26(5):387-452, 2019. Go to original source...
- Thomas Spooner and Rahul Savani. Robust market making via adversarial reinforcement learning. In Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, pages 4541-4547, 2020. Go to original source...
- Bruno Gasperov and Zvonko Kostanjcar. Market making with signals through deep reinforcement learning. IEEE Access, 9:61611-61622, 2021. Go to original source...
- Bruno Gasperov and Zvonko Kostanjcar. Deep reinforcement learning for market making under a Hawkes process-based limit order book model. IEEE Control Systems Letters, 6:2480-2485, 2022. Go to original source...
- Abbas Haider, Glenn I. Hawe, Hui Wang, and Bryan Scotney. Multi-asset market making via multi-task deep reinforcement learning. In Machine Learning and Knowledge Discovery in Databases. Research Track, pages 418-433. Springer, 2022. Go to original source...