This repo contains a detailed list of research papers using Transformers in Reinforcement Learning.
It is used for our survey paper Transformers in Reinforcement Learning: A Survey
If you use it, please cite:
@article{agarwal2023transformers,
title={Transformers in Reinforcement Learning: A Survey},
author={Agarwal, Pranav and Rahman, Aamer Abdul and St-Charles, Pierre-Luc and Prince, Simon JD and Kahou, Samira Ebrahimi},
journal={arXiv preprint arXiv:2307.05979},
year={2023}
}
The list of papers are divided into multiple categories as elaborated below taking into account the use of Transformers in the field of Reinforcement Learning.
- Representation Learning
- World Model
- Reward Learning
- Policy Learning
- Training Strategy
- Explainability
- Applications
-
My Body is a Cage: the Role of Morphology in Graph-Based Incompatible Control [Paper] Apr 2021
Vitaly Kurin, Maximilian Igl, Tim Rocktäschel, Wendelin Boehmer, Shimon Whiteson
-
The Sensory Neuron as a Transformer: Permutation-Invariant NeuralNetworks for Reinforcement Learning [Paper] Sept 2021
Yujin Tang, David Ha
-
CoBERL: Contrastive BERT for Reinforcement Learning [Paper] Feb 2022
Andrea Banino, Adrià Puidomenech Badia, Jacob Walker, Tim Scholtes, Jovana Mitrovic, Charles Blundell
-
Structure-Aware Transformer Policy for Inhomogeneous Multi-Task Reinforcement Learning [Paper] Mar 2022
Sunghoon Hong, Deunsol Yoon, Kee-Eung Kim
-
Efficient Spatiotemporal Transformer for Robotic Reinforcement Learning [Paper] June 2022
Yiming Yang, Dengpeng Xing, Bo Xu
-
Learning to Navigate in Interactive Environments with the Transformer-based Memory [Paper] June 2022
Weiyuan Li, Ruoxin Hong, Jiwei Shen, Yue Lu
-
CtrlFormer: Learning Transferable State Representation for Visual Control via Transformer [Paper] June 2022
Yao Mu, Shoufa Chen, Mingyu Ding, Jianyu Chen, Runjian Chen, Ping Luo
-
TVENet: Transformer-Based Visual Exploration Network for Mobile Robot in Unseen Environment [Paper] June 2022
Tianyao Zhang, Xiaoguang Hu, Jin Xiao, Guofeng Zhang
-
StARformer: Transformer with State-Action-Reward Representations for Visual Reinforcement Learning [Paper] July 2022
Jinghuan Shang, Kumara Kahatapitiya, Xiang Li, Michael S. Ryoo
-
Multi-granularity scenarios understanding network for trajectory prediction [Paper] Aug 2022
Biao Yang, Jicheng Yang, Rongrong Ni, Changchun Yang, Xiaofeng Liu
-
One for All: One-stage Referring Expression Comprehension with Dynamic Reasoning [Paper] Oct 2022
Zhipeng Zhang, Zhimin Wei, Zhongzhen Huang, Rui Niu, Peng Wang
-
Unsupervised Learning of Temporal Abstractions with Slot-based Transformers [Paper] Nov 2022
Anand Gopalakrishnan, Kazuki Irie, Jürgen Schmidhuber, Sjoerd van Steenkiste
-
Improving Sample Efficiency of Value Based Models Using Attention and Vision Transformers [Paper] Feb 2022
Amir Ardalan Kalantari, Mohammad Amini, Sarath Chandar, Doina Precup
-
Dreaming with Transformers [Paper] Feb 2022
Catherine Zeng, Jordan Docter, Alexander Amini, Igor Gilitschenski, Ramin Hasani and Daniela Rus
-
TransDreamer: Reinforcement Learning with Transformer World Models [Paper] Feb 2022
Chang Chen, Yi-Fu Wu, Jaesik Yoon, Sungjin Ahn
-
A model-based approach to meta-Reinforcement Learning: Transformers and tree search [Paper] Aug 2022
Brieuc Pinon, Jean-Charles Delvenne, Raphaël Jungers
-
Transformers are Sample Efficient World Models [Paper] Sep 2022
Vincent Micheli, Eloi Alonso, François Fleuret
-
Masked World Models for Visual Control [Paper] Nov 2022
Younggyo Seo, Danijar Hafner, Hao Liu, Fangchen Liu, Stephen James, Kimin Lee, Pieter Abbeel
-
Transformers-Based World Models Are Happy With 100k Interactions [Paper] Nov 2022
Jan Robine, Jan_Robine, Marc Höftmann, Tobias Uelwer, Stefan Harmeling
-
Comparing BERT-based Reward Functions for Deep Reinforcement Learning in Machine Translation [Paper] Oct 2022
Yuki Nakatani, Tomoyuki Kajiwara, Takashi Ninomiya
-
UPDeT: Universal Multi-agent Reinforcement Learning via Policy Decoupling with Transformers [Paper] Feb 2021
Siyi Hu, Fengda Zhu, Xiaojun Chang, Xiaodan Liang
-
Offline Reinforcement Learning as One Big Sequence Modeling Problem [Paper] Jun 2021
Michael Janner, Qiyang Li, Sergey Levine
-
Decision Transformer: Reinforcement Learning via Sequence Modeling [Paper] Jun 2021
Lili Chen, Kevin Lu, Aravind Rajeswaran, Kimin Lee, Aditya Grover, Michael Laskin, Pieter Abbeel, Aravind Srinivas, Igor Mordatch
-
A Novel Deep Reinforcement Learning Based Stock Direction Prediction using Knowledge Graph and Community Aware Sentiments [Paper] July 2021
Anil Berk Altuner, Zeynep Hilal Kilimci
-
Pretraining for Language Conditioned Imitation with Transformers [Paper] Sept 2021
Aaron L Putterman, Kevin Lu, Igor Mordatch, Pieter Abbeel
-
CubeTR: Learning to Solve the Rubik's Cube using Transformers [Paper] Sept 2021
Mustafa Ebrahim Chasmai
-
Transfer learning with causal counterfactual reasoning in Decision Transformers [Paper] Oct 2021
Ayman Boustati, Hana Chockler, Daniel C. McNamee
-
Offline Pre-trained Multi-Agent Decision Transformer: One Big Sequence Model Tackles All SMAC Tasks [Paper] Dec 2021
Linghui Meng, Muning Wen, Yaodong Yang, Chenyang Le, Xiyun Li, Weinan Zhang, Ying Wen, Haifeng Zhang, Jun Wang, Bo Xu
-
Switch Trajectory Transformer with Distributional Value Approximation for Multi-Task Reinforcement Learning [Paper] Mar 2022
Qinjie Lin, Han Liu, Biswa Sengupta
-
FedFormer: Contextual Federation with Attention in Reinforcement Learning [Paper] May 2022
Liam Hebert, Lukasz Golab, Pascal Poupart, Robin Cohen
-
Transformer with Memory Replay [Paper] May 2022
Rui Liu, Barzan Mozafari
-
Online Decision Transformer [Paper] July 2022
Qinqing Zheng, Amy Zhang, Aditya Grover
-
Multi-Game Decision Transformers [Paper] May 2022
Kuang-Huei Lee, Ofir Nachum, Mengjiao Yang, Lisa Lee, Daniel Freeman, Winnie Xu, Sergio Guadarrama, Ian Fischer, Eric Jang, Henryk Michalewski, Igor Mordatch
-
Prompting Decision Transformer for Few-Shot Policy Generalization [Paper] June 2022
Mengdi Xu, Yikang Shen, Shun Zhang, Yuchen Lu, Ding Zhao, Joshua B. Tenenbaum, Chuang Gan
-
Offline Pre-trained Multi-Agent Decision Transformer: One Big Sequence Model Tackles All SMAC Tasks [Paper] Jun 2022
Linghui Meng, Muning Wen, Yaodong Yang, Chenyang Le, Xiyun Li, Weinan Zhang, Ying Wen, Haifeng Zhang, Jun Wang, Bo Xu
-
Transformers are Meta-Reinforcement Learners [Paper] June 2022
Luckeciano C. Melo
-
Transformer-based Value Function Decomposition for Cooperative Multi-agent Reinforcement Learning in StarCraft [Paper] Aug 2022
Muhammad Junaid Khan, Syed Hammad Ahmed, Gita Sukthankar
-
Transformer-Based Deep Reinforcement Learning in VizDoom [Paper] Aug 2022
Vitalii Sopov, Ilya Makarov
-
Q-learning Decision Transformer: Leveraging Dynamic Programming for Conditional Sequence Modelling in Offline RL [Paper] Oct 2022
Taku Yamagata, Ahmed Khalil, Raul Santos-Rodriguez
-
Bootstrapped Transformer for Offline Reinforcement Learning [Paper] Oct 2022
Kerong Wang, Hanye Zhao, Xufang Luo, Kan Ren, Weinan Zhang, Dongsheng Li
-
Relational Reasoning via Set Transformers: Provable Efficiency and Applications to MARL [Paper] Oct 2022
Fengzhuo Zhang, Boyi Liu, Kaixin Wang, Vincent Y. F. Tan, Zhuoran Yang, Zhaoran Wang
-
Multi-Agent Reinforcement Learning is a Sequence Modeling Problem [Paper] Oct 2022
Muning Wen, Jakub Grudzien Kuba, Runji Lin, Weinan Zhang, Ying Wen, Jun Wang, Yaodong Yang
-
You Can't Count on Luck: Why Decision Transformers and RvS Fail in Stochastic Environments [Paper] Nov 2022
Keiran Paster, Sheila McIlraith, Jimmy Ba
-
Deep Transformer Q-Networks for Partially Observable Reinforcement Learning [Paper] Nov 2022
Kevin Esslinger, Robert Platt, Christopher Amato
-
Stabilizing Transformers for Reinforcement Learning [Paper] Oct 2019
Emilio Parisotto, H. Francis Song, Jack W. Rae, Razvan Pascanu, Caglar Gulcehre, Siddhant M. Jayakumar, Max Jaderberg, Raphael Lopez Kaufman, Aidan Clark, Seb Noury, Matthew M. Botvinick, Nicolas Heess, Raia Hadsell
-
Semi-supervised Offline Reinforcement Learning with Pre-trained Decision Transformers [Paper] Sept 2021
Catherine Cang, Kourosh Hakhamaneshi, Ryan Rudes, Igor Mordatch, Aravind Rajeswaran, Pieter Abbeel, Michael Laskin
-
Can Wikipedia Help Offline Reinforcement Learning? [Paper] Jan 2022
Machel Reid, Yutaro Yamada, Shixiang Shane Gu
-
Evaluating Vision Transformer Methods for Deep Reinforcement Learning from Pixels [Paper] Apr 2022
Tianxin Tao, Daniele Reda, Michiel van de Panne
-
ATTEXPLAINER: Explain Transformer via Attention by Reinforcement Learning [Paper] July 2022
Runliang Niu, Zhepei Wei, Yan Wang, Qi Wang
-
Vision Transformer for Learning Driving Policies in Complex Multi-Agent Environments [Paper] Sept 2021
Eshagh Kargar, Ville Kyrki
-
Neurosymbolic Learning for Robust and Reliable Intelligent Systems [Paper] Feb 2022
Jeevana Priya Inala
-
Object Memory Transformer for Object Goal Navigation [Paper] Mar 2022
Rui Fukushima, Kei Ota, Asako Kanezaki, Yoko Sasaki, Yusuke Yoshiyasu
-
End-to-End Trajectory Distribution Prediction Based on Occupancy Grid Maps [Paper] Mar 2022
Ke Guo, Wenxi Liu, Jia Pan
-
Learning Efficient Multi-agent Cooperative Visual Exploration [Paper] Oct 2022
Chao Yu, Xinyi Yang, Jiaxuan Gao, Huazhong Yang, Yu Wang, Yi Wu
-
Augmenting Reinforcement Learning with Transformer-based Scene Representation Learning for Decision-making of Autonomous Driving [Paper]Aug 2022
Haochen Liu, Zhiyu Huang, Xiaoyu Mo, Chen Lv
-
TransNav: spatial sequential transformer network for visual navigation [Paper] Aug 2022
Kang Zhou, Huyin Zhang, Fei Li
-
Reinforced Transformer for Medical Image Captioning [Paper] Oct 2019
Yuxuan Xiong, Bo Du, Pingkun Yan
-
Improving Factual Completeness and Consistency of Image-to-Text Radiology Report Generation [Paper] June 2021
Yasuhide Miura, Yuhao Zhang, Emily Tsai, Curtis Langlotz, Dan Jurafsky
-
Multi-constraint molecular generation based on conditional transformer, knowledge distillation and reinforcement learning [Paper] Oct 2021
Jike Wang, Chang-Yu Hsieh, Mingyang Wang, Xiaorui Wang, Zhenxing Wu, Dejun Jiang, Benben Liao, Xujun Zhang, Bo Yang, Qiaojun He, Dongsheng Cao, Xi Chen, Tingjun Hou
-
Deep Reinforcement Learning and Docking Simulations for autonomous molecule generation in de novo Drug Design [Paper] Dec 2021
Qian Wang, Hao Liu, Xiaotong Hu
-
Transformer-Based Generative Model Accelerating the Development of Novel BRAF Inhibitors [Paper] Dec 2021
Lijuan Yang, Guanghui Yang, Zhitong Bing, Yuan Tian, Yuzhen Niu, Liang Huang, Lei Yang
-
DrugEx v3: Scaffold-Constrained Drug Design with Graph Transformer-based Reinforcement Learning [Paper] Dec 2021
Xuhan Liu, Kai Ye, Herman W. T. van Vlijmen, Adriaan P. IJzerman, Gerard J. P. van Westen
-
Active Phase-Encode Selection for Slice-Specific Fast MR Scanning Using a Transformer-Based Deep Reinforcement Learning Framework [Paper] Mar 2022
Yiming Liu, Yanwei Pang, Ruiqi Jin, Zhenchang Wang
-
SIG-Former: monocular surgical instruction generation with transformers [Paper] July 2022
Jinglu Zhang, Yinyu Nie, Jian Chang, Jian Jun Zhang
-
Transformer-based Objective-reinforced Generative Adversarial Network to Generate Desired Molecules [Paper] July 2022
Chen Li, Chikashige Yamanaka, Kazuma Kaitoh, Yoshihiro Yamanishi
-
RL Based Unsupervised Video Summarization Framework for Ultrasound Imaging [Paper] Sept 2022
Roshan P. Mathews, Mahesh Raveendranatha Panicker, Abhilash R. Hareendranathan, Yale Tung Chen, Jacob L. Jaremko, Brian Buchanan, Kiran Vishnu Narayan, Kesavadas Chandrasekharan , Greeta Mathews
-
DxFormer: A Decoupled Automatic Diagnostic System Based on Decoder-Encoder Transformer with Dense Symptom Representations [Paper] Dec 2022
Wei Chen, Cheng Zhong, Jiajie Peng, Zhongyu Wei
-
Towards Reinforcement Learning for Pivot-based Neural Machine Translation with Non-autoregressive Transformer [Paper] Sept 2021
Evgeniia Tokarchuk, Jan Rosendahl, Weiyue Wang, Pavel Petrushkov, Tomer Lancewicki, Shahram Khadivi, Hermann Ney
-
Corrective Guidance and Learning for Dialogue Management [Paper] Oct 2021
Mahdin Rohmatillah, Jen-Tzung Chien
-
Goal-Directed Story Generation: Augmenting Generative Language Models with Reinforcement Learning [Paper] Dec 2021
Amal Alabdulkarim, Winston Li, Lara J. Martin, Mark O. Riedl
-
Adversarial Conversational Shaping for Intelligent Agents [Paper] Dec 2021
Piotr Tarasiewicz, Sultan Kenjeyev, Ilana Sebag, Shehab Alshehabi
-
FCSF-TABS: two-stage abstractive summarization with fact-aware reinforced content selection and fusion [Paper] Jan 2022
Mengli Zhang, Gang Zhou, Wanting Yu, Wenfen Liu, Ningbo Huang, Ze Yu
-
Audio Embeddings Help to Learn Better Dialogue Policies [Paper] Feb 2022
Asier López Zorrilla, M. Inés Torres, Heriberto Cuayáhuitl
-
Reward Modeling for Mitigating Toxicity in Transformer-based Language Models [Paper] Feb 2022
Farshid Faal, Ketra Schmitt, Jia Yuan Yu
-
Active Audio-Visual Separation of Dynamic Sound Sources [Paper] Feb 2022
Sagnik Majumder, Kristen Grauman
-
Bailando: 3D Dance Generation by Actor-Critic GPT with Choreographic Memory [Paper] Mar 2022
Li Siyao, Weijiang Yu, Tianpei Gu, Chunze Lin, Quan Wang, Chen Qian, Chen Change Loy, Ziwei Liu
-
Multi-Modal Instruction based Reinforcement Learning using MoME Transformer [Paper] Apr 2022
Avish Kadakia, Sabah Mohammed
-
Meta Policy Learning for Cold-Start Conversational Recommendation [Paper] May 2022
Zhendong Chu, Hongning Wang, Yun Xiao, Bo Long, Lingfei Wu
-
SILVER-BULLET-3D AT MANISKILL 2021: LEARNING-FROM-DEMONSTRATIONS AND HEURISTIC RULE-BASED METHODS FOR OBJECT MANIPULATION [Paper] Jun 2022
Yingwei Pan, Yehao Li, Yiheng Zhang, Qi Cai, Fuchen Long, Zhaofan Qiu, Ting Yao, Tao Mei
-
TF-SOD: a novel transformer framework for salient object detection [Paper] July 2022
Yunzhou Zhang, Zhenyu Wang, Yan Liu, Zhuo Wang, Sonya Coleman, Dermot Kerr
-
Improving Multi-Document Summarization through Referenced Flexible Extraction with Credit-Awareness [Paper] July 2022
Yun-Zhu Song, Yi-Syuan Chen, Hong-Han Shuai
-
Emotion Aware Reinforcement Network for Visual Storytelling [Paper] Sept 2022
Xin Li, Hanqing Cai, Tianling Jiang, Chunping Liu, Yi Ji
-
SURF: Semantic-level Unsupervised Reward Function for Machine Translation [Paper] July 2022
Atijit Anuchitanukul, Julia Ive
-
GenTUS: Simulating User Behaviour and Language in Task-oriented Dialogues with Generative Transformers [Paper] Sept 2022
Hsien-chin Lin, Christian Geishauser, Shutong Feng, Nurul Lubis, Carel van Niekerk, Michael Heck, Milica Gasic
-
Dynamic Dialogue Policy for Continual Reinforcement Learning [Paper] Oct 2022
Christian Geishauser, Carel van Niekerk, Hsien-chin Lin, Nurul Lubis, Michael Heck, Shutong Feng, Milica Gašić
-
Selective Token Generation for Few-shot Natural Language Generation [Paper] Oct 2022
Daejin Jo, Taehwan Kwon, Eun-Sol Kim, Sungwoong Kim
-
Efficient Unsupervised Sentence Compression by Fine-tuning Transformers with Reinforcement Learning [Paper] Nov 2022
Demian Ghalandari, Chris Hokamp, Georgiana Ifrim
-
RLAS-BIABC: A Reinforcement Learning-Based Answer Selection Using the BERT Model Boosted by an Improved ABC Algorithm [Paper] Jan 2023
Hamid Gharagozlou, Javad Mohammadzadeh, Azam Bastanfard, Saeed Shiry Ghidary
-
Exploring reward efficacy in traffic management using deep reinforcement learning in intelligent transportation system [Paper] April 2022
Ananya Paul, Sulata Mitra
-
A Distributed Vehicle-assisted Computation Offloading Scheme based on DRL in Vehicular Networks [Paper] July 2022
Jiayue Wang, Hongbo Zhao, Haoqiang Liu, Liwei Geng, Zebin Sun
-
Reinforced Transformer Learning for VSI-DDoS Detection in Edge Clouds [Paper] Sept 2022
Adil Bin Bhutto, Xuan Son Vu, Erik Elmroth, Wee Peng Tay, Monowar Bhuyan
-
Actor-Critic with Transformer for Cloud Computing Resource Three Stage Job Scheduling [Paper] Sept 2022
Yanbo Xu, Jiakun Zhao
-
Attention-Based Learning for Combinatorial Optimization [Paper] May 2022
Carson Smith
-
A Deep Reinforcement Learning Algorithm Using A New Graph Transformer Model for Routing Problems [Paper] Sept 2022
Yang Wang, Zhibin Chen
-
Deep reinforcement learning for optimal well control in subsurface systems with uncertain geology [Paper] Mar 2022
Yusuf Nasir, Louis J. Durlofsky
-
Confidence Estimation Transformer for Long-term Renewable Energy Forecasting in Reinforcement Learning-based Power Grid Dispatching [Paper] Apr 2022
Xinhang Li, Zihao Li, Nan Yang, Zheng Yuan, Qinwen Wang, Yiying Yang, Yupeng Huang, Xuri Song, Lei Li, Lin Zhang
-
A Hybrid Data-Driven Method for Low-Carbon Economic Energy Management Strategy in Electricity-Gas Coupled Energy Systems Based on Transformer Network and Deep Reinforcement Learning [Paper] Apr 2022
Bin Zhang, Weihao Hu, Xiao Xu, Zhenyuan Zhang, Zhe Chen
-
Stabilizing Voltage in Power Distribution Networks via Multi-Agent Reinforcement Learning with Transformer [Paper] Jun 2022
Minrui Wang, Mingxiao Feng, Wengang Zhou, Houqiang Li
-
Toward Smart Multizone HVAC Control by Combining Context-Aware System and Deep Reinforcement Learning [Paper] May 2022
Xiangtian Deng, Yi Zhang, Yi Zhang, He Qi
-
A Deep Reinforcement Learning Framework Based on an Attention Mechanism and Disjunctive Graph Embedding for the Job-Shop Scheduling Problem [Paper] Apr 2022
Ruiqi Chen, Wenxin Li, Hongbing Yang
-
An End-to-End Deep Reinforcement Learning Approach for Job Shop Scheduling [Paper] May 2022
Linlin Zhao, Weiming Shen, Chunjiang Zhang, Kunkun Peng
-
Transformer-Based Reinforcement Learning for Pickup and Delivery Problems With Late Penalties [Paper] Aug 2022
Ke Zhang, Xi Lin, Meng Li
-
Attention-based model and deep reinforcement learning for distribution of event processing tasks [Paper] Aug 2022
Andriy Mazayev, Faroq Al-Tam, Noélia Correia
-
Relation-Aware Transformer for Portfolio Policy Learning [Paper] July 2020
Ke Xu1, Yifan Zhang1, Deheng Ye, Peilin Zhao, Mingkui Tan
-
A Novel Deep Reinforcement Learning Framework for Stock Portfolio Optimization [Paper] Dec 2021
Shaobo Hu, Hongying Zheng, and Jianyong Chen
-
AME: Attention and Memory Enhancement in Hyper-Parameter Optimization [Paper] June 2022
Nuo Xua, Jianlong Chang, Xing Nie, Chunlei Huo, Shiming Xiang and Chunhong Pan