Reinforcement Learning

10.

Advanced Topics

10.1.

Exploration Strategies

10.1.1.

Exploration-Exploitation Trade-off

10.1.2.

Random Exploration

10.1.2.1.

10.1.2.2.

Boltzmann Exploration

10.1.3.

Optimistic Exploration

10.1.3.1.

Upper Confidence Bounds

10.1.3.2.

Optimism in Face of Uncertainty

10.1.4.

Information-Directed Exploration

10.1.4.1.

Thompson Sampling

10.1.4.2.

Bayesian Approaches

10.1.5.

Curiosity-Driven Exploration

10.1.5.1.

Intrinsic Motivation

10.1.5.2.

Count-Based Exploration

10.1.5.3.

Prediction Error Methods

10.2.

Model-Based Reinforcement Learning

10.2.1.

Learning Environment Models

10.2.1.1.

Transition Model Learning

10.2.1.2.

Reward Model Learning

10.2.1.3.

Model Uncertainty

10.2.2.

Planning with Learned Models

10.2.2.1.

10.2.2.2.

Backward Search

10.2.2.3.

Tree Search Methods

10.2.3.

Dyna Architecture

10.2.3.1.

Integrated Learning and Planning

10.2.3.2.

Dyna-Q Algorithm

10.2.3.3.

10.2.4.

Model-Predictive Control

10.2.4.1.

Receding Horizon Control

10.2.4.2.

Optimization-Based Planning

10.3.

Hierarchical Reinforcement Learning

10.3.1.

Temporal Abstraction

10.3.2.

Options Framework

10.3.2.1.

Option Definition

10.3.2.2.

Option Discovery

10.3.2.3.

Semi-Markov Decision Processes

10.3.3.

Goal-Conditioned RL

10.3.3.1.

Universal Value Functions

10.3.3.2.

Hindsight Experience Replay

10.3.4.

Feudal Networks

10.3.4.1.

Manager-Worker Hierarchies

10.3.4.2.

Subgoal Generation

10.4.

Multi-Agent Reinforcement Learning

10.4.1.

Multi-Agent Environments

10.4.2.

Independent Learning

10.4.3.

Centralized Training, Decentralized Execution

10.4.4.

Communication and Coordination

10.4.5.

Competitive vs Cooperative Settings

10.4.6.

Nash Equilibria

10.4.7.

Multi-Agent Policy Gradients

10.5.

Partial Observability

10.5.1.

Partially Observable MDPs (POMDPs)

10.5.2.

Belief State Representation

10.5.3.

Memory-Augmented Policies

10.5.4.

Recurrent Neural Networks

10.5.5.

Attention Mechanisms

10.6.

Inverse Reinforcement Learning

10.6.1.

Learning from Demonstrations

10.6.2.

Reward Function Recovery

10.6.3.

Maximum Entropy IRL

10.6.4.

Apprenticeship Learning

10.6.5.

Imitation Learning

10.6.5.1.

Behavioral Cloning

10.6.5.2.

Dataset Aggregation (DAgger)

10.7.

Safe Reinforcement Learning

10.7.1.

Safety Constraints

10.7.2.

Risk-Sensitive Objectives

10.7.3.

Safe Exploration

10.7.4.

Constrained Policy Optimization

10.7.5.

Worst-Case Analysis

10.8.

Transfer Learning and Meta-Learning

10.8.1.

Domain Adaptation

10.8.2.

Multi-Task Learning

10.8.3.

Few-Shot Learning

10.8.4.

Model-Agnostic Meta-Learning (MAML)

Previous

9. Policy Gradient Methods

Go to top

Next

11. Implementation and Practical Considerations