Policy Gradient Methods

Direct Policy Optimization

Advantages over Value-Based Methods

Policy Parameterization

Mathematical Derivation

Score Function Estimator

Gradient Estimation

Monte Carlo Policy Gradient

Algorithm Implementation

Variance Issues

Baseline Methods

Combining Policy and Value Learning

Actor Network (Policy)

Critic Network (Value Function)

Training Procedures

Advantage Actor-Critic (A2C)

Asynchronous Advantage Actor-Critic (A3C)

Generalized Advantage Estimation (GAE)

Trust Region Policy Optimization (TRPO)

Proximal Policy Optimization (PPO)

Gaussian Policies

Policy Parameterization

Exploration in Continuous Spaces