强化学习脉络思维导图

方法对比矩阵

Method	Model-based	Policy-based	Value-based	Actor-Critic	On-policy	Off-policy	Bootstrapping	Continuous Action	Function Approx	Stable
Policy Iteration	✅	❌	✅	❌	❌	❌	❌	❌	❌	✅
Value Iteration	✅	❌	✅	❌	❌	❌	❌	❌	❌	✅
Monte Carlo	❌	❌	✅	❌	✅	❌	❌	❌	❌	❌
TD(0)	❌	❌	✅	❌	✅	❌	✅	❌	❌	❌
SARSA	❌	❌	✅	❌	✅	❌	✅	❌	❌	❌
Q-learning	❌	❌	✅	❌	❌	✅	✅	❌	❌	❌
DQN	❌	❌	✅	❌	❌	✅	✅	❌	✅	❌
REINFORCE	❌	✅	❌	❌	✅	❌	❌	✅	✅	❌
Actor-Critic	❌	✅	✅	✅	✅	❌	✅	✅	✅	❌
A2C	❌	✅	✅	✅	✅	❌	✅	✅	✅	❌
A3C	❌	✅	✅	✅	✅	❌	✅	✅	✅	❌
PPO	❌	✅	✅	✅	✅	❌	✅	✅	✅	✅
DDPG	❌	✅	✅	✅	❌	✅	✅	✅	✅	❌
TD3	❌	✅	✅	✅	❌	✅	✅	✅	✅	❌
SAC	❌	✅	✅	✅	❌	✅	✅	✅	✅	✅