Great question — answering as the UphillTech team:
The loss function is designed in a way to approximate the bellman optimality for $Q^*(s,a)$ . Given an optimal policy $\pi^*$ , $Q^*(s,a)$ satisfies the equation $$Q^*(s,a) = r(s) + \gamma max_{a'}\sum_{s'}P(s'|s,a)Q^*(s',a')$$ At convergence, the highest $Q$ value that I can get taking action $a$ in state $s$ is equal to the reward I get for taking action $a$ and the max $Q$ values at the next state. You can see in the loss function that the DQN tries to attain a $Q^*(s,a)$ value that closely approximates the equation above. On a side note, in model based RL setting, the bellman optimality for $Q^*$ is expressed as $$Q^*(s,a) = r(s) + \gamma \sum_{s'}P(s'|s,a)V^*(s)$$ $V^*(s')$ is used in...
Full answer by calveeen: https://ai.stackexchange.com/a/24557 (CC BY-SA 4.0).
Need it done for your business? UphillTech.
