Bug in baselines/ddpg/ddpg.py?

[MogicianWu]

When I tried to run popart and return normalization, the program crashed at setup_popart() method in line 195. After looking at models.output_vars method, I found no variable in the graph has 'output' in their name. Is this a possible bug?

[watts4speed]

I saw the same thing. There's also code that's looking for some additional sub-strings. A fix will need to account for these also. If I can get this running I'll post something for others.

    for vs in [self.critic.output_vars, self.target_critic.output_vars]:
        assert len(vs) == 2
        M, b = vs
        assert 'kernel' in M.name
        assert 'bias' in b.name
        assert M.get_shape()[-1] == 1
        assert b.get_shape()[-1] == 1

[bjmuld]

samesies

[kirk86]

Any progress on this? ddpg seems to have multiple problems. I found that using mpirun with np > 2 throws an error because of NaNs in the thetalocal variable inside the MpiAdam class although the var_list seems to contain variables and it's not empty.