A2C some feedback

[simoninithomas]

Hi!

First of all thank you very much for your awesome work, the baselines are really good, but please comment your code because some part of it are not intuitive at all.

I think it can be a good idea to have a sort of documentation for each architecture and a small tutorial to explain why your architecture is implemented like that..

By the way I wanted to know why you implement some functions instead of using Tensorflow? For instance cat_entropy. Is there a reason or it's just because you prefer implement them?

Thanks again for your work!
Have a great day!