SubprocVecEnv not receiving any data

[DavidS32]

I have a problem similar to issue #640

For a long time it worked without any problem and now the code get's stuck in the initilization of the SubprocVecEnv. The line with causes the problem is:

observation_space, action_space = self.remotes[0].recv()

There it gets stuck without any problem. it just receives no data....
The problem occurs with any anvironment. I want to use the SubprocVecEnv and not the DummyVecEnv.

I am on windows 10 with the following packeges:
`
Package Version

---

-tari-py 1.2.1
absl-py 0.9.0
alabaster 0.7.12
anaconda-client 1.7.2
anaconda-navigator 1.9.12
anaconda-project 0.8.3
argh 0.26.2
asn1crypto 1.3.0
astor 0.8.1
astroid 2.3.3
astropy 4.0
atari-py 0.2.6
atomicwrites 1.3.0
attrs 19.3.0
autopep8 1.4.4
Babel 2.8.0
backcall 0.1.0
backports.functools-lru-cache 1.6.1
backports.os 0.1.1
backports.shutil-get-terminal-size 1.0.0
backports.tempfile 1.0
backports.weakref 1.0.post1
bcrypt 3.1.7
beautifulsoup4 4.8.2
bitarray 1.2.1
bkcharts 0.2
bleach 3.1.0
blinker 1.4
bokeh 1.4.0
boto 2.49.0
Bottleneck 1.3.2
Box2D 2.3.10
Box2D-kengz 2.3.3
box2d-py 2.3.8
cachetools 4.0.0
certifi 2020.4.5.1
cffi 1.14.0
chardet 3.0.4
click 7.1.1
cloudpickle 1.3.0
clyent 1.2.2
colorama 0.4.3
comtypes 1.1.7
conda-package-handling 1.6.0
conda-verify 3.4.2
contextlib2 0.6.0.post1
cryptography 2.8
cycler 0.10.0
Cython 0.29.15
cytoolz 0.10.1
dask 2.12.0
decorator 4.4.2
defusedxml 0.6.0
diff-match-patch 20181111
distributed 2.12.0
docutils 0.16
entrypoints 0.3
et-xmlfile 1.0.1
fastcache 1.1.0
filelock 3.0.12
flake8 3.7.9
Flask 1.1.1
fsspec 0.6.3
future 0.18.2
gast 0.2.2
gevent 1.4.0
glfw 1.11.2
glob2 0.7
google-auth 1.12.0
google-auth-oauthlib 0.4.1
google-pasta 0.2.0
greenlet 0.4.15
grpcio 1.27.2
gym 0.17.1
h5py 2.10.0
HeapDict 1.0.1
html5lib 1.0.1
hypothesis 5.5.4
idna 2.9
imageio 2.8.0
imagesize 1.2.0
importlib-metadata 1.5.0
intervaltree 3.0.2
ipykernel 5.1.4
ipython 7.13.0
ipython-genutils 0.2.0
ipywidgets 7.5.1
isort 4.3.21
itsdangerous 1.1.0
jdcal 1.4.1
jedi 0.15.2
Jinja2 2.11.1
joblib 0.14.1
json5 0.9.3
jsonschema 3.2.0
jupyter 1.0.0
jupyter-client 6.1.0
jupyter-console 6.1.0
jupyter-core 4.6.1
jupyterlab 1.2.6
jupyterlab-server 1.0.7
Keras-Applications 1.0.8
Keras-Preprocessing 1.1.0
keyring 21.1.0
kiwisolver 1.1.0
lazy-object-proxy 1.4.3
libarchive-c 2.8
llvmlite 0.31.0
locket 0.2.0
lockfile 0.12.2
lxml 4.5.0
Markdown 3.2.1
MarkupSafe 1.1.1
matplotlib 3.1.3
mccabe 0.6.1
menuinst 1.4.16
mistune 0.8.4
mkl-fft 1.0.15
mkl-random 1.1.0
mkl-service 2.3.0
mock 4.0.1
more-itertools 8.2.0
mpmath 1.1.0
msgpack 1.0.0
multipledispatch 0.6.0
navigator-updater 0.2.1
nbconvert 5.6.1
nbformat 5.0.4
networkx 2.4
nltk 3.4.5
nose 1.3.7
notebook 6.0.3
numba 0.48.0
numexpr 2.7.1
numpy 1.18.1
numpydoc 0.9.2
oauthlib 3.1.0
olefile 0.46
opencv-python 4.2.0.34
openpyxl 3.0.3
opt-einsum 3.2.0
packaging 20.3
pandas 1.0.3
pandocfilters 1.4.2
paramiko 2.7.1
parso 0.5.2
partd 1.1.0
path 13.1.0
pathlib2 2.3.5
pathtools 0.1.2
patsy 0.5.1
pep8 1.7.1
pexpect 4.8.0
pickleshare 0.7.5
Pillow 7.0.0
pip 20.0.2
pkginfo 1.5.0.1
pluggy 0.13.1
ply 3.11
prometheus-client 0.7.1
prompt-toolkit 3.0.3
protobuf 3.11.4
psutil 5.7.0
py 1.8.1
pyasn1 0.4.8
pyasn1-modules 0.2.8
pycodestyle 2.5.0
pycosat 0.6.3
pycparser 2.20
pycrypto 2.6.1
pycurl 7.43.0.5
pydocstyle 4.0.1
pyflakes 2.1.1
pyglet 1.5.0
Pygments 2.6.1
PyJWT 1.7.1
pylint 2.4.4
PyNaCl 1.3.0
pyodbc 4.0.0-unsupported
pyOpenSSL 19.1.0
pyparsing 2.4.6
pyreadline 2.1
pyrsistent 0.15.7
PySocks 1.7.1
pytest 5.4.1
pytest-arraydiff 0.3
pytest-astropy 0.8.0
pytest-astropy-header 0.1.2
pytest-doctestplus 0.5.0
pytest-openfiles 0.4.0
pytest-remotedata 0.3.2
python-dateutil 2.8.1
python-jsonrpc-server 0.3.4
python-language-server 0.31.9
pytz 2019.3
PyWavelets 1.1.1
pywin32 227
pywin32-ctypes 0.2.0
pywinpty 0.5.7
PyYAML 5.3.1
pyzmq 18.1.1
QDarkStyle 2.8
QtAwesome 0.7.0
qtconsole 4.7.2
QtPy 1.9.0
requests 2.23.0
requests-oauthlib 1.3.0
rope 0.16.0
rsa 4.0
Rtree 0.9.3
ruamel-yaml 0.15.87
scikit-image 0.16.2
scikit-learn 0.22.1
scipy 1.1.0
seaborn 0.10.0
Send2Trash 1.5.0
setuptools 46.1.1.post20200323
simplegeneric 0.8.1
singledispatch 3.4.0.3
six 1.14.0
snowballstemmer 2.0.0
sortedcollections 1.1.2
sortedcontainers 2.1.0
soupsieve 2.0
Sphinx 2.4.4
sphinxcontrib-applehelp 1.0.2
sphinxcontrib-devhelp 1.0.2
sphinxcontrib-htmlhelp 1.0.3
sphinxcontrib-jsmath 1.0.1
sphinxcontrib-qthelp 1.0.3
sphinxcontrib-serializinghtml 1.1.4
sphinxcontrib-websupport 1.2.1
spyder 4.1.1
spyder-kernels 1.9.0
SQLAlchemy 1.3.15
stable-baselines 2.10.0
statsmodels 0.11.0
sympy 1.5.1
tables 3.6.1
tblib 1.6.0
tensorboard 1.15.0
tensorflow 1.15.0
tensorflow-estimator 1.15.1
tensorflow-gpu 1.15.0
termcolor 1.1.0
terminado 0.8.3
testpath 0.4.4
toolz 0.10.0
torch 1.4.0
torchvision 0.5.0
tornado 6.0.4
tqdm 4.43.0
traitlets 4.3.3
ujson 1.35
unicodecsv 0.14.1
urllib3 1.25.8
watchdog 0.10.2
wcwidth 0.1.8
webencodings 0.5.1
Werkzeug 0.16.0
wheel 0.34.2
widgetsnbextension 3.5.1
win-inet-pton 1.1.0
win-unicode-console 0.5
wincertstore 0.2
wrapt 1.12.1
xlrd 1.2.0
XlsxWriter 1.2.8
xlwings 0.18.0
xlwt 1.3.0
xmltodict 0.12.0
yapf 0.28.0
zict 2.0.0
zipp 2.2.0 `

[DavidS32]

Ok I've done a few more tests an it seems to be a bit randome if the vektorized environment gets initialized. Sometimes it just works but most of the time it's not working.

It's interesting to watch the task manager if the initialization fails. First he opens as much tasks as you configured and then they get closed. The code is then stuck at

observation_space, action_space = self.remotes[0].recv()

[pisiiki]

@DavidS32 I'm almost sure this is something related to your computer config. It seems that you are using anaconda. Please try to create a fresh new environment like this:

conda create -n new_env tensorflow-gpu==1.15 cloudpickle~=1.2.0

Then install baselines on the anaconda env. with pip. I have just tested this on windows10 and it works (github master branch). If it doesn't work for you you can also check your firewall/antivirus software. Hope this helps.

[DavidS32]

@DavidS32 I'm almost sure this is something related to your computer config. It seems that you are using anaconda. Please try to create a fresh new environment like this:

conda create -n new_env tensorflow-gpu==1.15 cloudpickle~=1.2.0

Then install baselines on the anaconda env. with pip. I have just tested this on windows10 and it works (github master branch). If it doesn't work for you you can also check your firewall/antivirus software. Hope this helps.

Thanks for the answer :)
With your advice I reinstalled anaconda completely and installed the specific cloudpickle version. Now it seams to work.
At least the cart pole environment. My own one is still not working....

BrokenPipeError: [WinError 109] Die Pipe wurde beendet


During handling of the above exception, another exception occurred:

Traceback (most recent call last):

  File "D:\ccc\Testumgebung\Selfmade\PPO\V1_Baseline\main.py", line 37, in <module>
    model.learn(total_timesteps=10000, tb_log_name=safeName)

  File "C:\Users\ccc\Anaconda3\envs\TF_1_15_0\lib\site-packages\stable_baselines\ppo2\ppo2.py", line 336, in learn
    rollout = self.runner.run(callback)

  File "C:\Users\ccc\Anaconda3\envs\TF_1_15_0\lib\site-packages\stable_baselines\common\runners.py", line 48, in run
    return self._run()

  File "C:\Users\ccc\Anaconda3\envs\TF_1_15_0\lib\site-packages\stable_baselines\ppo2\ppo2.py", line 482, in _run
    self.obs[:], rewards, self.dones, infos = self.env.step(clipped_actions)

  File "C:\Users\ccc\Anaconda3\envs\TF_1_15_0\lib\site-packages\stable_baselines\common\vec_env\base_vec_env.py", line 150, in step
    return self.step_wait()

  File "C:\Users\ccc\Anaconda3\envs\TF_1_15_0\lib\site-packages\stable_baselines\common\vec_env\subproc_vec_env.py", line 107, in step_wait
    results = [remote.recv() for remote in self.remotes]

  File "C:\Users\ccc\Anaconda3\envs\TF_1_15_0\lib\site-packages\stable_baselines\common\vec_env\subproc_vec_env.py", line 107, in <listcomp>
    results = [remote.recv() for remote in self.remotes]

  File "C:\Users\ccc\Anaconda3\envs\TF_1_15_0\lib\multiprocessing\connection.py", line 250, in recv
    buf = self._recv_bytes()

  File "C:\Users\ccc\Anaconda3\envs\TF_1_15_0\lib\multiprocessing\connection.py", line 321, in _recv_bytes
    raise EOFError

EOFError

[DavidS32]

I was able to fix the problem.
I recommend to use the check_env function from the baseline.

https://stable-baselines.readthedocs.io/en/master/guide/custom_env.html

[praetorianer777]

Hello, I got the same problem and check_env doesn't throw an error...
What exactly was your problem?

[DavidS32]

Hello, I got the same problem and check_env doesn't throw an error...
What exactly was your problem?

I still cant say where the problem comes from. Which IDE do you use?

[praetorianer777]

What exactly was your problem?
I use Spyder 4.1.3 installed by anaconda navigator.

[DavidS32]

What exactly was your problem?
I use Spyder 4.1.3 installed by anaconda navigator.

If i start multiple threads with SubprocVecEnv, all created precesses die after a few seconds.
I also have no problems with check_env becuse there is no error in my environment. I think the problem comes from spyder.
You can try to start the skript with the console that anaconda offers.
For me this works every time.

[praetorianer777]

Hmmm... indeed it works!
Thank you very much :D