SubprocVecEnv not working #640
Comments
|
Apparently even with the default envs ('Cartpole-V1' as depicted in the linked example) has the same problem. Both custom and default envs work fine for DummyVecEnv's though. Edited: After some testing, the problem seems to be at:
Is the process eternally waiting for a response? |
|
Update: 10/10/2018: So I tried digging up a bit deeper, and eventually came to the conclusion that a part of the problem is in calling functions in the default way: Does not work either, it just hangs in there forever. However, if
|
|
Hi @lhorus ! As you have already established, SubprocVecEnv's are nasty to debug, because if the worker process fails, the parent process is just left there waiting or gets an uninformative broken pipe. Making SubprocVecEnv more debug-friendly is on a TODO list; however, as a general rule of thumb for now, if the problem is with vecenv, I'd recommend replacing SubprocVecEnv with DummyVecEnv and debug that way. Now, in your particular case sounds like the error is actually specific to SubprocVecEnv - for some reason forking the process has troubles with open file descriptors. If you were to remove Monitor wrapper from make_env, would the resulting thing work? |
|
I also faced the same issue and got it to work following the below hack, note that this is not a fix to the problem, it is a hack so that you can use import os
from stable_baselines import logger
from stable_baselines.bench import Monitor
from stable_baselines.common import set_global_seeds
from stable_baselines.common.atari_wrappers import make_atari, wrap_deepmind
from stable_baselines.common.cmd_util import make_atari_env
from stable_baselines.common.vec_env import DummyVecEnv
from stable_baselines.common.policies import CnnPolicy
from stable_baselines.common.vec_env import VecFrameStack
from stable_baselines import ACER
def custom_atari_env(env_id, num_env, seed, wrapper_kwargs=None, start_index=0,
allow_early_resets=True):
"""
Create a wrapped, monitored SubprocVecEnv for Atari.
:param env_id: (str) the environment ID
:param num_env: (int) the number of environment you wish to have
in subprocesses
:param seed: (int) the inital seed for RNG
:param wrapper_kwargs: (dict) the parameters for wrap_deepmind function
:param start_index: (int) start rank index
:param allow_early_resets: (bool) allows early reset of the environment
:return: (Gym Environment) The atari environment
"""
if wrapper_kwargs is None:
wrapper_kwargs = {}
def make_env(rank):
def _thunk():
env = make_atari(env_id)
env.seed(seed + rank)
env = Monitor(env, logger.get_dir() and os.path.join(
logger.get_dir(), str(rank)),
allow_early_resets=allow_early_resets)
return wrap_deepmind(env, **wrapper_kwargs)
return _thunk
set_global_seeds(seed)
return DummyVecEnv([make_env(i + start_index) for i in range(num_env)])
def main():
env = custom_atari_env('BreakoutNoFrameskip-v4', num_env=4, seed=42)
env = make_atari_env('PongNoFrameskip-v4', num_env=4, seed=42)
env = VecFrameStack(env, n_stack=4)
model = ACER(CnnPolicy, env, verbose=1,
tensorboard_log='/private/home/akadian/teas-baselines/'
'simple-nav/stable-baselines-runs')
model.learn(total_timesteps=25)
model.save('acer.pong.model')
if __name__ == '__main__':
main() |
|
@abhiskk from what I understand, your hack does not actually use SubprocVecEnv (it is replaced with DummyVecEnv). In OP's particular problem that should help indeed (because no forking / pickling is going on in DummyVecEnv); however, this fix will make things quite a bit slower. |
|
It does use env = custom_atari_env('BreakoutNoFrameskip-v4', num_env=4, seed=42)
env = make_atari_env('PongNoFrameskip-v4', num_env=4, seed=42)The call to |
|
I was having a similar issue when creating a UnityEnv gym wrapper using SubprocVecEnv. For me, the fix was editing the default RPC server port because the port was already open. Editing: A computer restart would have likely fixed this as well, I recommend trying a hard reset of your computer if you face a similar issue. |
|
I have the same issue, this should get fixed. My code works perfectly fine with Dummy env but Subproc throws the exact same error |
|
@Coronon could you provide the code you are running and error stacktrace? The baselines code has changed quite a bit since the issue was opened; it would be helpful to have a recent stacktrace. |
* Pass build arguments to Docker * Fix Codacy downloader to work with latest releases * Add automatic Docker push for release * Update changelog * Print pytype version in CI * Fix type error * Fix other type false positive
|
I have the same issue, running on a Mac M3 from stable_baselines3.common.env_util import make_vec_env from stable_baselines3.common.monitor import Monitor import ale_py gym.register_envs(ale_py) environment_name = "SpaceInvadersNoFrameskip-v4" env = gym.make(environment_name) Error given: Traceback (most recent call last): |
I'm trying to use SubprocVecEnv to initialize multiple instances of one environment to use in ACKTR and A2C, however, there seem to be problems with SubprocVecEnv. Following this code, the following happens:
Meaning, env creation through the function works fine, the same goes for manual creation, however, when SubprocVecEnv is called the code gets stuck, nothing seems to happen and it doesn't stop running, nor does the environment get created.
I tried calling manually
env = SubprocVecEnv([make_env])and it yields a strange error:The text was updated successfully, but these errors were encountered: