refactor

[typoverflow]

Hi Haotian, I drafted this pull request to better illustrate the structural changes. I think there are some things we can do:

• 1. extract a package, spectralrl, to manage the codes and to distribute the codes conveniently with pypi.
• 2. use hydra for argument parsing, instead of argparse, since different algorithms require different sets of arguments
• 3. refactor those image-based algorithms (namely MuLVRep, DiffSR and DrQv2), so that they share the same pipeline and just differ in the implementation of the agents
• 4. refactor the codes for state-based algorithms.

Currently, I have partially completed 1-3 (see spectral/ and examples/main_visual.py). I am still testing the image-based algorithms to make sure they behave in the same way as the original implementations. Could you help with refactoring the state-based algorithms if possible?

[typoverflow]

Hi Haotian, I have been using ctrl sac these days, so I refactored the state-based algorithms by the way. I tried to keep the implementations identical to the original ones, but I am not 100% sure whether their performances are good. It would be great if you can have a look at them, especially ctrl_sac and speder_sac, thanks!

[haotiansun14]

Hi Haotian, I have been using ctrl sac these days, so I refactored the state-based algorithms by the way. I tried to keep the implementations identical to the original ones, but I am not 100% sure whether their performances are good. It would be great if you can have a look at them, especially ctrl_sac and speder_sac, thanks!

Thank you for the great effort! I am taking a look at the implementation.

[typoverflow]

Benchmarked:

• visual/drqv2
• visual/mulvrep_drqv2
• visual/diffsr_drqv2
• state/sac

Benchmark Ongoing:

• state/ctrl_sac
• state/lvrep_sac
• state/speder_sac

Not implemented:

• state/diffsr_sac