Question/Suggestion, Multi-GPU setups.

[phalexo]

I have been playing with DeepFloyd from Stable Diffusion and one of the capabilities is that it is possible to put different chunks of the whole model onto different GPUs. Quite a lot of people have older GPUs, but several of them. I have 4 Maxwell Titans with 12.3GiB each.
So one GPU is too small to run the largest models, but several would have enough VRAM.

DeepFloyd is using the "accelerate" python module to allow model weight migration.

Question: Is "accelerate" being used with Open Assistant?

Suggestion: If not, please, consider it, as it would open up OA to a lot more people with multiple but older GPUs.

[phalexo]

Models will necessarily grow, the ability to put multiple small models on different GPUs or splitting a large model across multiple GPUs is a necessity, not a luxury. How many people really have access to 40GiB or 80GiB VRAM GPUs?