fine-tuning/RLHF ESM models

[avilella]

I have seen a couple of recent preprints/publications on fine-tuning ESM models.

I was wondering if anyone has considered an RLHF approach whereby we would use external proteins (an internal database of antibodies) as RLHF input to tune the ESM models.

Would an approach where 5000 external proteins are given as positive RLHF records (out of a set of ~250,000 external proteins) be enough to make the fine-tuned model infer which novel proteins to choose from based on the choices made by humans in the 5000 protein dataset?

What do people think would be the compute requirements for this? Would a single 48Gb Nvidia GPU RTX A6000 be enough?