Slow download from HuggingFace Hub (capped at 10.5 MB/s)

[Andrei-Aksionov]

Bug description

Sometimes the download speed is pretty slow, no more than 10.5 MB/s.
For example, gemma-2-2b takes a lot of time to download.

In download.py the code sets hf_transfer variable via constants:

litgpt/litgpt/scripts/download.py

Lines 79 to 82 in fe96c63

	if _HF_TRANSFER_AVAILABLE and not previous:
	print("Setting HF_HUB_ENABLE_HF_TRANSFER=1")
	constants.HF_HUB_ENABLE_HF_TRANSFER = True
	download.HF_HUB_ENABLE_HF_TRANSFER = True

But it doesn't help.
Even if set the value through os.environ["HF_HUB_ENABLE_HF_TRANSFER"]="1".

But, if to export the env variable before running a script one can notice a significant speed-up:

export HF_HUB_ENABLE_HF_TRANSFER=1

Most likely, HF Hub checks the env variable upon initialization.
The code needs to be fixed.

What operating system are you using?

macOS

LitGPT Version

Version: 0.5.4.dev1

[rasbt]

I think this is fixed now via #1899

[Andrei-Aksionov]

Nope :)
Still the same: the downloading speed is capped at 10.5, but if I enable HF_HUB_ENABLE_HF_TRANSFER before downloading it is superfast.

[rasbt]

So weird, I wasn't able to reproduce!

[ahsannawazch]

still the issue is not fixed. I am downloading Qwen2 VL 2b instruct model in lightning.ai studio and takes a lot of time.
3x slower downloading than kaggle.

[Respaired]

it goes beyond these models, it's an HF thing i believe and not Litgpt

[ahsannawazch]

it goes beyond these models, it's an HF thing i believe and not Litgpt

well I tried following and it went upto 500-600 MB/s. Took few seconds to download the same model.

1. export HF_HUB_ENABLE_HF_TRANSFER=1
2. pip install hf_transfer