Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Able to configure the shm size in Playground #238

Open
3 tasks
kerthcet opened this issue Jan 16, 2025 · 0 comments
Open
3 tasks

Able to configure the shm size in Playground #238

kerthcet opened this issue Jan 16, 2025 · 0 comments
Labels
needs-kind Indicates a PR lacks a label and requires one. needs-priority Indicates a PR lacks a label and requires one. needs-triage Indicates an issue or PR lacks a label and requires one.

Comments

@kerthcet
Copy link
Member

What would you like to be added:

When serving with ray for multi-host cases, you should be able to set the /dev/shm size to improve the performance, see the log here:

2025-01-14 03:08:29,081	WARNING services.py:2022 -- WARNING: The object store is using /tmp instead of /dev/shm because /dev/shm has only 67108864 bytes available. This will harm performance! You may be able to free up space by deleting files in /dev/shm. If you are inside a Docker container, you can increase /dev/shm size by passing '--shm-size=5.04gb' to 'docker run' (or add it to the run_options list in a Ray cluster config). Make sure to set this to more than 30% of available RAM.

Ray natively supports a distributed shared memory store for large objects (objects that are bigger than 100KB) to efficiently manage the memory usage of the cluster. When using Linux, Ray tries to use /dev/shm for the shared memory. But if this is too small, Ray automatically fallback to /tmp folder, which usually can cause worse performance than using the shared memory.

Why is this needed:

Completion requirements:

This enhancement requires the following artifacts:

  • Design doc
  • API change
  • Docs update

The artifacts should be linked in subsequent comments.

@InftyAI-Agent InftyAI-Agent added needs-triage Indicates an issue or PR lacks a label and requires one. needs-kind Indicates a PR lacks a label and requires one. needs-priority Indicates a PR lacks a label and requires one. labels Jan 16, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
needs-kind Indicates a PR lacks a label and requires one. needs-priority Indicates a PR lacks a label and requires one. needs-triage Indicates an issue or PR lacks a label and requires one.
Projects
None yet
Development

No branches or pull requests

2 participants