[Core] Support reset_prefix_cache

[comaniac]

This PR adds the capability of resetting prefix cache for both v0 and v1. This feature could be useful for the following cases:

1. In RLHF after weight syncing.
2. In benchmarking.

The prefix cache can only be reset when all blocks are free. However, although this PR also provide /reset_prefix_cache in API server, we currently always return 200 whether the prefix cache is reset successfully or not. This is mainly because it's not trivial for engine client and server to communicate such information (no fundamental burden but just need some work).

cc @youkaichao @robertgshaw2-redhat

[github-actions]

👋 Hi! Thank you for contributing to the vLLM project.
Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can do one of these:

• Add ready label to the PR
• Enable auto-merge.

🚀

[youkaichao]

it's better to have a flag so that this endpoint is opt-in, users should not send this request.

we can add an env var VLLM_DEV_MODE, and only expose this endpoint when VLLM_DEV_MODE is set.

[comaniac]

Yeah we can just add an argument when launching an endpoint to allow this route. I supposed most production API server doesn't want to this be enabled.

[youkaichao]

thanks for the great work! I didn't imagine this is so involved (it touches quite a lot files). I think we should do something to refactor the way we create a new endpoint, cc @robertgshaw2-redhat