Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Document procedure for pipeline cost estimation on Hail Batch #14711

Open
kasittig opened this issue Oct 3, 2024 · 1 comment
Open

Document procedure for pipeline cost estimation on Hail Batch #14711

kasittig opened this issue Oct 3, 2024 · 1 comment

Comments

@kasittig
Copy link
Contributor

kasittig commented Oct 3, 2024

A potential research collaborator is evaluating data platforms for running analysis pipelines on their upcoming very large dataset. They're interested in estimating the cost of running an existing pipeline using the Hail Query framework.

I think that getting one number here is likely very difficult. I do also think that this is a completely reasonable question for them to ask and that it would greatly benefit us to have some kind of documentation on cost estimation. Other collaborators might also have ideas.

@chrisvittal
Copy link
Collaborator

Some notes from discussion:

  1. Maybe add a pricing page with up to date pricing for resources.
  2. It is difficult to determine all the work that will run just from a hail pipeline.
  3. Teach users how to inspect the work that hail actually does?

@chrisvittal chrisvittal self-assigned this Oct 7, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants