From 01c3123d595dae530b9bc8f9e8fc67eb0276f1a9 Mon Sep 17 00:00:00 2001 From: shane knapp Date: Tue, 22 Oct 2024 10:18:31 -0700 Subject: [PATCH 1/5] adding cookiecutter --- dev-requirements.txt | 1 + 1 file changed, 1 insertion(+) diff --git a/dev-requirements.txt b/dev-requirements.txt index 5736f9761..6c1f52069 100644 --- a/dev-requirements.txt +++ b/dev-requirements.txt @@ -1,6 +1,7 @@ attrs==23.1.0 chardet==5.2.0 chartpress==2.2.0 +cookiecutter==2.6.0 git+https://github.com/berkeley-dsep-infra/hubploy.git jupyter-repo2docker==2024.07.0 myst-parser==4.0.0 From 88a18535ea1fda366e512376c4e452e56b9f5e62 Mon Sep 17 00:00:00 2001 From: shane knapp Date: Tue, 22 Oct 2024 10:32:06 -0700 Subject: [PATCH 2/5] fix/update some links to requirements.txt in our repo --- docs/admins/howto/calendar-scaler.qmd | 2 +- docs/admins/howto/new-hub.qmd | 15 ++++++++++----- 2 files changed, 11 insertions(+), 6 deletions(-) diff --git a/docs/admins/howto/calendar-scaler.qmd b/docs/admins/howto/calendar-scaler.qmd index 35ed93eb6..0f4bc2eb2 100644 --- a/docs/admins/howto/calendar-scaler.qmd +++ b/docs/admins/howto/calendar-scaler.qmd @@ -91,7 +91,7 @@ can run the following commands to create one: ``` bash conda create -ny scalertest python=3.11 -pip install -r requirements.txt +pip install -r images/node-placeholder-scaler/requirements.txt ``` Any changes to the scaler code will require you to run diff --git a/docs/admins/howto/new-hub.qmd b/docs/admins/howto/new-hub.qmd index 2b598ef9c..fa3a39869 100644 --- a/docs/admins/howto/new-hub.qmd +++ b/docs/admins/howto/new-hub.qmd @@ -9,7 +9,8 @@ aliases: The major reasons for making a new hub are: 1. A new course wants to join the Berkeley DataHub community. -2. One of your *students* are course staff in another course and have *elevated access*, enabling them to see other students' work. +2. One of your *students* are course staff in another course and have + *elevated access*, enabling them to see other students' work. 3. You want to use a different kind of authenticator. 4. You are running in a different cloud, or using a different billing account. @@ -25,11 +26,15 @@ Please let us know if you have some other justification for creating a new hub. Working installs of the following utilities: - - [sops](https://github.com/mozilla/sops/releases) - - [hubploy](https://hubploy.readthedocs.io/en/latest/index.html) + - [chartpress](https://pypi.org/project/chartpress/) + - [cookiecutter](https://pypi.org/project/cookiecutter/) - [gcloud](https://cloud.google.com/sdk/docs/install) + - [hubploy](https://github.com/berkeley-dsep-infra/hubploy) - [kubectl](https://kubernetes.io/docs/tasks/tools/) - - [cookiecutter](https://github.com/audreyr/cookiecutter) + - [sops](https://github.com/mozilla/sops/releases) + +The easiest way to install `chartpress`, `cookiecutter` and `hubploy` is to +run `pip install -r dev-requirements.txt` from the root of the `datahub` repo. Proper access to the following systems: @@ -53,7 +58,7 @@ will be required for the course. Typically, there are three general Small courses will usually have one or two assignments per semester, and may only have 20 or fewer users. -General courses have up to \~500 users, but don't have large amount of +General courses have up to ~500 users, but don't have large amount of data or require upgraded compute resources. Heavy usage courses can potentially have thousands of users, require From 3103e2c454b144705128f2644fcbd3c7cc4d4de3 Mon Sep 17 00:00:00 2001 From: shane knapp Date: Tue, 22 Oct 2024 10:39:51 -0700 Subject: [PATCH 3/5] update the readme w/some new instructions + extra formatting --- README.md | 24 ++++++++++++++++++++++-- 1 file changed, 22 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index b8c80ce72..e56cd0814 100644 --- a/README.md +++ b/README.md @@ -19,7 +19,18 @@ or from links in the deployment's `image/README.md` The `staging` branch always reflects the state of the [staging JupyterHub](http://staging.datahub.berkeley.edu), and the `prod` branch reflects the state of the [production JupyterHub](http://datahub.berkeley.edu). +## Installing the required python packages for working with datahub + +In the root directory of this repo, install `dev-requirements.txt` with the +following command: `pip install -r dev-requirements.txt`. This will install +the base python packages that are required to perform the tasks associated with +editing, testing, building and deploying hubs. + +The other python package definition file, `requirements.txt` is used solely by +our Github Actions CI/CD pipeline. + ## Setting up your fork and clones + First, go to your [github profile settings](https://github.com/settings/keys) and make sure you have an SSH key uploaded. @@ -59,7 +70,6 @@ git rebase upstream/staging && \ git push origin staging ``` - ## Procedure When developing for this deployment, always work in a fork of this repo. @@ -67,6 +77,9 @@ You should also make sure that your repo is up-to-date with this one prior to making changes. This is because other contributors may have pushed changes after you last synced with this repo but before you upstreamed your changes. +### Syncing your repo + +The following commands will sync the local clone of your fork with `upstream`: ``` git checkout staging && \ git fetch --prune --all && \ @@ -74,14 +87,19 @@ git rebase upstream/staging && \ git push origin staging ``` -To create a new branch and switch to it, run the following command: +### Creating a feature branch + +To create a new feature branch and switch to it, run the following command: ``` git checkout -b ``` +### Checking the status and diffs of your local work + After you make your changes, you can use the following commands to see what's been modified and check out the diffs: `git status` and `git diff`. +### Adding, committing and pushing changes When you're ready to push these changes, first you'll need to stage them for a commit: @@ -99,6 +117,8 @@ Now push to your fork: git push origin ``` +### Creating a pull request + Once you've pushed to your fork, you can go to the [Datahub repo](https://github.com/berkeley-dsep-infra/datahub) and there should be a big green button on the top that says `Compare and pull request`. From e1432f1c4b0bc7778ad08385f0f0cc05a154010b Mon Sep 17 00:00:00 2001 From: shane knapp Date: Tue, 22 Oct 2024 10:47:16 -0700 Subject: [PATCH 4/5] add ruff as it's handy --- dev-requirements.txt | 1 + 1 file changed, 1 insertion(+) diff --git a/dev-requirements.txt b/dev-requirements.txt index 6c1f52069..a72bb44df 100644 --- a/dev-requirements.txt +++ b/dev-requirements.txt @@ -9,5 +9,6 @@ niquests==3.7.2 pre-commit==4.0.1 pygithub==2.4.0 pyrsistent==0.19.3 +ruff==0.7.0 urllib3==2.2.3 yamllint==1.35.1 From 728b9e9a0eeffaefab3e63d2eb4784ed07974c79 Mon Sep 17 00:00:00 2001 From: "Image Builder Bot[tm]" Date: Tue, 22 Oct 2024 22:08:14 +0000 Subject: [PATCH 5/5] update ischool image tag to bf3004dcb234: deployments/ischool/hubploy.yaml --- deployments/ischool/hubploy.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/deployments/ischool/hubploy.yaml b/deployments/ischool/hubploy.yaml index 1b08b012a..ec63ec112 100644 --- a/deployments/ischool/hubploy.yaml +++ b/deployments/ischool/hubploy.yaml @@ -1,6 +1,6 @@ images: images: - - name: us-central1-docker.pkg.dev/ucb-datahub-2018/user-images/ischool-user-image:0be26ba905cb + - name: us-central1-docker.pkg.dev/ucb-datahub-2018/user-images/ischool-user-image:bf3004dcb234 cluster: provider: gcloud