Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(dev): Add Flox for local environment setup #26805

Merged
merged 10 commits into from
Dec 18, 2024
Merged

chore(dev): Add Flox for local environment setup #26805

merged 10 commits into from
Dec 18, 2024

Conversation

Twixes
Copy link
Member

@Twixes Twixes commented Dec 10, 2024

Problem

To get started developing PostHog, you have to install an array of dependencies with the correct versions, running a dozen of commands from the Developing locally guide. And then xmlsec ruins your day anyway. What if it took just one command to install all the right components with actual version control of the environment, without relying on janky system-wide deps?

Changes

Flox (brew install flox) is a take on this problem – specifically, a dev environment manager backed by Nix. As opposed to Homebrew, Nix has real versioning (xmlsec issues begone).

This PR adds a Flox manifest for the PostHog local environment, which turns basically the whole "Developing locally" guide into these few steps:

  1. brew install flox ruff rustup && rustup-init && rustup default stable – to install the only system-level dependencies needed: flox itself, ruff for Python pre-commit checks, cargo via rustup for Rust checks:

  2. flox activate

That's it actually. This gets you a full-fledged environment under .flox/ in the repository directory's, with everything needed to run and develop PostHog. Once done, you'll see instructions for migrations and running the app.
On the first activation you'll also be prompted if you'd like the environment to be activated automatically.

As a bonus, this PR replaces the old bin/start with bin/start-mprocs, by popular demand.

Copy link
Contributor

github-actions bot commented Dec 10, 2024

Size Change: 0 B

Total Size: 1.11 MB

ℹ️ View Unchanged
Filename Size
frontend/dist/toolbar.js 1.11 MB

compressed-size-action

@Twixes Twixes force-pushed the flox branch 5 times, most recently from 65cd4a9 to 5e44092 Compare December 16, 2024 12:28
@Twixes Twixes marked this pull request as ready for review December 18, 2024 16:19
@@ -20,4 +25,4 @@ python manage.py run_async_migrations --complete-noop-migrations
# k8s pod deployments.
python manage.py run_async_migrations --check

python manage.py sync_replicated_schema
wait $(jobs -p) # Make sure CH migrations are done before we exit
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Log message letting us know what we're doing would be helpful, otherwise it might feel like we're stuck

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You still get CH migration logs as migrations progress, so there shouldn't be that feeling – have you seen otherwise?

Copy link
Member

@rafaeelaudibert rafaeelaudibert left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎉 (didnt test it myself, looks good)

@mariusandra
Copy link
Collaborator

mariusandra commented Dec 18, 2024

Tried running it, here's the output

marius@Mariuss-MBP-2 ~/P/P/posthog (flox)> flox activate
Flox collects basic usage metrics in order to improve the user experience.

Flox includes a record of the subcommand invoked along with a unique token.
It does not collect any personal information.

The collection of metrics can be disabled in the following ways:

  environment: FLOX_DISABLE_METRICS=true
    user-wide: flox config --set-bool disable_metrics true
  system-wide: update /etc/flox.toml as described in flox(1)

This is a one-time notice.


✅ You are now using the environment 'posthog'.
To stop using this environment, type 'exit'

Using Python 3.11.9 interpreter at: .flox/run/aarch64-darwin.posthog.dev/bin/python3
Creating virtualenv at: .flox/cache/venv
Activate with: source .flox/cache/venv/bin/activate.fish
Python virtual environment path: .flox/cache/venv
Python interpreter path, for your code editor: .flox/cache/venv/bin/python

👉 Use direnv (https://direnv.net) for automatic activation of this environment by your shell.
❓ Would you like direnv to be set up now? (y/n, default: y): 
🔄 Installing direnv using 'brew install direnv'...
==> Fetching dependencies for direnv: bash
==> Fetching bash
==> Fetching direnv
==> Installing dependencies for direnv: bash
==> Installing direnv dependency: bash
🍺  /opt/homebrew/Cellar/bash/5.2.37: 163 files, 12.4MB
==> Installing direnv
🍺  /opt/homebrew/Cellar/direnv/2.35.0: 12 files, 10.5MB
==> Running `brew cleanup direnv`...
Disable this behaviour by setting HOMEBREW_NO_INSTALL_CLEANUP.
Hide these hints with HOMEBREW_NO_ENV_HINTS (see `man brew`).
✅ Installed direnv
✅ Configured ~/.config/direnv/direnv.toml
✅ Injected direnv hook into /Users/marius/.config/fish/config.fish
💫 direnv is now active

IT'S DANGEROUS TO GO ALONE! RUN THIS:
1. bin/migrate - to run all migrations
2. bin/start - to start the entire stack
3. ./manage.py generate_demo_data - to create a user with demo data

fish: Unknown command: '[global]'
~/.config/fish/config.fish (line 18): 
[global]
^~~~~~~^
from sourcing file ~/.config/fish/config.fish
        called during startup
fish: Unknown command: warn_timeout
~/.config/fish/config.fish (line 19): 
warn_timeout = 0 # Ignore timeout from this issue: https://github.com/direnv/direnv/issues/1065 - added by PostHog's Flox activation hook (../posthog/.flox/env/manifest.toml)
^~~~~~~~~~~^
from sourcing file ~/.config/fish/config.fish
        called during startup
Welcome to fish, the friendly interactive shell
Type help for instructions on how to use fish
(venv) flox [posthog] marius@Mariuss-MacBook-Pro-2 ~/P/P/posthog (flox)> 

Not sure what that is. The end of my fish config looks like this:

status is-login; and pyenv init --path | source
status is-interactive; and pyenv init - | source

rvm default

[global]
warn_timeout = 0 # Ignore timeout from this issue: https://github.com/direnv/direnv/issues/1065 - added by PostHog's Flox activation hook (../posthog/.flox/env/manifest.toml)

# Initialize direnv - added by PostHog's Flox activation hook (../posthog/.flox/env/manifest.toml)
direnv hook fish | source

bin/start works in this shell, so I assume it works... though everything worked before as well:

image

@Twixes
Copy link
Member Author

Twixes commented Dec 18, 2024

Oops, you can remove lines [global] and warn_timeout = 0, that was a mishap in my script. Thanks for catching, fixed!

@Twixes Twixes enabled auto-merge (squash) December 18, 2024 23:13
@Twixes Twixes merged commit a99faee into master Dec 18, 2024
95 checks passed
@Twixes Twixes deleted the flox branch December 18, 2024 23:36
@daibhin
Copy link
Contributor

daibhin commented Dec 19, 2024

We should probably post about this in Slack and update the Developing Locally doc. I ran into an issue this morning because I didn't have mprocs installed. Trying to figure out the migration path to flox now

@Twixes
Copy link
Member Author

Twixes commented Dec 19, 2024

Oh, right, the switch to bin/start-mprocs has been by popular demand, but I neglected a migration path if you haven't used it before 😅 You can just brew install mprocs! Will put out a PR to make this automatic

@Twixes
Copy link
Member Author

Twixes commented Dec 19, 2024

Fix here: #27052

@t0rx
Copy link

t0rx commented Dec 20, 2024

Out of curiosity, why not also specify ruff and rustup in your Flox environment?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants