Skip to content

Commit

Permalink
Revert "Migrate driver adapters to prisma/prisma (#4380)"
Browse files Browse the repository at this point in the history
This reverts commit 6dda9d7.
  • Loading branch information
miguelff committed Oct 27, 2023
1 parent 685d9bc commit 7d2dac2
Show file tree
Hide file tree
Showing 101 changed files with 5,279 additions and 348 deletions.
131 changes: 131 additions & 0 deletions .github/workflows/driver-adapter-smoke-tests.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,131 @@
name: Driver Adapters, Smoke Tests
on:
push:
branches:
- main
pull_request:
paths-ignore:
- '.github/**'
- '!.github/workflows/driver-adapter-smoke-tests.yml'
- '.buildkite/**'
- '*.md'
- 'LICENSE'
- 'CODEOWNERS'
- 'renovate.json'

jobs:
driver-adapter-smoke-tests:
name: ${{ matrix.adapter }}

strategy:
fail-fast: false
matrix:
adapter: ['neon:ws', 'neon:http', planetscale, pg, libsql]

runs-on: ubuntu-latest

services:
postgres:
image: postgres
env:
POSTGRES_PASSWORD: postgres
options: >-
--health-cmd pg_isready
--health-interval 10s
--health-timeout 5s
--health-retries 5
ports:
- 5432:5432

# via package.json rewritten into DATABASE_URL before scripts are run
env:
JS_NEON_DATABASE_URL: ${{ secrets.JS_NEON_DATABASE_URL }}
JS_PLANETSCALE_DATABASE_URL: ${{ secrets.JS_PLANETSCALE_DATABASE_URL }}
JS_PG_DATABASE_URL: postgres://postgres:postgres@localhost:5432/test # ${{ secrets.JS_PG_DATABASE_URL }}
# TODO: test sqld and embedded replicas
JS_LIBSQL_DATABASE_URL: file:/tmp/libsql.db
# TODO: test all three of ("number", "bigint", "string") and conditionally skip some tests as appropriate
JS_LIBSQL_INT_MODE: bigint

steps:
- uses: actions/checkout@v4

- uses: dtolnay/rust-toolchain@stable

- uses: pnpm/action-setup@v2
with:
version: 8
- uses: actions/setup-node@v3
with:
node-version: 18
#cache: 'pnpm'

- name: Compile Query Engine
run: cargo build -p query-engine-node-api

- name: Install Dependencies (Driver Adapters)
run: pnpm install
working-directory: ./query-engine/driver-adapters/js
- name: Build Driver Adapters
run: pnpm build
working-directory: ./query-engine/driver-adapters/js

- run: pnpm prisma:${{ matrix.adapter }}
working-directory: ./query-engine/driver-adapters/js/smoke-test-js
- run: pnpm ${{ matrix.adapter }}:libquery
working-directory: ./query-engine/driver-adapters/js/smoke-test-js
- name: pnpm ${{ matrix.adapter }}:client (using @prisma/client - including engine! - from Npm)
run: pnpm ${{ matrix.adapter }}:client
if: always()
working-directory: ./query-engine/driver-adapters/js/smoke-test-js


driver-adapter-smoke-tests-errors:
name: Errors

runs-on: ubuntu-latest

# services:
# postgres:
# image: postgres
# env:
# POSTGRES_PASSWORD: postgres
# options: >-
# --health-cmd pg_isready
# --health-interval 10s
# --health-timeout 5s
# --health-retries 5
# ports:
# - 5432:5432

env:
# via package.json rewritten into DATABASE_URL before scripts are run
JS_PG_DATABASE_URL: postgres://postgres:postgres@localhost:5432/test

steps:
- uses: actions/checkout@v4

- uses: dtolnay/rust-toolchain@stable

- uses: pnpm/action-setup@v2
with:
version: 8
- uses: actions/setup-node@v3
with:
node-version: 18
#cache: 'pnpm'

- name: Compile Query Engine
run: cargo build -p query-engine-node-api

- name: Install Dependencies (Driver Adapters)
run: pnpm install
working-directory: ./query-engine/driver-adapters/js
- name: Build Driver Adapters
run: pnpm build
working-directory: ./query-engine/driver-adapters/js

- name: pnpm errors
run: pnpm errors
if: always()
working-directory: ./query-engine/driver-adapters/js/smoke-test-js
83 changes: 83 additions & 0 deletions .github/workflows/publish-driver-adapters.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
name: Build and publish Prisma Driver Adapters
run-name: npm - release Driver Adapters ${{ github.event.inputs.prismaVersion }} from ${{ github.event.inputs.enginesHash }} on ${{ github.event.inputs.npmDistTag }}

concurrency: publish-prisma-driver-adapters

on:
# usually triggered via GH Actions Workflow in prisma/prisma repo
workflow_dispatch:
inputs:
enginesHash:
description: Engine commit hash to checkout for publishing
required: true
prismaVersion:
description: Prisma version to use for publishing
required: true
npmDistTag:
description: npm dist-tag to use for publishing
required: true
default: 'latest'
dryRun:
description: 'Check to do a dry run (does not publish packages)'
type: boolean

jobs:
build:
name: Build and publish Prisma Driver Adapters
runs-on: ubuntu-latest
steps:
- name: Print input
env:
THE_INPUT: '${{ toJson(github.event.inputs) }}'
run: |
echo $THE_INPUT
- uses: actions/checkout@v4
with:
ref: ${{ github.event.inputs.enginesHash }}

- uses: pnpm/[email protected]
with:
version: 8

- uses: actions/setup-node@v3
with:
node-version: '20.x'
registry-url: 'https://registry.npmjs.org/'

- name: Install dependencies
run: pnpm i
working-directory: query-engine/driver-adapters/js

- name: Build
run: pnpm -r build
working-directory: query-engine/driver-adapters/js

- name: Update version in package.json
run: |
# find all files package.json, and for each use jq to write the version, then write to temp file and overwrite original file with result
find . -name "package.json" -exec bash -c 'jq --arg version "${{ github.event.inputs.prismaVersion }}" ".version = \$version" "{}" > tmpfile && mv tmpfile "{}"' \;
working-directory: query-engine/driver-adapters/js

- name: Publish Prisma Driver Adapters packages
run: |
pnpm -r publish --no-git-checks --tag ${{ github.event.inputs.npmDistTag }} ${{ env.DRY_RUN }}
working-directory: query-engine/driver-adapters/js
env:
NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}
DRY_RUN: ${{ github.event.inputs.dryRun == 'true' && '--dry-run' || '' }}

#
# Failure handlers
#

- name: Set current job url in SLACK_FOOTER env var
if: ${{ failure() }}
run: echo "SLACK_FOOTER=<$GITHUB_SERVER_URL/$GITHUB_REPOSITORY/actions/runs/$GITHUB_RUN_ID|Click here to go to the job logs>" >> $GITHUB_ENV
- name: Slack Notification on Failure
if: ${{ failure() }}
uses: rtCamp/[email protected]
env:
SLACK_TITLE: 'prisma driver adapters publishing failed :x:'
SLACK_COLOR: '#FF0000'
SLACK_WEBHOOK: ${{ secrets.SLACK_WEBHOOK_DRIVER_ADPATERS_FAILING }}
11 changes: 0 additions & 11 deletions .github/workflows/query-engine-driver-adapters.yml
Original file line number Diff line number Diff line change
Expand Up @@ -49,8 +49,6 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
ref: ${{ github.event.pull_request.head.sha }}

- name: 'Setup Node.js'
uses: actions/setup-node@v3
Expand Down Expand Up @@ -78,15 +76,6 @@ jobs:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}

- name: Extract Branch Name
id: extract-branch
run: |
branch="$(git show -s --format=%s | grep -o "DRIVER_ADAPTERS_BRANCH=[^ ]*" | cut -f2 -d=)"
if [ -n "$branch" ]; then
echo "Using $branch branch of driver adapters"
echo "DRIVER_ADAPTERS_BRANCH=$branch" >> "$GITHUB_ENV"
fi
- run: make ${{ matrix.adapter.setup_task }}

- uses: dtolnay/rust-toolchain@stable
Expand Down
3 changes: 0 additions & 3 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,3 @@ dmmf.json
graph.dot

prisma-schema-wasm/nodejs

# This symlink looks orphan here, but it comes from prisma/prisma where driver adapters reference a file in their parent directory
tsconfig.build.adapter.json
55 changes: 2 additions & 53 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,6 @@ CONFIG_PATH = ./query-engine/connector-test-kit-rs/test-configs
CONFIG_FILE = .test_config
SCHEMA_EXAMPLES_PATH = ./query-engine/example_schemas
DEV_SCHEMA_FILE = dev_datamodel.prisma
DRIVER_ADAPTERS_BRANCH ?= main

LIBRARY_EXT := $(shell \
case "$$(uname -s)" in \
Expand Down Expand Up @@ -45,13 +44,7 @@ release:
#################

test-qe:
ifndef DRIVER_ADAPTER
cargo test --package query-engine-tests
else
@echo "Executing query engine tests with $(DRIVER_ADAPTER) driver adapter"; \
# Add your actual command for the "test-driver-adapter" task here
$(MAKE) test-driver-adapter-$(DRIVER_ADAPTER);
endif

test-qe-verbose:
cargo test --package query-engine-tests -- --nocapture
Expand Down Expand Up @@ -87,10 +80,6 @@ dev-sqlite:
dev-libsql-sqlite: build-qe-napi build-connector-kit-js
cp $(CONFIG_PATH)/libsql-sqlite $(CONFIG_FILE)

test-libsql-sqlite: dev-libsql-sqlite test-qe-st

test-driver-adapter-libsql: test-libsql-sqlite

start-postgres9:
docker compose -f docker-compose.yml up --wait -d --remove-orphans postgres9

Expand Down Expand Up @@ -126,20 +115,12 @@ start-pg-postgres13: build-qe-napi build-connector-kit-js start-postgres13
dev-pg-postgres13: start-pg-postgres13
cp $(CONFIG_PATH)/pg-postgres13 $(CONFIG_FILE)

test-pg-postgres13: dev-pg-postgres13 test-qe-st

test-driver-adapter-pg: test-pg-postgres13

start-neon-postgres13: build-qe-napi build-connector-kit-js
docker compose -f docker-compose.yml up --wait -d --remove-orphans neon-postgres13

dev-neon-ws-postgres13: start-neon-postgres13
cp $(CONFIG_PATH)/neon-ws-postgres13 $(CONFIG_FILE)

test-neon-ws-postgres13: dev-neon-ws-postgres13 test-qe-st

test-driver-adapter-neon: test-neon-ws-postgres13

start-postgres14:
docker compose -f docker-compose.yml up --wait -d --remove-orphans postgres14

Expand Down Expand Up @@ -274,47 +255,15 @@ start-planetscale-vitess8: build-qe-napi build-connector-kit-js
dev-planetscale-vitess8: start-planetscale-vitess8
cp $(CONFIG_PATH)/planetscale-vitess8 $(CONFIG_FILE)

test-planetscale-vitess8: dev-planetscale-vitess8 test-qe-st

test-driver-adapter-planetscale: test-planetscale-vitess8

######################
# Local dev commands #
######################

build-qe-napi:
cargo build --package query-engine-node-api

build-connector-kit-js: build-driver-adapters symlink-driver-adapters
cd query-engine/driver-adapters/connector-test-kit-executor && pnpm i && pnpm build

build-driver-adapters: ensure-prisma-present
@echo "Building driver adapters..."
@cd ../prisma && pnpm --filter "*adapter*" i && pnpm --filter "*adapter*" build
@echo "Driver adapters build completed.";

symlink-driver-adapters: ensure-prisma-present
@echo "Creating symbolic links for driver adapters..."
@for dir in $(wildcard $(realpath ../prisma)/packages/*adapter*); do \
if [ -d "$$dir" ]; then \
dir_name=$$(basename "$$dir"); \
ln -sfn "$$dir" "$(realpath .)/query-engine/driver-adapters/$$dir_name"; \
echo "Created symbolic link for $$dir_name"; \
fi; \
done;
echo "Symbolic links creation completed.";

ensure-prisma-present:
@if [ -d ../prisma ]; then \
cd "$(realpath ../prisma)" && git fetch origin main; \
LOCAL_CHANGES=$$(git diff --name-only HEAD origin/main -- 'packages/*adapter*'); \
if [ -n "$$LOCAL_CHANGES" ]; then \
echo "⚠️ ../prisma diverges from prisma/prisma main branch. Test results might diverge from those in CI ⚠️ "; \
fi \
else \
echo "git clone --depth=1 https://github.com/prisma/prisma.git --branch=$(DRIVER_ADAPTERS_BRANCH) ../prisma"; \
git clone --depth=1 https://github.com/prisma/prisma.git --branch=$(DRIVER_ADAPTERS_BRANCH) "../prisma" && echo "Prisma repository has been cloned to ../prisma"; \
fi;
build-connector-kit-js:
cd query-engine/driver-adapters/js && pnpm i && pnpm build

# Quick schema validation of whatever you have in the dev_datamodel.prisma file.
validate:
Expand Down
26 changes: 0 additions & 26 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -203,7 +203,6 @@ integration tests.
- Alternatively: Load the defined environment in `./.envrc` manually in your shell.

**Setup:**

There are helper `make` commands to set up a test environment for a specific
database connector you want to test. The commands set up a container (if needed)
and write the `.test_config` file, which is picked up by the integration
Expand Down Expand Up @@ -235,31 +234,6 @@ Other variables may or may not be useful.

Run `cargo test` in the repository root.

### Testing driver adapters

Please refer to the [Testing driver adapters](./query-engine/connector-test-kit-rs/README.md#testing-driver-adapters) section in the connector-test-kit-rs README.

**ℹ️ Important note on developing features that require changes to the both the query engine, and driver adapters code**

As explained in [Testing driver adapters](./query-engine/connector-test-kit-rs/README.md#testing-driver-adapters), running `DRIVER_ADAPTER=$adapter make qe-test`
will ensure you have prisma checked out in your filesystem in the same directory as prisma-engines. This is needed because the driver adapters code is symlinked in prisma-engines.

When working on a feature or bugfix spanning adapters code and query-engine code, you will need to open sibling PRs in `prisma/prisma` and `prisma/prisma-engines` respectively.
Locally, each time you run `DRIVER_ADAPTER=$adapter make qe-test` tests will run using the driver adapters built from the source code in the working copy of prisma/prisma. All good.

In CI, tho', we need to denote which branch of prisma/prisma we want to use for tests. In CI, there's no working copy of prisma/prisma before tests run.
The CI jobs clones prisma/prisma `main` branch by default, which doesn't include your local changes. To test in integration, we can tell CI to use the branch of prisma/prisma containing
the changes in adapters. To do it, you can use a simple convention in commit messages. Like this:

```
git commit -m "DRIVER_ADAPTERS_BRANCH=prisma-branch-with-changes-in-adapters [...]"
```

GitHub actions will then pick up the branch name and use it to clone that branch's code of prisma/prisma, and build the driver adapters code from there.

When it's time to merge the sibling PRs, you'll need to merge the prisma/prisma PR first, so when merging the engines PR you have the code of the adapters ready in prisma/prisma `main` branch.


## Parallel rust-analyzer builds

When rust-analzyer runs `cargo check` it will lock the build directory and stop any cargo commands from running until it has completed. This makes the build process feel a lot longer. It is possible to avoid this by setting a different build path for
Expand Down
Loading

0 comments on commit 7d2dac2

Please sign in to comment.