Migrate driver adapters to prisma/prisma

[miguelff]

Sibling to prisma/prisma#21601 (merge that PR first, then this engines' one)

This PR:

• Assumes driver adapters code reside in prisma/prisma
• Defines convenient make tasks for testing the query engine against driver adapters, which does the following:
  • Ensure you've got a working copy of prisma/prisma in the filesystem at the same level than prisma-engines
  • Symlink driver adapter packages within query-engine/driver-adapters
  • Build the driver adapters and the connector kit to run queries through the driver adapters
  • Run tests using the connector kit for running queries
  • All this complexity is hidden behind make tasks. To run query-engine test through driver adapters:
    • DRIVER_ADAPTER=$adapter make test-qe where $adapter is one of pg, neon, libsql or planetscale

To ease working on changes to both driver adapters and rust code (if a feature or bugfix spans both codebases), we jus work normally locally, being prisma/prisma checked out in the same directory of prisma/prisma-engines. When running the make task above, the symlinks will be updated and hence the new code of driver adapters will be used.

And then to test in CI, we denote which branch of prisma/prisma do we want to use for tests, by applying a convention on the commit message. As such, if we are working on branch fix-planetscale-adapter in prisma/prisma and want to query engine tests to use the driver adapters code in that branch, we can include the DRIVER_ADAPTERS_BRANCH=fix-planetscale-adapter in the message of the commit we want to test. By default, driver adapters tests will use the main version of prisma/prisma driver adapters. a merge commit on prisma-engines will use main in prisma/prisma, so the order of merging PRs that span both codebases is:

• First merge changes in prisma/prisma
• Then merge changes in prisma-engines

[codspeed-hq]

CodSpeed Performance Report

Merging #4380 will not alter performance

_{Comparing driver-adapters-migration (7eb2831) with main (3305ecc)}

Summary

✅ 11 untouched benchmarks

[miguelff]

Commented on the more relevant changes in this PR

[miguelff]

This allows us to test the query engine code against a particular prisma/prisma branch in CI. So we are not depending on merge ordering of prisma/prisma changes.

Example:

• Feature requires changes in the engine and driver-adapters
• changes in driver adapters are in branch foo of prisma/prisma
• If we want to test a particular commit in the engines against those changes we just need to push a commit in here containing DRIVER_ADAPTERS_BRANCH=foo.

[miguelff]

convention over configuration to run driver adapters tests easily. See README.md changes.

[miguelff]

2. We ensure prisma/prisma is our FS, and build driver adapters from source.

[miguelff]

3. Then we symlink driver adapters code

[miguelff]

1. If locally we have a clone of prisma, we are done, if that clone diverges from main we will warn the user. If we don´t have a clone (f.i. in CI) we shallowly clone it -- takes less than 10 seconds

[miguelff]

ignoring symlinks from prisma/prisma

[miguelff]

Depending on symlinked driver adapters

[Jolg42]

(Still reviewing)

@miguelff can you remove this workflow? https://github.com/prisma/prisma-engines/blob/main/.github/workflows/publish-driver-adapters.yml

[jkomyno]

Tested and reviewed together with @miguelff, with a few minor improvements approved together + @Jolg42's suggestion to remove a deprecated workflow file.

[janpio]

Did a full review, generally looks good but 2 remarks:

1. Stating the obvious that tests are failing with:

git clone --depth=1 https://github.com/prisma/prisma.git --branch= ../prisma
Cloning into '../prisma'...
warning: Could not find remote branch  to clone.
fatal: Remote branch  not found in upstream origin
make: *** [Makefile:308: ensure-prisma-present] Error 128
Error: Process completed with exit code 2.

2. Per README this is supposed to check out main of prisma/prisma. But does that not mean that prisma-engines tests are not not deterministic any more, as a change to prisma/prisma can make the same commit here pass or fail?

[miguelff]

1. Stating the obvious that tests are failing with

Fixed

2. Per README this is supposed to check out main of prisma/prisma. But does that not mean that prisma-engines tests are not not deterministic any more, as a change to prisma/prisma can make the same commit here pass or fail?

That is true, but I don't consider very negative. Let me explain why:

• While pushing commits to an engines PR, we would understand if something fails, as soon as something buggy was merged in prisma, instead of waiting for any integration PR (that was also the case, but reversed from prisma -> prisma/engines) when the driver adapters where in here.

• To the best of my knowledge, the only edge case where the non-determinism can play against us, would be this sceanario:

  1. a PR in engines sitting inactive, while some changes in main where merged in prisma/prisma.
  2. then engines PR is merged
  3. merged commit is tested against newer main of prisma, which was never tested before, and fails.

  I think this can happen but should be odd enough. To happen it means:

  • that no changes: in prisma/prisma-engines main happened in the mean time, or they were ignored and merge of the engines PR was done before updating the branch first. Because otherwise, a new update commit will be added to the PR, thus exercising the new main.
  • prisma/prisma tests did not provide enough fidelity, i.e. to merge prisma/main integration tests should have passed. If that was the case, then it would not very expected to make integration tests for prisma-engines fail, If so, we would be signaled by a lack of coverage in prisma/prisma, and informed to act on it.

All in all, this is not a perfect solution, but we can try as an intermediate step towards monorepo, that would allow us to develop faster across the two codebases (more or less as fast for engines developers) and faster for prisma/prisma developers which will have test feeeback instantly after modifying any driver adapter code.