From 3bf763aac09081a6c977cf0539e37ae27fe19918 Mon Sep 17 00:00:00 2001 From: Jan Piotrowski Date: Mon, 6 Nov 2023 14:24:01 +0100 Subject: [PATCH 1/8] feat: PgBouncer => external connection pooler --- ...figure-for-external-connection-pooler.mdx} | 32 +++++++++++-------- 1 file changed, 19 insertions(+), 13 deletions(-) rename content/300-guides/100-performance-and-optimization/150-connection-management/{200-configure-pg-bouncer.mdx => 200-configure-for-external-connection-pooler.mdx} (78%) diff --git a/content/300-guides/100-performance-and-optimization/150-connection-management/200-configure-pg-bouncer.mdx b/content/300-guides/100-performance-and-optimization/150-connection-management/200-configure-for-external-connection-pooler.mdx similarity index 78% rename from content/300-guides/100-performance-and-optimization/150-connection-management/200-configure-pg-bouncer.mdx rename to content/300-guides/100-performance-and-optimization/150-connection-management/200-configure-for-external-connection-pooler.mdx index f30fbb9131..63d985ef4b 100644 --- a/content/300-guides/100-performance-and-optimization/150-connection-management/200-configure-pg-bouncer.mdx +++ b/content/300-guides/100-performance-and-optimization/150-connection-management/200-configure-for-external-connection-pooler.mdx @@ -1,24 +1,20 @@ --- -title: Configure Prisma Client with PgBouncer +title: Configure Prisma Client with an external connection pooler (like PgBouncer) --- -PgBouncer holds a connection pool to the database and proxies incoming client connections by sitting between Prisma Client and the database. This reduces the number of processes a database has to handle at any given time. PgBouncer passes on a limited number of connections to the database and queues additional connections for delivery when space becomes available. +An external connection pooler like PgBouncer holds a connection pool to the database, and proxies incoming client connections by sitting between Prisma Client and the database. This reduces the number of processes a database has to handle at any given time. + +Usually, this works transparently, but some connection poolers only support a limited set of functionality. One common feature that external connection poolers do not support are named prepared statements, which Prisma uses. For these cases, Prisma can be configured to behave differently. -## Add pgbouncer to the connection URL +## PgBouncer -To use Prisma Client with PgBouncer from a serverless function, add the `?pgbouncer=true` flag to the PostgreSQL connection URL: +TODO Note about this only applying to PgBouncer <1.21.0 -``` -postgresql://USER:PASSWORD@HOST:PORT/DATABASE?pgbouncer=true -``` - -> Note: `PORT` specified for PgBouncer pooling sometimes different from the default `5432` port. Check database provider docs for the exact port number, otherwise adding `?pgbouncer=true` won't work. - -## Set PgBouncer to transaction mode +### Set PgBouncer to transaction mode Additionally, for Prisma Client to work reliably, PgBouncer must run in [**Transaction mode**](https://www.pgbouncer.org/features.html). @@ -32,7 +28,17 @@ _Transaction mode_ offers a connection for every transaction – a requirement f -## Prisma Migrate and PgBouncer workaround +### Add pgbouncer=true to the connection URL + +To use Prisma Client with PgBouncer from a serverless function, add the `?pgbouncer=true` flag to the PostgreSQL connection URL: + +``` +postgresql://USER:PASSWORD@HOST:PORT/DATABASE?pgbouncer=true +``` + +> Note: `PORT` specified for PgBouncer pooling sometimes different from the default `5432` port. Check database provider docs for the exact port number, otherwise adding `?pgbouncer=true` won't work. + +### Prisma Migrate and PgBouncer workaround Prisma Migrate uses **database transactions** to check out the current state of the database and the migrations table. However, the Schema Engine is designed to use a **single connection to the database**, and does not support connection pooling with PgBouncer. If you attempt to run Prisma Migrate commands in any environment that uses PgBouncer for connection pooling, you might see the following error: @@ -57,7 +63,7 @@ The block above uses a PgBouncer connection string as the primary URL using `url It also provides a connection string directly to the database, without PgBouncer, using the `directUrl` field. This connection string will be used when [commands that require a single connection](/data-platform/classic-projects/data-proxy/prisma-cli-with-data-proxy#prisma-cli-commands-that-require-a-direct-database-connection) to the database, such as `prisma migrate dev` or `prisma db push`, are invoked. -## PgBouncer with different database providers +### PgBouncer with different database providers There are sometimes minor differences in how to connect directly to a Postgres database that depend on the provider hosting the database. From 63f984e97aba8683a0e527231a2ed663b703482d Mon Sep 17 00:00:00 2001 From: Jan Piotrowski Date: Mon, 6 Nov 2023 14:30:17 +0100 Subject: [PATCH 2/8] add redirect to vercel.json --- vercel.json | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/vercel.json b/vercel.json index ef1da8dceb..3930998740 100644 --- a/vercel.json +++ b/vercel.json @@ -1714,6 +1714,10 @@ { "source": "/docs/data-platform/classic-projects/contact-support", "destination": "/docs/data-platform/platform-console/support" + }, + { + "source": "/docs/guides/performance-and-optimization/connection-management/configure-pg-bouncer", + "destination": "/docs/guides/performance-and-optimization/connection-management/configure-for-external-connection-pooler" } ] } From 904fb12ad4c1d9855f7df079c9059b3966364c37 Mon Sep 17 00:00:00 2001 From: Jan Piotrowski Date: Mon, 6 Nov 2023 14:35:05 +0100 Subject: [PATCH 3/8] fix some stuff --- ...nfigure-for-external-connection-pooler.mdx | 28 +++++++++---------- 1 file changed, 14 insertions(+), 14 deletions(-) diff --git a/content/300-guides/100-performance-and-optimization/150-connection-management/200-configure-for-external-connection-pooler.mdx b/content/300-guides/100-performance-and-optimization/150-connection-management/200-configure-for-external-connection-pooler.mdx index 63d985ef4b..de89b01c01 100644 --- a/content/300-guides/100-performance-and-optimization/150-connection-management/200-configure-for-external-connection-pooler.mdx +++ b/content/300-guides/100-performance-and-optimization/150-connection-management/200-configure-for-external-connection-pooler.mdx @@ -12,31 +12,31 @@ Usually, this works transparently, but some connection poolers only support a li ## PgBouncer -TODO Note about this only applying to PgBouncer <1.21.0 - ### Set PgBouncer to transaction mode -Additionally, for Prisma Client to work reliably, PgBouncer must run in [**Transaction mode**](https://www.pgbouncer.org/features.html). - -_Transaction mode_ offers a connection for every transaction – a requirement for the Prisma query engine to work with PgBouncer. - -
- How `pgbouncer` mode works in Prisma - -- Prisma cleans up already present prepared statements in the connection by running `DEALLOCATE ALL` before preparing and executing Prisma Client queries. -- Prisma opens a transaction for every query case – even when just reading data, allowing Prisma to use prepared statements. +For Prisma Client to work reliably, PgBouncer must run in [**Transaction mode**](https://www.pgbouncer.org/features.html). -
+_Transaction mode_ offers a connection for every transaction – a requirement for the Prisma Client to work with PgBouncer. ### Add pgbouncer=true to the connection URL -To use Prisma Client with PgBouncer from a serverless function, add the `?pgbouncer=true` flag to the PostgreSQL connection URL: +To use Prisma Client with PgBouncer, add the `?pgbouncer=true` flag to the PostgreSQL connection URL: ``` postgresql://USER:PASSWORD@HOST:PORT/DATABASE?pgbouncer=true ``` -> Note: `PORT` specified for PgBouncer pooling sometimes different from the default `5432` port. Check database provider docs for the exact port number, otherwise adding `?pgbouncer=true` won't work. +> Note: `PORT` specified for PgBouncer pooling is sometimes different from the default `5432` port. Check your database provider docs for the correct port number. + +
+ How `pgbouncer` mode works in Prisma + +- Prisma opens a transaction for every query – even when just reading data, allowing Prisma to use prepared statements. +- Prisma does not try to set the `search_path`, which is not supported by PgBouncer. +- Prisma cleans up already present prepared statements in the connection by running `DEALLOCATE ALL` before preparing and executing Prisma Client queries. +- Prisma also disables any prepared statement or type query caches. + +
### Prisma Migrate and PgBouncer workaround From 2608fe40c55e77310ead652cc47ba807ea12eb88 Mon Sep 17 00:00:00 2001 From: Jan Piotrowski Date: Mon, 6 Nov 2023 14:41:45 +0100 Subject: [PATCH 4/8] Update 200-troubleshooting-development.mdx remove outdated link --- .../200-troubleshooting-development.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/300-guides/025-migrate/100-developing-with-prisma-migrate/200-troubleshooting-development.mdx b/content/300-guides/025-migrate/100-developing-with-prisma-migrate/200-troubleshooting-development.mdx index 60d1e8b94e..8d4d1202b2 100644 --- a/content/300-guides/025-migrate/100-developing-with-prisma-migrate/200-troubleshooting-development.mdx +++ b/content/300-guides/025-migrate/100-developing-with-prisma-migrate/200-troubleshooting-development.mdx @@ -134,4 +134,4 @@ Error: undefined: Database error Error querying the database: db error: ERROR: prepared statement "s0" already exists ``` -See [Prisma Migrate and PgBouncer workaround](/guides/performance-and-optimization/connection-management/configure-pg-bouncer) for further information and a workaround. Follow [GitHub issue #6485](https://github.com/prisma/prisma/issues/6485) for updates. +See [Prisma Migrate and PgBouncer workaround](/guides/performance-and-optimization/connection-management/configure-pg-bouncer) for further information and a workaround. From 9c2be214ba6d37deba9ee9df0de70a685d926a09 Mon Sep 17 00:00:00 2001 From: Jan Piotrowski Date: Mon, 6 Nov 2023 19:15:22 +0100 Subject: [PATCH 5/8] explain supavisor and others --- .../200-configure-for-external-connection-pooler.mdx | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/content/300-guides/100-performance-and-optimization/150-connection-management/200-configure-for-external-connection-pooler.mdx b/content/300-guides/100-performance-and-optimization/150-connection-management/200-configure-for-external-connection-pooler.mdx index de89b01c01..8eba8c748a 100644 --- a/content/300-guides/100-performance-and-optimization/150-connection-management/200-configure-for-external-connection-pooler.mdx +++ b/content/300-guides/100-performance-and-optimization/150-connection-management/200-configure-for-external-connection-pooler.mdx @@ -71,3 +71,11 @@ Below are links to information on how to set up these connections with providers - [Connecting directly to a PostgreSQL database hosted on Digital Ocean](https://github.com/prisma/prisma/issues/6157) - [Connecting directly to a PostgreSQL database hosted on ScaleGrid](https://github.com/prisma/prisma/issues/6701#issuecomment-824387959) + +## Supabase Supavisor + +Supabase's Supavisor behaves similar to [PgBouncer](#pgbouncer). You can add `pgbouncer=true` to your connection pooled connection string. + +## Other external connection poolers + +Although Prisma does not have explicit support for other connection poolers, if the limitations are similar to the ones of [PgBouncer](#pgbouncer) you can usually also use `pgbouncer=true` in your connection string to put Prisma in a mode that works with them as well. From dc52b27fde850806aa4aa2ebbd624dc6eb10b8dd Mon Sep 17 00:00:00 2001 From: Alex Ruheni <33921841+ruheni@users.noreply.github.com> Date: Thu, 16 Nov 2023 13:43:19 +0000 Subject: [PATCH 6/8] remove trailing comma --- vercel.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/vercel.json b/vercel.json index 4ec825c3d0..8bbfac1511 100644 --- a/vercel.json +++ b/vercel.json @@ -1726,6 +1726,6 @@ { "source": "/docs/data-platform/accelerate/testing", "destination": "/docs/data-platform/accelerate/evaluating" - }, + } ] } From 891f5161b7dcc25fbb031677eec95c1abb3e928f Mon Sep 17 00:00:00 2001 From: Jan Piotrowski Date: Thu, 16 Nov 2023 20:45:45 +0100 Subject: [PATCH 7/8] Update content/300-guides/100-performance-and-optimization/150-connection-management/200-configure-for-external-connection-pooler.mdx Co-authored-by: Alex Ruheni <33921841+ruheni@users.noreply.github.com> --- .../200-configure-for-external-connection-pooler.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/300-guides/100-performance-and-optimization/150-connection-management/200-configure-for-external-connection-pooler.mdx b/content/300-guides/100-performance-and-optimization/150-connection-management/200-configure-for-external-connection-pooler.mdx index 8eba8c748a..2d818122cf 100644 --- a/content/300-guides/100-performance-and-optimization/150-connection-management/200-configure-for-external-connection-pooler.mdx +++ b/content/300-guides/100-performance-and-optimization/150-connection-management/200-configure-for-external-connection-pooler.mdx @@ -16,7 +16,7 @@ Usually, this works transparently, but some connection poolers only support a li For Prisma Client to work reliably, PgBouncer must run in [**Transaction mode**](https://www.pgbouncer.org/features.html). -_Transaction mode_ offers a connection for every transaction – a requirement for the Prisma Client to work with PgBouncer. +Transaction mode offers a connection for every transaction – a requirement for the Prisma Client to work with PgBouncer. ### Add pgbouncer=true to the connection URL From 5a18782c2f318ea032491e22bbcfd3673a7d76f5 Mon Sep 17 00:00:00 2001 From: Jan Piotrowski Date: Thu, 16 Nov 2023 20:48:15 +0100 Subject: [PATCH 8/8] Update cSpell.json --- cSpell.json | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/cSpell.json b/cSpell.json index 13b402dc94..5210c20fe2 100644 --- a/cSpell.json +++ b/cSpell.json @@ -72,7 +72,8 @@ "Vitess", "libgcc", "libc", - "Distroless" + "Distroless", + "Supavisor" ], "patterns": [ {