Skip to content

Commit

Permalink
Doc: add clatification for cross-region PL
Browse files Browse the repository at this point in the history
  • Loading branch information
bobbyiliev committed Jan 13, 2025
1 parent 5d66a0c commit 9df7a53
Show file tree
Hide file tree
Showing 4 changed files with 57 additions and 17 deletions.
2 changes: 1 addition & 1 deletion doc/user/content/ingest-data/postgres/amazon-aurora.md
Original file line number Diff line number Diff line change
Expand Up @@ -331,7 +331,7 @@ start by selecting the relevant option.
```mzsql
CREATE CONNECTION privatelink_svc TO AWS PRIVATELINK (
SERVICE NAME 'com.amazonaws.vpce.us-east-1.vpce-svc-0356210a8a432d9e9',
AVAILABILITY ZONES ('use1-az1', 'use1-az2', 'use1-az3')
AVAILABILITY ZONES ('use1-az1', 'use1-az2', 'use1-az4')
);
```
Expand Down
2 changes: 1 addition & 1 deletion doc/user/content/ingest-data/postgres/amazon-rds.md
Original file line number Diff line number Diff line change
Expand Up @@ -381,7 +381,7 @@ start by selecting the relevant option.
```mzsql
CREATE CONNECTION privatelink_svc TO AWS PRIVATELINK (
SERVICE NAME 'com.amazonaws.vpce.us-east-1.vpce-svc-0356210a8a432d9e9',
AVAILABILITY ZONES ('use1-az1', 'use1-az2', 'use1-az3')
AVAILABILITY ZONES ('use1-az1', 'use1-az2', 'use1-az4')
);
```
Expand Down
36 changes: 28 additions & 8 deletions doc/user/layouts/shortcodes/network-security/privatelink-kafka.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,14 +52,34 @@ and retrieve the AWS principal needed to configure the AWS PrivateLink service.
1. #### Create an AWS PrivateLink connection
In Materialize, create an [AWS PrivateLink connection](/sql/create-connection/#aws-privatelink) that references the endpoint service that you created in the previous step.

```mzsql
CREATE CONNECTION privatelink_svc TO AWS PRIVATELINK (
SERVICE NAME 'com.amazonaws.vpce.<region_id>.vpce-svc-<endpoint_service_id>',
AVAILABILITY ZONES ('use1-az1', 'use1-az2', 'use1-az3')
);
```
**Availability Zone Selection**:

* For in-region connections, the AZs **must match** between the NLB and the consumer VPC.
* For cross-region connections, AZ alignment is not required. You can choose any AZs in your consumer VPC.

For cross-region connections, avoid using `use1-az3` in `us-east-1` due to known resource constraints. For details, refer to [this article](https://wolfman.dev/posts/exclude-use1-az3/).

* In region connection example:

```mzsql
CREATE CONNECTION privatelink_svc TO AWS PRIVATELINK (
SERVICE NAME 'com.amazonaws.vpce.<region_id>.vpce-svc-<endpoint_service_id>',
AVAILABILITY ZONES ('use1-az1', 'use1-az2', 'use1-az4')
);
```
Update the list of the availability zones to match the ones in your AWS account.
* Cross-region connection example:
```mzsql
CREATE CONNECTION privatelink_svc TO AWS PRIVATELINK (
SERVICE NAME 'com.amazonaws.vpce.us-west-2.vpce-svc-<endpoint_service_id>',
AVAILABILITY ZONES ('use1-az1', 'use1-az2')
);
```
Update the list of the availability zones to match the ones in your AWS account.
Note that the service name region refers to where the endpoint service was created, while the availability zones correspond to your Materialize region. This setup allows you to connect to the endpoint service from a different region than the one where your Materialize instance is running.
## Configure the AWS PrivateLink service
Expand Down Expand Up @@ -105,7 +125,7 @@ CREATE CONNECTION kafka_connection TO KAFKA (
BROKERS (
'b-1.hostname-1:9096' USING AWS PRIVATELINK privatelink_svc (PORT 9001, AVAILABILITY ZONE 'use1-az2'),
'b-2.hostname-2:9096' USING AWS PRIVATELINK privatelink_svc (PORT 9002, AVAILABILITY ZONE 'use1-az1'),
'b-3.hostname-3:9096' USING AWS PRIVATELINK privatelink_svc (PORT 9003, AVAILABILITY ZONE 'use1-az3')
'b-3.hostname-3:9096' USING AWS PRIVATELINK privatelink_svc (PORT 9003, AVAILABILITY ZONE 'use1-az4')
),
-- Authentication details
-- Depending on the authentication method the Kafka cluster is using
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -39,14 +39,34 @@
1. #### Create an AWS PrivateLink Connection
In Materialize, create an [AWS PrivateLink connection](/sql/create-connection/#aws-privatelink) that references the endpoint service that you created in the previous step.

```mzsql
CREATE CONNECTION privatelink_svc TO AWS PRIVATELINK (
SERVICE NAME 'com.amazonaws.vpce.<region_id>.vpce-svc-<endpoint_service_id>',
AVAILABILITY ZONES ('use1-az1', 'use1-az2', 'use1-az3')
);
```
**Availability Zone Selection**:

* For in-region connections, the AZs **must match** between the NLB and the connection definition in Materialize.
* For cross-region connections, AZ alignment is not required. You can choose any AZs from the region where your Materialize instance is running.

For cross-region connections, avoid using `use1-az3` in `us-east-1` due to known resource constraints. For details, refer to [this article](https://wolfman.dev/posts/exclude-use1-az3/).

* **In region connection example**:

```mzsql
CREATE CONNECTION privatelink_svc TO AWS PRIVATELINK (
SERVICE NAME 'com.amazonaws.vpce.<region_id>.vpce-svc-<endpoint_service_id>',
AVAILABILITY ZONES ('use1-az1', 'use1-az2', 'use1-az4')
);
```
Update the list of the availability zones to match the ones in your AWS account.
* **Cross-region connection example**:
```mzsql
CREATE CONNECTION privatelink_svc TO AWS PRIVATELINK (
SERVICE NAME 'com.amazonaws.vpce.us-west-2.vpce-svc-<endpoint_service_id>',
AVAILABILITY ZONES ('use1-az1', 'use1-az2')
);
```
Update the list of the availability zones to match the ones that you are using in your AWS account.
Note that the service name region refers to where the endpoint service was created, while the availability zones correspond to your Materialize region. This setup allows you to connect to the endpoint service from a different region than the one where your Materialize instance is running.
## Configure the AWS PrivateLink service
Expand Down

0 comments on commit 9df7a53

Please sign in to comment.