Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add support for sts regional interface endpoints #227

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

nimble00
Copy link

@nimble00 nimble00 commented Jun 29, 2022

Issue #, Not available

Issue Description

  1. Current code does NOT work when it is hosted in a VPC without PublicSubnets (has i-gw) or PrivateSubnets (has nat-gw). It is unable to call AWS STS even when the VPC has InterfaceEndpoints to access AWS services like ECR, STS, S3, DynamoDB.
  2. InterfaceEndpoints allow access ONLY to regional endpoints like - sts.eu-west-1.amazonaws.com etc.
  3. Root cause: the boto3 STS client has a peculiar behavior - even when region_name parameter is specified, it hits the global endpoint - sts.amazon.com instead of regional endpoints. There are two reasons behind this behavior -
    a. Some of the regions don't have regional STS endpoints itself
    b. I read somewhere (can't verify correctness!) that it does that for historical reasons - some critical infra was breaking when they tried to move it to regional endpoints.
  4. Solution looks easy - add a PrivateSubnet (NAT-GW)! But what if we don't want that due to security reasons and multitude of other reasons?

Changes / Solution

  1. This issue can be solved by specifying endpoint_url parameter while creating the sts client using boto3. This change has been implemented in this CR.

Testing

  1. The changes are successfully running in Prod in an Enterprise Service. Tested by copying the relevant class (Endpoints) from this codebase to my service package, and implementing the same changes there.
  2. Build is Successful locally -
nimbalr@88665a11506c neptune-python-utils % ./build.sh
...
...
...
  adding: yarl/py.typed (stored 0%)
  adding: yarl/_quoting_c.pyi (deflated 61%)
+ mv neptune_python_utils.zip ../../../../target/neptune_python_utils.zip
+ deactivate
+ unset -f pydoc
+ '[' -z _ ']'
+ PATH=/usr/local/opt/node@14/bin:/Users/nimbalr/.toolbox/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin
+ export PATH
+ unset _OLD_VIRTUAL_PATH
+ '[' -z '' ']'
+ hash -r
+ '[' -z _ ']'
+ PS1=
+ export PS1
+ unset _OLD_VIRTUAL_PS1
+ unset VIRTUAL_ENV
+ '[' '!' '' = nondestructive ']'
+ unset -f deactivate
+ popd
~/Documents/codes/amazon-neptune-tools/neptune-python-utils
+ rm -rf temp
nimbalr@88665a11506c neptune-python-utils %

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@triggan triggan self-requested a review June 29, 2022 14:05
@nimble00 nimble00 force-pushed the master branch 3 times, most recently from fff90ad to 472f7ef Compare June 30, 2022 08:06
@nimble00 nimble00 requested a review from triggan July 4, 2022 07:21
Copy link
Contributor

@abhishekpradeepmishra abhishekpradeepmishra left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

kindly make changes as requested

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants