Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add AWS SSO native credential support #52

Merged

Conversation

masteinhauser
Copy link
Contributor

Issue #51

Description of changes:
Enables support for AWS SSO credentials natively, without relying on credential_process or the many other available workarounds.

This also fixes mock generation, although I cannot explain why this was needed. I could not find any viable way to generate the mocks required to pass tests without this change, and the vendor directory removal.

AWS Docs for configuration, and example:

[profile my-dev-profile]
sso_start_url = https://my-sso-portal.awsapps.com/start
sso_region = us-east-1
sso_account_id = 123456789011
sso_role_name = readOnly
region = us-west-2
output = json

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

github.com/aws/aws-sdk-go v1.37.8+ adds native support for AWS SSO credentials.
This is crucial for users who have adopted AWS SSO, and allows for user without workarounds via `credential_process`.

Example ~/.aws/config:
```
[profile sso]
region = us-east-2
sso_start_url = https://d-1234567890.awsapps.com/start/
sso_region = us-east-1
sso_account_id = 123456789012
sso_role_name = AWSPowerUserAccess
```
This is necessary for tests to pass.
@masteinhauser
Copy link
Contributor Author

make test output from local:

amazon-ecs-local-container-endpoints % make test
go test -mod=vendor -timeout=120s -v -cover ./local-container-endpoints/...
?       github.com/awslabs/amazon-ecs-local-container-endpoints/local-container-endpoints/clients/docker        [no test files]
?       github.com/awslabs/amazon-ecs-local-container-endpoints/local-container-endpoints/clients/docker/mock_docker    [no test files]
?       github.com/awslabs/amazon-ecs-local-container-endpoints/local-container-endpoints/clients/iam   [no test files]
?       github.com/awslabs/amazon-ecs-local-container-endpoints/local-container-endpoints/clients/iam/mock_iamiface     [no test files]
?       github.com/awslabs/amazon-ecs-local-container-endpoints/local-container-endpoints/clients/sts   [no test files]
?       github.com/awslabs/amazon-ecs-local-container-endpoints/local-container-endpoints/clients/sts/mock_stsiface     [no test files]
?       github.com/awslabs/amazon-ecs-local-container-endpoints/local-container-endpoints/clients/useragent     [no test files]
?       github.com/awslabs/amazon-ecs-local-container-endpoints/local-container-endpoints/config        [no test files]
=== RUN   TestGetRoleCredentials                                                                                                                                             
--- PASS: TestGetRoleCredentials (0.00s)                                                                                                                                     
=== RUN   TestGetRoleCredentialsGetRoleError        
--- PASS: TestGetRoleCredentialsGetRoleError (0.00s)        
=== RUN   TestGetRoleCredentialsSTSError                                              
--- PASS: TestGetRoleCredentialsSTSError (0.00s)
=== RUN   TestGetTemporaryCredentials                                                 
--- PASS: TestGetTemporaryCredentials (0.00s)                                                                                                                                
=== RUN   TestGetTemporaryCredentialsErrorCase                                        
--- PASS: TestGetTemporaryCredentialsErrorCase (0.00s)                                
=== RUN   TestGetTemporaryCredentialsExistingTempCreds                                
--- PASS: TestGetTemporaryCredentialsExistingTempCreds (0.00s)
=== RUN   TestFindContainerWithIdentifierID 
=== RUN   TestFindContainerWithIdentifierID/e18ab3d25b38                                                                                                                     
=== RUN   TestFindContainerWithIdentifierID/457129ed3bd03f1fc70125c3be7bcbee760d5edf092e32155a5c6a730cd32020
=== RUN   TestFindContainerWithIdentifierID/0756a2371cad1976b07954490660f07d240a6a6f52d17594ed691799915695f7
--- PASS: TestFindContainerWithIdentifierID (0.00s)
    --- PASS: TestFindContainerWithIdentifierID/e18ab3d25b38 (0.00s)                                                                                                         
    --- PASS: TestFindContainerWithIdentifierID/457129ed3bd03f1fc70125c3be7bcbee760d5edf092e32155a5c6a730cd32020 (0.00s)                             
    --- PASS: TestFindContainerWithIdentifierID/0756a2371cad1976b07954490660f07d240a6a6f52d17594ed691799915695f7 (0.00s)       
=== RUN   TestFindContainerWithIdentifierName                                                                                                                       [51/4600]
=== RUN   TestFindContainerWithIdentifierName/container1-puddles                                                                                                             
=== RUN   TestFindContainerWithIdentifierName/container2-pudding                                                                                                             
=== RUN   TestFindContainerWithIdentifierName/clyde-container3-dumpling               
--- PASS: TestFindContainerWithIdentifierName (0.00s)
    --- PASS: TestFindContainerWithIdentifierName/container1-puddles (0.00s)
    --- PASS: TestFindContainerWithIdentifierName/container2-pudding (0.00s)
    --- PASS: TestFindContainerWithIdentifierName/clyde-container3-dumpling (0.00s)
=== RUN   TestFindContainerWithCallerIP
=== RUN   TestFindContainerWithCallerIP/172.17.0.2
=== RUN   TestFindContainerWithCallerIP/172.17.0.3
=== RUN   TestFindContainerWithCallerIP/172.17.0.4
--- PASS: TestFindContainerWithCallerIP (0.00s)
    --- PASS: TestFindContainerWithCallerIP/172.17.0.2 (0.00s)
    --- PASS: TestFindContainerWithCallerIP/172.17.0.3 (0.00s)
    --- PASS: TestFindContainerWithCallerIP/172.17.0.4 (0.00s)
=== RUN   TestFindContainerWithCallerIPAndNetworks
--- PASS: TestFindContainerWithCallerIPAndNetworks (0.00s)
=== RUN   TestFindContainerWithCallerIPAndNetworksFailure
--- PASS: TestFindContainerWithCallerIPAndNetworksFailure (0.00s)
=== RUN   TestFindContainerWithAllChecks
--- PASS: TestFindContainerWithAllChecks (0.00s)
=== RUN   TestFindContainerFailureMoreThanOneMatches
--- PASS: TestFindContainerFailureMoreThanOneMatches (0.00s)
=== RUN   TestFindContainerWithIdentifierFailure
--- PASS: TestFindContainerWithIdentifierFailure (0.00s)
=== RUN   TestGetTaskContainers
--- PASS: TestGetTaskContainers (0.00s)
=== RUN   TestGetTaskContainersNilTest
time="2021-02-13T13:54:21-05:00" level=warning msg="Failed to find the container which the request came from. Narrowed down search to 2 containers"
time="2021-02-13T13:54:21-05:00" level=info msg="Will use all containers to represent one 'local task'"
--- PASS: TestGetTaskContainersNilTest (0.00s)
=== RUN   TestGetTaskContainersOneContainerReturned
--- PASS: TestGetTaskContainersOneContainerReturned (0.00s)
PASS
coverage: 31.6% of statements
ok      github.com/awslabs/amazon-ecs-local-container-endpoints/local-container-endpoints/handlers      (cached)        coverage: 31.6% of statements
=== RUN   TestGetRoleCredentials
--- PASS: TestGetRoleCredentials (0.00s)
=== RUN   TestGetTemporaryCredentials
--- PASS: TestGetTemporaryCredentials (0.00s)
=== RUN   TestV2Handler_TaskMetadata
time="2021-02-13T13:54:21-05:00" level=warning msg="Failed to find the container which the request came from. Narrowed down search to 0 containers"
time="2021-02-13T13:54:21-05:00" level=info msg="Will use all containers to represent one 'local task'"
--- PASS: TestV2Handler_TaskMetadata (0.00s)
=== RUN   TestV2Handler_TaskMetadata_TrailingSlash
time="2021-02-13T13:54:21-05:00" level=warning msg="Failed to find the container which the request came from. Narrowed down search to 0 containers"
time="2021-02-13T13:54:21-05:00" level=info msg="Will use all containers to represent one 'local task'"
--- PASS: TestV2Handler_TaskMetadata_TrailingSlash (0.00s)
=== RUN   TestV2Handler_TaskMetadata_DockerAPIError
time="2021-02-13T13:54:21-05:00" level=error msg="HTTP 500 - Some API Error"
--- PASS: TestV2Handler_TaskMetadata_DockerAPIError (0.00s)
=== RUN   TestV2Handler_ContainerMetadata
--- PASS: TestV2Handler_ContainerMetadata (0.00s)
=== RUN   TestV2Handler_TaskMetadata_InvalidURL
--- PASS: TestV2Handler_TaskMetadata_InvalidURL (0.00s)
=== RUN   TestV2Handler_ContainerMetadata_TrailingSlash
--- PASS: TestV2Handler_ContainerMetadata_TrailingSlash (0.00s)
=== RUN   TestV2Handler_ContainerStats
--- PASS: TestV2Handler_ContainerStats (0.00s)
=== RUN   TestV2Handler_ContainerStats_TrailingSlash
--- PASS: TestV2Handler_ContainerStats_TrailingSlash (0.00s)
=== RUN   TestV2Handler_TaskStats
--- PASS: TestV2Handler_TaskStats (0.00s)
=== RUN   TestV2Handler_TaskStats_TrailingSlash
--- PASS: TestV2Handler_TaskStats_TrailingSlash (0.00s)
=== RUN   TestV2Handler_TaskStats_DockerAPIError
time="2021-02-13T13:54:21-05:00" level=error msg="HTTP 500 - Some error"
--- PASS: TestV2Handler_TaskStats_DockerAPIError (0.00s)
=== RUN   TestV3Handler_TaskMetadata
--- PASS: TestV3Handler_TaskMetadata (0.00s)
=== RUN   TestV3Handler_TaskMetadata_TrailingSlash
--- PASS: TestV3Handler_TaskMetadata_TrailingSlash (0.00s)
=== RUN   TestV3Handler_TaskMetadata_DockerAPIError
time="2021-02-13T13:54:21-05:00" level=error msg="HTTP 500 - Some API Error"
--- PASS: TestV3Handler_TaskMetadata_DockerAPIError (0.00s)
=== RUN   TestV3Handler_TaskMetadata_InvalidURL
--- PASS: TestV3Handler_TaskMetadata_InvalidURL (0.00s)
=== RUN   TestV3Handler_ContainerStats
--- PASS: TestV3Handler_ContainerStats (0.00s)
=== RUN   TestV3Handler_ContainerStats_TrailingSlash
--- PASS: TestV3Handler_ContainerStats_TrailingSlash (0.00s)
=== RUN   TestV3Handler_TaskStats
--- PASS: TestV3Handler_TaskStats (0.00s)
=== RUN   TestV3Handler_TaskStats_TrailingSlash
--- PASS: TestV3Handler_TaskStats_TrailingSlash (0.00s)
=== RUN   TestV3Handler_TaskStats_DockerAPIError
time="2021-02-13T13:54:21-05:00" level=error msg="HTTP 500 - Some error"
--- PASS: TestV3Handler_TaskStats_DockerAPIError (0.00s)
PASS
coverage: 100.0% of statements
ok      github.com/awslabs/amazon-ecs-local-container-endpoints/local-container-endpoints/handlers/functional_tests     (cached)        coverage: 100.0% of statements
=== RUN   TestGetTaskMetadata
--- PASS: TestGetTaskMetadata (0.00s)
PASS
coverage: 95.3% of statements
ok      github.com/awslabs/amazon-ecs-local-container-endpoints/local-container-endpoints/metadata      (cached)        coverage: 95.3% of statements
?       github.com/awslabs/amazon-ecs-local-container-endpoints/local-container-endpoints/testingutils  [no test files]
?       github.com/awslabs/amazon-ecs-local-container-endpoints/local-container-endpoints/utils [no test files]
?       github.com/awslabs/amazon-ecs-local-container-endpoints/local-container-endpoints/version       [no test files]
?       github.com/awslabs/amazon-ecs-local-container-endpoints/local-container-endpoints/version/gen   [no test files]

@masteinhauser
Copy link
Contributor Author

make functional-test:

amazon-ecs-local-container-endpoints % make functional-test
go test -mod=vendor -timeout=120s -v -tags functional -cover ./local-container-endpoints/handlers/functional_tests/...
=== RUN   TestGetRoleCredentials                                                      
--- PASS: TestGetRoleCredentials (0.00s)                                              
=== RUN   TestGetTemporaryCredentials                                                 
--- PASS: TestGetTemporaryCredentials (0.00s)
=== RUN   TestV2Handler_TaskMetadata
time="2021-02-13T13:54:21-05:00" level=warning msg="Failed to find the container which the request came from. Narrowed down search to 0 containers"                   
time="2021-02-13T13:54:21-05:00" level=info msg="Will use all containers to represent one 'local task'"
--- PASS: TestV2Handler_TaskMetadata (0.00s)
=== RUN   TestV2Handler_TaskMetadata_TrailingSlash
time="2021-02-13T13:54:21-05:00" level=warning msg="Failed to find the container which the request came from. Narrowed down search to 0 containers"
time="2021-02-13T13:54:21-05:00" level=info msg="Will use all containers to represent one 'local task'"
--- PASS: TestV2Handler_TaskMetadata_TrailingSlash (0.00s)
=== RUN   TestV2Handler_TaskMetadata_DockerAPIError
time="2021-02-13T13:54:21-05:00" level=error msg="HTTP 500 - Some API Error"
--- PASS: TestV2Handler_TaskMetadata_DockerAPIError (0.00s)
=== RUN   TestV2Handler_ContainerMetadata
--- PASS: TestV2Handler_ContainerMetadata (0.00s)
=== RUN   TestV2Handler_TaskMetadata_InvalidURL
--- PASS: TestV2Handler_TaskMetadata_InvalidURL (0.00s)
=== RUN   TestV2Handler_ContainerMetadata_TrailingSlash
--- PASS: TestV2Handler_ContainerMetadata_TrailingSlash (0.00s)
=== RUN   TestV2Handler_ContainerStats
--- PASS: TestV2Handler_ContainerStats (0.00s)
=== RUN   TestV2Handler_ContainerStats_TrailingSlash
--- PASS: TestV2Handler_ContainerStats_TrailingSlash (0.00s)
=== RUN   TestV2Handler_TaskStats
--- PASS: TestV2Handler_TaskStats (0.00s)
=== RUN   TestV2Handler_TaskStats_TrailingSlash
--- PASS: TestV2Handler_TaskStats_TrailingSlash (0.00s)
=== RUN   TestV2Handler_TaskStats_DockerAPIError
time="2021-02-13T13:54:21-05:00" level=error msg="HTTP 500 - Some error"
--- PASS: TestV2Handler_TaskStats_DockerAPIError (0.00s)
=== RUN   TestV3Handler_TaskMetadata
--- PASS: TestV3Handler_TaskMetadata (0.00s)
=== RUN   TestV3Handler_TaskMetadata_TrailingSlash
--- PASS: TestV3Handler_TaskMetadata_TrailingSlash (0.00s)
=== RUN   TestV3Handler_TaskMetadata_DockerAPIError
time="2021-02-13T13:54:21-05:00" level=error msg="HTTP 500 - Some API Error"
--- PASS: TestV3Handler_TaskMetadata_DockerAPIError (0.00s)
=== RUN   TestV3Handler_TaskMetadata_InvalidURL
--- PASS: TestV3Handler_TaskMetadata_InvalidURL (0.00s)
=== RUN   TestV3Handler_ContainerStats
--- PASS: TestV3Handler_ContainerStats (0.00s)
=== RUN   TestV3Handler_ContainerStats_TrailingSlash
--- PASS: TestV3Handler_ContainerStats_TrailingSlash (0.00s)
=== RUN   TestV3Handler_TaskStats
--- PASS: TestV3Handler_TaskStats (0.00s)
=== RUN   TestV3Handler_TaskStats_TrailingSlash
--- PASS: TestV3Handler_TaskStats_TrailingSlash (0.00s)
=== RUN   TestV3Handler_TaskStats_DockerAPIError
time="2021-02-13T13:54:21-05:00" level=error msg="HTTP 500 - Some error"
--- PASS: TestV3Handler_TaskStats_DockerAPIError (0.00s)
PASS
coverage: 100.0% of statements
ok      github.com/awslabs/amazon-ecs-local-container-endpoints/local-container-endpoints/handlers/functional_tests     (cached)        coverage: 100.0% of statements

@masteinhauser
Copy link
Contributor Author

I cannot get make integ to pass for me locally, but I suspect my current AWS account is missing the role ecs-local-endpoints-integ-role (as I have not created it manually). I am happy to continue testing if necessary and useful, otherwise I am looking forward to your review and feedback.

@PettitWesley
Copy link
Contributor

@masteinhauser I get errors when I try to build:

$ ./scripts/build_binary.sh ./bin/
go: inconsistent vendoring in /Users/wesley/local-dev/amazon-ecs-local-container-endpoints:

I was able to fix this very simply by running go mod vendor and then build and test work fine. Can you please do that and then push an update to this PR?

@masteinhauser
Copy link
Contributor Author

@PettitWesley Thanks for the response!

After running go mod vendor locally, I agree that make build and make test (etc.) pass correctly, but make generate still fails in the same fashion as before. This may not be critical for the project now, but it will be detrimental to the project later on. I don't know enough of newer Golang to help resolve that situation.

Please re-review with the new commit 49f5649

@masteinhauser
Copy link
Contributor Author

Hello @PettitWesley! What else is needed here to get this PR tested + merged?

@PettitWesley PettitWesley merged commit f324041 into awslabs:mainline Mar 2, 2021
@PettitWesley PettitWesley mentioned this pull request Mar 2, 2021
@SantuchoSJ
Copy link

SantuchoSJ commented Aug 14, 2022

Sorry to bring this back up, so what is the way of using ecs local endpoints with sso? I have my aws configured correctly and i have this as the docker.compose.override:

version: "3.7"
networks:

This special network is configured so that the local metadata
service can bind to the specific IP address that ECS uses
in development
credentials_network:
driver: bridge
ipam:
config:
- subnet: "169.254.170.0/24"
services:

This container vends credentials to your containers
ecs-local-endpoints:
# The Amazon ECS Local Container Endpoints Docker Image
image: amazon/amazon-ecs-local-container-endpoints
volumes:
# Mount /var/run so we can access docker.sock and talk to Docker
- /var/run:/var/run
# Mount the shared configuration directory, used by the AWS CLI and AWS SDKs
# On Windows, this directory can be found at "%UserProfile%.aws"
- $HOME/.aws/:/home/.aws/
environment:
# define the home folder; credentials will be read from $HOME/.aws
HOME: "/home"
# You can change which AWS CLI Profile is used
AWS_PROFILE: "dev"
networks:
credentials_network:
# This special IP address is recognized by the AWS SDKs and AWS CLI
ipv4_address: "169.254.170.2"

Here we reference the application container that we are testing
You can test multiple containers at a time, simply duplicate this section
and customize it for each container, and give it a unique IP in 'credentials_network'.
local:
depends_on:
- ecs-local-endpoints
networks:
credentials_network:
ipv4_address: "169.254.170.3"
environment:
AWS_DEFAULT_REGION: "sa-east-1"
AWS_CONTAINER_CREDENTIALS_RELATIVE_URI: "/role/my-ecs-role"

I have to use version 3.7 because thats what my previous and main app is built on and i cannot (or i am not allowed) to step back to v2, maybe that's an issue. I have inspected the docker container and the volumes are set up correctly, i can access the aws credentials from inside the container. I have also set up the trust policy in my ecs role, but it seems that i cannot reach the host that returns the role name. I keep on getting this error:

image

Btw, i am trying to access dynamodb from the nodejs aws sdk

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants