Skip to content

Commit

Permalink
Merge branch 'main' into edgarrmondragon/fix/schema-hidden-sensitive
Browse files Browse the repository at this point in the history
  • Loading branch information
pnadolny13 authored Feb 16, 2024
2 parents 5705861 + ef21b1b commit bdb8dcc
Show file tree
Hide file tree
Showing 27 changed files with 310 additions and 51 deletions.
17 changes: 9 additions & 8 deletions .github/workflows/build-api.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,13 +8,14 @@ on:
branches:
- main
workflow_dispatch:
hub-url:
description: The base URL of the Meltano Hub site
required: true
default: 'https://hub.meltano.com'
api-url:
description: The base URL of the Hub API
required: true
inputs:
hub-url:
description: The base URL of the Meltano Hub site
required: true
default: 'https://hub.meltano.com'
api-url:
description: The base URL of the Hub API
required: true

jobs:
build:
Expand Down Expand Up @@ -83,7 +84,7 @@ jobs:
steps:
- uses: actions/[email protected]
- name: Configure AWS credentials
uses: aws-actions/[email protected].1
uses: aws-actions/[email protected].2
with:
role-to-assume: ${{ secrets.AWS_ROLE_TO_ASSUME }}
aws-region: us-west-2
Expand Down
6 changes: 3 additions & 3 deletions .github/workflows/metadata-extract.yml
Original file line number Diff line number Diff line change
Expand Up @@ -74,7 +74,7 @@ jobs:
- uses: actions/[email protected]

- name: Configure AWS credentials
uses: aws-actions/[email protected].1
uses: aws-actions/[email protected].2
with:
role-to-assume: ${{ secrets.AWS_ROLE_TO_ASSUME }}
aws-region: us-west-2
Expand Down Expand Up @@ -132,7 +132,7 @@ jobs:
- uses: actions/[email protected]

- name: Configure AWS credentials
uses: aws-actions/[email protected].1
uses: aws-actions/[email protected].2
with:
role-to-assume: ${{ secrets.AWS_ROLE_TO_ASSUME }}
aws-region: us-west-2
Expand Down Expand Up @@ -211,7 +211,7 @@ jobs:
- uses: actions/[email protected]

- name: Configure AWS credentials
uses: aws-actions/[email protected].1
uses: aws-actions/[email protected].2
with:
role-to-assume: ${{ secrets.AWS_ROLE_TO_ASSUME }}
aws-region: us-west-2
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/metadata-refresh.yml
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ jobs:
- uses: actions/[email protected]

- name: Configure AWS credentials
uses: aws-actions/[email protected].1
uses: aws-actions/[email protected].2
with:
role-to-assume: ${{ secrets.AWS_ROLE_TO_ASSUME }}
aws-region: us-west-2
Expand Down
8 changes: 8 additions & 0 deletions _data/meltano/extractors/tap-athena/meltanolabs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,14 @@ settings:
kind: string
label: Batch Config Storage Root
name: batch_config.storage.root
- description: 'One or more LCID locale strings to produce localized output for: https://faker.readthedocs.io/en/master/#localization'
kind: string
label: Faker Config Locale
name: faker_config.locale
- description: 'Value to seed the Faker generator for deterministic output: https://faker.readthedocs.io/en/master/#seeding-the-generator'
kind: string
label: Faker Config Seed
name: faker_config.seed
- description: "'True' to enable schema flattening and automatically expand nested
properties."
kind: boolean
Expand Down
8 changes: 8 additions & 0 deletions _data/meltano/extractors/tap-bitso/edgarrmondragon.yml
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,14 @@ settings:
name: books
value:
- btc_mxn
- description: 'One or more LCID locale strings to produce localized output for: https://faker.readthedocs.io/en/master/#localization'
kind: string
label: Faker Config Locale
name: faker_config.locale
- description: 'Value to seed the Faker generator for deterministic output: https://faker.readthedocs.io/en/master/#seeding-the-generator'
kind: string
label: Faker Config Seed
name: faker_config.seed
- description: "'True' to enable schema flattening and automatically expand nested
properties."
kind: boolean
Expand Down
8 changes: 8 additions & 0 deletions _data/meltano/extractors/tap-circle-ci/meltanolabs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,14 @@ settings:
kind: string
label: Batch Config Storage Root
name: batch_config.storage.root
- description: 'One or more LCID locale strings to produce localized output for: https://faker.readthedocs.io/en/master/#localization'
kind: string
label: Faker Config Locale
name: faker_config.locale
- description: 'Value to seed the Faker generator for deterministic output: https://faker.readthedocs.io/en/master/#seeding-the-generator'
kind: string
label: Faker Config Seed
name: faker_config.seed
- description: "'True' to enable schema flattening and automatically expand nested
properties."
kind: boolean
Expand Down
8 changes: 8 additions & 0 deletions _data/meltano/extractors/tap-csv/meltanolabs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,14 @@ settings:
label: Csv Files Definition
name: csv_files_definition
placeholder: Ex. files-def.json
- description: 'One or more LCID locale strings to produce localized output for: https://faker.readthedocs.io/en/master/#localization'
kind: string
label: Faker Config Locale
name: faker_config.locale
- description: 'Value to seed the Faker generator for deterministic output: https://faker.readthedocs.io/en/master/#seeding-the-generator'
kind: string
label: Faker Config Seed
name: faker_config.seed
- description: |
Array of objects with `entity`, `path`, `keys`, and `encoding` [Optional] keys:
Expand Down
23 changes: 16 additions & 7 deletions _data/meltano/extractors/tap-dbf/edgarrmondragon.yml
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,14 @@ settings:
kind: string
label: Batch Config Storage Root
name: batch_config.storage.root
- description: 'One or more LCID locale strings to produce localized output for: https://faker.readthedocs.io/en/master/#localization'
kind: string
label: Faker Config Locale
name: faker_config.locale
- description: 'Value to seed the Faker generator for deterministic output: https://faker.readthedocs.io/en/master/#seeding-the-generator'
kind: string
label: Faker Config Seed
name: faker_config.seed
- description: "'True' to enable schema flattening and automatically expand nested
properties."
kind: boolean
Expand All @@ -55,33 +63,34 @@ settings:
kind: integer
label: Flattening Max Depth
name: flattening_max_depth
- description: Fs Root
- description: The root of the filesystem to read from.
kind: string
label: Fs Root
name: fs_root
value: file://
- description: ''
- description: OAuth 2.0 token for GCS.
kind: password
label: Gcs Token
name: gcs.token
- description: A boolean whether to ignore missing memofile.
- description: Whether to proceed reading the file even if the [memofile] is not present.
kind: boolean
label: Ignore Missing Memofile
name: ignore_missing_memofile
value: false
- description: The file path.
- description: Glob expression where the files are located. Stream names will be extracted
from the file name.
kind: string
label: Path
name: path
- description: ''
- description: The S3 endpoint URL.
kind: string
label: S3 Endpoint URL
name: s3.endpoint_url
- description: ''
- description: The AWS key ID.
kind: password
label: S3 Key
name: s3.key
- description: ''
- description: The AWS secret key.
kind: password
label: S3 Secret
name: s3.secret
Expand Down
10 changes: 9 additions & 1 deletion _data/meltano/extractors/tap-dbt/meltanolabs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,14 @@ settings:
kind: string
label: Batch Config Storage Root
name: batch_config.storage.root
- description: 'One or more LCID locale strings to produce localized output for: https://faker.readthedocs.io/en/master/#localization'
kind: string
label: Faker Config Locale
name: faker_config.locale
- description: 'Value to seed the Faker generator for deterministic output: https://faker.readthedocs.io/en/master/#seeding-the-generator'
kind: string
label: Faker Config Seed
name: faker_config.seed
- description: "'True' to enable schema flattening and automatically expand nested
properties."
kind: boolean
Expand Down Expand Up @@ -86,7 +94,7 @@ settings:
kind: string
label: User Agent
name: user_agent
value: tap-dbt/0.9.1.dev30+9f90f0c Singer tap for the dbt Cloud API.
value: tap-dbt/0.10.1.dev8+c41fe9f Singer tap for the dbt Cloud API.
settings_group_validation:
- - account_ids
- api_key
Expand Down
8 changes: 8 additions & 0 deletions _data/meltano/extractors/tap-delighted/ticketswap.yml
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,14 @@ settings:
kind: string
label: Batch Config Storage Root
name: batch_config.storage.root
- description: 'One or more LCID locale strings to produce localized output for: https://faker.readthedocs.io/en/master/#localization'
kind: string
label: Faker Config Locale
name: faker_config.locale
- description: 'Value to seed the Faker generator for deterministic output: https://faker.readthedocs.io/en/master/#seeding-the-generator'
kind: string
label: Faker Config Seed
name: faker_config.seed
- description: "'True' to enable schema flattening and automatically expand nested
properties."
kind: boolean
Expand Down
8 changes: 8 additions & 0 deletions _data/meltano/extractors/tap-f1/reubenf.yml
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,14 @@ settings:
kind: string
label: Batch Config Storage Root
name: batch_config.storage.root
- description: 'One or more LCID locale strings to produce localized output for: https://faker.readthedocs.io/en/master/#localization'
kind: string
label: Faker Config Locale
name: faker_config.locale
- description: 'Value to seed the Faker generator for deterministic output: https://faker.readthedocs.io/en/master/#seeding-the-generator'
kind: string
label: Faker Config Seed
name: faker_config.seed
- description: "'True' to enable schema flattening and automatically expand nested
properties."
kind: boolean
Expand Down
2 changes: 1 addition & 1 deletion _data/meltano/extractors/tap-mongodb/meltanolabs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -185,7 +185,7 @@ settings_preamble: |
```yaml
metadata:
'*':
replication-key: _id
replication-key: replication_key
replication-method: LOG_BASED
```
Expand Down
8 changes: 8 additions & 0 deletions _data/meltano/extractors/tap-mongodb/z3z1ma.yml
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,14 @@ settings:
kind: array
label: Database Includes
name: database_includes
- description: 'One or more LCID locale strings to produce localized output for: https://faker.readthedocs.io/en/master/#localization'
kind: string
label: Faker Config Locale
name: faker_config.locale
- description: 'Value to seed the Faker generator for deterministic output: https://faker.readthedocs.io/en/master/#seeding-the-generator'
kind: string
label: Faker Config Seed
name: faker_config.seed
- description: "'True' to enable schema flattening and automatically expand nested
properties."
kind: boolean
Expand Down
8 changes: 8 additions & 0 deletions _data/meltano/extractors/tap-mysql/meltanolabs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,14 @@ settings:
kind: string
label: Database
name: database
- description: 'One or more LCID locale strings to produce localized output for: https://faker.readthedocs.io/en/master/#localization'
kind: string
label: Faker Config Locale
name: faker_config.locale
- description: 'Value to seed the Faker generator for deterministic output: https://faker.readthedocs.io/en/master/#seeding-the-generator'
kind: string
label: Faker Config Seed
name: faker_config.seed
- description: If an array of schema names is provided, the tap will only process
the specified MySQL schemas and ignore others. If left blank, the tap automatically
determines ALL available MySQL schemas.
Expand Down
8 changes: 8 additions & 0 deletions _data/meltano/extractors/tap-postgres/meltanolabs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,14 @@ settings:
- label: Log Based
value: LOG_BASED
value: FULL_TABLE
- description: 'One or more LCID locale strings to produce localized output for: https://faker.readthedocs.io/en/master/#localization'
kind: string
label: Faker Config Locale
name: faker_config.locale
- description: 'Value to seed the Faker generator for deterministic output: https://faker.readthedocs.io/en/master/#seeding-the-generator'
kind: string
label: Faker Config Seed
name: faker_config.seed
- description: If an array of schema names is provided, the tap will only process
the specified Postgres schemas and ignore others. If left blank, the tap automatically
determines ALL available Postgres schemas.
Expand Down
8 changes: 8 additions & 0 deletions _data/meltano/extractors/tap-pulumi-cloud/meltanolabs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,14 @@ settings:
kind: string
label: Batch Config Storage Root
name: batch_config.storage.root
- description: 'One or more LCID locale strings to produce localized output for: https://faker.readthedocs.io/en/master/#localization'
kind: string
label: Faker Config Locale
name: faker_config.locale
- description: 'Value to seed the Faker generator for deterministic output: https://faker.readthedocs.io/en/master/#seeding-the-generator'
kind: string
label: Faker Config Seed
name: faker_config.seed
- description: "'True' to enable schema flattening and automatically expand nested
properties."
kind: boolean
Expand Down
8 changes: 8 additions & 0 deletions _data/meltano/extractors/tap-qualified/z3z1ma.yml
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,14 @@ settings:
kind: string
label: Batch Config Storage Root
name: batch_config.storage.root
- description: 'One or more LCID locale strings to produce localized output for: https://faker.readthedocs.io/en/master/#localization'
kind: string
label: Faker Config Locale
name: faker_config.locale
- description: 'Value to seed the Faker generator for deterministic output: https://faker.readthedocs.io/en/master/#seeding-the-generator'
kind: string
label: Faker Config Seed
name: faker_config.seed
- description: "'True' to enable schema flattening and automatically expand nested
properties."
kind: boolean
Expand Down
8 changes: 8 additions & 0 deletions _data/meltano/extractors/tap-shortcut/edgarrmondragon.yml
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,14 @@ settings:
kind: string
label: Batch Config Storage Root
name: batch_config.storage.root
- description: 'One or more LCID locale strings to produce localized output for: https://faker.readthedocs.io/en/master/#localization'
kind: string
label: Faker Config Locale
name: faker_config.locale
- description: 'Value to seed the Faker generator for deterministic output: https://faker.readthedocs.io/en/master/#seeding-the-generator'
kind: string
label: Faker Config Seed
name: faker_config.seed
- description: "'True' to enable schema flattening and automatically expand nested
properties."
kind: boolean
Expand Down
8 changes: 8 additions & 0 deletions _data/meltano/extractors/tap-showpad/z3z1ma.yml
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,14 @@ settings:
kind: string
label: Batch Config Storage Root
name: batch_config.storage.root
- description: 'One or more LCID locale strings to produce localized output for: https://faker.readthedocs.io/en/master/#localization'
kind: string
label: Faker Config Locale
name: faker_config.locale
- description: 'Value to seed the Faker generator for deterministic output: https://faker.readthedocs.io/en/master/#seeding-the-generator'
kind: string
label: Faker Config Seed
name: faker_config.seed
- description: "'True' to enable schema flattening and automatically expand nested
properties."
kind: boolean
Expand Down
8 changes: 8 additions & 0 deletions _data/meltano/extractors/tap-smoke-test/meltano.yml
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,14 @@ settings:
kind: string
label: Batch Config Storage Root
name: batch_config.storage.root
- description: 'One or more LCID locale strings to produce localized output for: https://faker.readthedocs.io/en/master/#localization'
kind: string
label: Faker Config Locale
name: faker_config.locale
- description: 'Value to seed the Faker generator for deterministic output: https://faker.readthedocs.io/en/master/#seeding-the-generator'
kind: string
label: Faker Config Seed
name: faker_config.seed
- description: "'True' to enable schema flattening and automatically expand nested
properties."
kind: boolean
Expand Down
8 changes: 8 additions & 0 deletions _data/meltano/extractors/tap-snowflake/meltanolabs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,14 @@ settings:
kind: string
label: Database
name: database
- description: 'One or more LCID locale strings to produce localized output for: https://faker.readthedocs.io/en/master/#localization'
kind: string
label: Faker Config Locale
name: faker_config.locale
- description: 'Value to seed the Faker generator for deterministic output: https://faker.readthedocs.io/en/master/#seeding-the-generator'
kind: string
label: Faker Config Seed
name: faker_config.seed
- description: "'True' to enable schema flattening and automatically expand nested
properties."
kind: boolean
Expand Down
8 changes: 8 additions & 0 deletions _data/meltano/extractors/tap-stackexchange/meltanolabs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,14 @@ settings:
kind: string
label: Batch Config Storage Root
name: batch_config.storage.root
- description: 'One or more LCID locale strings to produce localized output for: https://faker.readthedocs.io/en/master/#localization'
kind: string
label: Faker Config Locale
name: faker_config.locale
- description: 'Value to seed the Faker generator for deterministic output: https://faker.readthedocs.io/en/master/#seeding-the-generator'
kind: string
label: Faker Config Seed
name: faker_config.seed
- description: Custom API filter to apply to all requests
documentation: https://api.stackexchange.com/docs/filters
kind: string
Expand Down
Loading

0 comments on commit bdb8dcc

Please sign in to comment.