Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

stage_external_sources fails after update from dbt 1.4.5 to 1.7.10 and dbt-redshift 1.4.0 to 1.7.7 #338

Open
1 of 3 tasks
lgwoodmark opened this issue Jan 15, 2025 · 0 comments
Labels
bug Something isn't working triage

Comments

@lgwoodmark
Copy link

lgwoodmark commented Jan 15, 2025

Describe the bug

When running stage_external_sources for dbt-core 1.7.10 and dbt-redshift 1.7.7, the macro fails, claiming an error in redshift__refresh_external_table when running from lines 17 on, claiming that partition is a string object. This does not happen in dbt 1.4.5 and dbt-redshift 1.4.0, which was out previous version before updating.

Steps to reproduce

None of the source definitions containing a partitions block work anymore.

partitions:
  - name: load_date
    data_type: date
    vals:
      macro: dbt.dates_in_range
      args:
        start_date_str: '2022-01-01'
        end_date_str: "{{ modules.datetime.date.today().strftime('%Y-%m-%d') }}"
        in_fmt: "%Y-%m-%d"
        out_fmt: "%Y-%m-%d"
    path_macro: dbt_external_tables.year_month_day

--->

Expected results

Running stage_external_sources works as it did before the update.

Actual results

The script errors out with Compilation Error 'str object' has no attribute 'vals'

Screenshots and log output

$ dbt run-operation stage_external_sources -t lab
22:23:11  Running with dbt=1.7.10
22:23:12  Registered adapter: redshift=1.7.7
22:24:32  Found 1401 models, 2277 tests, 6 seeds, 11 operations, 4 analyses, 840 sources, 3 exposures, 0 metrics, 835 macros, 0 groups, 0 semantic models
22:24:34  1 of 1 START external xxx.yyy
22:24:35  Encountered an error while running operation: Compilation Error
  'str object' has no attribute 'vals'

  > in macro refresh_external_table (macros\common\refresh_external_table.sql)
  > called by macro redshift__get_external_build_plan (macros\plugins\redshift\get_external_build_plan.sql)
  > called by macro get_external_build_plan (macros\common\get_external_build_plan.sql)
  > called by macro stage_external_sources (macros\common\stage_external_sources.sql)
  > called by <Unknown>

System information

The contents of your packages.yml file:

packages:

  • package: dbt-labs/dbt_external_tables
    version: "0.11.1"
  • package: dbt-labs/dbt_utils
    version: "1.1.1"
  • package: dbt-labs/logging
    version: "0.8.0"
  • package: dbt-labs/codegen
    version: "0.10.0"
  • git: "https://github.com/calogica/dbt-date.git"
    revision: "0.10.1"

Which database are you using dbt with?

  • redshift
  • snowflake
  • other (specify: ____________)

The output of dbt --version:

Core:
  - installed: 1.7.10
  - latest:    1.9.1  - Update available!

  Your version of dbt-core is out of date!
  You can find instructions for upgrading here:
  https://docs.getdbt.com/docs/installation

Plugins:
  - postgres: 1.7.10 - Update available!
  - redshift: 1.7.7  - Update available!

  At least one plugin is out of date or incompatible with dbt-core.
  You can find instructions for upgrading here:
  https://docs.getdbt.com/docs/installation

The operating system you're using:
Windows 11 Enterprise 24H2

The output of python --version:
Python 3.9.13

@lgwoodmark lgwoodmark added bug Something isn't working triage labels Jan 15, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working triage
Projects
None yet
Development

No branches or pull requests

1 participant