Skip to content

Commit

Permalink
dex test refactor (#7463)
Browse files Browse the repository at this point in the history
* refactor(dex): move USD amount validation to dbt test

- Add reasonable_usd_amount test
- Apply test to chain level views
- Remove CI-specific view and steps

* refactor(dex): add USD amount validation to chain-level base trades

- Add test_reasonable_usd_amount test to each chain's base_trades model
- Set max USD amount to $1B for all chains

* refactor(dex): update test_reasonable_usd_amount macro

- Use enrich_dex_trades macro to calculate amount_usd
- Only check trades from last 24h
- Keep consistent with existing amount_usd calculation logic

* refactor(dex): move test_reasonable_usd_amount to tests/generic

- Move test to standard dbt test location
- Fix test macro name to follow dbt convention

* Recover schema

* Delete dbt_subprojects/dex/tests/test_reasonable_usd_amount.sql

* refactor(dex): simplify test_reasonable_usd_amount

- Use add_amount_usd macro directly instead of enrich_dex_trades
- Only check trades from last 24h

* refactor(dex): update test_reasonable_usd_amount to use enrich_dex_trades

- Use enrich_dex_trades macro to get token information and USD amount
- Only check trades from last 24h

* Clean up comments

* Add missing columns

* refactor(dex): simplify test_reasonable_usd_amount\n\n- Combine all trade types into a single enrichments CTE\n- Handle Curve trades with CASE statements\n- Add time filter to prices table\n- Optimize JOIN conditions

* feat(dex): add days_back parameter to test_reasonable_usd_amount\n\n- Add days_back parameter with default value of 30 days\n- Update time filters in base_trades and prices CTEs

* Rename macro

* refactor(dex): optimize test_acceptable_usd_amount merge points\n\n- Split price calculation for regular trades and Balancer V3\n- Prioritize ERC4626 prices for Balancer V3 trades\n- Remove redundant project checks after pre-filtering\n- Improve query optimization hints through early filtering
  • Loading branch information
Hosuke authored Jan 23, 2025
1 parent 6ea4e4f commit cd38fbc
Show file tree
Hide file tree
Showing 26 changed files with 322 additions and 0 deletions.
3 changes: 3 additions & 0 deletions dbt_subprojects/dex/models/trades/arbitrum/_schema.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,9 @@ models:
- name: dex_arbitrum_base_trades
data_tests:
- check_dex_info_relationship
- test_acceptable_usd_amount:
column_name: amount_usd
max_value: 1000000000

- name: uniswap_v3_arbitrum_base_trades
meta:
Expand Down
3 changes: 3 additions & 0 deletions dbt_subprojects/dex/models/trades/avalanche_c/_schema.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,9 @@ models:
- name: dex_avalanche_c_base_trades
data_tests:
- check_dex_info_relationship
- test_acceptable_usd_amount:
column_name: amount_usd
max_value: 1000000000

- name: uniswap_v3_avalanche_c_base_trades
meta:
Expand Down
3 changes: 3 additions & 0 deletions dbt_subprojects/dex/models/trades/base/_schema.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,9 @@ models:
- name: dex_base_base_trades
data_tests:
- check_dex_info_relationship
- test_acceptable_usd_amount:
column_name: amount_usd
max_value: 1000000000

- name: uniswap_v3_base_base_trades
meta:
Expand Down
3 changes: 3 additions & 0 deletions dbt_subprojects/dex/models/trades/blast/_schema.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,9 @@ models:
- name: dex_blast_base_trades
data_tests:
- check_dex_info_relationship
- test_acceptable_usd_amount:
column_name: amount_usd
max_value: 1000000000

- name: uniswap_v3_blast_base_trades
meta:
Expand Down
3 changes: 3 additions & 0 deletions dbt_subprojects/dex/models/trades/bnb/_schema.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,9 @@ models:
- name: dex_bnb_base_trades
data_tests:
- check_dex_info_relationship
- test_acceptable_usd_amount:
column_name: amount_usd
max_value: 1000000000

- name: uniswap_v3_bnb_base_trades
meta:
Expand Down
3 changes: 3 additions & 0 deletions dbt_subprojects/dex/models/trades/boba/_schema.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,9 @@ models:
- name: dex_boba_base_trades
data_tests:
- check_dex_info_relationship
- test_acceptable_usd_amount:
column_name: amount_usd
max_value: 1000000000

- name: icecreamswap_boba_base_trades
meta:
Expand Down
3 changes: 3 additions & 0 deletions dbt_subprojects/dex/models/trades/celo/_schema.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,9 @@ models:
- name: dex_celo_base_trades
data_tests:
- check_dex_info_relationship
- test_acceptable_usd_amount:
column_name: amount_usd
max_value: 1000000000

- name: uniswap_v3_celo_base_trades
meta:
Expand Down
3 changes: 3 additions & 0 deletions dbt_subprojects/dex/models/trades/ethereum/_schema.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,9 @@ models:
- name: dex_ethereum_base_trades
data_tests:
- check_dex_info_relationship
- test_acceptable_usd_amount:
column_name: amount_usd
max_value: 1000000000

- name: uniswap_v1_ethereum_base_trades
meta:
Expand Down
3 changes: 3 additions & 0 deletions dbt_subprojects/dex/models/trades/fantom/_schema.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,9 @@ models:
- name: dex_fantom_base_trades
data_tests:
- check_dex_info_relationship
- test_acceptable_usd_amount:
column_name: amount_usd
max_value: 1000000000

- name: sushiswap_v1_fantom_base_trades
meta:
Expand Down
3 changes: 3 additions & 0 deletions dbt_subprojects/dex/models/trades/flare/_schema.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,9 @@ models:
Flare DEX trades
data_tests:
- check_dex_info_relationship
- test_acceptable_usd_amount:
column_name: amount_usd
max_value: 1000000000

- name: enosys_v2_flare_base_trades
meta:
Expand Down
3 changes: 3 additions & 0 deletions dbt_subprojects/dex/models/trades/gnosis/_schema.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,9 @@ models:
- name: dex_gnosis_base_trades
data_tests:
- check_dex_info_relationship
- test_acceptable_usd_amount:
column_name: amount_usd
max_value: 1000000000

- name: sushiswap_v1_gnosis_base_trades
meta:
Expand Down
3 changes: 3 additions & 0 deletions dbt_subprojects/dex/models/trades/kaia/_schema.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,9 @@ models:
- name: dex_kaia_base_trades
data_tests:
- check_dex_info_relationship
- test_acceptable_usd_amount:
column_name: amount_usd
max_value: 1000000000

- name: dragon_swap_v2_kaia_base_trades
meta:
Expand Down
3 changes: 3 additions & 0 deletions dbt_subprojects/dex/models/trades/linea/_schema.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,9 @@ models:
- name: dex_linea_base_trades
data_tests:
- check_dex_info_relationship
- test_acceptable_usd_amount:
column_name: amount_usd
max_value: 1000000000

- name: sushiswap_v2_linea_base_trades
meta:
Expand Down
3 changes: 3 additions & 0 deletions dbt_subprojects/dex/models/trades/mantle/_schema.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,9 @@ models:
- name: dex_mantle_base_trades
data_tests:
- check_dex_info_relationship
- test_acceptable_usd_amount:
column_name: amount_usd
max_value: 1000000000

- name: merchant_moe_mantle_base_trades
meta:
Expand Down
3 changes: 3 additions & 0 deletions dbt_subprojects/dex/models/trades/nova/_schema.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,9 @@ models:
- name: dex_nova_base_trades
data_tests:
- check_dex_info_relationship
- test_acceptable_usd_amount:
column_name: amount_usd
max_value: 1000000000

- name: sushiswap_nova_base_trades
meta:
Expand Down
3 changes: 3 additions & 0 deletions dbt_subprojects/dex/models/trades/optimism/_schema.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,9 @@ models:
- name: dex_optimism_base_trades
data_tests:
- check_dex_info_relationship
- test_acceptable_usd_amount:
column_name: amount_usd
max_value: 1000000000

- name: uniswap_v3_optimism_base_trades
meta:
Expand Down
3 changes: 3 additions & 0 deletions dbt_subprojects/dex/models/trades/polygon/_schema.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,9 @@ models:
- name: dex_polygon_base_trades
data_tests:
- check_dex_info_relationship
- test_acceptable_usd_amount:
column_name: amount_usd
max_value: 1000000000

- name: uniswap_v3_polygon_base_trades
meta:
Expand Down
3 changes: 3 additions & 0 deletions dbt_subprojects/dex/models/trades/ronin/_schema.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,9 @@ models:
- name: dex_ronin_base_trades
data_tests:
- check_dex_info_relationship
- test_acceptable_usd_amount:
column_name: amount_usd
max_value: 1000000000

- name: katana_v2_ronin_base_trades
meta:
Expand Down
3 changes: 3 additions & 0 deletions dbt_subprojects/dex/models/trades/scroll/_schema.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,9 @@ models:
- name: dex_scroll_base_trades
data_tests:
- check_dex_info_relationship
- test_acceptable_usd_amount:
column_name: amount_usd
max_value: 1000000000

- name: uniswap_v3_scroll_base_trades
meta:
Expand Down
3 changes: 3 additions & 0 deletions dbt_subprojects/dex/models/trades/sei/_schema.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,9 @@ models:
- name: dex_sei_base_trades
data_tests:
- check_dex_info_relationship
- test_acceptable_usd_amount:
column_name: amount_usd
max_value: 1000000000

- name: jelly_swap_sei_base_trades
meta:
Expand Down
3 changes: 3 additions & 0 deletions dbt_subprojects/dex/models/trades/sonic/_schema.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,9 @@ models:
- name: dex_sonic_base_trades
data_tests:
- check_dex_info_relationship
- test_acceptable_usd_amount:
column_name: amount_usd
max_value: 1000000000

- name: beets_sonic_base_trades
meta:
Expand Down
3 changes: 3 additions & 0 deletions dbt_subprojects/dex/models/trades/worldchain/_schema.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,9 @@ models:
- name: dex_worldchain_base_trades
data_tests:
- check_dex_info_relationship
- test_acceptable_usd_amount:
column_name: amount_usd
max_value: 1000000000

- name: uniswap_v3_worldchain_base_trades
meta:
Expand Down
3 changes: 3 additions & 0 deletions dbt_subprojects/dex/models/trades/zkevm/_schema.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,9 @@ models:
- name: dex_zkevm_base_trades
data_tests:
- check_dex_info_relationship
- test_acceptable_usd_amount:
column_name: amount_usd
max_value: 1000000000

- name: balancer_v2_zkevm_base_trades
meta:
Expand Down
3 changes: 3 additions & 0 deletions dbt_subprojects/dex/models/trades/zksync/_schema.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,9 @@ models:
- name: dex_zksync_base_trades
data_tests:
- check_dex_info_relationship
- test_acceptable_usd_amount:
column_name: amount_usd
max_value: 1000000000

- name: maverick_zksync_base_trades
meta:
Expand Down
3 changes: 3 additions & 0 deletions dbt_subprojects/dex/models/trades/zora/_schema.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,9 @@ models:
- name: dex_zora_base_trades
data_tests:
- check_dex_info_relationship
- test_acceptable_usd_amount:
column_name: amount_usd
max_value: 1000000000

- name: uniswap_v2_zora_base_trades
meta:
Expand Down
Loading

0 comments on commit cd38fbc

Please sign in to comment.