Zeroex performance #6203
Zeroex performance #6203
Conversation
|
@RantumBits could you please review this to make sure I'm not messing up the logic on BNB here? |
dex/models/_projects/zeroex/bnb/zeroex_bnb_api_fills_uni_v2_pair_creation.sql
Show resolved
Hide resolved
| schema = 'zeroex_bnb', | ||
| alias = 'fills_uni_v2_pair_creation', | ||
| materialized='incremental', | ||
| unique_key = ['pair', 'makerToken', 'takerToken'], |
There was a problem hiding this comment.
not sure all three of these should be unique keys, otherwise only block_time is left to be updated. likely just pair?
There was a problem hiding this comment.
or potentially this could be append-only or something rather than merge? if so, we'd have to left join {{ this }} to make sure not already processed. merge may still be easier with correct keys
There was a problem hiding this comment.
My understanding of how this works is that you get the latest events for each pair, the old query implied there could be duplicates so we'd want to replace those values with the latest. This could be wrong however.
There was a problem hiding this comment.
block time might actually not even be needed here
There was a problem hiding this comment.
i think pair should be the only unique key, if we partition by that field in the rn and then the maker/taker tokens can change?
|
fyi on this one @Hosuke |
Pluies
force-pushed
the
zeroex-performance
branch
from
d3bda87 to
d6855d3
Compare
|
(Rebasing this to trigger a new CI run and hopefully reproduce issues that happened before for investigation 👍 ) |
…tions into zeroex-performance
|
Overlaps with #6192 |
@aalan3 looks good ser, thanks for your help with this 🫡 |
|
@RantumBits thanks for reviewing this! |
This PR hopes to improve performance on some zeroex models.
For ethereum I added an incremental filter on prices where it was missing.
For BNB I broke out the part of the query that's doing a full scan on bnb logs into its own model. We should then be able to run that model incrementally and joining on it should be much lighter as well.