feat(API): Split gas component cron jobs up into different files #1372
Conversation
Updating the gas costs and gas prices in the same cron job makes the cache resets slower due to the memory constraints of a single file, and when updating the gas price every 5s, this makes a difference. This PR splits the gas cost updates, which are 10s and 30s for l1 data fees and native gas costs, separate from the gas price updates so that the gas price updates can be closer to 5s. Currently its anywhere from 5s to 10s. Additionally, this PR adds try-catches around the `await cache.set()` calls so that the a failure due to an upstream RPC call (e.g. I've seen this happen with linea_estimateGas calls a lot) doesn't take down the cron job for the full run.
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎
|
api/cron-cache-gas-costs.ts
Outdated
| const gasCost = await gasCostCache.get(); | ||
| if (utils.chainIsOPStack(chainId)) { | ||
| const cache = getCachedOpStackL1DataFee(depositArgs, gasCost); | ||
| try { |
There was a problem hiding this comment.
This try-catch is added around the cache.set
| if (diff >= maxDurationSec * 1000) { | ||
| break; | ||
| } | ||
| try { |
There was a problem hiding this comment.
This try-catch is added around the cache.set()
api/cron-cache-gas-costs.ts
Outdated
| // The above promises are supposed to complete after maxDurationSec seconds, but there are so many of them | ||
| // (one per route) that they last one can run quite a bit longer, so force the function to stop after maxDurationSec | ||
| // so that the serverless function can exit successfully. | ||
| await Promise.race([cacheUpdatePromises, utils.delay(maxDurationSec)]); |
There was a problem hiding this comment.
Force the many promises to stop after 60s, since there are so many of them this Promise.all wasn't completing in under 90s before, evevn though each promise was designed to finish in 60s
api/cron-cache-gas-costs.ts
Outdated
| // Set lower than TTL in getCachedNativeGasCost. This should rarely change so we should just make sure | ||
| // we keep this cache warm. | ||
| const updateNativeGasCostIntervalsSecPerChain = { | ||
| default: 20, |
There was a problem hiding this comment.
Lowered this to 20s from 30s since there are so many routes to update, the real latency of updating this per route is practically quite a bit higher (5-10s) than whatever is set here. We want to make sure this cache gets reset once per run so that the cache stays warm. The TTL is 120s so one update per run should be totally fine
nicholaspai
changed the title
|
|
||
| const cacheUpdatePromise = Promise.all( | ||
| mainnetChains | ||
| .filter((chain) => utils.chainIsOPStack(chain.chainId)) |
There was a problem hiding this comment.
Added this filter here since we don't need to update l1 date fees for non op stack chains
nicholaspai
requested review from
mrice32,
dohaki,
james-a-morris and
pxrl
as code owners
| availableRoutes | ||
| .filter(({ destinationChainId }) => destinationChainId === chainId) | ||
| .forEach(({ destinationToken }) => { | ||
| if (!destinationTokens.has(destinationToken)) { |
There was a problem hiding this comment.
Using a set here cuts down computation a lot for duplicate routes
| } | ||
| try { | ||
| await cache.set(); | ||
| updateCounts[chainId][outputTokenAddress]++; |
There was a problem hiding this comment.
This updateCount helped debug a lot that I was sending tons of duplicate queries for chain+outputToken combinations
Updating the gas costs and gas prices in the same cron job makes the cache resets slower due to the memory constraints of a single file, and when updating the gas price every 5s, this makes a difference.
This PR splits the gas cost updates, which are 10s and 30s for l1 data fees and native gas costs, separate from the gas price updates so that the gas price updates can be closer to 5s. Currently its anywhere from 5s to 10s.
Additionally, this PR adds try-catches around the
await cache.set()calls so that the a failure due to an upstream RPC call (e.g. I've seen this happen with linea_estimateGas calls a lot) doesn't take down the cron job for the full run.Finally, there was an issue with the current cron job where all combinations of chainId+outputToken were repeated many times, leading to tons of wasteful computation. This PR uses
Set'sto make sure costs are computed once for each destinationChain+outputToken combination