bug: Non-Finalised State not returned by Zebra's ReadStateService in Regtest mode

[AloeareV]

What happened?

I expected to see this happen:
Transparent receipts to be visible in the non-finalized state.
Instead, this happened:
Transparent receipts in the latest 100 blocks (non-finalized state) do not appear in calls to ReadStateService get_taddr_transactions. Once those transactions are 100+blocks old, they appear in the get_taddr_transactions output, with all their details correct.

Original issue here: zingolabs/zaino#145

Zebra logs

2025-01-23T19:58:56.712709Z  INFO zebrad::components::tracing::component: started tracing component filter="info" TRACING_STATIC_MAX_LEVEL=LevelFilter::INFO LOG_STATIC_MAX_LEVEL=Info
2025-01-23T19:58:56.712729Z  INFO zebrad::components::tracing::component: set 'tracing.progress_bar ="summary"' in zebrad.toml to activate progress bars
2025-01-23T19:58:56.712731Z  INFO zebrad::application: Diagnostic Metadata:
version: 2.1.0+27.g8b84017
Zcash network: Regtest
running state version: 26.0.0
initial disk state version: creating.new.database
features: default,getblocktemplate_rpcs,howudoin,indicatif,progress_bar,release_max_level_info
branch: main
git commit: 8b84017
commit timestamp: 2025-01-23T09:54:05.000000000Z
target triple: x86_64-unknown-linux-gnu
rust compiler: 1.83.0
rust release date: 2024-11-26
optimization level: 3
debug checks: false
2025-01-23T19:58:56.712732Z  INFO zebrad::application: loaded zebrad config config_path=Some("/tmp/.tmphuEsWV/zebrad.toml") config=ZebradConfig { consensus: Config { checkpoint_sync: true }, metrics: Config { endpoint_addr:
2025-01-23T19:58:56.712774Z  INFO {zebrad="8b84017" net="Regtest"}: zebrad::application: initialized rayon thread pool for CPU-bound tasks num_threads=16
2025-01-23T19:58:56.713426Z  INFO {zebrad="8b84017" net="Regtest"}: zebrad::commands::start: Starting zebrad
2025-01-23T19:58:56.713477Z  INFO {zebrad="8b84017" net="Regtest"}: zebrad::commands::start: initializing node state
2025-01-23T19:58:56.713499Z  INFO {zebrad="8b84017" net="Regtest"}: zebrad::commands::start: opening database, this may take a few minutes
2025-01-23T19:58:56.713640Z  WARN zebra_state::service::finalized_state::disk_db: could not canonicalize "/tmp/.tmpicgQfZ/state/v25/regtest": No such file or directory (os error 2)
2025-01-23T19:58:56.713659Z  INFO zebra_state::service::finalized_state::disk_format::upgrade: creating new database with the current format running_version=26.0.0
2025-01-23T19:58:56.713867Z  INFO zebra_state::service::finalized_state::disk_db: the open file limit is high enough for Zebra current_limit=1024 min_limit=512 ideal_limit=1024
2025-01-23T19:58:56.722981Z  INFO zebra_state::service::finalized_state::disk_db: Opened Zebra state cache at /tmp/.tmpicgQfZ/state/v26/regtest
2025-01-23T19:58:56.723395Z  INFO zebra_state::service::finalized_state: loaded Zebra state cache tip=None
2025-01-23T19:58:56.723471Z  INFO zebra_state::service: starting legacy chain check
2025-01-23T19:58:56.723481Z  INFO zebra_state::service: cached state consensus branch is valid: no legacy chain found
2025-01-23T19:58:56.723504Z  INFO {zebrad="8b84017" net="Regtest"}: zebrad::commands::start: logging database metrics on startup
2025-01-23T19:58:56.723530Z  INFO {zebrad="8b84017" net="Regtest"}: zebra_state::service::finalized_state::disk_db: the open file limit is high enough for Zebra current_limit=1024 min_limit=512 ideal_limit=1024
2025-01-23T19:58:56.723526Z  INFO zebra_state::service::finalized_state::disk_format::upgrade: marked database format as newly created running_version=26.0.0 disk_version=26.0.0
2025-01-23T19:58:56.723583Z  INFO zebra_state::service::finalized_state::disk_format::upgrade: database format is valid running_version=26.0.0 inital_disk_version=None
2025-01-23T19:58:56.723653Z  INFO {zebrad="8b84017" net="Regtest"}: zebra_state::service::finalized_state::disk_db: Total Database Disk Size: 0 B
2025-01-23T19:58:56.723656Z  INFO {zebrad="8b84017" net="Regtest"}: zebra_state::service::finalized_state::disk_db: Total Live Data Disk Size: 0 B
2025-01-23T19:58:56.723657Z  INFO {zebrad="8b84017" net="Regtest"}: zebra_state::service::finalized_state::disk_db: Total Database Memory Size: 49.2 KB
2025-01-23T19:58:56.723679Z  INFO {zebrad="8b84017" net="Regtest"}: zebrad::commands::start: initializing network
2025-01-23T19:58:56.723695Z  INFO {zebrad="8b84017" net="Regtest"}:open_listener{addr=127.0.0.1:15842}: zebra_network::peer_set::initialize: Trying to open Zcash protocol endpoint at 127.0.0.1:15842...
2025-01-23T19:58:56.723724Z  INFO {zebrad="8b84017" net="Regtest"}:open_listener{addr=127.0.0.1:15842}: zebra_network::peer_set::initialize: Opened Zcash protocol endpoint at 127.0.0.1:15842
2025-01-23T19:58:56.723750Z  INFO {zebrad="8b84017" net="Regtest"}: zebra_network::address_book_updater: starting the address book updater
2025-01-23T19:58:56.723838Z  INFO {zebrad="8b84017" net="Regtest"}:add_initial_peers: zebra_network::peer_set::initialize: connecting to initial peer set ipv4_peer_count=0 ipv6_peer_count=0
2025-01-23T19:58:56.723843Z  INFO {zebrad="8b84017" net="Regtest"}:add_initial_peers: zebra_network::peer_set::initialize: finished connecting to initial seed and disk cache peers handshake_success_total=0 handshake_error_t
2025-01-23T19:58:56.723863Z  INFO {zebrad="8b84017" net="Regtest"}: zebra_network::peer_set::initialize: sending initial request for peers active_initial_peer_count=0
2025-01-23T19:58:56.723873Z  INFO {zebrad="8b84017" net="Regtest"}: zebrad::commands::start: initializing verifiers
2025-01-23T19:58:56.723880Z  INFO {zebrad="8b84017" net="Regtest"}:crawl_and_dial{new_peer_interval=61s}: zebra_network::peer_set::initialize: starting the peer address crawler crawl_new_peer_interval=61s outbound_connectio
2025-01-23T19:58:56.723895Z  INFO {zebrad="8b84017" net="Regtest"}:init{config=Config { checkpoint_sync: true } network=Regtest { activation_heights: {Height(0): Genesis, Height(1): Nu6} }}: zebra_consensus::router: startin
2025-01-23T19:58:56.723995Z  INFO {zebrad="8b84017" net="Regtest"}:init{config=Config { checkpoint_sync: true } network=Regtest { activation_heights: {Height(0): Genesis, Height(1): Nu6} }}: zebra_consensus::router: initial
2025-01-23T19:58:56.724003Z  INFO {zebrad="8b84017" net="Regtest"}: zebrad::commands::start: initializing syncer
2025-01-23T19:58:56.724033Z  INFO {zebrad="8b84017" net="Regtest"}: zebrad::commands::start: initializing mempool
2025-01-23T19:58:56.724041Z  INFO {zebrad="8b84017" net="Regtest"}: zebrad::commands::start: fully initializing inbound peer request handler
2025-01-23T19:58:56.724044Z  INFO {zebrad="8b84017" net="Regtest"}: zebrad::commands::start: spawning RPC server
2025-01-23T19:58:56.724044Z  INFO {zebrad="8b84017" net="Regtest"}: zebrad::commands::start: Trying to open RPC endpoint at 127.0.0.1:24538...
2025-01-23T19:58:56.724084Z  INFO {zebrad="8b84017" net="Regtest"}: zebra_rpc::server: Opened RPC endpoint at 127.0.0.1:24538
2025-01-23T19:58:56.724117Z  INFO {zebrad="8b84017" net="Regtest"}: zebrad::commands::start: spawning block gossip task
2025-01-23T19:58:56.724123Z  INFO {zebrad="8b84017" net="Regtest"}: zebrad::commands::start: spawning mempool queue checker task
2025-01-23T19:58:56.724126Z  INFO {zebrad="8b84017" net="Regtest"}: zebrad::commands::start: spawning mempool transaction gossip task
2025-01-23T19:58:56.724129Z  INFO {zebrad="8b84017" net="Regtest"}: zebrad::commands::start: spawning delete old databases task
2025-01-23T19:58:56.724133Z  INFO {zebrad="8b84017" net="Regtest"}: zebrad::components::sync::gossip: initializing block gossip task
2025-01-23T19:58:56.724136Z  INFO {zebrad="8b84017" net="Regtest"}: zebrad::components::mempool::queue_checker: initializing mempool queue checker task
2025-01-23T19:58:56.724141Z  INFO {zebrad="8b84017" net="Regtest"}: zebrad::components::mempool::gossip: initializing transaction gossip task
2025-01-23T19:58:56.724207Z  INFO {zebrad="8b84017" net="Regtest"}: zebrad::commands::start: spawning progress logging task
2025-01-23T19:58:56.724211Z  INFO {zebrad="8b84017" net="Regtest"}: zebrad::commands::start: spawning end of support checking task
2025-01-23T19:58:56.724216Z  INFO {zebrad="8b84017" net="Regtest"}: zebrad::commands::start: spawning mempool crawler task
2025-01-23T19:58:56.724219Z  INFO {zebrad="8b84017" net="Regtest"}: zebrad::components::sync::end_of_support: Starting end of support task
2025-01-23T19:58:56.724223Z  INFO {zebrad="8b84017" net="Regtest"}: zebrad::commands::start: spawning syncer task
2025-01-23T19:58:56.724224Z  INFO {zebrad="8b84017" net="Regtest"}: zebra_state::config: checking for old database versions db_kind="state"
2025-01-23T19:58:56.724229Z  INFO {zebrad="8b84017" net="Regtest"}: zebrad::components::mempool::crawler: initializing mempool crawler task
2025-01-23T19:58:56.724234Z  INFO {zebrad="8b84017" net="Regtest"}: zebrad::components::sync::progress: initial sync is waiting to download the genesis block sync_percent=0.000 % current_height=None
2025-01-23T19:58:56.724242Z  INFO {zebrad="8b84017" net="Regtest"}:init{config=Config { checkpoint_sync: true } network=Regtest { activation_heights: {Height(0): Genesis, Height(1): Nu6} }}: zebra_consensus::router: state i
2025-01-23T19:58:56.724246Z  INFO {zebrad="8b84017" net="Regtest"}: zebra_state::config: finished old database version cleanup task
2025-01-23T19:58:56.724246Z  INFO {zebrad="8b84017" net="Regtest"}:init{config=Config { checkpoint_sync: true } network=Regtest { activation_heights: {Height(0): Genesis, Height(1): Nu6} }}: zebra_consensus::router: finishe
2025-01-23T19:58:56.724331Z  INFO {zebrad="8b84017" net="Regtest"}:checkpoint: zebra_consensus::checkpoint: verified checkpoint range block_count=1 current_range=(Unbounded, Included(Height(0)))
2025-01-23T19:58:56.724338Z  INFO {zebrad="8b84017" net="Regtest"}:checkpoint: zebra_consensus::checkpoint: verified final checkpoint: starting full validation final_checkpoint_height=Height(0)
2025-01-23T19:58:56.786974Z  INFO {zebrad="8b84017" net="Regtest"}: zebrad::commands::start: spawned initial Zebra tasks
2025-01-23T19:59:01.725842Z  INFO {zebrad="8b84017" net="Regtest"}: zebrad::components::mempool: enabling mempool for debugging best_tip_height=Height(0) enable_at_height=Height(0)
2025-01-23T19:59:01.725855Z  INFO {zebrad="8b84017" net="Regtest"}: zebrad::components::mempool: activating mempool: Zebra is close to the tip tip_height=Height(0)
2025-01-23T19:59:06.725102Z  INFO {zebrad="8b84017" net="Regtest"}: zebrad::components::sync::end_of_support: Release always valid in Testnet
2025-01-23T19:59:11.829172Z  INFO zebra_rpc::methods::get_block_template_rpcs: submit block accepted block_hash=block::Hash("6095670f4d9a72fec27443be51da4b330db4a494be5dac413d27e4a412bacae2") block_height="1"
2025-01-23T19:59:16.729647Z  INFO {zebrad="8b84017" net="Regtest"}: zebrad::components::mempool: resetting mempool: switched best chain, skipped blocks, or activated network upgrade tip_height=Height(1)
2025-01-23T19:59:16.729672Z  INFO {zebrad="8b84017" net="Regtest"}: zebrad::components::mempool: enabling mempool for debugging best_tip_height=Height(1) enable_at_height=Height(0)
2025-01-23T19:59:16.729675Z  INFO {zebrad="8b84017" net="Regtest"}: zebrad::components::mempool: activating mempool: Zebra is close to the tip tip_height=Height(1)
2025-01-23T19:59:16.729678Z  INFO {zebrad="8b84017" net="Regtest"}: zebrad::components::mempool: re-verifying mempool transactions after a chain fork transactions=0
2025-01-23T19:59:20.070696Z  INFO zebra_rpc::methods::get_block_template_rpcs: submit block accepted block_hash=block::Hash("437372d8a9bf638b459b59a952fdf35b5f92784dfb98d064403c3c059fb55dff") block_height="2"
2025-01-23T19:59:20.071337Z  INFO zebra_rpc::methods::get_block_template_rpcs: submit block accepted block_hash=block::Hash("8cc5647b7d858cb398d226b56bc39519868fdd7304a9d61198a49f39da06bc21") block_height="3"
2025-01-23T19:59:20.072222Z  INFO zebra_rpc::methods::get_block_template_rpcs: submit block accepted block_hash=block::Hash("5af6e95aaf5469d2b950d5f22ef6a56d28f50f318e18c63f545ee87312bfab67") block_height="4"
2025-01-23T19:59:20.073036Z  INFO zebra_rpc::methods::get_block_template_rpcs: submit block accepted block_hash=block::Hash("3bdae8cb9b186c62d825cbc6a2b55607c52a4028c93936316b0cf70b0cf7ea6b") block_height="5"

... (some mostly identical lines omitted)

2025-01-23T19:59:20.137058Z  INFO zebra_rpc::methods::get_block_template_rpcs: submit block accepted block_hash=block::Hash("d84547736b54c8aba9c365f0a395947959e6e4e4686346da2a4b9947dc293c81") block_height="100"
2025-01-23T19:59:20.137977Z  INFO zebra_rpc::methods::get_block_template_rpcs: submit block accepted block_hash=block::Hash("e71a1ecb1b59f04ebd56bc17d17a27130ab119efb3c50dccd8886bba033a49ba") block_height="101"
2025-01-23T19:59:27.297181Z  INFO {zebrad="8b84017" net="Regtest"}: zebrad::components::mempool::gossip: sending mempool transaction broadcast request=AdvertiseTransactionIds(1) changes=1
2025-01-23T19:59:27.348109Z  INFO zebra_rpc::methods::get_block_template_rpcs: submit block accepted block_hash=block::Hash("a21ff3fe49271e53f6cc583d4a3ce64586698ea29634a53291af23377c28837a") block_height="102"
2025-01-23T19:59:31.909192Z  INFO zebra_rpc::methods::get_block_template_rpcs: submit block accepted block_hash=block::Hash("44ea3817a3c26e223cf6f3f220086b61745f25211be31998047736badd7f4c53") block_height="103"

... (some mostly identical lines omitted)

2025-01-23T19:59:32.013616Z  INFO zebra_rpc::methods::get_block_template_rpcs: submit block accepted block_hash=block::Hash("8877f9b077658ce82a10a56741eebda89284f51c8d512cb4c7be2307deeadbe7") block_height="201"
2025-01-23T19:59:32.014365Z  INFO zebra_rpc::methods::get_block_template_rpcs: submit block accepted block_hash=block::Hash("b6176f812ba362bfc05c9e4ea856dd112b9ed6b9b7f8c7c564c898bb793edd39") block_height="202"

Zebra Version

version: 2.1.0+27.g8b84017

Which operating systems does the issue happen on?

• Linux
• macOS
• Windows
• Other OS

OS details

No response

Additional information

No response