Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fee Upgrade #1530

Merged
merged 58 commits into from
Jun 17, 2024
Merged
Show file tree
Hide file tree
Changes from 8 commits
Commits
Show all changes
58 commits
Select commit Hold shift + click to select a range
c067572
add ResolvableChainConfig to ValidatedState
imabdulbasit May 28, 2024
cfd2662
get_chain_config
imabdulbasit May 28, 2024
9515679
Merge branch 'main' into abdul/move-chain-config
imabdulbasit May 28, 2024
6242d27
chain config route
imabdulbasit May 28, 2024
3c5f368
Merge remote-tracking branch 'origin/main' into abdul/move-chain-config
imabdulbasit May 28, 2024
870858c
Merge branch 'main' into abdul/move-chain-config
imabdulbasit May 28, 2024
e9bde5a
add test
imabdulbasit May 28, 2024
1f8d73f
clippy
imabdulbasit May 29, 2024
e49ec5c
Merge branch 'main' into abdul/move-chain-config
imabdulbasit May 29, 2024
581ac67
Merge remote-tracking branch 'origin/main' into abdul/move-chain-config
imabdulbasit May 30, 2024
3568098
point hotshot to main and others to daily-build
imabdulbasit May 30, 2024
0b6bd2b
pass simple builder config
imabdulbasit May 30, 2024
48ce9bd
merge abdul/move-chain-config, hotshot ss/fee-upgrade others abdul/up…
imabdulbasit May 30, 2024
8f5df88
make clippy happy
imabdulbasit May 30, 2024
f8c817b
Merge branch 'abdul/move-chain-config' into abdul/fee-upgrade
imabdulbasit May 30, 2024
b924031
update lockfile to point to specific hotshot commit
imabdulbasit May 30, 2024
05629c1
update builder-core
imabdulbasit May 30, 2024
f0cf3ef
Merge remote-tracking branch 'origin/abdul/move-chain-config' into ab…
imabdulbasit May 30, 2024
434969b
arc SystemContextHandle
imabdulbasit May 30, 2024
375e34a
add upgrades to Genesis and Nodestate
imabdulbasit Jun 3, 2024
727a7cd
use fee_contract address from updated chain_config
imabdulbasit Jun 3, 2024
cc80ce3
rename upgrade block field to view
imabdulbasit Jun 3, 2024
c88dcea
Chain config persistence
imabdulbasit Jun 4, 2024
510da55
fix: use instance chain config for genesis validated_state
imabdulbasit Jun 4, 2024
5cd07e9
test for chain config catchup
imabdulbasit Jun 4, 2024
3b15b05
remove version from NodeState
imabdulbasit Jun 4, 2024
32eeb8e
move proposal.chain_config and state chainconfig check below upgrade
imabdulbasit Jun 5, 2024
8079e64
move current version back to NodeState
imabdulbasit Jun 5, 2024
f2d92d5
get upgrade for current_version
imabdulbasit Jun 5, 2024
1b53a5a
Merge remote-tracking branch 'origin/main' into abdul/fee-upgrade
imabdulbasit Jun 5, 2024
81c8292
update lockfile
imabdulbasit Jun 5, 2024
59a337c
add test for chain config upgrade
imabdulbasit Jun 5, 2024
366f91c
Merge remote-tracking branch 'origin/main' into abdul/fee-upgrade
imabdulbasit Jun 5, 2024
4ab7e3c
pass None for builder port
imabdulbasit Jun 5, 2024
f186c45
merge main
imabdulbasit Jun 5, 2024
353a80d
fix test
imabdulbasit Jun 6, 2024
af27263
update migration version to V31 and cleanup
imabdulbasit Jun 6, 2024
d9366a0
apply_upgrade() and get_chain_config()
imabdulbasit Jun 7, 2024
b585bac
update only if chain config is different
imabdulbasit Jun 7, 2024
c54d166
return early
imabdulbasit Jun 7, 2024
625d4c4
Merge remote-tracking branch 'origin/main' into abdul/fee-upgrade
imabdulbasit Jun 13, 2024
9b3952a
use Base::instance()
imabdulbasit Jun 13, 2024
ef2e491
add propose_window, commit and revert in caller, remove header cf com…
imabdulbasit Jun 13, 2024
c3625aa
lint
imabdulbasit Jun 13, 2024
17d9277
lint
imabdulbasit Jun 13, 2024
56bbcfc
fix migrations version and use propose_window
imabdulbasit Jun 14, 2024
1f49651
fix: enforce_max_block_size test
imabdulbasit Jun 14, 2024
7f976ff
change TestConfig nodes to 5
imabdulbasit Jun 14, 2024
03b4e3c
set propose window to 10
imabdulbasit Jun 14, 2024
119f60a
use transaction()
imabdulbasit Jun 14, 2024
4d16988
remove header cf commit check
imabdulbasit Jun 14, 2024
e9fbe2d
update hotshot and cdn
imabdulbasit Jun 14, 2024
1ac5cf2
Update CDN in `ab/fee-upgrade` (#1606)
rob-maron Jun 14, 2024
f6b1d6c
cargo sort
imabdulbasit Jun 14, 2024
98ad78b
add commit check back
imabdulbasit Jun 14, 2024
257727b
`try_cast`
rob-maron Jun 14, 2024
8d9d7a7
clippy
rob-maron Jun 14, 2024
57e9cfb
bump
jparr721 Jun 14, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
451 changes: 348 additions & 103 deletions Cargo.lock

Large diffs are not rendered by default.

23 changes: 12 additions & 11 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -47,19 +47,19 @@ dotenvy = "0.15"
ethers = { version = "2.0", features = ["solc"] }
futures = "0.3"

hotshot = { git = "https://github.com/EspressoSystems/hotshot", branch = "ss/fee-upgrade" }
hotshot = { git = "https://github.com/EspressoSystems/hotshot", branch = "main" }
# Hotshot imports
hotshot-builder-api = { git = "https://github.com/EspressoSystems/HotShot.git", branch = "ss/fee-upgrade" }
hotshot-builder-core = { git = "https://github.com/EspressoSystems/hotshot-builder-core", branch = "abdul/update-hotshot" }
hotshot-contract-adapter = { version = "0.1.0", path = "contracts/rust/adapter" }
hotshot-events-service = { git = "https://github.com/EspressoSystems/hotshot-events-service.git", branch = "abdul/update-hotshot" }
hotshot-orchestrator = { git = "https://github.com/EspressoSystems/hotshot", branch = "ss/fee-upgrade" }
hotshot-query-service = { git = "https://github.com/EspressoSystems/hotshot-query-service", branch = "abdul/update-hotshot" }
hotshot-stake-table = { git = "https://github.com/EspressoSystems/hotshot", branch = "ss/fee-upgrade" }
hotshot-builder-api = { git = "https://github.com/EspressoSystems/HotShot.git", branch = "main" }
hotshot-builder-core = { git = "https://github.com/EspressoSystems/hotshot-builder-core", branch = "daily-build" }
hotshot-orchestrator = { git = "https://github.com/EspressoSystems/hotshot", branch = "main" }
hotshot-stake-table = { git = "https://github.com/EspressoSystems/hotshot", branch = "main" }
hotshot-state-prover = { version = "0.1.0", path = "hotshot-state-prover" }
hotshot-task = { git = "https://github.com/EspressoSystems/hotshot", branch = "ss/fee-upgrade" }
hotshot-testing = { git = "https://github.com/EspressoSystems/hotshot", branch = "ss/fee-upgrade" }
hotshot-types = { git = "https://github.com/EspressoSystems/hotshot", branch = "ss/fee-upgrade" }
hotshot-task = { git = "https://github.com/EspressoSystems/hotshot", branch = "main" }
hotshot-testing = { git = "https://github.com/EspressoSystems/hotshot", branch = "main" }
hotshot-types = { git = "https://github.com/EspressoSystems/hotshot", branch = "main" }
hotshot-contract-adapter = { version = "0.1.0", path = "contracts/rust/adapter" }
hotshot-events-service = { git = "https://github.com/EspressoSystems/hotshot-events-service.git", branch = "daily-build" }
hotshot-query-service = { git = "https://github.com/EspressoSystems/hotshot-query-service", branch = "daily-build" }

# Push CDN imports
cdn-broker = { git = "https://github.com/EspressoSystems/Push-CDN", features = [
Expand Down Expand Up @@ -115,6 +115,7 @@ serde = { version = "1.0.195", features = ["derive"] }
toml = "0.8"
url = "2.3"
vbs = "0.1"
vec1 = "1.12"
vergen = { version = "8.3", features = ["git", "gitcl"] }
zeroize = "1.7"
committable = "0.2"
Expand Down
1 change: 1 addition & 0 deletions builder/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,7 @@ tide-disco = { workspace = true }
tracing = { workspace = true }
url = { workspace = true }
vbs = { workspace = true }
vec1 = { workspace = true }

[dev-dependencies]
sequencer = { path = "../sequencer", features = ["testing"] }
6 changes: 3 additions & 3 deletions builder/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -88,7 +88,7 @@ pub fn run_builder_api_service(url: Url, source: ProxyGlobalState<SeqTypes>) {
app.register_module("txn_submit", private_mempool_api)
.expect("Failed to register the private mempool API");

async_spawn(app.serve(url, vbs::version::StaticVersion::<0, 1> {}));
async_spawn(app.serve(url, Base::instance()));
}

#[cfg(test)]
Expand Down Expand Up @@ -217,7 +217,7 @@ pub mod testing {
data_request_delay: Duration::from_millis(200),
view_sync_timeout: Duration::from_secs(5),
fixed_leader_for_gpuvid: 0,
builder_url,
builder_urls: vec1::vec1![builder_url],
builder_timeout: Duration::from_secs(1),
start_threshold: (
known_nodes_with_stake.clone().len() as u64,
Expand Down Expand Up @@ -446,7 +446,7 @@ pub mod testing {
app.register_module("hotshot-events", hotshot_events_api)
.expect("Failed to register hotshot events API");

async_spawn(app.serve(url, vbs::version::StaticVersion::<0, 1> {}));
async_spawn(app.serve(url, Base::instance()));
}
// enable hotshot event streaming
pub fn enable_hotshot_node_event_streaming<P: SequencerPersistence>(
Expand Down
2 changes: 1 addition & 1 deletion builder/src/non_permissioned.rs
Original file line number Diff line number Diff line change
Expand Up @@ -290,7 +290,7 @@ mod test {
);

// builder api url
let hotshot_builder_api_url = hotshot_config.config.builder_url.clone();
let hotshot_builder_api_url = hotshot_config.config.builder_urls[0].clone();

let builder_config = NonPermissionedBuilderTestConfig::init_non_permissioned_builder(
&hotshot_config,
Expand Down
2 changes: 1 addition & 1 deletion builder/src/permissioned.rs
Original file line number Diff line number Diff line change
Expand Up @@ -604,7 +604,7 @@ mod test {
let state_signer = handles[node_id].1.take().unwrap();

// builder api url
let hotshot_builder_api_url = hotshot_config.config.builder_url.clone();
let hotshot_builder_api_url = hotshot_config.config.builder_urls[0].clone();
let builder_config = PermissionedBuilderTestConfig::init_permissioned_builder(
hotshot_config,
hotshot_context_handle,
Expand Down
10 changes: 1 addition & 9 deletions data/genesis/demo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -11,18 +11,10 @@ fee_contract = '0xa15bb66138824a1c7167f5e85b957d04dd34e468'
[header]
timestamp = "1970-01-01T00:00:00Z"

[network]
# bootstrap_nodes = [
# "/ip4/127.0.0.1/udp/7003/quic-v1/p2p/12D3KooWFLKhKr61FhtdcMNR2wV4XHnceJQRFo6KKwQvwvRCRvy6",
# "/ip4/127.0.0.1/udp/7001/quic-v1/p2p/12D3KooWR9caJf3eqwyB3rZQX7DFUYTjM1jvXZJJxYbwhkAnPf4w",
# "/ip4/127.0.0.1/udp/7002/quic-v1/p2p/12D3KooWGbEJ2VsfX8KdUZgVaLYFndAxmtdScC91EritK57FbE2N",
# "/ip4/127.0.0.1/udp/7000/quic-v1/p2p/12D3KooWDtGECieXrqKoVxfDhU7afYnS6toj1GqWXuEDfcaGPDxa",
# "/ip4/127.0.0.1/udp/1769/quic-v1/p2p/12D3KooWCzRqAGWkUib7EqHgaWsJsGvKnujXSLsm4xf8u36nJKhH"
# ]

[[upgrade]]
version = "0.2"
view = 5
propose_window = 100


[upgrade.chain_config]
jbearer marked this conversation as resolved.
Show resolved Hide resolved
Expand Down
2 changes: 1 addition & 1 deletion docker-compose.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,7 @@ services:
ports:
- "$ESPRESSO_ORCHESTRATOR_PORT:$ESPRESSO_ORCHESTRATOR_PORT"
environment:
- ESPRESSO_ORCHESTRATOR_BUILDER_URL=http://permissionless-builder:$ESPRESSO_BUILDER_SERVER_PORT
- ESPRESSO_ORCHESTRATOR_BUILDER_URLS=http://permissionless-builder:$ESPRESSO_BUILDER_SERVER_PORT
- ESPRESSO_ORCHESTRATOR_PORT
- ESPRESSO_ORCHESTRATOR_NUM_NODES
- ESPRESSO_ORCHESTRATOR_START_DELAY
Expand Down
30 changes: 13 additions & 17 deletions hotshot-state-prover/src/mock_ledger.rs
Original file line number Diff line number Diff line change
Expand Up @@ -477,23 +477,19 @@ pub fn gen_plonk_proof_for_test(
let mut proofs = vec![];
let mut extra_msgs = vec![];

circuits
.iter()
.zip(prove_keys.iter())
.enumerate()
.for_each(|(_, (cs, pk))| {
let extra_msg = Some(vec![]); // We set extra_msg="" for the contract tests to pass
proofs.push(
PlonkKzgSnark::<Bn254>::prove::<_, _, SolidityTranscript>(
rng,
cs,
pk,
extra_msg.clone(),
)
.unwrap(),
);
extra_msgs.push(extra_msg);
});
circuits.iter().zip(prove_keys.iter()).for_each(|(cs, pk)| {
let extra_msg = Some(vec![]); // We set extra_msg="" for the contract tests to pass
proofs.push(
PlonkKzgSnark::<Bn254>::prove::<_, _, SolidityTranscript>(
rng,
cs,
pk,
extra_msg.clone(),
)
.unwrap(),
);
extra_msgs.push(extra_msg);
});

let public_inputs: Vec<Vec<F>> = circuits
.iter()
Expand Down
2 changes: 1 addition & 1 deletion process-compose.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,7 @@ processes:
orchestrator:
command: orchestrator
environment:
- ESPRESSO_ORCHESTRATOR_BUILDER_URL=http://localhost:$ESPRESSO_BUILDER_SERVER_PORT
- ESPRESSO_ORCHESTRATOR_BUILDER_URLS=http://localhost:$ESPRESSO_BUILDER_SERVER_PORT
readiness_probe:
http_get:
scheme: http
Expand Down
1 change: 1 addition & 0 deletions sequencer/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -113,6 +113,7 @@ typenum = { version = "1.15.0", default-features = false, features = [
] }
url = { workspace = true }
vbs = { workspace = true }
vec1 = { workspace = true }
zeroize = { workspace = true }

[package.metadata.cargo-udeps.ignore]
Expand Down
2 changes: 1 addition & 1 deletion sequencer/api/public-env-vars.toml
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ variables = [
"ESPRESSO_NASTY_CLIENT_WEIGHT_QUERY_NAMESPACE",
"ESPRESSO_NASTY_CLIENT_WEIGHT_QUERY_WINDOW",
"ESPRESSO_ORCHESTRATOR_BUILDER_TIMEOUT",
"ESPRESSO_ORCHESTRATOR_BUILDER_URL",
"ESPRESSO_ORCHESTRATOR_BUILDER_URLS",
"ESPRESSO_ORCHESTRATOR_LIBP2P_MESH_N",
"ESPRESSO_ORCHESTRATOR_NEXT_VIEW_TIMEOUT",
"ESPRESSO_ORCHESTRATOR_NUM_NODES",
Expand Down
36 changes: 24 additions & 12 deletions sequencer/src/api.rs
Original file line number Diff line number Diff line change
Expand Up @@ -413,7 +413,7 @@ pub mod test_helpers {

let (builder_task, builder_url) = run_test_builder(builder_port).await;

cfg.set_builder_url(builder_url);
cfg.set_builder_urls(vec1::vec1![builder_url]);

let mut upgrades_map = BTreeMap::default();
if let Some(upgrades) = upgrades {
Expand Down Expand Up @@ -477,9 +477,7 @@ pub mod test_helpers {
let handle_0 = &nodes[0];

// Hook the builder up to the event stream from the first node
if let Some(builder_task) = builder_task {
builder_task.start(Box::new(handle_0.event_stream().await));
}
builder_task.start(Box::new(handle_0.event_stream().await));

for ctx in &nodes {
ctx.start_consensus().await;
Expand Down Expand Up @@ -1247,7 +1245,11 @@ mod test {
let states = std::array::from_fn(|_| state.clone());

let mut network = TestNetwork::with_state(
Options::from(options::Http { port }).catchup(Default::default()),
Options::from(options::Http {
port,
max_connections: None,
})
jbearer marked this conversation as resolved.
Show resolved Hide resolved
.catchup(Default::default()),
states,
[no_storage::Options; TestConfig::NUM_NODES],
std::array::from_fn(|_| {
Expand Down Expand Up @@ -1319,7 +1321,11 @@ mod test {
states[0] = state2;

let mut network = TestNetwork::with_state(
Options::from(options::Http { port }).catchup(Default::default()),
Options::from(options::Http {
port,
max_connections: None,
})
.catchup(Default::default()),
states,
[no_storage::Options; TestConfig::NUM_NODES],
std::array::from_fn(|_| {
Expand Down Expand Up @@ -1368,10 +1374,13 @@ mod test {
..Default::default()
};
let mut map = std::collections::BTreeMap::new();
let view = 5;
let propose_window = 100;
map.insert(
Version { major: 0, minor: 2 },
Upgrade {
view: 5,
view,
propose_window,
upgrade_type: UpgradeType::ChainConfig {
chain_config: chain_config_upgrade,
},
Expand All @@ -1381,16 +1390,19 @@ mod test {
let stop_voting_view = 100;
let upgrades = TestNetworkUpgrades {
upgrades: map,
start_proposing_view: 5,
stop_proposing_view: 6,
start_proposing_view: view,
stop_proposing_view: view + propose_window,
start_voting_view: 1,
stop_voting_view,
};

let mut network = TestNetwork::with_state(
Options::from(options::Http { port })
.catchup(Default::default())
.status(Default::default()),
Options::from(options::Http {
port,
max_connections: None,
})
.catchup(Default::default())
.status(Default::default()),
Default::default(),
[no_storage::Options; TestConfig::NUM_NODES],
std::array::from_fn(|_| {
Expand Down
7 changes: 4 additions & 3 deletions sequencer/src/api/data_source.rs
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@ use hotshot_types::{
use serde::Serialize;
use tide_disco::Url;
use vbs::version::StaticVersionType;
use vec1::Vec1;

pub trait DataSourceOptions: persistence::PersistenceOptions {
type DataSource: SequencerDataSource<Options = Self>;
Expand Down Expand Up @@ -215,7 +216,7 @@ pub struct PublicHotShotConfig {
pub num_bootstrap: usize,
pub builder_timeout: Duration,
pub data_request_delay: Duration,
pub builder_url: Url,
pub builder_urls: Vec1<Url>,
pub start_proposing_view: u64,
pub stop_proposing_view: u64,
pub start_voting_view: u64,
Expand Down Expand Up @@ -247,7 +248,7 @@ impl From<HotShotConfig<PubKey>> for PublicHotShotConfig {
num_bootstrap,
builder_timeout,
data_request_delay,
builder_url,
builder_urls,
start_proposing_view,
stop_proposing_view,
start_voting_view,
Expand All @@ -274,7 +275,7 @@ impl From<HotShotConfig<PubKey>> for PublicHotShotConfig {
num_bootstrap,
builder_timeout,
data_request_delay,
builder_url,
builder_urls,
start_proposing_view,
stop_proposing_view,
start_voting_view,
Expand Down
2 changes: 1 addition & 1 deletion sequencer/src/api/endpoints.rs
Original file line number Diff line number Diff line change
Expand Up @@ -268,7 +268,7 @@ where
state
.get_chain_config(commitment)
.await
.map_err(|err| Error::catch_all(StatusCode::NotFound, format!("{err:#}")))
.map_err(|err| Error::catch_all(StatusCode::NOT_FOUND, format!("{err:#}")))
}
.boxed()
})?;
Expand Down
3 changes: 0 additions & 3 deletions sequencer/src/api/sql.rs
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,6 @@ use hotshot_query_service::{
data_source::{
sql::{Config, Query, SqlDataSource},
storage::SqlStorage,
VersionedDataSource,
},
merklized_state::{MerklizedStateDataSource, Snapshot},
Resolvable,
Expand Down Expand Up @@ -142,8 +141,6 @@ impl ChainConfigPersistence for SqlStorage {
)
.await?;

self.commit().await?;

Ok(())
}

Expand Down
7 changes: 4 additions & 3 deletions sequencer/src/bin/orchestrator.rs
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ use std::num::{NonZeroUsize, ParseIntError};
use std::str::FromStr;
use std::time::Duration;
use url::Url;
use vec1::Vec1;

#[derive(Parser)]
struct Args {
Expand Down Expand Up @@ -76,8 +77,8 @@ struct Args {
keygen_seed: [u8; 32],

/// HotShot builder URL
#[arg(long, env = "ESPRESSO_ORCHESTRATOR_BUILDER_URL")]
builder_url: Url,
#[arg(long, env = "ESPRESSO_ORCHESTRATOR_BUILDER_URLS", num_args = 1.., value_delimiter = ',')]
builder_urls: Vec<Url>,

/// The maximum amount of time a leader can wait to get a block from a builder.
///
Expand Down Expand Up @@ -196,7 +197,7 @@ async fn main() {
config.config.start_delay = args.start_delay.as_millis() as u64;
config.config.da_staked_committee_size = args.num_nodes.get();
config.config.da_non_staked_committee_size = 0;
config.config.builder_url = args.builder_url;
config.config.builder_urls = Vec1::try_from_vec(args.builder_urls).unwrap();
config.config.builder_timeout = args.builder_timeout;
run_orchestrator(
config,
Expand Down
Loading
Loading