Skip to content

Commit

Permalink
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
docs: move readme docs to lib.rs rust docs
Browse files Browse the repository at this point in the history
samlaf committed Jan 7, 2025
1 parent e61ddcf commit 194b32b
Showing 12 changed files with 119 additions and 86 deletions.
16 changes: 7 additions & 9 deletions Cargo.toml
Original file line number Diff line number Diff line change
@@ -8,17 +8,16 @@ description = "This library offers a set of functions for generating and interac
readme = "README.md"
repository = "https://github.com/Layr-Labs/rust-kzg-bn254"
license-file = "LICENSE"
exclude = [
"tests/*",
"benches/*",
]
exclude = ["tests/*", "benches/*"]
# TODO: is this needed for the image to show up in the rust docs?
include = ["./kzg_commitment_diagram.png"]

[dependencies]
ark-bn254 = "0.5.0"
ark-ec = {version = "0.5.0", features = ["parallel"]}
ark-ff = {version = "0.5.0", features = ["parallel"]}
ark-ec = { version = "0.5.0", features = ["parallel"] }
ark-ff = { version = "0.5.0", features = ["parallel"] }
ark-serialize = "0.5.0"
ark-std = {version = "0.5.0", features = ["parallel"]}
ark-std = { version = "0.5.0", features = ["parallel"] }
directories = "5.0.1"
hex-literal = "0.4.1"
rand = "0.8.5"
@@ -28,7 +27,7 @@ num-bigint = "0.4"
rayon = "^1.5"
num-traits = "0.2"
byteorder = "1.4"
ark-poly = {version = "0.5.0", features = ["parallel"]}
ark-poly = { version = "0.5.0", features = ["parallel"] }
crossbeam-channel = "0.5"
num_cpus = "1.13.0"
sys-info = "0.9"
@@ -109,4 +108,3 @@ panic = 'unwind'
incremental = false
codegen-units = 16
rpath = false

58 changes: 9 additions & 49 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,18 +1,10 @@
# rust-kzg-bn254

## Description
[![Docs](https://docs.rs/rust-kzg-bn254/badge.svg)](https://docs.rs/rust-kzg-bn254/latest/rust_kzg_bn254/)
[![Crate](https://img.shields.io/crates/v/rust-kzg-bn254.svg)](https://crates.io/crates/rust-kzg-bn254)

This library offers a set of functions for generating and interacting with bn254 KZG commitments and proofs in rust, with the motivation of supporting fraud and validity proof logic in EigenDA rollup integrations.

## Warning & Disclaimer

This code is unaudited and under construction. This is experimental software and is provided on an "as is" and "as available" basis and may not work at all. It should not be used in production.

## Setup for testing

1. To test, please download the provided G1 and G2 points from [DA Resources](https://github.com/Layr-Labs/eigenda/tree/master/inabox/resources/kzg),
2. Specify these files in the `kzg.setup()` function, leave the `g2_power_of2_path` empty, and specify `srs_order` to be 3000.

## Configuring with the EigenDA KZG trusted setup

1. Follow the setup instructions to download the G1 and G2 powers of 2 points from the [Operator Setup Guide](https://github.com/Layr-Labs/eigenda-operator-setup)
@@ -21,47 +13,15 @@ This code is unaudited and under construction. This is experimental software and

## Quick Start

See the `test_compute_kzg_proof` function in [./tests/kzg_test.rs](./tests/kzg_test.rs#) for an end to end usage of the library.

## Requirements

1. SRS points required are in the same format as provided by EigenDA.
2. Committing is performed in Lagrange format. The required IFFT is done within the function and is not required to be performed separately.
3. For proof generation, the data is treated as evaluation of polynomial. The required (I)FFT is performed by the compute function and is not required to be performed separately.

## Library Design / Architecture
See the `test_compute_kzg_proof` function in [./tests/kzg_test.rs](./tests/kzg_test.rs) for an end to end usage of the library.

The main purpose of this library is to allow taking a piece of data, committing to it, and then generating and verifying proofs against that commitment.
Also make sure to check out the examples in our [docs](https://docs.rs/rust-kzg-bn254/latest/rust_kzg_bn254/).

### Data Types

The main data pipeline goes:
> user data -> Blob -> Polynomial -> KZG Commitment / Proof
- User Data: bytes array
- meaningful to users (typically will be a rollup batch)
- Blob: bn254 field elements array
- meaningful to EigenDA network
- Obtained from User Data by inserting zeroes every 31 bytes to make every 32 byte an element of bn254.
- Polynomial: bn254 field elements array, interpreted as coefficients or evaluations of a polynomial
- meaningful when committing and generating/verifying proofs
- Obtained from Blob by appending zeroes to make the length a power of 2, and then interpreting the array as coefficients or evaluations of a polynomial.
- Kzg: struct storing the SRS points used to generate commitments and proofs
- SRS points: bn254 group elements
- inner producted with the polynomial to generate commitments

The [Blob](./src/blob.rs) and [Polynomial](./src/polynomial.rs) structs are mostly [Plain Old Data](https://en.wikipedia.org/wiki/Passive_data_structure) with constructor and few helper methods. The interesting stuff happens in the [KZG](./src/kzg.rs) struct, which has methods for committing to a blob, polynomial in coeff or eval form, and generating and verifying proofs.

Our current codebase has the types PolynomialEvalForm and PolynomialCoeffForm to represent the polynomial in evaluation and coefficient form respectively. However, we do not have types to represent the two forms of srs points. They are implicitly assumed to be in monomial form when loaded, and an IFFT is performed before taking the inner product with the polynomial in evaluation form.

### KZG Commitments

Below diagram explains the different types involved between polynomials, SRS points, and KZG commitments.
A KZG commitment can be taken by an inner product between (poly_coeff, srs_monomial) or (poly_eval, srs_lagrange). FFT and IFFT operations can be performed to convert between these forms.

![KZG Commitments](./kzg_commitment_diagram.png)
## Setup for testing

1. To test, please download the provided G1 and G2 points from [DA Resources](https://github.com/Layr-Labs/eigenda/tree/master/inabox/resources/kzg),
2. Specify these files in the `kzg.setup()` function, leave the `g2_power_of2_path` empty, and specify `srs_order` to be 3000.

### KZG Proofs
## Warning & Disclaimer

TODO: Add diagram for KZG Proofs
This code is unaudited and under construction. This is experimental software and is provided on an "as is" and "as available" basis and may not work at all. It should not be used in production.
4 changes: 2 additions & 2 deletions benches/bench_g1_ifft.rs
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
use criterion::{black_box, criterion_group, criterion_main, Criterion};
use rust_kzg_bn254::kzg::Kzg;
use rust_kzg_bn254::kzg::KZG;
use std::time::Duration;

fn generate_powers_of_2(limit: u64) -> Vec<usize> {
@@ -16,7 +16,7 @@ fn generate_powers_of_2(limit: u64) -> Vec<usize> {

fn bench_g1_ifft(c: &mut Criterion) {
c.bench_function("bench_g1_ifft", |b| {
let kzg = Kzg::setup(
let kzg = KZG::setup(
"tests/test-files/mainnet-data/g1.131072.point",
"",
"tests/test-files/mainnet-data/g2.point.powerOf2",
4 changes: 2 additions & 2 deletions benches/bench_kzg_commit.rs
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
use criterion::{criterion_group, criterion_main, Criterion};
use rand::Rng;
use rust_kzg_bn254::{blob::Blob, kzg::Kzg};
use rust_kzg_bn254::{blob::Blob, kzg::KZG};
use std::time::Duration;

fn bench_kzg_commit(c: &mut Criterion) {
let mut rng = rand::thread_rng();
let mut kzg = Kzg::setup(
let mut kzg = KZG::setup(
"tests/test-files/mainnet-data/g1.131072.point",
"",
"tests/test-files/mainnet-data/g2.point.powerOf2",
4 changes: 2 additions & 2 deletions benches/bench_kzg_commit_large_blobs.rs
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
use criterion::{criterion_group, criterion_main, Criterion};
use rand::Rng;
use rust_kzg_bn254::{blob::Blob, kzg::Kzg};
use rust_kzg_bn254::{blob::Blob, kzg::KZG};
use std::time::Duration;

fn bench_kzg_commit(c: &mut Criterion) {
let mut rng = rand::thread_rng();
let mut kzg = Kzg::setup(
let mut kzg = KZG::setup(
"tests/test-files/mainnet-data/g1.32mb.point",
"",
"tests/test-files/mainnet-data/g2.point.powerOf2",
4 changes: 2 additions & 2 deletions benches/bench_kzg_proof.rs
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
use criterion::{criterion_group, criterion_main, Criterion};
use rand::Rng;
use rust_kzg_bn254::{blob::Blob, kzg::Kzg};
use rust_kzg_bn254::{blob::Blob, kzg::KZG};
use std::time::Duration;

fn bench_kzg_proof(c: &mut Criterion) {
let mut rng = rand::thread_rng();
let mut kzg = Kzg::setup(
let mut kzg = KZG::setup(
"tests/test-files/mainnet-data/g1.131072.point",
"",
"tests/test-files/mainnet-data/g2.point.powerOf2",
6 changes: 3 additions & 3 deletions benches/bench_kzg_setup.rs
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
use criterion::{criterion_group, criterion_main, Criterion};
use rust_kzg_bn254::kzg::Kzg;
use rust_kzg_bn254::kzg::KZG;
use std::time::Duration;

fn bench_kzg_setup(c: &mut Criterion) {
c.bench_function("bench_kzg_setup", |b| {
b.iter(|| {
Kzg::setup(
KZG::setup(
"tests/test-files/g1.point",
"tests/test-files/g2.point",
"tests/test-files/g2.point.powerOf2",
@@ -16,7 +16,7 @@ fn bench_kzg_setup(c: &mut Criterion) {
});

b.iter(|| {
Kzg::setup(
KZG::setup(
"tests/test-files/mainnet-data/g1.131072.point",
"",
"tests/test-files/mainnet-data/g2.point.powerOf2",
4 changes: 2 additions & 2 deletions benches/bench_kzg_verify.rs
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
use criterion::{criterion_group, criterion_main, Criterion};
use rand::Rng;
use rust_kzg_bn254::{blob::Blob, kzg::Kzg};
use rust_kzg_bn254::{blob::Blob, kzg::KZG};
use std::time::Duration;

fn bench_kzg_verify(c: &mut Criterion) {
let mut rng = rand::thread_rng();
let mut kzg = Kzg::setup(
let mut kzg = KZG::setup(
"tests/test-files/mainnet-data/g1.131072.point",
"",
"tests/test-files/mainnet-data/g2.point.powerOf2",
2 changes: 1 addition & 1 deletion src/blob.rs
Original file line number Diff line number Diff line change
@@ -5,7 +5,7 @@ use crate::{

/// A blob which is Eigen DA spec aligned.
/// TODO: we should probably move to a transparent repr like
/// https://docs.rs/alloy-primitives/latest/alloy_primitives/struct.FixedBytes.html
/// <https://docs.rs/alloy-primitives/latest/alloy_primitives/struct.FixedBytes.html>
#[derive(Clone, Debug, PartialEq, Eq)]
pub struct Blob {
blob_data: Vec<u8>,
12 changes: 10 additions & 2 deletions src/kzg.rs
Original file line number Diff line number Diff line change
@@ -19,8 +19,16 @@ use std::{
io::{self, BufReader},
};

/// Main interesting struct of the rust-kzg-bn254 crate.
/// [Kzg] is a struct that holds the SRS points in monomial form, and
/// provides methods for committing to a blob, (either via a [Blob] itself,
/// or a [PolynomialCoeffForm] or [PolynomialEvalForm]), and generating and verifying proofs.
///
/// The [Blob] and [PolynomialCoeffForm]/[PolynomialEvalForm] structs are mostly
/// <https://en.wikipedia.org/wiki/Passive_data_structure> with
/// constructor and few helper methods.
#[derive(Debug, PartialEq, Clone)]
pub struct Kzg {
pub struct KZG {
// SRS points are stored in monomial form, ready to be used for commitments with polynomials
// in coefficient form. To commit against a polynomial in evaluation form, we need to transform
// the SRS points to lagrange form using IFFT.
@@ -39,7 +47,7 @@ struct Params {
completed_setup: bool,
}

impl Kzg {
impl KZG {
pub fn setup(
path_to_g1_points: &str,
path_to_g2_points: &str,
67 changes: 67 additions & 0 deletions src/lib.rs
Original file line number Diff line number Diff line change
@@ -1,3 +1,70 @@
/*!
## Library Design / Architecture
The main purpose of this library is to allow taking a piece of data, committing to it, and then generating and verifying proofs against that commitment.
### Data Types
The main data pipeline goes:
> user data -> [blob::Blob] -> [polynomial::PolynomialEvalForm]/[polynomial::PolynomialCoeffForm] -> KZG Commitment / Proof
- User Data: bytes array
- meaningful to users (typically will be a rollup batch)
- Blob: bn254 field elements array
- meaningful to EigenDA network
- Obtained from User Data by inserting zeroes every 31 bytes to make every 32 byte an element of bn254.
- Polynomial: bn254 field elements array, interpreted as coefficients or evaluations of a polynomial
- meaningful when committing and generating/verifying proofs
- Obtained from Blob by appending zeroes to make the length a power of 2, and then interpreting the array as coefficients or evaluations of a polynomial.
- KZG: struct storing the SRS points used to generate commitments and proofs
- SRS points: bn254 group elements
- inner producted with the polynomial to generate commitments
The Blob and Polynomial structs are mostly
[Plain Old Data](https://en.wikipedia.org/wiki/Passive_data_structure) with constructor and few helper methods.
The interesting stuff happens in the [kzg::KZG] struct,
which has methods for committing to a blob, polynomial in coeff or eval form,
and generating and verifying proofs.
Our current codebase has the types PolynomialEvalForm and PolynomialCoeffForm to represent the polynomial in evaluation and coefficient form respectively. However, we do not have types to represent the two forms of srs points. They are implicitly assumed to be in monomial form when loaded, and an IFFT is performed before taking the inner product with the polynomial in evaluation form.
### KZG Commitments
A KZG commitment can be taken by an inner product between (poly_coeff, srs_monomial) or (poly_eval, srs_lagrange). FFT and IFFT operations can be performed to convert between these forms.
![KZG](../kzg_commitment_diagram.png)
### KZG Proofs
TODO
## Examples
### Commit to a some user data
```rust
use rust_kzg_bn254::{blob::Blob, kzg::Kzg};
let kzg = Kzg::setup(
"tests/test-files/mainnet-data/g1.131072.point",
"",
"tests/test-files/mainnet-data/g2.point.powerOf2",
268435456,
131072,
).unwrap();
let rollup_data: &[u8] = "some rollup batcher data".as_bytes();
let blob = Blob::from_raw_data(rollup_data);
let poly = blob.to_polynomial_eval_form();
let commitment = kzg.commit_eval_form(&poly).unwrap();
```
### Generate a proof for a piece of data
```rust
// TODO
```
*/

mod arith;
pub mod blob;
pub mod consts;
24 changes: 12 additions & 12 deletions tests/kzg_test.rs
Original file line number Diff line number Diff line change
@@ -6,7 +6,7 @@ mod tests {
blob::Blob,
errors::KzgError,
helpers,
kzg::Kzg,
kzg::KZG,
polynomial::{PolynomialCoeffForm, PolynomialEvalForm},
};
use std::{
@@ -18,17 +18,17 @@ mod tests {
use ark_std::{str::FromStr, One};

// Function to determine the setup based on an environment variable
fn determine_setup() -> Kzg {
fn determine_setup() -> KZG {
match env::var("KZG_ENV") {
Ok(val) if val == "mainnet-data" => Kzg::setup(
Ok(val) if val == "mainnet-data" => KZG::setup(
"tests/test-files/mainnet-data/g1.131072.point",
"",
"tests/test-files/mainnet-data/g2.point.powerOf2",
268435456,
131072,
)
.unwrap(),
_ => Kzg::setup(
_ => KZG::setup(
"tests/test-files/g1.point",
"tests/test-files/g2.point",
"tests/test-files/g2.point.powerOf2",
@@ -41,8 +41,8 @@ mod tests {

// Define a static variable for setup
lazy_static! {
static ref KZG_INSTANCE: Kzg = determine_setup();
static ref KZG_3000: Kzg = Kzg::setup(
static ref KZG_INSTANCE: KZG = determine_setup();
static ref KZG_3000: KZG = KZG::setup(
"tests/test-files/g1.point",
"tests/test-files/g2.point",
"tests/test-files/g2.point.powerOf2",
@@ -71,15 +71,15 @@ mod tests {

#[test]
fn test_kzg_setup_errors() {
let kzg1 = Kzg::setup("tests/test-files/g1.point", "", "", 3000, 3000);
let kzg1 = KZG::setup("tests/test-files/g1.point", "", "", 3000, 3000);
assert_eq!(
kzg1,
Err(KzgError::GenericError(
"both g2 point files are empty, need the proper file specified".to_string()
))
);

let mut kzg2 = Kzg::setup(
let mut kzg2 = KZG::setup(
"tests/test-files/g1.point",
"tests/test-files/g2.point",
"tests/test-files/g2.point.powerOf2",
@@ -97,7 +97,7 @@ mod tests {
))
);

let kzg3 = Kzg::setup(
let kzg3 = KZG::setup(
"tests/test-files/g1.point",
"tests/test-files/g2.point",
"tests/test-files/g2.point.powerOf2",
@@ -118,7 +118,7 @@ mod tests {
use rust_kzg_bn254::helpers::is_on_curve_g2;
use std::io::BufRead;

let kzg = Kzg::setup(
let kzg = KZG::setup(
"tests/test-files/g1.point",
"",
"tests/test-files/g2.point.powerOf2",
@@ -159,8 +159,8 @@ mod tests {
use rand::Rng;
let mut rng = rand::thread_rng();

let mut kzg_clone1: Kzg = KZG_3000.clone();
let mut kzg_clone2: Kzg = KZG_3000.clone();
let mut kzg_clone1: KZG = KZG_3000.clone();
let mut kzg_clone2: KZG = KZG_3000.clone();

(0..10000).for_each(|_| {
let blob_length: u64 = rand::thread_rng().gen_range(35..40000);

0 comments on commit 194b32b

Please sign in to comment.