Update spec for test changes, add "bench" subcommand

[paf31]

Updates ndc-spec and adds a bench subcommand for simple benchmarks of connectors built with the SDK:

Usage: ndc_hub_example bench [OPTIONS] --configuration <DIRECTORY> --snapshots-dir <DIRECTORY>

Options:
      --configuration <DIRECTORY>  [env: HASURA_CONFIGURATION_DIRECTORY=]
      --samples <COUNT>            the number of samples to collect per test [default: 100]
      --tolerance <TOLERANCE>      tolerable deviation from previous report, in standard deviations from the mean
      --snapshots-dir <DIRECTORY>  [env: HASURA_SNAPSHOTS_DIR=]
  -h, --help                       Print help

[SamirTalwar]

I think that in the future, we should consider moving the ndc-test-related stuff to a separate, optional crate.

I want this testing and benchmarking but I don't want to ship it to prod.

FWIW we don't use the test command in ndc-postgres; we wrap the test runner in an integration test.

[paf31]

We could put it behind a feature flag.

[paf31]

% cargo run
Usage: ndc_hub_example <COMMAND>

Commands:
  serve         
  check-health  
  help          Print this message or the help of the given subcommand(s)

% cargo run --features ndc-test

Usage: ndc_hub_example <COMMAND>

Commands:
  serve         
  test          
  replay        
  bench         
  check-health  
  help          Print this message or the help of the given subcommand(s)

[SamirTalwar]

Yeah, that's what I was thinking, but I'm fine with doing it later.

[SamirTalwar]

Love to see it.

[SamirTalwar]

Why is this necessary? Doesn't the ConsoleReporter handle this now?

(Same applies to the replay function.)

[paf31]

The reporter only responds to the traversal of the nodes of the test tree structure, it doesn't have a hook for when the tests are fully complete. ConsoleReporter writes the tree out to stdout without allocating, but TestResults collects up the failures. Another approach would be to add a finish method to the trait, but as it is, TestResults is useful for cases where you don't want to write to stdout at all, or immediately.

[SamirTalwar]

Let's ship this!