Skip to content

Commit

Permalink
Only run tests that Kontrol's integration tests job runs (#3879)
Browse files Browse the repository at this point in the history
- consolidate `pytest` options into a bash variable
- select only the test that are run by the integration tests CI
[job](https://github.com/runtimeverification/kontrol/blob/921dfe8ddd0774265d59122d0ace19ac5889007a/.github/workflows/test-pr.yml#L119).

The deselected tests need to run in isolation, as they fail if ran
together. This has something to do with proof artifacts produced by
other tests. Ideally, we should run the extra tests too.

Co-authored-by: rv-jenkins <[email protected]>
  • Loading branch information
geo2a and rv-jenkins authored May 20, 2024
1 parent 6e57788 commit b373660
Showing 1 changed file with 7 additions and 3 deletions.
10 changes: 7 additions & 3 deletions scripts/performance-tests-kontrol.sh
Original file line number Diff line number Diff line change
Expand Up @@ -114,14 +114,18 @@ cp -r $PYTEST_TEMP_DIR/foundry/* $FOUNDRY_DIR

mkdir -p $SCRIPT_DIR/logs

feature_shell "make test-integration TEST_ARGS='--foundry-root $FOUNDRY_DIR --maxfail=0 --numprocesses=$PYTEST_PARALLEL -vv $BUG_REPORT' | tee $SCRIPT_DIR/logs/kontrol-$KONTROL_VERSION-$FEATURE_BRANCH_NAME.log"
# set test arguments and select which tests to run
QUOTE='"'
TEST_ARGS="--foundry-root $FOUNDRY_DIR --maxfail=0 --numprocesses=$PYTEST_PARALLEL -vv $BUG_REPORT -k 'not (test_kontrol_cse or test_foundry_minimize_proof or test_kontrol_end_to_end)'"

feature_shell "make test-integration TEST_ARGS=$QUOTE$TEST_ARGS$QUOTE | tee $SCRIPT_DIR/logs/kontrol-$KONTROL_VERSION-$FEATURE_BRANCH_NAME.log"
killall kore-rpc-booster || echo "no zombie processes found"

if [ -z "$BUG_REPORT" ]; then
if [ ! -e "$SCRIPT_DIR/logs/kontrol-$KONTROL_VERSION-master-$MASTER_COMMIT_SHORT.log" ]; then
# remove proofs so that they are not reused by the master shell call
rm -r $FOUNDRY_DIR/out/proofs
master_shell "make test-integration TEST_ARGS='--foundry-root $FOUNDRY_DIR --maxfail=0 --numprocesses=$PYTEST_PARALLEL -vv' | tee $SCRIPT_DIR/logs/kontrol-$KONTROL_VERSION-master-$MASTER_COMMIT_SHORT.log"
rm -rf $FOUNDRY_DIR/out/proofs
master_shell "make test-integration TEST_ARGS=$QUOTE$TEST_ARGS$QUOTE | tee $SCRIPT_DIR/logs/kontrol-$KONTROL_VERSION-master-$MASTER_COMMIT_SHORT.log"
killall kore-rpc-booster || echo "no zombie processes found"
fi

Expand Down

0 comments on commit b373660

Please sign in to comment.