Skip to content

Commit

Permalink
Marchive Object Scanner (Pusher/Flusher)
Browse files Browse the repository at this point in the history
Scan MarFS DAL and write out paths for flushing or pushing.

Added config file to set up
    default file age reference time (can be overwritten in commandline)
    filesystem utilization thresholds
    object regex blacklist

An SQLite database file is placed into each leaf directory to store
paths of files that have been pushed but not flushed. Flushing them
will remove the records from the database.

Added single whitelist regexp that can be passed in for matching.
Added --force flag to force flush/push even if not eligible.
Added --leaf to process a single leaf dir instead of an entire DAL.

This is an update to flush_ls.
  • Loading branch information
calccrypto committed Nov 26, 2024
1 parent f6ebd04 commit ac01311
Show file tree
Hide file tree
Showing 9 changed files with 1,633 additions and 731 deletions.
6 changes: 3 additions & 3 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@ jobs:
cd $GITHUB_WORKSPACE
# https://doc.rust-lang.org/cargo/guide/continuous-integration.html
flush_ls:
obj_scanner:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
Expand All @@ -60,8 +60,8 @@ jobs:

- name: Build
run: cargo build --verbose
working-directory: flush_ls
working-directory: obj_scanner

- name: Test
run: cargo test --verbose
working-directory: flush_ls
working-directory: obj_scanner
Loading

0 comments on commit ac01311

Please sign in to comment.