eigerco · Rqnsom · Nov 26, 2024 · Nov 18, 2024
diff --git a/README.md b/README.md
@@ -61,6 +61,8 @@ $ cargo install --git https://github.com/eigerco/move-spec-testing.git --locked
 $ RUSTFLAGS="--cfg tokio_unstable" cargo install --git https://github.com/eigerco/move-spec-testing.git --locked move-mutation-test
 ```
 
+Note: we don't recommend using the `debug` build since this tool is very resource-intensive. For any development purposes, we recommend using the `release` mode only.
+
 That will install the tools into `~/.cargo/bin` directory (at least on MacOS and Linux).
 Ensure to have this path in your `PATH` environment. This step can be done with the below command.
 ```bash
@@ -78,6 +80,7 @@ $ cargo uninstall move-mutation-test
 
 The basic tool overview is shown in the below chapters. To dive more deeply into each tool, please check out the documentation here:
 
+ - [`How to use guide`](docs/How-to-use.md)
  - [`move-mutator` documentation](move-mutator/README.md)
  - [`move-spec-test` documentation](move-spec-test/README.md)
  - [`move-mutation-test` documentation](move-mutation-test/README.md)

diff --git a/docs/1.png b/docs/1.png
diff --git a/docs/10.png b/docs/10.png
diff --git a/docs/2.png b/docs/2.png
diff --git a/docs/3.png b/docs/3.png
diff --git a/docs/4.png b/docs/4.png
diff --git a/docs/5.png b/docs/5.png
diff --git a/docs/6.png b/docs/6.png
diff --git a/docs/7.png b/docs/7.png
diff --git a/docs/8.png b/docs/8.png
diff --git a/docs/9.png b/docs/9.png
diff --git a/docs/How-to-use.md b/docs/How-to-use.md
@@ -0,0 +1,90 @@
+# How to use mutation tools
+
+This is a quick tutorial on how the `move-mutation-test` tool should be used.
+
+The  `move-mutation-test` can be used to improve unit test test suites. A similar tool, the `move-spec-test`, can be used to enhance specification tests.
+
+_In this quick showcase below, only the `move-mutation-test` tool is presented, but the commands are the same for the `move-spec-test` tool, so it works in a similar way._
+
+## Using the tool in the `aptos-framework` project
+
+The tool mutates the original code and then reruns the tests for each mutation. Each code mutation is named the _mutant_.
+- If the test suite is passing for the mutant, it indicates the test suite should be improved, or in some rare cases - that the original code should be improved.
+- If the test suite fails for the generated mutant, that's an indication the test suite is well written.
+
+The tool is relatively slow, so the recommended way to use it is on a per-module or per-function basis.
+
+The tools are started by using the `run` subcommand. The other subcommand is called `display-report`, which can be used to parse the results after the `run` subcommand finishes.
+
+Let's try the tool in the `aptos-framework` project and let's select the `fixed_point64` module to scan the mutants. We'll use the `--coverage` flag to ensure mutated code is generated only on pieces of the code that have proper unit test coverage:
+
+```bash
+move-mutation-test run --coverage --output report.txt --mutate-modules fixed_point64
+```
+
+> [!IMPORTANT]  
+> To use the `-coverage` flag, the user first needs to run the `aptos move test --coverage` command to generate the coverage report stored locally within the project files.
+
+<p align="center"><img src="1.png" width="60%"/></p>
+
+Once the execution is done, we should see this short summary that tells us the number of alive mutants per function in the module.
+
+<p align="center"><img src="2.png" width="50%"/></p>
+
+It seems like the function `round` has nine surviving mutants. Use the following command to see the results more clearly:
+
+```bash
+move-mutation-test display-report coverage --path-to-report report.txt
+```
+
+<p align="center"><img src="3.png" width="50%"/></p>
+
+If we scroll further down, we should find the `round` function where we can see the lines with info regarding the alive and killed mutants:
+
+<p align="center"><img src="4.png" width="40%"/></p>
+
+This is a cool overview of the state of the function, but it doesn't tell us which mutants survived. For that purpose, we can use the `mutants` subcommand:
+
+```bash
+move-mutation-test display-report mutants --modules fixed_point64 --functions round
+```
+
+<p align="center"><img src="5.png" width="50%"/></p>
+
+The next step would be to check the current state of the tests and see how we can write some tests that would ensure these mutants would fail the test suite.
+
+<p align="center"><img src="6.png" width="40%"/></p>
+
+From the above, we can see that the tests that are trying to test the `round` function could be improved a lot.
+Let's try to improve these tests with the below:
+
+<p align="center"><img src="7.png" width="40%"/></p>
+
+Now, let's rerun the tool, but let's be more specific this time to make the execution shorter, let's mutate only the `round` function with the command:
+
+```bash
+move-mutation-test run --coverage --output report.txt --mutate-modules fixed_point64 --mutate-functions round
+```
+
+<p align="center"><img src="8.png" width="45%"/></p>
+
+We can already see from the summary report that the stats for this function have improved!
+
+Let's again check the coverage with the `display-report coverage` command:
+
+```bash
+move-mutation-test display-report coverage
+```
+
+<p align="center"><img src="9.png" width="40%"/></p>
+
+And that's it! We just used the tool to improve our test suite.
+
+If you want to learn more, feel free to explore the tool.
+Or even maybe try to improve the test in order to kill the remaining mutant here:
+
+```bash
+move-mutation-test display-report mutants --modules fixed_point64 --functions round
+```
+
+<p align="center"><img src="10.png" width="60%"/></p>
diff --git a/move-mutator/doc/design.md b/move-mutator/doc/design.md
@@ -102,8 +102,7 @@ Diff:
 
 It contains an API module that exposes function calls to other modules
 (internal or external). It is mainly used to expose the `run_move_mutator`
-function, which is the entry point to the Move mutator tool. It takes the
-configuration file path as an argument and returns the list of mutants.
+function, which is the entry point to the Move mutator tool.
 
 ### Main logic layer
 
@@ -147,10 +146,6 @@ mutation operator has categories like arithmetic, bitwise, and shifts. When the
 If so, it chooses an appropriate category based on the expression. ALL
 mutations within the category are applied.
 
-The above means that the operator tool can produce many mutants within one
-place in the source code for each mutation. Concrete operators or can be 
-filtered using the configuration file described in the data layer section.
-
 Once generated, mutants can be checked to see if they are valid. It's possible
 to run the Move compiler to check if the mutant is valid, as some of the
 mutations can create mutants that cannot be compiled properly.
@@ -163,46 +158,6 @@ There is additional behaviour:
 - mutator tries to return at least one mutant per file,
 - mutator tries to return at least one mutant per mutation operator.
 
-### Data layer
-
-This layer handles data sources - reading Move projects (source code) and
-configuration files. It also provides data to the other layers.
-
-The configuration file is a JSON or TOML (both supported) file that contains
-the configuration of the Move mutator tool. It includes information on the
-project to mutate, mutation operators to use, and so on.
-
-Sample configuration file:
-```json
-{
-    "project": {
-        "path": "path/to/project",
-        "files": [
-            "path/to/file1",
-            "path/to/file2"
-        ]
-    },
-    "mutation": {
-        "operators": [
-            "binary_operator_replacement",
-            "unary_operator_replacement"
-        ]
-    }
-}
-```
-
-```toml
-[project]
-move_sources = ["/path/to/move/source"]
-mutate_modules = {"Selected" = ["module1", "module2"]}      # Alternative: mutate_modules = "All"
-[mutation]
-operators = ["operator1", "operator2"]
-[[individual]]
-file = "/path/to/file"
-verify_mutants = true
-include_functions = ["function1", "function2"]
-```
-
 ### Cross layer
 
 The layer is used to provide a common function set to other layers. None of
@@ -231,7 +186,6 @@ the proving process as it is not the goal of the tool.
 The specification verification tool placed inside the `move-spec-test` tool,
 which does the following:
 1. Takes arguments both for the Move Prover tool and for the Move mutator tool.
-It can also read the configuration from the JSON configuration file.
 2. Run the Move mutator tool (`mutate`) to generate mutants with the previously
 specified parameters.
 3. Run the Move Prover tool, passing the generated mutants one by one.

diff --git a/move-mutator/src/lib.rs b/move-mutator/src/lib.rs
@@ -132,8 +132,6 @@ pub fn run_move_mutator(
 
     // If the downsample ratio is set, we need to downsample the mutants.
     if let Some(percentage) = mutator_configuration.project.downsampling_ratio_percentage {
-        //TODO: currently we are downsampling the mutants after they are generated. This is not
-        // ideal as we are generating all mutants and then removing some of them.
         let total_mutants = transformed_mutants.len();
 
         let no_of_mutants_to_keep =