Feature Proposal: Add Plots and Graphs to zbench

[hendriknielaender]

Summary

This feature proposal suggests the enhancement of zbench to include the generation of useful charts and graphs similar to what is available in criterion.rs. The addition of these visual aids will provide a better understanding of the behavior of benchmarks, making it easier for developers to analyze performance metrics.

Motivation

Visual representation of benchmark results significantly enhances the ability to interpret and analyze performance data. By incorporating plots and graphs into zbench, users can quickly identify trends, anomalies, and overall performance characteristics. This improvement aligns with our goal of making zbench a more robust and user-friendly benchmarking tool.

Detailed Design

Chart Types

The following charts and graphs are proposed to be added:

https://github.com/bheisler/criterion.rs/tree/master/src/plot/gnuplot_backend

Implementation Details

• Default Backend: The primary backend for generating these charts will be gnuplot. Gnuplot is a highly capable and flexible plotting tool that is well-suited for our needs.
• Fallback Backend: In the absence of gnuplot, a zig plotting library will be used as a fallback. This ensures that the plotting functionality remains available even in environments where gnuplot is not installed.

Integration Points

• Benchmarking API: Modify the existing benchmarking API to include options for generating and displaying these charts post-benchmarking.
• Configuration Options: Provide configuration options to enable or disable specific types of charts, and to select between gnuplot and the Zig native plotting library.

Drawbacks

• Increased Dependencies: Adding gnuplot as a dependency might be seen as a drawback for environments that prefer minimal external dependencies.
• Complexity: Introducing graphical representations adds complexity to the codebase, requiring maintenance and updates to ensure compatibility with both gnuplot and the Zig native plotting library.