From 239cb5f7edc4ea123d4208dbdbfc76fbe604fa59 Mon Sep 17 00:00:00 2001 From: Puskar Basu Date: Fri, 27 Sep 2024 11:59:35 +0530 Subject: [PATCH] remove stuff from reference --- docs/reference/cli/check.md | 312 --------- docs/reference/cli/dashboard.md | 223 ------ docs/reference/cli/mod.md | 88 --- docs/reference/cli/overview.md | 13 +- docs/reference/cli/variable.md | 48 -- .../env-vars/steampipe_introspection.md | 79 --- .../env-vars/steampipe_mod_location.md | 20 - docs/reference/mod-resources/benchmark.md | 128 ---- docs/reference/mod-resources/card.md | 288 -------- docs/reference/mod-resources/category.md | 210 ------ docs/reference/mod-resources/chart.md | 444 ------------ docs/reference/mod-resources/container.md | 64 -- docs/reference/mod-resources/control.md | 96 --- docs/reference/mod-resources/dashboard.md | 513 -------------- docs/reference/mod-resources/edge.md | 154 ----- docs/reference/mod-resources/flow.md | 576 ---------------- docs/reference/mod-resources/graph.md | 634 ------------------ docs/reference/mod-resources/hierarchy.md | 303 --------- docs/reference/mod-resources/image.md | 101 --- docs/reference/mod-resources/input.md | 396 ----------- docs/reference/mod-resources/locals.md | 49 -- docs/reference/mod-resources/mod.md | 151 ----- docs/reference/mod-resources/node.md | 173 ----- docs/reference/mod-resources/overview.md | 50 -- docs/reference/mod-resources/query.md | 307 --------- docs/reference/mod-resources/table.md | 197 ------ docs/reference/mod-resources/text.md | 60 -- docs/reference/mod-resources/variable.md | 60 -- docs/reference/mod-resources/with.md | 142 ---- docs/reference/overview.md | 1 - 30 files changed, 2 insertions(+), 5878 deletions(-) delete mode 100644 docs/reference/cli/check.md delete mode 100644 docs/reference/cli/dashboard.md delete mode 100644 docs/reference/cli/mod.md delete mode 100644 docs/reference/cli/variable.md delete mode 100644 docs/reference/env-vars/steampipe_introspection.md delete mode 100644 docs/reference/env-vars/steampipe_mod_location.md delete mode 100644 docs/reference/mod-resources/benchmark.md delete mode 100644 docs/reference/mod-resources/card.md delete mode 100644 docs/reference/mod-resources/category.md delete mode 100644 docs/reference/mod-resources/chart.md delete mode 100644 docs/reference/mod-resources/container.md delete mode 100644 docs/reference/mod-resources/control.md delete mode 100644 docs/reference/mod-resources/dashboard.md delete mode 100644 docs/reference/mod-resources/edge.md delete mode 100644 docs/reference/mod-resources/flow.md delete mode 100644 docs/reference/mod-resources/graph.md delete mode 100644 docs/reference/mod-resources/hierarchy.md delete mode 100644 docs/reference/mod-resources/image.md delete mode 100644 docs/reference/mod-resources/input.md delete mode 100644 docs/reference/mod-resources/locals.md delete mode 100644 docs/reference/mod-resources/mod.md delete mode 100644 docs/reference/mod-resources/node.md delete mode 100644 docs/reference/mod-resources/overview.md delete mode 100644 docs/reference/mod-resources/query.md delete mode 100644 docs/reference/mod-resources/table.md delete mode 100644 docs/reference/mod-resources/text.md delete mode 100644 docs/reference/mod-resources/variable.md delete mode 100644 docs/reference/mod-resources/with.md diff --git a/docs/reference/cli/check.md b/docs/reference/cli/check.md deleted file mode 100644 index bbd95d6..0000000 --- a/docs/reference/cli/check.md +++ /dev/null @@ -1,312 +0,0 @@ ---- -title: Steampipe Check -sidebar_label: steampipe check ---- - - -# steampipe check - -> ***Powerpipe is now the recommended way to run dashboards and benchmarks!*** -> Mods still work as normal in Steampipe for now, but they are deprecated and will be removed in a future release: -> - [Steampipe Unbundled →](https://steampipe.io/blog/steampipe-unbundled) -> - [Powerpipe for Steampipe users →](https://powerpipe.io/blog/migrating-from-steampipe) - - - -Execute one or more Steampipe benchmarks and controls. - -You may specify one or more benchmarks or controls to run, or run `steampipe check all` to run all controls in the workspace. - -## Usage -Run benchmarks/controls: -```bash -steampipe check [item,item,...] [flags] -``` - -List available benchmarks: -```bash -steampipe check list -``` - - -## Flags: - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Argument Description
--cloud-host Sets the Turbot Pipes host used when connecting to Turbot Pipes workspaces. DEPRECATED - Use --pipes-host.
--cloud-token Sets the Turbot Pipes authentication token used when connecting to Turbot Pipes workspaces. DEPRECATED - Use --pipes-token.
--dry-run If specified, prints the controls that would be run by the command, but does not execute them.
--export string Export control output to a file. You may export multiple output formats for a single control run by entering multiple --export arguments. If a file path is specified as an argument, its type will be inferred by the suffix. Supported export formats are asff, csv, html, json, md,nunit3, sps (snapshot) -
--header string Specify whether to include column headers in csv output/export (default true).
--help Help for steampipe check.
--input Enable/Disable interactive prompts for missing variables. To disable prompts and fail on missing variables, use --input=false. This is useful when running from scripts. (default true)
--max-parallel integer Set the maximum number of database connections to open. When running steampipe check, Steampipe will attempt to run up to this many controls in parallel. See the STEAMPIPE_MAX_PARALLEL environment variable documentation for details. (default 10)
--mod-install Specify whether to install mod dependencies before running the check (default true)
--mod-location Sets the Steampipe workspace working directory. If not specified, the workspace directory will be set to the current working directory. See STEAMPIPE_MOD_LOCATION for details.
--output string Select the console output format. Defaults to text. Possible values are brief, csv, html, json, md, sps (snapshot), text, none
--pipes-host Sets the Turbot Pipes host used when connecting to Turbot Pipes workspaces. See PIPES_HOST for details.
--pipes-token Sets the Turbot Pipes authentication token used when connecting to Turbot Pipes workspaces. See PIPES_TOKEN for details.
--progress Enable or disable progress information. By default, progress information is shown - set --progress=false to hide the progress bar.
--query-timeout int The query timeout, in seconds. The default is 240.
--search-path strings Set a comma-separated list of connections to use as a custom search path for the control run.
--search-path-prefix strings Set a comma-separated list of connections to use as a prefix to the current search path for the control run.
--separator string A single character to use as a separator string for csv output (defaults to ",")
--share Create snapshot in Turbot Pipes with anyone_with_link visibility.
--snapshot Create snapshot in Turbot Pipes with the default (workspace) visibility.
--snapshot-location string The location to write snapshots - either a local file path or a Turbot Pipes workspace
--snapshot-tag string=string Specify tags to set on the snapshot. Multiple --snapshot-tag arguments may be passed.
--snapshot-title string=string The title to give a snapshot when uploading to Turbot Pipes.
--tag string=string Filter the list of controls to run by one or more tag values. Multiple --tag arguments may be passed. Discrete keys are and'ed and duplicate keys are or'ed. For example, steampipe check all --tag pci=true --tag service=ec2 --tag service=iam will run only controls with a service tag equal to either ec2 or iam that also are tagged with pci=true. -
--theme Select output theme (color scheme, etc). Defaults to dark. Possible values are light,dark, plain
--timing=string Enable or disable query execution timing: off (default) or on
--var string=string Specify the value of a mod variable. Multiple --var arguments may be passed. -
--var-file string Specify an .spvars file containing mod variable values. -
--where Filter the list of controls to run, using a sql where clause against the steampipe_control reflection table. -
--workspace-database Sets the database that Steampipe will connect to. This can be local (the default) or a remote Turbot Pipes database. See STEAMPIPE_WORKSPACE_DATABASE for details.
- - -## Output Formats -| Format | Description -|-|- -| `asff` | [Findings](https://docs.aws.amazon.com/securityhub/latest/userguide/securityhub-findings.html) in [asff](https://docs.aws.amazon.com/securityhub/latest/userguide/securityhub-findings-format-syntax.html) json format. Only used with AWS controls. -| `brief` | Text based output that shows only actionable items (errors and alarms) as well as a summary. -| `csv` | Comma-separated output with full control details. -| `html` | Single-page HTML output with full control details and group summaries. -| `json` | Hierarchical json output with full control details and group summaries. -| `md` | Single-page markdown output with full control details and group summaries. -| `none` | Don't send any output to stdout. -| `nunit3` | Results in [nunit3](https://docs.nunit.org/articles/nunit/technical-notes/usage/Test-Result-XML-Format.html) xml format. -| `snapshot` | Steampipe snapshot json (alias for `sps`) -| `sps` | Steampipe snapshot json. -| `text` | Full text based output with details and summary. This is the default console output format. - -## Examples - -Run all controls: -```bash -steampipe check all -``` - -List the benchmarks available to run in the current mod context: - -```bash -steampipe check list -``` - -Run the cis_v130 benchmark: -```bash -steampipe check benchmark.cis_v130 -``` - -Run a benchmark and save a [snapshot](/docs/snapshots/batch-snapshots): -```bash -steampipe check --snapshot benchmark.cis_v130 -``` - -Run a benchmark and share a [snapshot](/docs/snapshots/batch-snapshots): -```bash -steampipe check --share benchmark.cis_v130 -``` - -Only show "failed" items (alarm, error) -```bash -steampipe check all --output=brief -``` - -Run all controls and pass variable values on the command line: -```bash -steampipe check all --var='mandatory_tags=["Owner","Application","Environment"]' --var='sensitive_tags=["password","key"]' -``` - -Run all controls and pass a .spvars file that contains variable values to use -```bash -steampipe check all --var-file='tags.spvars' -``` - - -Run the controls that have tags cis_level=1 and cis=true: -```bash -steampipe check all --tag cis_level=1 --tag cis=true -``` - -Preview the controls that would run in the cis_v130 benchmark with the cis_level=1 tag filter: -```bash -steampipe check benchmark.cis_v130 --tag cis_level=1 --dry-run -``` - -Run controls with the a benchmark=pci tag that are either high or critical severity: -```bash -steampipe check all --where "severity in ('critical', 'high') and tags ->> 'pci' = 'true'" -``` - -Run the cis_v130 benchmark with light mode output: -```bash -steampipe check benchmark.cis_v130 --theme=light -``` - -Run the cis_v130_1_4 and cis_v130_2_1_1 controls: -```bash -steampipe check control.cis_v130_1_4 control.cis_v130_2_1_1 -``` - -Run the foundational_security benchmark, but suppress items: -```bash -steampipe check benchmark.foundational_security --where "tags ->> 'foundational_security_item_id' != all(ARRAY['cloudformation_1','s3_11'])" -``` - -Use plain text and no progress (typical for CI or batch jobs) -```bash -steampipe check all --theme=plain --progress=false -``` - -Export to html (with default file name) -```bash -steampipe check all --export=html -``` - -Export to csv with default file name and json as `output.json` -```bash -steampipe check all --export=csv --export=output.json -``` - -Export to markdown and json with default file names, asff as `output.asff.json`, nunit3 as `output.nunit3.xml` - -```bash -steampipe check all --export=md --export=json --export=output.asff.json --export=output.nunit3.xml -``` - -Send json output to stdout and pipe to `jq ` -```bash -steampipe check all --output=json | jq -``` diff --git a/docs/reference/cli/dashboard.md b/docs/reference/cli/dashboard.md deleted file mode 100644 index 956d699..0000000 --- a/docs/reference/cli/dashboard.md +++ /dev/null @@ -1,223 +0,0 @@ ---- -title: steampipe dashboard -sidebar_label: steampipe dashboard ---- - -# steampipe dashboard - -> ***Powerpipe is now the recommended way to run dashboards and benchmarks!*** -> Mods still work as normal in Steampipe for now, but they are deprecated and will be removed in a future release: -> - [Steampipe Unbundled →](https://steampipe.io/blog/steampipe-unbundled) -> - [Powerpipe for Steampipe users →](https://powerpipe.io/blog/migrating-from-steampipe) - - -Run the Steampipe Dashboard server. - -The Dashboard loads the `mod` in the current working directory or the `--mod-location` and listens for changes to dashboards defined in the `mod`. - -## Usage -Run [Steampipe Dashboard](/docs/dashboard/overview) interactively: -```bash -steampipe dashboard [flags] -``` - -Take a snapshot or export of a single dashboard (non-interactively): -```bash -steampipe dashboard {dashboard name} [flags] -``` - -List available dashboards: -```bash -steampipe dashboard list -``` - -## Flags - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Argument Description
--browser bool Specify whether to launch the browser after starting the dashboard server (default true).
--cloud-host Sets the Turbot Pipes host used when connecting to Turbot Pipes workspaces. DEPRECATED - Use --pipes-host.
--cloud-token Sets the Turbot Pipes authentication token used when connecting to Turbot Pipes workspaces. DEPRECATED - Use --pipes-token.
--dashboard-input string=string Specify the value of a dashboard input. Multiple --dashboard-input arguments may be passed. -
--dashboard-listen string Accept connections from local (localhost only) or network (default local).
--dashboard-port int Dashboard webserver port (default 9194).
--export string Export dashboard output to a file. Supported export formats are none, sps (snapshot). -
--help Help for steampipe dashboard.
--input Enable/Disable interactive prompts for missing variables. To disable prompts and fail on missing variables, use --input=false. This is useful when running from scripts. (default true)
--max-parallel integer Set the maximum number of database connections to open. See the STEAMPIPE_MAX_PARALLEL environment variable documentation for details. (default 10).
--mod-install bool Specify whether to install mod dependencies before running the dashboard (default true).
--mod-location Sets the Steampipe workspace working directory. If not specified, the workspace directory will be set to the current working directory. See STEAMPIPE_MOD_LOCATION for details.
--output string Select the console output format. Possible values are none, sps (snapshot) (default none).
--pipes-host Sets the Turbot Pipes host used when connecting to Turbot Pipes workspaces. See PIPES_HOST for details.
--pipes-token Sets the Turbot Pipes authentication token used when connecting to Turbot Pipes workspaces. See PIPES_TOKEN for details.
--progress Enable or disable progress information. By default, progress is shown - set --progress=false to hide the progress information.
--search-path strings Set a comma-separated list of connections to use as a custom search path for the dashboard run.
--search-path-prefix strings Set a comma-separated list of connections to use as a prefix to the current search path for the dashboard run.
--share Create snapshot in Turbot Pipes with anyone_with_link visibility.
--snapshot Create snapshot in Turbot Pipes with the default (workspace) visibility.
--snapshot-location string The location to write snapshots - either a local file path or a Turbot Pipes workspace
--snapshot-tag string=string Specify tags to set on the snapshot. Multiple --snapshot-tag arguments may be passed.
--snapshot-title string=string The title to give a snapshot when uploading to Turbot Pipes.
--var string=string Specify the value of a mod variable. Multiple --var arguments may be passed. -
--var-file string Specify an .spvars file containing mod variable values. -
--workspace-database Sets the database that Steampipe will connect to. This can be local (the default) or a remote Turbot Pipes database. See STEAMPIPE_WORKSPACE_DATABASE for details.
- -### Examples - -Start the dashboard server and launch the browser to the dashboard home page: - -```bash -steampipe dashboard -``` - - -Start the dashboard server, but don't open the browser: - -```bash -steampipe dashboard --browser=false -``` - -List the dashboards available to run in the current mod context: - -```bash -steampipe dashboard list -``` - -Run a dashboard and save a [snapshot](/docs/snapshots/batch-snapshots): - -```bash -steampipe dashboard --snapshot aws_insights.dashboard.aws_account_report -``` - -Run a dashboard and share a [snapshot](/docs/snapshots/batch-snapshots): - -```bash -steampipe dashboard --share aws_insights.dashboard.aws_account_report -``` - - -Run a dashboard and save a [snapshot](/docs/snapshots/batch-snapshots), specifying inputs: - -```bash -steampipe dashboard --snapshot --dashboard-input vpc_id=vpc-9d7ae1e7 \ - aws_insights.dashboard.aws_vpc_detail -``` \ No newline at end of file diff --git a/docs/reference/cli/mod.md b/docs/reference/cli/mod.md deleted file mode 100644 index 85e97d0..0000000 --- a/docs/reference/cli/mod.md +++ /dev/null @@ -1,88 +0,0 @@ ---- -title: steampipe mod -sidebar_label: steampipe mod ---- - - -# steampipe mod -Steampipe mod management. - -Mods provide an easy way to share Steampipe queries, controls, and benchmarks. Find mods using the public registry at [hub.steampipe.io](https://hub.steampipe.io/mods). - - -## Usage -```bash -steampipe mod [command] -``` - -## Available Commands: - -| Command | Description -|-|- -| `init` | Initialize the current directory with a `mod.sp` file -| `install` | Install one or more mods and their dependencies -| `list` | List currently installed mods -| `uninstall` | Uninstall a mod and its dependencies -| `update ` | Update one or more mods and their dependencies - - -| Flag | Description -|-|- -|` --dry-run` | Show which mods would be installed/updated/uninstalled without modifying them (default `false`). -|` --mod-location` | Sets the Steampipe workspace working directory. If not specified, the workspace directory will be set to the current working directory. See STEAMPIPE_MOD_LOCATION for details. -|` --prune` | Remove unused mods and dependencies when doing `mod update` and `mod install` (default `true`). - - - -## Examples -List installed mods: -```bash -steampipe mod list -``` - -Install a mod and add the `require` statement to your `mod.sp`: -```bash -steampipe mod install github.com/turbot/steampipe-mod-aws-compliance -``` - -Install an exact version of a mod and update the `require` statement to your `mod.sp`. This may upgrade or downgrade the mod if it is already installed: -```bash -steampipe mod install github.com/turbot/steampipe-mod-aws-compliance@0.1 -``` - -Install a version of a mod using a semver constraint and update the `require` statement to your `mod.sp`. This may upgrade or downgrade the mod if it is already installed: -```bash -steampipe mod install github.com/turbot/steampipe-mod-aws-compliance@'^1' -``` - -Install all mods specified in the `mod.sp` and their dependencies: -```bash -steampipe mod install -``` - -Preview what `steampipe mod install` will do, without actually installing anything: -```bash -steampipe mod install --dry-run -``` - - -Update a mod to the latest version allowed by its current constraint: -```bash -steampipe mod update github.com/turbot/steampipe-mod-aws-compliance -``` - -Update all mods specified in the `mod.sp` and their dependencies to the latest versions that meet their constraints, and install any that are missing: -```bash -steampipe mod update -``` - - -Uninstall a mod: -```bash -steampipe mod uninstall github.com/turbot/steampipe-mod-azure-compliance -``` - -Preview uninstalling a mod, but don't uninstall it: -```bash -steampipe mod uninstall github.com/turbot/steampipe-mod-gcp-compliance --dry-run -``` diff --git a/docs/reference/cli/overview.md b/docs/reference/cli/overview.md index 36c4a69..39af924 100644 --- a/docs/reference/cli/overview.md +++ b/docs/reference/cli/overview.md @@ -9,16 +9,12 @@ sidebar_label: Steampipe CLI | Command | Description |-|- -| [steampipe check](reference/cli/check) | Run Steampipe benchmarks and controls | [steampipe completion](reference/cli/completion)| Generate the autocompletion script for the specified shell -| [steampipe dashboard](reference/cli/dashboard)| Steampipe dashboards | [steampipe help](reference/cli/help) | Help about any command -| [steampipe login](reference/cli/login) | Log in to Steampipe CLoud -| [steampipe mod](reference/cli/mod) | Steampipe mod management +| [steampipe login](reference/cli/login) | Log in to Steampipe CLoud | [steampipe plugin](reference/cli/plugin) | Steampipe plugin management | [steampipe query](reference/cli/query) | Execute SQL queries interactively or by argument | [steampipe service](reference/cli/service)| Steampipe service management -| [steampipe variable](reference/cli/variable)| Steampipe variable management ## Global Flags @@ -63,9 +59,7 @@ sidebar_label: Steampipe CLI | Value | Name | Description |---------|---------------------------------------|---------------------------------------- -| **0** | `ExitCodeSuccessful` | Steampipe ran successfully, with no runtime errors, control errors, or alarms -| **1** | `ExitCodeControlsAlarm` | `steampipe check` completed with no runtime or control errors, but there were one or more alarms -| **2** | `ExitCodeControlsError` | `steampipe check` completed with no runtime errors, but one or more control errors occurred +| **0** | `ExitCodeSuccessful` | Steampipe ran successfully, with no runtime errors | **11** | `ExitCodePluginLoadingError` | Plugin loading error | **12** | `ExitCodePluginListFailure` | Plugin listing failed | **13** | `ExitCodePluginNotFound` | Plugin not found @@ -77,12 +71,9 @@ sidebar_label: Steampipe CLI | **33** | `ExitCodeServiceStopFailure` | Service stop failed | **41** | `ExitCodeQueryExecutionFailed` | One or more queries failed for `steampipe query` | **51** | `ExitCodeLoginCloudConnectionFailed` | Connecting to cloud failed -| **61** | `ExitCodeModInitFailed` | Mod init failed -| **62** | `ExitCodeModInstallFailed` | Mod install failed | **249** | `ExitCodeInvalidExecutionEnvironment` | Steampipe was run in an unsupported environment | **250** | `ExitCodeInitializationFailed` | Initialization failed | **251** | `ExitCodeBindPortUnavailable` | Network port binding failed -| **252** | `ExitCodeNoModFile` | The command requires a mod, but no mod file was found | **253** | `ExitCodeFileSystemAccessFailure` | File system access failed | **254** | `ExitCodeInsufficientOrWrongInputs` | Runtime error - insufficient or incorrect input | **255** | `ExitCodeUnknownErrorPanic` | Runtime error - an unknown panic occurred diff --git a/docs/reference/cli/variable.md b/docs/reference/cli/variable.md deleted file mode 100644 index de894f7..0000000 --- a/docs/reference/cli/variable.md +++ /dev/null @@ -1,48 +0,0 @@ ---- -title: steampipe variable -sidebar_label: steampipe variable ---- - -# steampipe variable - -> ***Powerpipe is now the recommended way to run dashboards and benchmarks!*** -> Mods still work as normal in Steampipe for now, but they are deprecated and will be removed in a future release: -> - [Steampipe Unbundled →](https://steampipe.io/blog/steampipe-unbundled) -> - [Powerpipe for Steampipe users →](https://powerpipe.io/blog/migrating-from-steampipe) - - -Manage steampipe variables in the current mod and its direct dependents. - - -## Usage -```bash -steampipe variable [command] [flags] -``` - -## Sub-Commands - -| Command | Description -|-|- -| `list` | List variables for the the current mod and its direct dependents. - -## Flags - -| Flag | Applies to | Description -|-|-|- -| `--mod-location string` | `list` | -| `--output string` | `list` | Select a console output format: `table` or `json` (default `table`) - -## Examples - -List variables: - -```bash -steampipe variable list -``` - - -List variables in json format: - -```bash -steampipe variable list --output json -``` \ No newline at end of file diff --git a/docs/reference/env-vars/steampipe_introspection.md b/docs/reference/env-vars/steampipe_introspection.md deleted file mode 100644 index 87e9473..0000000 --- a/docs/reference/env-vars/steampipe_introspection.md +++ /dev/null @@ -1,79 +0,0 @@ ---- -title: STEAMPIPE_INTROSPECTION -sidebar_label: STEAMPIPE_INTROSPECTION ---- - -# STEAMPIPE_INTROSPECTION - -> ***Powerpipe is now the recommended way to run dashboards and benchmarks!*** -> Mods still work as normal in Steampipe for now, but they are deprecated and will be removed in a future release: -> - [Steampipe Unbundled →](https://steampipe.io/blog/steampipe-unbundled) -> - [Powerpipe for Steampipe users →](https://powerpipe.io/blog/migrating-from-steampipe) - -Steampipe can create a set of introspection tables that allow you to query the mod resources in the workspace. For performance reasons, introspection is disabled by default, however you can enable it by setting the `STEAMPIPE_INTROSPECTION` environment variable. - -Once enabled, you can query the introspection tables. For example, you can list all the benchmarks in the workspace: - -``` -> select resource_name from steampipe_benchmark order by resource_name -+----------------------+ -| resource_name | -+----------------------+ -| cis_v130 | -| cis_v130_1 | -| cis_v130_2 | -| cis_v130_2_1 | -| cis_v130_2_2 | -| cis_v130_3 | -| cis_v130_4 | -| cis_v130_5 | -| pci_v321 | -| pci_v321_autoscaling | -| pci_v321_cloudtrail | -| pci_v321_kms | -+----------------------+ -``` - - - - -When introspection is enabled, the following tables are available to query: -- `steampipe_benchmark` -- `steampipe_control` -- `steampipe_dashboard` -- `steampipe_dashboard_card` -- `steampipe_dashboard_chart` -- `steampipe_dashboard_container` -- `steampipe_dashboard_flow` -- `steampipe_dashboard_graph` -- `steampipe_dashboard_hierarchy` -- `steampipe_dashboard_image` -- `steampipe_dashboard_input` -- `steampipe_dashboard_table` -- `steampipe_dashboard_text` -- `steampipe_mod` -- `steampipe_query` -- `steampipe_reference` -- `steampipe_variable` - - - -## Usage - - -Enable introspection data - -```bash -export STEAMPIPE_INTROSPECTION=info -``` - - -Disable introspection data (the default): - -```bash -unset STEAMPIPE_INTROSPECTION - -``` \ No newline at end of file diff --git a/docs/reference/env-vars/steampipe_mod_location.md b/docs/reference/env-vars/steampipe_mod_location.md deleted file mode 100644 index 834b29f..0000000 --- a/docs/reference/env-vars/steampipe_mod_location.md +++ /dev/null @@ -1,20 +0,0 @@ ---- -title: STEAMPIPE_MOD_LOCATION -sidebar_label: STEAMPIPE_MOD_LOCATION ---- - -# STEAMPIPE_MOD_LOCATION - -> ***Powerpipe is now the recommended way to run dashboards and benchmarks!*** -> Mods still work as normal in Steampipe for now, but they are deprecated and will be removed in a future release: -> - [Steampipe Unbundled →](https://steampipe.io/blog/steampipe-unbundled) -> - [Powerpipe for Steampipe users →](https://powerpipe.io/blog/migrating-from-steampipe) - -Sets the Steampipe workspace directory. If not specified, the workspace directory will be set to the current working directory. - - -## Usage -Set your mod location: -```bash -export STEAMPIPE_MOD_LOCATION=~/my-steampipe-mods -``` \ No newline at end of file diff --git a/docs/reference/mod-resources/benchmark.md b/docs/reference/mod-resources/benchmark.md deleted file mode 100644 index 25fe2a3..0000000 --- a/docs/reference/mod-resources/benchmark.md +++ /dev/null @@ -1,128 +0,0 @@ ---- -title: benchmark -sidebar_label: benchmark ---- - -# benchmark - -> ***Powerpipe is now the recommended way to run dashboards and benchmarks!*** -> Mods still work as normal in Steampipe for now, but they are deprecated and will be removed in a future release: -> - [Steampipe Unbundled →](https://steampipe.io/blog/steampipe-unbundled) -> - [Powerpipe for Steampipe users →](https://powerpipe.io/blog/migrating-from-steampipe) - -The benchmark block provides a mechanism for grouping controls into control benchmarks, and into sections within a control benchmark. For instance, the AWS mod has separate top-level benchmarks for CIS, NIST, PCI, etc. Each of these benchmarks may have sub-level benchmarks to organize controls in a way that reflects that particular control framework - one for each section or subsection in CIS, one for each CSF Core Function and subcategory for NIST, etc. - -A `benchmark` may specify `control` or `benchmark` resources as children, enabling you to create flexible hierarchies of any depth. By default, controls will be grouped with aggregated totals at each benchmark. - -You can run benchmarks or refer to them with hcl syntax as `{mod}.benchmark.{name}`. The name must be unique in the namespace (mod). Typically, controls and benchmarks in a given benchmark should be named in a way that mimics the hierarchy in order to provide an easy to follow structure. This is a convention that should be followed, but not a strict requirement. - - -You can [run controls and benchmarks](check/overview) with the [steampipe check](reference/cli/check) command. - -You can also [view benchmarks as dashboards](dashboard/overview) with the [steampipe dashboard](reference/cli/dashboard) command. -## Example Usage - -```hcl -benchmark "cisv130" { - title = "CIS v1.3.0" - description = "The CIS Amazon Web Services Foundations Benchmark provides prescriptive guidance for configuring security options for a subset of Amazon Web Services with an emphasis on foundational, testable, and architecture agnostic settings." - documentation = file("./docs/cis-overview.md") - - children = [ - benchmark.cis_v130_1, - benchmark.cis_v130_2, - benchmark.cis_v130_3, - benchmark.cis_v130_4, - benchmark.cis_v130_5 - ] - - tags = { - cloud_provider = "aws" - framework = "cis" - cis_version = "v1.3.0" - } -} - -benchmark "cisv130_1" { - title = "1 Identity and Access Management" - documentation = file("./docs/cisv130_1.md") - - children = [ - control.cis_v130_1_1, - control.cis_v130_1_2, - control.cis_v130_1_3, - control.cis_v130_1_4, - control.cis_v130_1_5, - control.cis_v130_1_6 - ] - - tags = { - cloud_provider = "aws" - framework = "cis" - cis_version = "v1.3.0" - } -} - -control "cisv130_1_4" { - title = "1.4 Ensure no root user account access key exists (Automated)" - description = "The root user account is the most privileged user in an AWS account. AWS Access Keys provide programmatic access to a given AWS account. It is recommended that all access keys associated with the root user account be removed." - sql = query.iam_credential_report_root_user_access_key.sql - documentation = file("./docs/cisv130_1_4.md") - - - tags = { - cloud_provider = "aws" - framework = "cis" - cis_version = "v1.3.0" - cis_item_id = "1.4" - cis_control = "4.3" - cis_type = "automated" - cis_level = "1" - } -} - -``` - - -## Argument Reference -| Argument |Type | Required? | Description -|-|-|-|- -| `base` | Benchmark Reference | Optional | A reference to a named `benchmark` that this `benchmark` should source its definition from. Typically, `base` is used embed an existing benchmark in a dashboard. -| `children` | List | Optional| An ordered list of `control` and/or `benchmark` references that are members (direct descendants) of the benchmark. -| `description` | String | Optional| A description of the benchmark -| `documentation` | String (Markdown)| Optional | A markdown string containing a long form description, used as documentation for the mod on hub.steampipe.io. -| `tags` | Map | Optional | A map of key:value metadata for the benchmark, used to categorize, search, and filter. The structure is up to the mod author and varies by benchmark and provider. -| `title` | String | Optional | A display title for the benchmark -| `type` | String | Optional | When running in `steampipe dashboard`, the type of the benchmark view. Can be `benchmark` (the default) or `table`. - - -## More Examples -### Embedding an existing benchmark in a dashboard - - - -
- -```hcl -dashboard "my_dashboard" { - benchmark { - base = aws_compliance.benchmark.cis_v140_2_1 - } -} -``` - - -### Embedding a table view of a benchmark in a dashboard - - - -
- -```hcl -dashboard "my_dashboard" { - benchmark { - base = aws_compliance.benchmark.cis_v140_2_1 - type = "table" - } -} -``` \ No newline at end of file diff --git a/docs/reference/mod-resources/card.md b/docs/reference/mod-resources/card.md deleted file mode 100644 index b5ed386..0000000 --- a/docs/reference/mod-resources/card.md +++ /dev/null @@ -1,288 +0,0 @@ ---- -title: Card -sidebar_label: card ---- - -# card - -> ***Powerpipe is now the recommended way to run dashboards and benchmarks!*** -> Mods still work as normal in Steampipe for now, but they are deprecated and will be removed in a future release: -> - [Steampipe Unbundled →](https://steampipe.io/blog/steampipe-unbundled) -> - [Powerpipe for Steampipe users →](https://powerpipe.io/blog/migrating-from-steampipe) - -A card is used to show a value to the user, or some change in value. A card can also present itself in different types e.g. show me a count of public S3 buckets and if the value is greater than 0 show as an `alert`, else as `ok`. - -Cards can be declared as named resources at the top level of a mod, or be declared as anonymous blocks inside a `dashboard` or `container`, or be re-used inside a `dashboard` or `container` by using a `card` with `base = .card.`. - - - -## Example Usage - - - -
- -```hcl -card { - sql = <<-EOQ - select - count(*) as "Buckets" - from - aws_s3_bucket - EOQ - - icon = "hashtag" - width = 2 -} - -``` - - -## Argument Reference -| Argument | Type | Optional? | Description -|-|-|-|- -| `args` | Map | Optional| A map of arguments to pass to the query. -| `base` | Card Reference | Optional | A reference to a named `card` resource that this `card` should source its definition from. `label`, `title`, `value`, `type` and `width` can be overridden after sourcing via `base`. -| `icon` | String | Optional | An [icon](reference/mod-resources/dashboard#icon) to use for the elements with this category. -| `href` | String |Optional | A url that the card should link to. The `href` may use a [jq template](#jq-templates) to dynamically generate the link the card. | -| `label` | String | Optional | Inferred from the first column name in simple data format. Else can be set explicitly in HCL, or returned by the query in the `label` column in the formal data format. -| `param` | Block | Optional| A [param](reference/mod-resources/query#param) block that defines the parameters that can be passed in to the query. `param` blocks may only be specified for cards that specify the `sql` argument. -| `query` | Query Reference | Optional | A reference to a [query](reference/mod-resources/query) resource that defines the query to run. A card may either specify the `query` argument or the `sql` argument, but not both. -| `sql` | String | Optional | An SQL string to provide data for the card. A card may either specify the `query` argument or the `sql` argument, but not both. -| `title` | String | Optional | A plain text [title](/docs/reference/mod-resources/dashboard#title) to display for this card. -| `type` | String | Optional | `plain` (default), `alert`, `info` or `ok`. You can also use `table` to review the raw data. -| `value` | String | Optional | Inferred from the first column's value in simple data format. -| `width` | Number | Optional | The [width](/docs/reference/mod-resources/dashboard#width) as a number of grid units that this item should consume from its parent. - -## Data Structure - -A card supports 2 data structures. - -1. A simple structure where column 1's name is the card `label` and column 1's value is the card `value`. -2. A formal data structure where the column names map to properties of the `card`. - -Simple data structure: - -| | -|----------| -| | - -For example: - -| Unencrypted Buckets | -|---------------------| -| 25 | - - -Formal data structure: - -| label | value | type | -| ------------------- | ----- | ----- | -| Unencrypted Buckets | 10 | alert | - - -#### JQ Templates -The `href` argument allows you to specify a [jq](https://stedolan.github.io/jq/) template to dynamically generate a hyperlink from the data in the row. To use a jq template, enclose the jq in double curly braces (`{{ }}`). - -Steampipe will pass the first row of data to jq in the same format that is returned by [steampipe query json mode output](reference/dot-commands/output), where the keys are the column names and the values are the data for that row. - -For example, this query: -```sql -select - s.volume_id as value, - 'Source Volume' as label, - 'info' as type, - v.arn -from - aws_ebs_snapshot as s, - aws_ebs_volume as v -where - s.volume_id = v.volume_id - and s.snapshot_id = 'snap-0cc613495a9fe5c1c'; - -``` - -will present rows to the jq template in this format: -```json -{ - "arn": "arn:aws:ec2:us-east-2:123456789012:volume/vol-0566e02dcc2c08e77", - "label": "Source Volume", - "type": "info", - "value": "vol-0566e02dcc2c08e77" - } -``` - -which you can then use in a jq template in the `href` argument: -```hcl -card { - sql = <<-EOQ - select - s.volume_id as value, - 'Source Volume' as label, - 'info' as type, - v.arn - from - aws_ebs_snapshot as s, - aws_ebs_volume as v - where - s.volume_id = v.volume_id - and s.snapshot_id = 'snap-0cc613495a9fe5c1c'; - EOQ - - width = 3 - href = "/aws_insights.dashboard.aws_ebs_volume_detail?input.volume_arn={{.arn | @uri}}" -} -``` - -Note that for a `card`, we pass `label` , `value` , `type` or `icon` HCL attributes in the JQ context, but the columns from the SQL query will overwrite any of the statically-defined HCL attributes. - - -Refer to [JQ Escaping & Interpolation ](/docs/reference/mod-resources/dashboard#jq-escaping--interpolation) for more advanced examples. - - -## More Examples - - - -### Alert Card - - - -
- -```hcl -card { - sql = "select 0 as alert" - type = "alert" - width = 2 -} -``` - -### OK Card - - - -
- -```hcl -card { - sql = "select 0 as ok" - type = "ok" - width = 2 -} -``` - - -### Info Card - - - -
- -```hcl -card { - sql = "select 0 as info" - type = "info" - width = 2 -} -``` - - - -### Dynamic Styling via formal query data structure - - - -
- -```hcl -card { - sql = <<-EOQ - select - 'Unencrypted Buckets' as label, - count(*) as value, - case - when count(*) > 0 then 'alert' - else 'ok' - end as type - from - aws_s3_bucket - where - server_side_encryption_configuration is null; - EOQ - width = 2 -} -``` - - - -### Static data and static (external) link - - - -
- -```hcl -card { - value = "github" - label = "site" - width = 2 - href = "https://github.com" -} -``` - - -### Dynamic data and static (internal) link - - -
- -```hcl -card { - sql = <<-EOQ - select - count(*) as value, - 'Has Public Bucket Policy' as label, - case - count(*) - when 0 then 'ok' - else 'alert' - end as "type" - from - aws_s3_bucket - where - bucket_policy_is_public; - EOQ - - icon = "hashtag" - width = 2 - href = "${dashboard.aws_s3_bucket_public_access_report.url_path}" -} -``` - - -### Dynamic link with JQ template - - -
- -```hcl -card { - sql = <<-EOQ - select - s.volume_id as value, - 'Source Volume' as label, - 'info' as type, - v.arn - from - aws_ebs_snapshot as s, - aws_ebs_volume as v - where - s.volume_id = v.volume_id - and s.snapshot_id = 'snap-0cc613495a9fe5c1c'; - EOQ - - width = 3 - href = "/aws_insights.dashboard.aws_ebs_volume_detail?input.volume_arn={{.arn | @uri}}" -} -``` diff --git a/docs/reference/mod-resources/category.md b/docs/reference/mod-resources/category.md deleted file mode 100644 index ae7f517..0000000 --- a/docs/reference/mod-resources/category.md +++ /dev/null @@ -1,210 +0,0 @@ ---- -title: Category -sidebar_label: category ---- - -# category - -> ***Powerpipe is now the recommended way to run dashboards and benchmarks!*** -> Mods still work as normal in Steampipe for now, but they are deprecated and will be removed in a future release: -> - [Steampipe Unbundled →](https://steampipe.io/blog/steampipe-unbundled) -> - [Powerpipe for Steampipe users →](https://powerpipe.io/blog/migrating-from-steampipe) - -The `category` block defines a category for a graph node or edge. Categories are used to define display properties for the item, such as the color, icon, and folding options. - - -## Example Usage - -```hcl -category "ec2_instance" { - title = "EC2 Instance" - href = "/aws_insights.dashboard.ec2_instance_detail?input.instance_arn={{.properties.'ARN' | @uri}}" - icon = "memory" - color = "orange" -} -``` - -A `category` may be defined as a top-level, named resource block, or as a block inside a `graph`, `flow`, or `hierarchy`. If defined as a top-level resource, the category will be available to all graph nodes and edges, and you can set it on a node or edge using the `category` HCL property: - -```hcl - -dashboard "categories_ex1" { - graph { - node "iam_policy" { - category = category.iam_policy - - sql = <<-EOQ - select - arn as id, - name as title, - jsonb_build_object( - 'ARN', arn, - 'AWS Managed', is_aws_managed::text, - 'Attached', is_attached::text, - 'Create Date', create_date, - 'Account ID', account_id - ) as properties - from - aws_iam_policy - where - arn = 'arn:aws:iam::aws:policy/ReadOnlyAccess'; - EOQ - - } - } -} - -category "iam_policy" { - title = "IAM Policy" - color = "red" - href = "/aws_insights.dashboard.iam_policy_detail?input.policy_arn={{.properties.'ARN' | @uri}}" - icon = "policy" -} -``` - -If defined as a block inside a `graph`, `flow`, or `hierarchy`, the category will only be available to that `graph`, `flow`, or `hierarchy`. In this case, you cannot set the category with `category` HCL property, but can set it dynamically for each row in the SQL result set using the `category` column: - -```hcl -dashboard "categories_ex2" { - graph { - category "iam_policy" { - title = "IAM Policy" - color = "red" - href = "/aws_insights.dashboard.iam_policy_detail?input.policy_arn={{.properties.'ARN' | @uri}}" - icon = "policy" - } - - node "iam_policy" { - sql = <<-EOQ - select - arn as id, - name as title, - 'iam_policy' as category, - jsonb_build_object( - 'ARN', arn, - 'AWS Managed', is_aws_managed::text, - 'Attached', is_attached::text, - 'Create Date', create_date, - 'Account ID', account_id - ) as properties - from - aws_iam_policy - where - arn = 'arn:aws:iam::aws:policy/ReadOnlyAccess'; - EOQ - } - } -} -``` - -The inline `category` block is useful when you want to define a category dynamically based on the SQL result set. You can even use the `base` property in the inline category to inherit properties from a named category: - -```hcl -dashboard "categories_ex3" { - graph { - category "iam_policy" { - base = category.iam_policy - } - - node "iam_policy" { - sql = <<-EOQ - select - arn as id, - name as title, - 'iam_policy' as category, - jsonb_build_object( - 'ARN', arn, - 'AWS Managed', is_aws_managed::text, - 'Attached', is_attached::text, - 'Create Date', create_date, - 'Account ID', account_id - ) as properties - from - aws_iam_policy - where - arn = 'arn:aws:iam::aws:policy/ReadOnlyAccess'; - EOQ - } - } -} - -category "iam_policy" { - title = "IAM Policy" - color = "red" - href = "/aws_insights.dashboard.iam_policy_detail?input.policy_arn={{.properties.'ARN' | @uri}}" - icon = "policy" -} -``` - - - -## Argument Reference -| Argument | Type | Optional? | Description -|-|-|-|- -| `color` | String | The matching color from the default theme for the data series index. | A [valid color value](reference/mod-resources/dashboard#color). This may be a named color, RGB or RGBA string, or a control status color. | The color to display for this category. | -| `href` | String | Optional | A url that the item should link to. The `href` may use a [jq template](reference/mod-resources/dashboard#jq-templates) to dynamically generate the link. | -| `icon` | String | Optional | An [icon](reference/mod-resources/dashboard#icon) to use for the elements with this category. -| `title` | String | Optional | A plain text [title](/docs/reference/mod-resources/dashboard#title) to display for this category. -| `fold` | Block | Optional | A `fold` block for this category. - - - -## Folding - -The `graph` resource supports *folding*, allowing you to collapse multiple nodes that have the same category into a single node. You can click to expand or collapse the folded nodes. To be considered for folding, the nodes must have the same category and the same edges. - -By default, folding is enabled with a threshold of `3`. This means that if there are 3 or more nodes with the same category and edges, they will be folded into a single node. You can change the fold options in the `fold` block in the category definition. The `fold` block has the following properties: - -| Argument | Type | Optional? | Description -|-|-|-|- -| `icon` | String | Optional | An [icon](reference/mod-resources/dashboard#icon) to use when this category is folded. If not specified, the `category` icon will be used. -| `title` | String | Optional | A plain text [title](/docs/reference/mod-resources/dashboard#title) to display for this category. If not specified, the `category` title will be used. -| `threshold` | Number | Optional | The number of items that should be displayed before folding. The default is `3`. - - - -## More Examples - - -### Category with Material Symbol Icon - -```hcl -category "ec2_instance" { - title = "EC2 Instance" - color = "orange" - icon = "memory" -} -``` - -### Category with Heroicons Icon -```hcl -category "ec2_instance" { - title = "EC2 Instance" - color = "orange" - icon = "heroicons-outline:cpu-chip" -} -``` - -### Category with Text Icon -```hcl -category "ec2_instance" { - title = "EC2 Instance" - color = "orange" - icon = "text:Instance" -} -``` - -### Category with `fold` properties - -```hcl -category "aws_vpc_subnet" { - title = "Subnet" - icon = "lan" - color = "#FF9900" - - fold { - title = "Subnets..." - icon = "more-horiz" - threshold = 2 - } -``` \ No newline at end of file diff --git a/docs/reference/mod-resources/chart.md b/docs/reference/mod-resources/chart.md deleted file mode 100644 index da9c744..0000000 --- a/docs/reference/mod-resources/chart.md +++ /dev/null @@ -1,444 +0,0 @@ ---- -title: Chart -sidebar_label: chart ---- - -# chart - -> ***Powerpipe is now the recommended way to run dashboards and benchmarks!*** -> Mods still work as normal in Steampipe for now, but they are deprecated and will be removed in a future release: -> - [Steampipe Unbundled →](https://steampipe.io/blog/steampipe-unbundled) -> - [Powerpipe for Steampipe users →](https://powerpipe.io/blog/migrating-from-steampipe) - -A chart enables visualisation of queries in a variety of charting types such as `bar`, `column`, `donut`, `line` or `pie`. - -The chart types share key properties such as shape of the data and configuration. So, for example, if you change the type of chart from `bar` to `line` it just works. - -Chart blocks can be declared as named resources at the top level of a mod, or be declared as anonymous blocks inside a `dashboard` or `container`, or be re-used inside a `dashboard` or `container` by using a `chart` with `base = .chart.`. - - - -## Example Usage - - - - -```hcl - -chart { - type = "bar" - title = "AWS S3 Buckets by Region" - - sql = <<-EOQ - select - region as Region, - count(*) as Total - from - aws_s3_bucket - group by - region - order by - Total desc - EOQ -} - -``` - - - -## Argument Reference -| Argument | Type | Optional? | Description -|-|-|-|- -| `args` | Map | Optional| A map of arguments to pass to the query. -| `axes` | Block | Optional | See [axes](#axes). -| `base` | Chart Reference | Optional | A reference to a named `chart` resource that this `chart` should source its definition from. `title` and `width` can be overridden after sourcing via `base`. -| `grouping` | Block | Optional | The layout for multi-series charts. Can be `stack` (the default) or `compare`. -| `legend` | Block | Optional | See [legend](#legend). -| `param` | Block | Optional| A [param](reference/mod-resources/query#param) block that defines the parameters that can be passed in to the query. `param` blocks may only be specified for charts that specify the `sql` argument. -| `query` | Query Reference | Optional | A reference to a [query](reference/mod-resources/query) resource that defines the query to run. A chart may either specify the `query` argument or the `sql` argument, but not both. -| `series` | Block | Optional | A named block matching the name of the series you wish to configure. See [series](#series). -| `sql` | String | Optional | An SQL string to provide data for the chart. A chart may either specify the `query` argument or the `sql` argument, but not both. -| `title` | String | Optional | A plain text [title](/docs/reference/mod-resources/dashboard#title) to display for this chart. -| `transform` | String | Optional | See [transform](#transform). -| `type` | String | Optional | The type of the chart. Can be `bar`, `column`, `donut`, `line` or `pie`. You can also use `table` to review the raw data. -| `width` | Number | Optional | The [width](/docs/reference/mod-resources/dashboard#width) as a number of grid units that this item should consume from its parent. - - -## Data Format - -Data can be provided in 2 formats. Either in classic "Excel-like" column format, where each series data is contained in its own column: - -| X-Axis | Y-Axis Series 1 | Y-Axis Series 2 | ... | Y-Axis Series N | -| ------- | ---------------- | ---------------- | --- | ---------------- | -| Label 1 | Value 1 Series 1 | Value 1 Series 2 | ... | Value 1 Series N | -| Label 2 | Value 2 Series 1 | Value 2 Series 2 | ... | Value 2 Series N | -| ... | ... | ... | ... | ... | -| Label N | Value N Series 1 | Value 1 Series 2 | ... | Value N Series N | - -Alternatively, data can be provided with the series data in rows. - -| region | series_name | count | -|-----------|-------------|-------| -| us-east-1 | foo | 4 | -| us-east-2 | bar | 1 | -| us-west-1 | foo | 1 | -| us-west-1 | bar | 2 | - -The chart will automatically crosstab the data into the below format. See [transform](#transform): - -| region | foo | bar | -|-----------|------|------| -| us-east-1 | 4 | NULL | -| us-east-2 | NULL | 1 | -| us-west-1 | 1 | 2 | - - -## Common Chart Properties - -### axes - -Applicable to `bar`, `column`, `line` and `scatter`. - -#### `x` - -| Property | Type | Default | Values | Description | -| -------- | ---------------------------- | ------- | ------ | ----------- | -| `title` | See [axis title](#axis-title) | | | | -| `labels` | See [labels](#labels) | | | | - -#### `y` - -| Property | Type | Default | Values | Description | -| -------- | ---------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------- | ----------- | -| `title` | See [axis title](#axis-title) | | | | -| `labels` | See [labels](#labels) | | | | -| `min` | number | Determined by the range of values. For positive ranges, this will be `0`. For negative ranges, this will be scaled to the next appropriate value below the range min. | Any valid number. | | -| `max` | number | Determined by the range of values. For positive ranges, this will be scaled to the next appropriate value above the range max `0`. For negative ranges, this will be `0`. | Any valid number. | | - -### axis title - -| Property | Type | Default | Values | Description | -| --------- | ------ | -------- | ------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------- | -| `display` | string | `none` | `always` or `none` (default). | `always` will ensure the axis title is `always` shown, or `none` will never show it. | -| `align` | string | `center` | `start`, `center` (default) or `end`. | By default the chart will align the axis title in the `center` of the chart, but this can be overridden to `start` or `end` if required. | -| `value` | string | | Max 50 characters. | | - -### transform - -What data transform to apply. - -Defaults to `auto`, which will automatically crosstab row series data into column series data if it detects a 3-column dataset, with the first 2 columns non-numeric and the 3rd column numeric. - -Alternative values are `none`, which applies no data transforms, or `crosstab` which explicitly applies the crosstab transform that `auto` may apply. - -### labels - -| Property | Type | Default | Values | Description | -| --------- | ------ | ------- | ------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -| `display` | string | `auto` | `auto` (default), `always` or `none`. | `auto` will display as many labels as possible for the size of the chart. `always` will always show all labels, but will truncate them with an ellipsis as necessary. `none` will never show labels for this axis. | -| `format` | TBD | | | | - -### legend - -| Property | Type | Default | Values | Description | -| ---------- | ------ | ------- | ----------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------- | -| `display` | string | `auto` | `auto`, `all` or `none`. | `auto` will display a legend if there are multiple data series. `all` will ensure a legend is always shown, or `none` will never show a legend. | -| `position` | string | `top` | `top`, `right`, `bottom` or `left`. | By default the chart will display a legend at the `top` of the chart, but this can be overridden to `right`, `bottom` or `left` if required. | - -### series - -| Property | Type | Default | Values | Description | -| -------- | ------ | -------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------- | ----------- | -| `title` | string | The column name that the series data resides in. | Max 50 characters | | -| `color` | string | The matching color from the default theme for the data series index. |A [valid color value](reference/mod-resources/dashboard#color). This may be a named color, RGB or RGBA string, or a control status color. | | -| `point` | string | An element of a series. |A [point](reference/mod-resources/chart#point). | | - -### point - -| Property | Type | Default | Values | Description | -| -------- | ------ | -------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------- | ----------- | -| `color` | string | The matching color from the default theme for the data series index. |A [valid color value](reference/mod-resources/dashboard#color). This may be a named color, RGB or RGBA string, or a control status color. | | - - -## More Examples - - -#### Multi-series bar chart with custom properties - - - - -```hcl -chart { - type = "bar" - title = "Bucket Versioning by Region" - width = 4 - - legend { - display = "auto" - position = "top" - } - - series versioned { - title = "Versioned Buckets" - color = "green" - } - series nonversioned { - title = "Non-Versioned Buckets" - color = "red" - } - axes { - x { - title { - value = "Regions" - } - labels { - display = "auto" - } - } - y { - title { - value = "Totals" - } - labels { - display = "show" - } - min = 0 - max = 100 - } - } - sql = <<-EOQ - with versioned_buckets_by_region as ( - select - region, - count(*) as versioned - from - aws_s3_bucket - where - versioning_enabled - group by - region - ), - nonversioned_buckets_by_region as ( - select - region, - count(*) as nonversioned - from - aws_s3_bucket - where - not versioning_enabled - group by - region - ) - select - v.region, - v.versioned, - n.nonversioned - from - versioned_buckets_by_region as v - full join nonversioned_buckets_by_region n on v.region = n.region; - EOQ -} -``` - -### Bar Chart - - - -```hcl - -chart { - type = "bar" - title = "AWS S3 Buckets by Region" - - sql = <<-EOQ - select - region as Region, - count(*) as Total - from - aws_s3_bucket - group by - region - order by - Total desc - EOQ -} - -``` - -### Column Chart - - -```hcl -chart { - type = "column" - title = "AWS S3 Buckets by Region" - - sql = <<-EOQ - select - region as Region, - count(*) as Total - from - aws_s3_bucket - group by - region - order by - Total desc - EOQ -} -``` - -### Donut Chart - - - -```hcl -chart { - type = "donut" - title = "AWS S3 Buckets by Region" - - sql = <<-EOQ - select - region as Region, - count(*) as Total - from - aws_s3_bucket - group by - region - order by - Total desc - EOQ -} -``` -### Multiple donut charts - - - -
- -```hcl -chart "db_base" { - series "mentions" { - point "Citus" { - color = "green" - } - point "MongoDB" { - color = "gray" - } - point "MySQL|MariaDB" { - color = "orange" - } - point "Oracle" { - color = "red" - } - point "Postgres" { - color = "lightblue" - } - point "SQL Server" { - color = "blue" - } - point "Supabase" { - color = "yellow" - } - point "Redis" { - color = "#065E5B" - } - point "SQLite" { - color = "purple" - } - - } -``` - -### Line Chart - - -```hcl -chart { - type = "line" - title = "AWS S3 Buckets by Region" - - sql = <<-EOQ - select - region as Region, - count(*) as Total - from - aws_s3_bucket - group by - region - order by - Total desc - EOQ -} -``` - -### Pie Chart - - -```hcl -chart { - type = "pie" - title = "AWS S3 Buckets by Region" - - sql = <<-EOQ - select - region as Region, - count(*) as Total - from - aws_s3_bucket - group by - region - order by - Total desc - EOQ -} -``` - - -### Stack Chart - - -```hcl -chart { - type = "column" - title = "EBS total Storage by Region" - width = 4 - - sql = <<-EOQ - select - region, - state, - sum(size) - from - aws_ebs_volume - group by - region, - state - EOQ -} -``` - - -### Comparison Chart - - -```hcl -chart { - type = "column" - title = "EBS total Storage by Region" - grouping = "compare" - width = 4 - - sql = <<-EOQ - select - region, - state, - sum(size) - from - aws_ebs_volume - group by - region, - state - EOQ -} -``` diff --git a/docs/reference/mod-resources/container.md b/docs/reference/mod-resources/container.md deleted file mode 100644 index 50436c4..0000000 --- a/docs/reference/mod-resources/container.md +++ /dev/null @@ -1,64 +0,0 @@ ---- -title: Container -sidebar_label: container ---- - -# container - -> ***Powerpipe is now the recommended way to run dashboards and benchmarks!*** -> Mods still work as normal in Steampipe for now, but they are deprecated and will be removed in a future release: -> - [Steampipe Unbundled →](https://steampipe.io/blog/steampipe-unbundled) -> - [Powerpipe for Steampipe users →](https://powerpipe.io/blog/migrating-from-steampipe) - -A container groups and arranges related items within a dashboard. For example, you may want to group together a number of different charts for a specific AWS service. - -Containers can only be declared as anonymous blocks inside a `dashboard` or `container`. - - -## Example Usage - - - -```hcl - -container { - title = "Side by Side Container Example" - container { - width = 6 - - card { - sql = "select 1 as \"Left Side\"" - } - card { - sql = "select 2 as \"Left Side\"" - } - } - - container { - width = 6 - card { - sql = "select 1 as \"Right Side\"" - } - card { - sql = "select 2 as \"Right Side\"" - } - } -} -``` - - -## Argument Reference -| Argument | Type | Optional? | Description -|-|-|-|- -| `base` | Dashboard or Container Reference | Optional | A reference to a named `dashboard` or `container` that this `container` should source its definition from. `title` and `width` can be overridden after sourcing via `base`. -| `benchmark` | Block | Optional | [benchmark](/docs/reference/mod-resources/benchmark) blocks to embed benchmarks in the dashboard. -| `chart` | Block | Optional | [chart](/docs/reference/mod-resources/chart) blocks to visualize SQL data in a number of ways e.g. `bar`, `column`, `line`, `pie` -| `container` | Block | Optional | [container](/docs/reference/mod-resources/container) blocks to lay out related components together in a dashboard. -| `flow` | Block | Optional | [flow](/docs/reference/mod-resources/flow) blocks to visualize flows using types such as `sankey`. -| `hierarchy` | Block | Optional | [hierarchy](/docs/reference/mod-resources/hierarchy) blocks to visualize hierarchical data using types such as `tree`. -| `image` | Block | Optional | [image](/docs/reference/mod-resources/image) blocks to embed images in dashboards. Supports static URLs, or can be derived from SQL. -| `input` | Block | Optional | [input](/docs/reference/mod-resources/input) blocks to make dynamic dashboards based on user-provided input. -| `table` | Block | Optional | [table](/docs/reference/mod-resources/table) blocks to show tabular data in a dashboard. -| `text` | Block | Optional | [text](/docs/reference/mod-resources/text) blocks to add GitHub-flavoured markdown to a dashboard. -| `title` | String | Optional | A plain text [title](/docs/reference/mod-resources/dashboard#title) to display for this container. -| `width` | Number | Optional | The [width](/docs/reference/mod-resources/dashboard#width) as a number of grid units that this item should consume from its parent. diff --git a/docs/reference/mod-resources/control.md b/docs/reference/mod-resources/control.md deleted file mode 100644 index 1d2d191..0000000 --- a/docs/reference/mod-resources/control.md +++ /dev/null @@ -1,96 +0,0 @@ ---- -title: control -sidebar_label: control ---- - -# control - -> ***Powerpipe is now the recommended way to run dashboards and benchmarks!*** -> Mods still work as normal in Steampipe for now, but they are deprecated and will be removed in a future release: -> - [Steampipe Unbundled →](https://steampipe.io/blog/steampipe-unbundled) -> - [Powerpipe for Steampipe users →](https://powerpipe.io/blog/migrating-from-steampipe) - -**Controls** provide a defined structure and interface for queries that draw a specific conclusion (e.g. 'OK', 'Alarm') about each row (typically, each row represents a single evaluated 'block'). This pattern is common across multiple control benchmarks (CIS, etc) and unit testing benchmarks (JUnit, NUnit, etc). - -Controls use **queries** to gather data, but contain specific metadata and return specific columns with specified values in a specific format. - - -## Example Usage - -```hcl -control "cisv130_2_1_2" { - title = "2.1.2 Ensure S3 Bucket Policy allows HTTPS requests (Manual)" - description = "At the Amazon S3 bucket level, you can configure permissions through a bucket policy making the objects accessible only through HTTPS." - documentation = file("docs/cisv130/2_1_2.md") - sql = query.s3_bucket_encryption_in_transit_control.sql - - tags = { - cloud_provider = "aws" - framework = "cis" - cis_version = "v1.3.0" - cis_item_id = "1.4" - cis_control = "4.3" - cis_type = "automated" - cis_level = "1" - } - -} - -``` - -You may run all controls in a group: - -```bash -# run all the cis controls -steampipe check aws.benchmark.cisv130 - -# run all cis section 2 controls -steampipe check aws.benchmark.cisv130_2 - -# run all cis section 2.1 controls -steampipe check aws.benchmark.cisv130_2_1 - -``` - -## Argument Reference -| Argument | Type | Optional? | Description -|-|-|-|- -| `args` | Map | Optional| A map of arguments to pass to the query. The `args` argument may only be specified for controls that specify the `query` argument. -| `description` | String| Optional| A description of the control. -| `documentation` | String (Markdown)| Optional | A markdown string containing a long form description, used as documentation for the mod on hub.steampipe.io. -| `param` | Block | Optional| A [param](reference/mod-resources/query#param) block that defines the parameters that can be passed in to the control's query. `param` blocks may only be specified for controls that specify the `sql` argument. -| `query` | Query Reference | Optional | A reference to a [query](reference/mod-resources/query) resource that defines the control query to run. The referenced query must conform to the control interface - it must return the [required control columns](#required-control-columns). A control must either specify the `query` argument or the `sql` argument, but not both. -| `severity`| String | Optional | The potential impact of given control. Allowed values are `none`,`low`,`medium`,`high`,`critical`. The severity names are taken from the [Common Vulnerability Scoring System version 3.1: Specification Document](https://www.first.org/cvss/specification-document). This document may be used as guidance for classifying your controls. -| `sql` | String | Required | An SQL string that returns rows that conform to the control interface - it must return the [required control columns](#required-control-columns). A control must either specify the `query` argument or the `sql` argument, but not both. -| `tags` | Map | Optional | A map of key:value metadata for the benchmark, used to categorize, search, and filter. The structure is up to the mod author and varies by benchmark and provider. -| `title` | String | Optional | Display title for the control. - - - -#### Required Control Columns - -ALL controls must specify queries that return rows that contain the following standard columns: - -| Column | Type | Description -|-|-|- -| **status** | String | The control status for this row. Only the defined [control statuses](#control-statuses) should be used. -| **reason** | String | A description that details WHY the row has the specified status. The reason should always be set, even if the control is in an 'ok' state. It should be a sentence ending with a period. -| **resource** | String | A unique identifier that identifies the targeted resource for this control check. Ideally, this should be a ***globally unique identifier*** such as an arn. - - - -##### Control Statuses - -| Status | Description -|-|- -| **ok** | The control is compliant for the resource/row. -| **alarm** | The control is not compliant for the resource/row, and should be remediated. -| **skip** | This control was not evaluated because Steampipe was requested to skip it. -| **info** | This control is not automated. The control may provide instructions to manually verify, and/or may provide additional context. -| **error** | Used when an error has occurred in calculating the control state. - - - -#### Additional Control Columns & Dimensions -A control's query MUST return the required columns, but it may also return additional columns. These additional columns are referred to as `dimensions`, and can be used to specify additional provider-specific columns that are included in control reports and outputs to provide additional context. For example, a benchmark that runs controls against AWS resources may specify dimensions for `account_id` and `region` to help locate the evaluated resource. The `account_id` and `region` columns will be added as dimensions to the control output for any control whose query returns those columns. - diff --git a/docs/reference/mod-resources/dashboard.md b/docs/reference/mod-resources/dashboard.md deleted file mode 100644 index c75f2bb..0000000 --- a/docs/reference/mod-resources/dashboard.md +++ /dev/null @@ -1,513 +0,0 @@ ---- -title: Dashboard -sidebar_label: dashboard ---- - -# dashboard - -> ***Powerpipe is now the recommended way to run dashboards and benchmarks!*** -> Mods still work as normal in Steampipe for now, but they are deprecated and will be removed in a future release: -> - [Steampipe Unbundled →](https://steampipe.io/blog/steampipe-unbundled) -> - [Powerpipe for Steampipe users →](https://powerpipe.io/blog/migrating-from-steampipe) - -A dashboard is designed to be used by consumers of your mod to answer specific questions, such as "How many public AWS buckets do I have?" or "Show me the number of aging Zendesk tickets by owner". - -Dashboards can be declared as named resources at the top-level of a mod, or be nested inside another `dashboard` or `container` by using a named `dashboard` with `base = .dashboard.`. - -For layout, a dashboard consists of 12 grid units, where items inside it will consume the full 12 grid units, unless they specify an explicit [width](#width). - - - -## Example Usage - - - -```hcl - -dashboard "my_s3_dashboard" { - - title = "My S3 Dashboard" - - container { - card { - sql = <<-EOQ - select - count(*) as "Total Buckets" - from - aws_s3_bucket - EOQ - width = 2 - } - - card { - sql = <<-EOQ - select - count(*) as "Unencrypted Buckets" - from - aws_s3_bucket - where - server_side_encryption_configuration is null; - EOQ - type = "alert" - width = 2 - } - } - - - container { - title = "Analysis" - - chart { - title = "Buckets by Account" - sql = <<-EOQ - select - a.title as "account", - count(i.*) as "total" - from - aws_s3_bucket as i, - aws_account as a - where - a.account_id = i.account_id - group by - account - order by count(i.*) desc - EOQ - type = "column" - width = 6 - } - - - chart { - title = "Buckets by Region" - sql = <<-EOQ - select - region, - count(i.*) as total - from - aws_s3_bucket as i - group by - region - EOQ - type = "column" - width = 6 - } - } - -} -``` - - - -### Existing dashboard re-used with `base` -```hcl -dashboard "compose_other" { - dashboard "reused_hello_world" { - base = dashboard.hello_world - } -} -``` - - - -## Argument Reference -| Argument | Type | Optional? | Description -|-|-|-|- -| `benchmark` | Block | Optional | [benchmark](/docs/reference/mod-resources/benchmark) blocks to embed benchmarks in the dashboard. -| `chart` | Block | Optional | [chart](/docs/reference/mod-resources/chart) blocks to visualize SQL data in a number of ways e.g. `bar`, `column`, `line`, `pie`. -| `container` | Block | Optional | [container](/docs/reference/mod-resources/container) blocks to lay out related components together in a dashboard. -| `description` | String | Optional| A description of the dashboard -| `documentation` | String (Markdown)| Optional | A markdown string containing a long form description, used as documentation for the dashboard on hub.steampipe.io. -| `flow` | Block | Optional | [flow](/docs/reference/mod-resources/flow) blocks to visualize flows using types such as `sankey`. -| `hierarchy` | Block | Optional | [hierarchy](/docs/reference/mod-resources/hierarchy) blocks to visualize hierarchical data using types such as `tree`. -| `image` | Block | Optional | [image](/docs/reference/mod-resources/image) blocks to embed images in dashboards. Supports static URLs, or can be derived from SQL. -| `input` | Block | Optional | [input](/docs/reference/mod-resources/input) blocks to make dynamic dashboards based on user-provided input. -| `table` | Block | Optional | [table](/docs/reference/mod-resources/table) blocks to show tabular data in a dashboard. -| `tags` | Map | Optional | A map of key:value metadata for the dashboard, used to categorize, search, and filter. The structure is up to the mod author. -| `text` | Block | Optional | [text](/docs/reference/mod-resources/text) blocks to add GitHub-flavoured markdown to a dashboard. -| `title` | String | Optional | Plain text [title](/docs/reference/mod-resources/dashboard#title) used to display in lists, page title etc. When viewing the dashboard in a browser, will be rendered as a `h1`. -| `with` | Block | Optional| [with](/docs/reference/mod-resources/with) blocks that define prerequisite queries to run. - - -## Common Properties - -### title - -Optional `title` for an item. Provided as plain text, but will be rendered as `text` with a `type` of `markdown` using h1 (for `dashboard`), h2 (for `container`) or h3 (for any leaf nodes e.g. `chart`). This `text` block and the item it is titling will be wrapped by a `container`. - -E.g. a `container` that defines a title: - -```hcl -container { - title = "AWS S3 Metrics" - width = 6 - - # Other dashboard components -} -``` - -...is really just shorthand for: - -```hcl -container { - width = 6 - - text { - value = "## AWS S3 Metrics" - } - - container { - # Other dashboard components - } -} -``` - -A `chart` that defines a title: - -```hcl -chart { - title = "AWS S3 Buckets by Region" - width = 4 - - # Other chart options -} -``` - -...is really just shorthand for: - -```hcl -container { - width = 4 - - text { - value = "### AWS S3 Buckets by Region" - } - - chart { - # Other chart options - } -} -``` - -### width - -The number of grid units that this item should consume from its parent. - -A dashboard has 12 grid columns. By default, any dashboard component will consume the full width of its parent e.g. 12 grid columns, regardless of what `width` it specifies. As more viewport space becomes available in the browser, we provide more grid space up to and including the specified width. - -| width | width on mobile (`<768px`) | width on tablet (`768-1023px`) | width on desktop (`>=1024px`) | -|-------|----------------------------|--------------------------------|-------------------------------| -| 0 | 0 | 0 | 0 | -| 1 | 12 | 3 | 1 | -| 2 | 12 | 3 | 2 | -| 3 | 12 | 3 | 3 | -| 4 | 12 | 6 | 4 | -| 5 | 12 | 6 | 5 | -| 6 | 12 | 6 | 6 | -| 7 | 12 | 7 | 7 | -| 8 | 12 | 8 | 8 | -| 9 | 12 | 9 | 9 | -| 10 | 12 | 10 | 10 | -| 11 | 12 | 11 | 11 | -| 12 | 12 | 12 | 12 | - -For example, both of these dashboard components will consume the full width of the report at all viewport sizes, as they either implicitly or explicitly define 12 grid units: - -```hcl -dashboard "width_example" { - text { - value = "## I am implictly 12 grid units" - } - - card { - sql = query.aws_s3_bucket_unencrypted_count.sql - width = 12 # Explicitly 12 grid units - } -} -``` - -In this example we have 3 cards, each specifying a width of 4. As specified in the table above, they will consume either 12, 6 or 4 grid units according to the viewport size: - -```hcl -dashboard "width_example" { - card { - sql = query.aws_s3_bucket_public_count.sql - width = 4 # 12 on mobile, 6 on tablet and 4 on desktop - } - - card { - sql = query.aws_s3_bucket_unencrypted_count.sql - width = 4 # 12 on mobile, 6 on tablet and 4 on desktop - } - - card { - sql = query.aws_s3_bucket_unversioned_count.sql - width = 4 # 12 on mobile, 6 on tablet and 4 on desktop - } -} -``` - -In this example we have a card with a width of 2. As specified in the table above, it will consume either 12, 3 or 2 grid units according to the viewport size: - -```hcl -dashboard "width_example" { - card { - sql = query.aws_s3_bucket_public_count.sql - width = 2 # 12 on mobile, 3 on tablet and 2 on desktop - } -} -``` - -As a dashboard component always consumes the grid units of its parent, consider the following example. - -On mobiles each container will consume 12 grid units, with each card inside it also consuming 12 grid units of its parent (its respective parent container), meaning you effectively have 2 full-width cards, 1 below the other. - -On tablets and desktop each container will consume 6 grid units, with each card inside it also consuming 6 grid units of its parent (its respective parent container), meaning you have 4 cards side-by-side on the page. - -```hcl -dashboard "width_example" { - container { - width = 6 - - card { - sql = query.aws_s3_bucket_private_count.sql - width = 6 # 12 on mobile, 6 on tablet and 6 on desktop - } - - card { - sql = query.aws_s3_bucket_public_count.sql - width = 6 # 12 on mobile, 6 on tablet and 6 on desktop - } - } - - container { - width = 6 - - card { - sql = query.aws_s3_bucket_encrypted_count.sql - width = 6 # 12 on mobile, 6 on tablet and 6 on desktop - } - - card { - sql = query.aws_s3_bucket_unencrypted_count.sql - width = 6 # 12 on mobile, 6 on tablet and 6 on desktop - } - } -} -``` - - -### color - -Many dashboard elements contain a `color` argument. The color arguments support a standard set of functionality and options, and may be: - -- A [standard HTML color](https://www.w3schools.com/tags/ref_colornames.asp): `color = "green"` -- A control status value (`alert`, `info`, `ok`): `color = "alert"` -- An RGB hexadecimal color value: `color = "#AABBCC"` -- An RGB color value of the format `color = "rgb(128, 0, 128)"` -- An [RGBA color value](https://www.w3schools.com/css/css3_colors.asp#:~:text=RGBA%20color%20values%20are%20an,and%201.0%20(fully%20opaque).) of the format `color = "rgb(128, 0, 128, 0.5)"` - - -### icon - -Many dashboard elements contain an `icon` argument. The icon arguments support a standard set of functionality and options, and may be: -- A google [material symbols icon](https://fonts.google.com/icons). If no prefix is specified, the icon will be chosen from google material symbols. For example, `icon = "verified-user"` will use the `verified_user` icon from google material symbols. -- A [heroicons](https://heroicons.com/) icon. To use heroicons, prefix the icon name with `heroicons-outline:` or `heroicons-solid:`. For example, `icon = "heroicons-outline:shield-check"` will use the `shield-check` icon from heroicons outline. -- A custom icon. To use a custom icon, specify the icon URL, for example `icon = "https://steampipe.io/images/steampipe-logo.png"`. -- Text. To use text as an icon, prefix the icon with `text:`, for example `icon = "text:OK"` will display the text "OK" in place of an icon. Any unicode characters may appear, so you can even use `icon = "text:👍"`. - - -### jq Templates -Some elements ( `card`, `column` in a `table`) allow you to specify a [jq](https://stedolan.github.io/jq/) template in the `href` argument to dynamically generate a hyperlink from the data in the row. To use a jq template, enclose the jq in double curly braces (`{{ }}`). - -Steampipe will pass each row of data to jq in the same format that is returned by [steampipe query json mode output](reference/dot-commands/output), where the keys are the column names and the values are the data for that row. - -For example, this query: -```sql -select - instance_id, - region, - sg->>'GroupId' as security_group -from - aws_ec2_instance, - jsonb_array_elements(security_groups) as sg - -``` - -will present rows to the jq template in this format: -```json -{ - "instance_id": "i-03d11d111b1407bbc", - "region": "us-east-2", - "security_group": "sg-01ee40ea54e0fa089" - } -``` - -which you can then use in a jq template in the `href` argument: - -```hcl -table { - title = "Attached Security Groups" - width = 4 - sql = <<-EOQ - select - instance_id, - region, - sg->>'GroupId' as security_group - from - aws_ec2_instance, - jsonb_array_elements(security_groups) as sg - EOQ - - column "security_group" { - href = "${dashboard.aws_vpc_security_group_detail.url_path}?input.security_group_id={{.'security_group' | @uri}}" - } -} -``` - -#### JQ Escaping & Interpolation - -At a high level, templates have string components and `{{ interpolated }}` components. - -For example: - -```hcl -href = "/region/{{ .region }}" -``` - -becomes: - -```hcl -href = "/region/us-east-2" -``` - -Interpolation is fairly straightforward when column names are simple and don't contain spaces. Complex field names require escaping, however. - -Take the following example: - -```sql -select - instance_id as "Unique ID", - region, - sg->>'GroupId' as security_group -from - aws_ec2_instance, - jsonb_array_elements(security_groups) as sg -``` - -will present rows to the jq template in this format: - -```json -{ - "Unique ID": "i-03d11d111b1407bbc", - "region": "us-east-2", - "security_group": "sg-01ee40ea54e0fa089" - } -``` - -Note that the query returns a `Unique ID` field. To refer to this in a jq interpolated field, you need to quote it (jq uses double quotes): - -```hcl -/detail/{{ ."Unique ID" }} -``` - -In HCL, this is easy with HEREDOC: - -```hcl -href = < ***Powerpipe is now the recommended way to run dashboards and benchmarks!*** -> Mods still work as normal in Steampipe for now, but they are deprecated and will be removed in a future release: -> - [Steampipe Unbundled →](https://steampipe.io/blog/steampipe-unbundled) -> - [Powerpipe for Steampipe users →](https://powerpipe.io/blog/migrating-from-steampipe) - -The `edge` block represents a connection between 2 nodes (vertices) in a `graph`, `hierarchy` or `flow`. - -Anonymous `edge` blocks can be declared inside a `graph`, `hierarchy` or `flow`. They may also may be declared as named resources at the top level of a mod and referenced via `base` from other edges in a `graph`, `hierarchy` or `flow`. - - -## Example Usage - -```hcl -edge "plugin_to_version" { - title = "version" - - sql = <<-EOQ - select - name as from_id, - digest as to_id - from - steampipe_registry_plugin_version - where - name = $1 - EOQ - - param "plugin_name" {} -} -``` - - - -## Argument Reference -| Argument | Type | Optional? | Description -|-|-|-|- -| `args` | Map | Optional| A map of arguments to pass to the query. -| `base` | flow Reference | Optional | A reference to a named `edge` resource that this `edge` should source its definition from. -| `category` | Block | Optional| [category](/docs/reference/mod-resources/category) blocks that specify display options for edges with that category. -| `param` | Block | Optional| [param](reference/mod-resources/query#param) blocks that defines the parameters that can be passed in to the `sql`. `param` blocks may only be specified when the edge is defined as a top-level (mod level), named resource. -| `query` | Query Reference | Optional | A reference to a [query](reference/mod-resources/query) resource that defines the query to run. You must either specify the `query` argument or the `sql` argument, but not both. -| `sql` | String | Optional | A SQL string to provide data for the `edge`. You must either specify the `query` argument or the `sql` argument, but not both. -| `title` | String | Optional | A plain text [title](/docs/reference/mod-resources/dashboard#title) to display for this edge. - - -## Data Format -Data must be provided in a format where each row represents an *edge*. Significant columns are: - -| Name | Description -|------------|--------------------------------------------------- -| `title` | An optional title to display for the edge. -| `category` | An optional display category. This must be the name of a `category` declared in the parent `graph`, `flow`, or `hierarchy`. -| `from_id` | The `id` of the source `node` of an edge. `from_id` is required for edges -| `properties`| A jsonb key/value map of properties to display for the node/edge when the user hovers over it. The `properties` column is optional for nodes and edges. -| `to_id` | The `id` of the destination `node` of an edge. `to_id` is required for edges - -The `category` and `title` may be specified either in the SQL results or in HCL. If both are specified, the value in the SQL result set has precedence. - - - -## More Examples - -### An inline edge in a graph - -```hcl -dashboard "edge_ex_1" { - graph { - - edge { - title = "version" - - sql = <<-EOQ - select - name as from_id, - digest as to_id - from - steampipe_registry_plugin_version - where - name = 'turbot/ldap' - EOQ - } - - } -} -``` - - -### Reusable edge with `base` - -```hcl -dashboard "edge_ex_2" { - graph { - - edge { - base = edge.plugin_to_version - } - - } -} - -edge "plugin_to_version" { - title = "version" - - sql = <<-EOQ - select - name as from_id, - digest as to_id - from - steampipe_registry_plugin_version - EOQ -} - -``` - - -### Reusable edge with `base` passing args - -```hcl -dashboard "edge_ex_2" { - graph { - - edge { - base = edge.plugin_to_version - args = { - plugin_name = self.input.plugin_name.value - } - } - - } -} - -edge "plugin_to_version" { - title = "version" - - sql = <<-EOQ - select - name as from_id, - digest as to_id - from - steampipe_registry_plugin_version - where - name = $1 - EOQ - - param "plugin_name" {} -} - -``` \ No newline at end of file diff --git a/docs/reference/mod-resources/flow.md b/docs/reference/mod-resources/flow.md deleted file mode 100644 index 9600482..0000000 --- a/docs/reference/mod-resources/flow.md +++ /dev/null @@ -1,576 +0,0 @@ ---- -title: Flow -sidebar_label: flow ---- - -# flow - -> ***Powerpipe is now the recommended way to run dashboards and benchmarks!*** -> Mods still work as normal in Steampipe for now, but they are deprecated and will be removed in a future release: -> - [Steampipe Unbundled →](https://steampipe.io/blog/steampipe-unbundled) -> - [Powerpipe for Steampipe users →](https://powerpipe.io/blog/migrating-from-steampipe) - -A flow allows visualization of queries using types such as `sankey`. Flows are [node/edge visualizations](/docs/reference/mod-resources/graph#nodeedge-visualizations). The data to be displayed is specified using a series of nodes and edges. The nodes define the vertices of the graph, and the edges define the connections between them. - -Flow blocks can be declared as named resources at the top level of a mod, or can be declared as anonymous blocks inside a `dashboard` or `container`, or be re-used inside a `dashboard` or `container` by using a `flow` with `base = .flow.`. - - - -## Example Usage - - - - -```hcl -dashboard "flow_ex_node_edge" { - - input "vpc_input" { - width = 4 - - sql = <<-EOQ - select - title as label, - vpc_id as value - from - aws_vpc - EOQ - } - - flow { - title = "AWS VPC Subnets by AZ" - - node { - sql = <<-EOQ - select - vpc_id as id, - vpc_id as title - from - aws_vpc - where - vpc_id = $1 - EOQ - - args = [self.input.vpc_input.value] - } - - - node { - sql = <<-EOQ - select - distinct on (availability_zone) - availability_zone as id, - availability_zone as title - from - aws_vpc_subnet - where - vpc_id = $1 - EOQ - - args = [self.input.vpc_input.value] - } - - node { - sql = <<-EOQ - select - subnet_id as id, - subnet_id as title - from - aws_vpc_subnet - where - vpc_id = $1 - EOQ - - args = [self.input.vpc_input.value] - } - - edge { - sql = <<-EOQ - select - distinct on (availability_zone) - vpc_id as from_id, - availability_zone as to_id - from - aws_vpc_subnet - where - vpc_id = $1 - EOQ - - args = [self.input.vpc_input.value] - - } - - edge { - sql = <<-EOQ - select - availability_zone as from_id, - subnet_id as to_id - from - aws_vpc_subnet - where - vpc_id = $1 - EOQ - args = [self.input.vpc_input.value] - } - } - -} -``` - - - -## Argument Reference -| Argument | Type | Optional? | Description -|-|-|-|- -| `args` | Map | Optional| A map of arguments to pass to the query. -| `base` | flow Reference | Optional | A reference to a named `flow` resource that this `flow` should source its definition from. `title` and `width` can be overridden after sourcing via `base`. -| `category` | Block | Optional| [category](#category) blocks that specify display options for nodes with that category. -| `edge` | Block | Optional| [edge](/docs/reference/mod-resources/edge) blocks that define the edges in the flow. -| `node` | Block | Optional| [node](/docs/reference/mod-resources/node) blocks that define the nodes in the flow. -| `param` | Block | Optional| [param](reference/mod-resources/query#param) blocks that defines the parameters that can be passed in to the query. `param` blocks may only be specified for hierarchies that specify the `sql` argument. -| `query` | Query Reference | Optional | A reference to a [query](reference/mod-resources/query) resource that defines the query to run. A `flow` may either specify the `query` argument or the `sql` argument, but not both. -| `sql` | String | Optional | An SQL string to provide data for the `flow`. A `flow` may either specify the `query` argument or the `sql` argument, but not both. -| `title` | String | Optional | A plain text [title](/docs/reference/mod-resources/dashboard#title) to display for this flow. -| `type` | String | Optional | The type of the flow. Can be `sankey` or `table`. -| `width` | Number | Optional | The [width](/docs/reference/mod-resources/dashboard#width) as a number of grid units that this item should consume from its parent. -| `with` | Block | Optional| [with](/docs/reference/mod-resources/with) blocks that define prerequisite queries to run. `with` blocks may only be specified when the flow is defined as a top-level (mod level), named resource. - - - - -## Common Flow Properties - -### category - -| Property | Type | Default | Values | Description | -| -------- | ------ |----------------------------------------------------------------------| --------------------------------------------------------------------------------------------------------------------------------------- | ----------- | -| `color` | string | The matching color from the default theme for the data series index. | A [valid color value](reference/mod-resources/dashboard#color). This may be a named color, RGB or RGBA string, or a control status color. | The color to display for this category. | - - - -## Data Format -Flow data must be provided in a format where each row represents a *node* (vertex), an *edge* (connecting 2 vertices), or both. - -Note that both column *names* and their *relative position* are important in flow queries; Steampipe looks for columns *by name* in the result set, however Postgres union queries will *append the rows based on the column's position*, not the name of the column. ***All the `union` queries must return the same columns, in the same order.*** - -Significant columns are: - -| Name | Description -|------------|--------------------------------------------------------------- -| `id` | A unique identifier for the node. Nodes have an `id`, edges do not. -| `title` | A title to display for the node. -| `category` | A display category name. Both nodes and edges may specify a `category` to dictate how the item is displayed. By default, items of the same category are displayed with the same appearance (color), distinct from other categories. You can specify display options with a [category](#category) block. -| `depth` | An integer to set the position of the node. The layout of the nodes is inferred from the query, however you can force placement with the `depth` column if you need to override the default behavior. -| `from_id` | The `id` of the source side of an edge. -| `to_id` | The `id` of the destination side of an edge. - - - - -Generally speaking, there are 2 data formats commonly used for flows. If the data is hierarchical, it is often simpler to specify results where each row species a node (with an `id`, and optionally `title`, `category`, and/or `depth`) and an edge, by specifying a `from_id`: - -| from_id | id | title | category | -|---------|------------------|------------------|------------------| -| | 1 | foo | root | -| 1 | 2 | bar | widget | -| 1 | 3 | baz | widget | -| 2 | 4 | foobar | fidget | - -For flows that do not conform to a single-parent hierarchical structure, its usually easier to specify nodes and edges as separate rows. In this case, nodes will have an `id` and optionally `title`, `category`, and/or `depth`, but `to_id` and `from_id` will be null. Edges will populate `to_id` and `from_id` and optionally `category`, and will have null `id`, `depth`, and `title`: - - -| from_id | to_id | id | title | category | -|---------|-----------|------------------|------------------|------------------| -| | | 1 | foo | root | -| | | 2 | bar | widget | -| | | 3 | baz | widget | -| | | 4 | foobar | fidget | -| 1 | 2 | | | widget | -| 1 | 3 | | | widget | -| 2 | 4 | | | fidget | -| 3 | 4 | | | fidget | - - - -## More Examples - -### Sankey with color by category - - - -```hcl -dashboard "flow_ex_node_edge_category" { - - input "vpc_input" { - width = 4 - - sql = <<-EOQ - select - title as label, - vpc_id as value - from - aws_vpc - EOQ - } - - flow { - title = "AWS VPC Subnets by AZ" - - node { - category = category.aws_vpc - - sql = <<-EOQ - select - vpc_id as id, - vpc_id as title - from - aws_vpc - where - vpc_id = $1 - EOQ - - args = [self.input.vpc_input.value] - } - - - node { - category = category.aws_availability_zone - - sql = <<-EOQ - select - distinct on (availability_zone) - availability_zone as id, - availability_zone as title - from - aws_vpc_subnet - where - vpc_id = $1 - EOQ - - args = [self.input.vpc_input.value] - } - - node { - category = category.aws_vpc_subnet - - sql = <<-EOQ - select - subnet_id as id, - subnet_id as title - from - aws_vpc_subnet - where - vpc_id = $1 - EOQ - - args = [self.input.vpc_input.value] - } - - edge { - sql = <<-EOQ - select - distinct on (availability_zone) - vpc_id as from_id, - availability_zone as to_id - from - aws_vpc_subnet - where - vpc_id = $1 - EOQ - - args = [self.input.vpc_input.value] - - } - - edge { - sql = <<-EOQ - select - availability_zone as from_id, - subnet_id as to_id - from - aws_vpc_subnet - where - vpc_id = $1 - EOQ - args = [self.input.vpc_input.value] - } - } - -} - -category "aws_vpc" { - color = "orange" -} - -category "aws_availability_zone" { - color = "tan" -} - -category "aws_vpc_subnet" { - color = "green" -} -``` - -### sankey with monolithic query - - - -```hcl -flow { - type = "sankey" - title = "AWS VPC Subnets by AZ" - width = 6 - - sql = <<-EOQ - - with vpc as - (select 'vpc-9d7ae1e7' as vpc_id) - - select - null as from_id, - vpc_id as id, - vpc_id as title, - 0 as depth, - 'aws_vpc' as category - from - aws_vpc - where - vpc_id in (select vpc_id from vpc) - - union all - select - distinct on (availability_zone) - vpc_id as from_id, - availability_zone as id, - availability_zone as title, - 1 as depth, - 'aws_availability_zone' as category - from - aws_vpc_subnet - where - vpc_id in (select vpc_id from vpc) - - - union all - select - availability_zone as from_id, - subnet_id as id, - subnet_id as title, - 2 as depth, - 'aws_vpc_subnet' as category - from - aws_vpc_subnet - where - vpc_id in (select vpc_id from vpc) - - EOQ -} -``` - -### Sankey with color by category (monolithic query) - - - -```hcl -flow { - type = "sankey" - title = "AWS VPC Subnets by AZ" - width = 6 - - category "aws_vpc" { - color = "orange" - } - - category "aws_availability_zone" { - color = "tan" - } - - category "aws_vpc_subnet" { - color = "green" - } - - sql = <<-EOQ - - with vpc as - (select 'vpc-9d7ae1e7' as vpc_id) - - select - null as from_id, - vpc_id as id, - vpc_id as title, - 0 as depth, - 'aws_vpc' as category - from - aws_vpc - where - vpc_id in (select vpc_id from vpc) - - union all - select - distinct on (availability_zone) - vpc_id as from_id, - availability_zone as id, - availability_zone as title, - 1 as depth, - 'aws_availability_zone' as category - from - aws_vpc_subnet - where - vpc_id in (select vpc_id from vpc) - - - union all - select - availability_zone as from_id, - subnet_id as id, - subnet_id as title, - 2 as depth, - 'aws_vpc_subnet' as category - from - aws_vpc_subnet - where - vpc_id in (select vpc_id from vpc) - - EOQ -} - -``` - -### Sankey with node / edge data format, color by category, depth - - - - -```hcl - -flow { - type = "sankey" - title = "AWS IAM Managed Policies for User" - width = 6 - - category "direct" { - color = "alert" - } - - category "indirect" { - color = "ok" - } - - sql = <<-EOQ - with user_list as - (select 'arn:aws:iam::111111111111:user/jsmyth' as arn) - - -- User Nodes - select - arn as id, - name as title, - 0 as depth, - 'user' as category, - null as from_id, - null as to_id - from - aws_iam_user - where - arn in (select arn from user_list) - - -- Group Nodes - union select - g ->> 'Arn' as id, - g ->> 'GroupName' as title, - 1 as depth, - 'group' as category, - null as from_id, - null as to_id - from - aws_iam_user, - jsonb_array_elements(groups) as g - where - arn in (select arn from user_list) - - - -- Policy Nodes (attached to groups) - union select - p.arn as id, - p.name as title, - 2 as depth, - 'policy' as category, - null as from_id, - null as to_id - from - aws_iam_user as u, - jsonb_array_elements(groups) as g, - aws_iam_group as grp, - jsonb_array_elements_text(grp.attached_policy_arns) as pol_arn, - aws_iam_policy as p - where - g ->> 'Arn' = grp.arn - and pol_arn = p.arn - and u.arn in (select arn from user_list) - - - -- Policy Nodes (attached to user) - union select - p.arn as id, - p.name as title, - 2 as depth, - 'policy' as category, - null as from_id, - null as to_id - from - aws_iam_user as u, - jsonb_array_elements_text(attached_policy_arns) as pol_arn, - aws_iam_policy as p - where - pol_arn = p.arn - and u.arn in (select arn from user_list) - - -- User-> Group Edge - union select - null as id, - null as title, - null as depth, - 'indirect' as category, - arn as from_id, - g ->> 'Arn' as to_id - from - aws_iam_user, - jsonb_array_elements(groups) as g - where - arn in (select arn from user_list) - - -- User -> Policy Edge - union select - null as id, - null as title, - null as depth, - 'direct' as category, - arn as from_id, - pol_arn - from - aws_iam_user, - jsonb_array_elements_text(attached_policy_arns) as pol_arn - where - arn in (select arn from user_list) - - - -- Group -> Policy Edge - union select - null as id, - null as title, - null as depth, - 'indirect' as category, - grp.arn as from_id, - pol_arn - from - aws_iam_user as u, - jsonb_array_elements(groups) as g, - aws_iam_group as grp, - jsonb_array_elements_text(grp.attached_policy_arns) as pol_arn, - aws_iam_policy as p - where - g ->> 'Arn' = grp.arn - and pol_arn = p.arn - and u.arn in (select arn from user_list) - - EOQ -} -``` \ No newline at end of file diff --git a/docs/reference/mod-resources/graph.md b/docs/reference/mod-resources/graph.md deleted file mode 100644 index dac1de4..0000000 --- a/docs/reference/mod-resources/graph.md +++ /dev/null @@ -1,634 +0,0 @@ ---- -title: Graph -sidebar_label: graph ---- - -# graph - -> ***Powerpipe is now the recommended way to run dashboards and benchmarks!*** -> Mods still work as normal in Steampipe for now, but they are deprecated and will be removed in a future release: -> - [Steampipe Unbundled →](https://steampipe.io/blog/steampipe-unbundled) -> - [Powerpipe for Steampipe users →](https://powerpipe.io/blog/migrating-from-steampipe) - -A graph is a dashboard component that allows you to visualize the relationships between different resources and understand how they are connected. - -Graphs can be declared as a block inside a dashboard or container. The data to be displayed in the graph is specified using a series of nodes and edges. The nodes define the vertices of the graph, and the edges define the connections between them. - -## Example Usage - - - -```hcl -dashboard "tables" { - - graph { - title = "Schemas & Tables" - - node { - category = category.catalog - - sql = <<-EOQ - select - distinct on (catalog_name) - concat('catalog:',catalog_name) as id, - catalog_name as title - from - information_schema.schemata - where - schema_name = 'net' - EOQ - } - - node { - category = category.schema - sql = <<-EOQ - select - concat('schema:',schema_name) as id, - schema_name as title, - json_build_object( - 'catalog', catalog_name, - 'schema', schema_name, - 'owner', schema_owner - ) as properties - from - information_schema.schemata - where - schema_name = 'net' - EOQ - } - - node { - category = category.table - sql = <<-EOQ - select - concat('table:',table_name) as id, - table_name as title, - json_build_object( - 'catalog', table_catalog, - 'schema', table_schema, - 'type', table_type - ) as properties - from - information_schema.tables - where - table_schema = 'net' - EOQ - } - - edge { - sql = <<-EOQ - select - concat('catalog:',catalog_name) as from_id, - concat('schema:',schema_name) as to_id - from - information_schema.schemata - EOQ - } - - edge { - sql = <<-EOQ - select - concat('schema:',table_schema) as from_id, - concat('table:',table_name) as to_id - from - information_schema.tables - EOQ - } - } -} - -category "catalog" { - title = "catalog" - icon = "library-books" - color = "orange" -} - -category "schema" { - title = "schema" - icon = "schema" - color = "blue" -} - -category "table" { - title = "table" - icon = "table" - color = "green" -} -``` - -## Argument Reference - -| Argument | Type | Optional? | Description -|-------------|--------|-----------|------------ -| `args` | Map | Optional| A map of arguments to pass to the query. -| `base` | Graph Reference | Optional | A reference to a named `graph` resource that this `graph` should source its definition from. -| `category` | Block | Optional| [category](/docs/reference/mod-resources/category) blocks that specify display options for nodes and edges with that category. -| `direction` | String | Optional | The direction of the graph layout. Valid options are `left_right` and `top_down`. The default is `top_down`. -| `edge` | Block | Optional| [edge](/docs/reference/mod-resources/edge) blocks that define the edges in the graph. -| `node` | Block | Optional| [node](/docs/reference/mod-resources/node) blocks that define the nodes in the graph. -| `param` | Block | Optional| A [param](reference/mod-resources/query#param) block that defines the parameters that can be passed in to the graph. You can only specify `param` blocks when the graph is defined as a top-level, named resource. -| `query` | Query Reference | Optional | A reference to a [query](reference/mod-resources/query) resource that defines the query to run. A graph may either specify the `query` argument or the `sql` argument, but not both. -| `sql` | String | Optional | A SQL string to provide data for the graph. A graph may either specify the `query` argument or the `sql` argument, but not both. -| `title` | String | Optional | The title to display above the graph. -| `type` | String | Optional | The type of graph to display. Currently, only `graph` is supported. The default is `graph`. -| `width` | Number | Optional | The [width](/docs/reference/mod-resources/dashboard#width) as a number of grid units that this item should consume from its parent. -| `with` | Block | Optional| [with](/docs/reference/mod-resources/with) blocks that define prerequisite queries to run. `with` blocks may only be specified when the graph is defined as a top-level (mod level), named resource. - - - ----- - - -## Node/Edge Visualizations - -Some steampipe dashboard elements can include `node` and `edge` blocks. These elements are sometimes referred to as **node/edge visualizations** and include `graph`, `flow`, and `hierarchy`. These resources essentially implement the same interface: - - They support `node` and `edge` blocks as children - - They are also [query-based resources](/docs/reference/mod-resources/query#query-based-resources), and support using the `sql` and `query` arguments instead of `node` and `edge` blocks - - They must appear in a dashboard to be displayed, but may be defined as top level resources and referenced with `base` - - They support `param` and `with` blocks, but only when in a top level resource - - They support `category` blocks - - They have similar data formats - - -### Monolithic Query v/s Node & Edge Blocks - -Node/edge visualizations allow you to specify a monolithic query that returns a row for each node and edge, or you can specify `node` and `edge` blocks to define the nodes and edges separately. - -In either case, the concept of *nodes* and *edges* is the same. *Nodes* and *edges* represent points and connections in a `graph`, `hierarchy` or `flow`. A *node* is a vertex in the diagram, whereas as an *edge* is a relationship between 2 nodes (usually represented with a line connecting them). - -Key differences between nodes and edges are: -- A `node` MUST have an `id`. An `edge` CANNOT have an `id`. -- An `edge` must have a `from_id` and a `to_id`. A `node` CANNOT have a `to_id`. Nodes USUALLY do not have a `from_id` either, however for simple single-parent hierarchies it is often simpler to create a simple edge by specifying `from_id` on the `node` instead of creating separate node and edge blocks / rows. - - -#### Node & Edge Blocks -Typically, it is preferable to specify `node` and `edge` blocks than to use the monolithic query format: -- Using `node` and `edge` results in simpler, more readable, maintainable configuration. -- Developing large union queries is difficult. Errors are hard to find and fix. The `node` and `edge` model provides smaller, simpler queries that can be run and tested independently. -- You can reuse nodes and edges in multiple node/edge visualizations. -- Steampipe can run the node/edge queries in parallel, and can provide better status information while the visualization is loading. - -In the node / edge model, your `graph`, `flow`, or `hierarchy` block will not specify the `sql` or `query` argument, but instead will contain one or more `node` blocks and `edge` blocks. The `node` and `edge` blocks will specify the `sql` or `query` argument to retrieve the data for the node or edge. - -The sql column names are identical to the monolithic query format. Note that some fields (`category`, `title`) may be specified in HCL *or* in the query results. When *both* are specified, the SQL value takes precedence. - - -##### Example: Graph with node/edge blocks - -```hcl -dashboard "plugin_versions" { - - graph { - title = "LDAP Plugin Versions" - - node { - category = category.plugin - - sql = <<-EOQ - select - name as id, - name as title, - jsonb_build_object( - 'name', name, - 'created', create_time, - 'updated', update_time - ) as properties - from - steampipe_registry_plugin - where - name = 'turbot/ldap' - EOQ - } - - node { - category = category.plugin_version - - sql = <<-EOQ - select - digest as id, - left(split_part(digest,':',2),12) as title, - jsonb_build_object( - 'digest', digest, - 'created', create_time, - 'updated', update_time - ) as properties - from - steampipe_registry_plugin_version - where - name = 'turbot/ldap' - EOQ - } - - node { - category = category.plugin_tag - - sql = <<-EOQ - select - concat(digest,':',tag) as id, - tag as title - from - steampipe_registry_plugin_version, - jsonb_array_elements(tags) as tag - where - name = 'turbot/ldap' - EOQ - } - - edge { - title = "version" - - sql = <<-EOQ - select - name as from_id, - digest as to_id - from - steampipe_registry_plugin_version - where - name = 'turbot/ldap' - EOQ - } - - - edge { - title = "tag" - - sql = <<-EOQ - select - digest as from_id, - concat(digest,':',tag) as to_id - from - steampipe_registry_plugin_version, - jsonb_array_elements(tags) as tag - where - name = 'turbot/ldap' - EOQ - } - - } -} - - -category "plugin" { - title = "plugin" - icon = "extension" - color = "darkred" -} - -category "plugin_version" { - title = "version" - icon = "difference" - color = "darkred" -} - -category "plugin_tag" { - title = "tag" - icon = "sell" - color = "black" -} -``` - -Including the full node and edge block definitions within the graph can become unwieldy as the graph becomes more complex. It is often preferable to define the node and edge blocks as top-level resources and include them in the graph with the `base` argument. This allows you to reuse the nodes and edges in multiple node/edge visualizations. You can even define parameters in the nodes and edges, and pass arguments from the graph. - -```hcl - -dashboard "plugin_versions_example" { - - input "plugin_name" { - query = query.plugin_input - width = 4 - } - - graph { - title = "Plugin Versions" - - node { - base = node.plugin - args = { - plugin_name = self.input.plugin_name.value - } - } - - node { - base = node.plugin_version - args = { - plugin_name = self.input.plugin_name.value - } - } - - node { - base = node.plugin_tag - args = { - plugin_name = self.input.plugin_name.value - } - } - - edge { - base = edge.plugin_to_version - args = { - plugin_name = self.input.plugin_name.value - } - } - - edge { - base = edge.version_to_tag - args = { - plugin_name = self.input.plugin_name.value - } - } - - } -} - -query "plugin_input" { - sql = <<-EOQ - select - name as value, - name as label - from - steampipe_registry_plugin - EOQ -} - -node "plugin"{ - category = category.plugin - - sql = <<-EOQ - select - name as id, - name as title, - json_build_object( - 'name', name, - 'created', create_time, - 'updated', update_time - ) as properties - from - steampipe_registry_plugin - where - name = $1 - EOQ - - param "plugin_name" {} -} - -node "plugin_version" { - category = category.plugin_version - - sql = <<-EOQ - select - digest as id, - left(split_part(digest,':',2),12) as title, - json_build_object( - 'digest', digest, - 'created', create_time, - 'updated', update_time - ) as properties - from - steampipe_registry_plugin_version - where - name = $1 - EOQ - - param "plugin_name" {} -} - -node "plugin_tag" { - category = category.plugin_tag - - sql = <<-EOQ - select - concat(digest,':',tag) as id, - tag as title - from - steampipe_registry_plugin_version, - jsonb_array_elements(tags) as tag - where - name = $1 - EOQ - - param "plugin_name" {} -} - -edge "plugin_to_version" { - title = "version" - - sql = <<-EOQ - select - name as from_id, - digest as to_id - from - steampipe_registry_plugin_version - where - name = $1 - EOQ - - param "plugin_name" {} -} - - -edge "version_to_tag" { - title = "tag" - - sql = <<-EOQ - select - digest as from_id, - concat(digest,':',tag) as to_id - from - steampipe_registry_plugin_version, - jsonb_array_elements(tags) as tag - where - name = $1 - EOQ - - param "plugin_name" {} -} - -category "plugin" { - title = "plugin" - icon = "extension" - color = "darkred" -} - -category "plugin_version" { - title = "version" - icon = "difference" - color = "darkred" -} - -category "plugin_tag" { - title = "tag" - icon = "sell" - color = "black" -} -``` - - - -#### Monolithic query - -Node/edge visualizations are also query-based resources, and support using either the `sql` or `query` argument (but not both). When using a monolithic query, the query must return a row for each node and edge. Note that using a single query is an older format - generally it is simpler to use `node` and `edge` blocks instead. - - -Significant columns are: - -| Name | Applies To | Description -|------------|------------|--------------------------------------------------- -| `id` | node | A unique identifier for the node. Nodes have an `id`, edges do not. `id` is required for nodes. -| `title` | node, edge | A title to display for the node. -| `category` | node, edge | A display category. This can be a `category` block or a reference to a named `category`. -| `depth` | node (`flow` only) | An integer to set the position of the node in a flow. The layout of the nodes is inferred from the query, however you can force placement with the `depth` column if you need to override the default behavior. -| `from_id` | node, edge | The `id` of the source side of an edge. `from_id` is required for edges, optional for nodes. -| `properties`| node, edge (graph only) | A jsonb key/value map of properties to display for the node/edge when the user hovers over it. The `properties` column is optional for nodes and edges. -| `to_id` | edge | The `id` of the destination side of an edge. `to_id` is required for edges. - -Typically, the monolithic query will be a large `union` query. Note that both column *names* and their *relative position* are important! Steampipe looks for columns *by name* in the result set, however Postgres union queries will *append the rows based on the column's position*, not the name of the column. ***All the `union` queries must return the same columns, in the same order.*** - - -Most commonly, you should specify nodes and edges as separate rows. In this case, nodes will have an `id` and optionally `title`, `category`, and/or `depth`, but `to_id` and `from_id` will be null. Edges will populate `to_id` and `from_id` and optionally `category`, and will have null `id`, `depth`, and `title`: - - -| from_id | to_id | id | title | category | -|---------|-----------|------------------|------------------|------------------| -| | | 1 | foo | root | -| | | 2 | bar | widget | -| | | 3 | baz | widget | -| | | 4 | foobar | fidget | -| 1 | 2 | | | widget | -| 1 | 3 | | | widget | -| 2 | 4 | | | fidget | -| 3 | 4 | | | fidget | - - -If the data is strictly hierarchical (where each node can only have a single parent, as for a `hierarchy`), it may be simpler to format the results such that each row species a node (with an `id`, and optionally `title`, `category`, and/or `depth`) and an edge, by specifying a `from_id`: - -| from_id | id | title | category | -|---------|------------------|------------------|------------------| -| | 1 | foo | root | -| 1 | 2 | bar | widget | -| 1 | 3 | baz | widget | -| 2 | 4 | foobar | fidget | - - - -##### Example: Graph with monolithic query - - -```hcl -dashboard "plugin_versions_mono" { - - graph { - title = "LDAP Plugin Versions" - - category "plugin" { - title = "plugin" - icon = "extension" - color = "darkred" - } - - category "plugin_version" { - title = "version" - icon = "difference" - color = "darkred" - } - - category "plugin_tag" { - title = "tag" - icon = "sell" - color = "black" - } - - sql = <<-EOQ - -- plugin nodes - select - name as id, - null as from_id, - null as to_id, - name as title, - 'plugin' as category, - jsonb_build_object( - 'name', name, - 'created', create_time, - 'updated', update_time - ) as properties - from - steampipe_registry_plugin - where - name = 'turbot/ldap' - - -- plugin version nodes - union all - select - digest as id, - null as from_id, - null as to_id, - left(split_part(digest,':',2),12) as title, - 'plugin_version' as category, - jsonb_build_object( - 'digest', digest, - 'created', create_time, - 'updated', update_time - ) as properties - from - steampipe_registry_plugin_version - where - name = 'turbot/ldap' - - -- plugin tag nodes - union all - select - concat(digest,':',tag) as id, - null as from_id, - null as to_id, - tag as title, - 'plugin_tag' as category, - null as properties - from - steampipe_registry_plugin_version, - jsonb_array_elements_text(tags) as tag - where - name = 'turbot/ldap' - - -- plugin version edges - union all - select - null as id, - name as from_id, - digest as to_id, - 'version' as title, - null as category, - null as properties - from - steampipe_registry_plugin_version - where - name = 'turbot/ldap' - - -- plugin tag edges - union all - select - null as id, - digest as from_id, - concat(digest,':',tag) as to_id, - 'tag' as title, - null as category, - null as properties - from - steampipe_registry_plugin_version, - jsonb_array_elements_text(tags) as tag - where - name = 'turbot/ldap' - EOQ - } -} -``` - - -### Categories - -Node/Edge visualizations allow you to specify a [category](/docs/reference/mod-resources/category) for each node and edge. Categories are used to define display properties such as color, title, and icon to provide a consistent look and feel across panels and dashboards. - -Categories may be defined either at the mod level or in a `graph`, `flow`, or `hierarchy`. When using `node` and `edge` blocks, it is typically preferable to define the categories as top-level, named mod resources. This allows you to reference them via the `category` HCL argument in a node or edge. When specifying a `category` column in SQL, such as when using the monolithic query approach, you will need to define the category in the graph, flow, or hierarchy. - -### With blocks - -Node/Edge visualizations support [`with` blocks](/docs/reference/mod-resources/with). Similar to a `with` clause in a postgres CTE, the `with` block allows you to specify additional queries or SQL statements to run **first**, and then pass the query results as arguments to `sql`, `query`, and `node` & `edge` blocks. - -You can only specify `with` blocks on **top-level named resources** in your mod. The results of the `with` query can be referenced only within the resource in which it is defined (including any sub-blocks). diff --git a/docs/reference/mod-resources/hierarchy.md b/docs/reference/mod-resources/hierarchy.md deleted file mode 100644 index ff5f523..0000000 --- a/docs/reference/mod-resources/hierarchy.md +++ /dev/null @@ -1,303 +0,0 @@ ---- -title: Hierarchy -sidebar_label: hierarchy ---- - -# hierarchy - -> ***Powerpipe is now the recommended way to run dashboards and benchmarks!*** -> Mods still work as normal in Steampipe for now, but they are deprecated and will be removed in a future release: -> - [Steampipe Unbundled →](https://steampipe.io/blog/steampipe-unbundled) -> - [Powerpipe for Steampipe users →](https://powerpipe.io/blog/migrating-from-steampipe) - -A hierarchy allows visualization of queries using types such as `tree`. Hierarchies are [node/edge visualizations](/docs/reference/mod-resources/graph#nodeedge-visualizations). The data to be displayed is specified using a series of nodes and edges. The nodes define the vertices of the graph, and the edges define the connections between them. - -Hierarchy blocks can be declared as named resources at the top level of a mod, or be declared as anonymous blocks inside a `dashboard` or `container`, or be re-used inside a `dashboard` or `container` by using a `hierarchy` with `base = .hierarchy.`. - - -## Example Usage - - - -```hcl -dashboard "tree_ex_nodeonly" { - - input "vpc" { - width = 4 - - sql = <<-EOQ - select - title as label, - vpc_id as value - from - aws_vpc - EOQ - } - - hierarchy { - title = "AWS VPC Subnets by AZ" - - node { - sql = <<-EOQ - select - vpc_id as id, - vpc_id as title - from - aws_vpc - where - vpc_id = $1 - EOQ - args = [self.input.vpc.value] - } - - - node { - sql = <<-EOQ - select - distinct on (availability_zone) - vpc_id as from_id, - availability_zone as id, - availability_zone as title - from - aws_vpc_subnet - where - vpc_id = $1 - EOQ - args = [self.input.vpc.value] - - } - - node { - sql = <<-EOQ - select - availability_zone as from_id, - subnet_id as id, - subnet_id as title - from - aws_vpc_subnet - where - vpc_id = $1 - EOQ - args = [self.input.vpc.value] - } - - } - -} -``` - - -## Argument Reference -| Argument | Type | Optional? | Description -|-|-|-|- -| `args` | Map | Optional| A map of arguments to pass to the query. -| `base` | Hierarchy Reference | Optional | A reference to a named `hierarchy` resource that this `hierarchy` should source its definition from. `title` and `width` can be overridden after sourcing via `base`. -| `category` | Block | Optional| [category](#category) blocks that specify display options for nodes with that category. -| `edge` | Block | Optional| [edge](/docs/reference/mod-resources/edge) blocks that define the edges in the hierarchy. -| `node` | Block | Optional| [node](/docs/reference/mod-resources/node) blocks that define the nodes in the hierarchy. -| `param` | Block | Optional| [param](reference/mod-resources/query#param) blocks that defines the parameters that can be passed in to the query. `param` blocks may only be specified for hierarchies that specify the `sql` argument. -| `query` | Query Reference | Optional | A reference to a [query](reference/mod-resources/query) resource that defines the query to run. A `hierarchy` may either specify the `query` argument or the `sql` argument, but not both. -| `sql` | String | Optional | An SQL string to provide data for the `hierarchy`. A `hierarchy` may either specify the `query` argument or the `sql` argument, but not both. -| `title` | String | Optional | A plain text [title](/docs/reference/mod-resources/dashboard#title) to display for this hierarchy. -| `type` | String | Optional | The type of the hierarchy. Can be `tree` or `table`. -| `width` | Number | Optional | The [width](/docs/reference/mod-resources/dashboard#width) as a number of grid units that this item should consume from its parent. -| `with` | Block | Optional| [with](/docs/reference/mod-resources/with) blocks that define prerequisite queries to run. `with` blocks may only be specified when the hierarchy is defined as a top-level (mod level), named resource. - - - - -## Common Hierarchy Properties - -### category - -| Property | Type | Default | Values | Description | -| -------- | ------ |----------------------------------------------------------------------| --------------------------------------------------------------------------------------------------------------------------------------- | ----------- | -| `color` | string | The matching color from the default theme for the data series index. | A [valid color value](reference/mod-resources/dashboard#color). This may be a named color, RGB or RGBA string, or a control status color. | The color to display for this category. | - -## Data Format - - -## Data Format -Hierarchy data must be provided in a format where each row represents a *node* (vertex), an *edge* (connecting 2 vertices), or both. Hierarchy queries have the same basic structure as flow queries, but unlike flows hierarchies are restricted to a strict single parent structure; a given node may have only a single `from_id`. - -Note that both column *names* and their *relative position* are important in hierarchy queries; Steampipe looks for columns *by name* in the result set, however Postgres union queries will *append the rows based on the column's position*, not the name of the column. ***All the `union` queries must return the same columns, in the same order.*** - -Significant columns are: - -| Name | Description -|------------|--------------------------------------------------------------- -| `id` | A unique identifier for the node. Nodes have an `id`, edges do not. -| `title` | A title to display for the node. -| `category` | A display category name. Both nodes and edges may specify a `category` to dictate how the item is displayed. By default, items of the same category are displayed with the same appearance (color), distinct from other categories. You can specify display options with a [category](#category) block. -| `from_id` | The `id` of the source side of an edge. -| `to_id` | The `id` of the destination side of an edge. - - - - -Generally speaking, there are 2 data formats commonly used for hierarchies. It is usually simplest to specify results where each row species a node (with an `id`, and optionally `title`, `category`, and/or `depth`) and an edge, by specifying a `from_id`: - -| from_id | id | title | category | -|---------|------------------|------------------|------------------| -| | 1 | foo | root | -| 1 | 2 | bar | widget | -| 1 | 3 | baz | widget | -| 2 | 4 | foobar | fidget | - -Alternately, you may specify nodes and edges as separate rows. In this case, nodes will have an `id` and optionally `title`, `category`, and/or `depth`, but `to_id` and `from_id` will be null. Edges will populate `to_id` and `from_id` and optionally `category`, and will have null `id`, `depth`, and `title`: - - -| from_id | to_id | id | title | category | -|---------|-----------|------------------|------------------|------------------| -| | | 1 | foo | root | -| | | 2 | bar | widget | -| | | 3 | baz | widget | -| | | 4 | foobar | fidget | -| 1 | 2 | | | widget | -| 1 | 3 | | | widget | -| 2 | 4 | | | fidget | - - -## Common Hierarchy Properties - -### category - -| Property | Type | Default | Values | Description | -| -------- | ------ |----------------------------------------------------------------------| --------------------------------------------------------------------------------------------------------------------------------------- | ----------- | -| `color` | string | The matching color from the default theme for the data series index. | A [valid color value](reference/mod-resources/dashboard#color). This may be a named color, RGB or RGBA string, or a control status color. | The color to display for this category. | - - -## More Examples - - - -### Tree via monolithic query - - - - - -```hcl -hierarchy { - type = "tree" - title = "AWS VPC Subnets by AZ" - width = 6 - - sql = <<-EOQ - - with vpc as - (select 'vpc-9d7ae1e7' as vpc_id) - - select - null as from_id, - vpc_id as id, - vpc_id as title, - 0 as depth, - 'aws_vpc' as category - from - aws_vpc - where - vpc_id in (select vpc_id from vpc) - - union all - select - distinct on (availability_zone) - vpc_id as from_id, - availability_zone as id, - availability_zone as title, - 1 as depth, - 'aws_availability_zone' as category - from - aws_vpc_subnet - where - vpc_id in (select vpc_id from vpc) - - - union all - select - availability_zone as from_id, - subnet_id as id, - subnet_id as title, - 2 as depth, - 'aws_vpc_subnet' as category - from - aws_vpc_subnet - where - vpc_id in (select vpc_id from vpc) - - EOQ -} -``` - - -### Tree with color by category [monolithic] - - - -```hcl - - -hierarchy { - type = "tree" - title = "AWS VPC Subnets by AZ" - width = 6 - - category "aws_vpc" { - color = "blue" - } - - category "aws_availability_zone" { - color = "red" - } - - category "aws_vpc_subnet" { - color = "green" - } - - sql = <<-EOQ - - with vpc as - (select 'vpc-9d7ae1e7' as vpc_id) - - select - null as from_id, - vpc_id as id, - vpc_id as title, - 0 as depth, - 'aws_vpc' as category - from - aws_vpc - where - vpc_id in (select vpc_id from vpc) - - union all - select - distinct on (availability_zone) - vpc_id as from_id, - availability_zone as id, - availability_zone as title, - 1 as depth, - 'aws_availability_zone' as category - from - aws_vpc_subnet - where - vpc_id in (select vpc_id from vpc) - - - union all - select - availability_zone as from_id, - subnet_id as id, - subnet_id as title, - 2 as depth, - 'aws_vpc_subnet' as category - from - aws_vpc_subnet - where - vpc_id in (select vpc_id from vpc) - - EOQ -} - -``` - diff --git a/docs/reference/mod-resources/image.md b/docs/reference/mod-resources/image.md deleted file mode 100644 index 5c19fd9..0000000 --- a/docs/reference/mod-resources/image.md +++ /dev/null @@ -1,101 +0,0 @@ ---- -title: Image -sidebar_label: image ---- - -# image - -> ***Powerpipe is now the recommended way to run dashboards and benchmarks!*** -> Mods still work as normal in Steampipe for now, but they are deprecated and will be removed in a future release: -> - [Steampipe Unbundled →](https://steampipe.io/blog/steampipe-unbundled) -> - [Powerpipe for Steampipe users →](https://powerpipe.io/blog/migrating-from-steampipe) - -Display images in a dashboard, either from a known publicly-accessible `src` URL, or from a `src` returned in a SQL query. - -Image blocks can be declared as named resources at the top level of a mod, or be declared as anonymous blocks inside a `dashboard` or `container`, or be re-used inside a `dashboard` or `container` by using an `image` with `base = .image.`. - - -## Example Usage - - - -```hcl -image { - src = "https://steampipe.io/images/steampipe_logo_wordmark_color.svg" - alt = "Steampipe Logo" - width = 2 -} -``` - - -## Argument Reference -| Argument | Type | Optional? | Description -|-|-|-|- -| `alt` | String | Optional | Alternative text for the image. -| `args` | Map | Optional| A map of arguments to pass to the query. -| `base` | Text Reference | Optional | A reference to a named `text` resource that this `text` should source its definition from. `title` and `width` can be overridden after sourcing via `base`. -| `param` | Block | Optional| A [param](reference/mod-resources/query#param) block that defines the parameters that can be passed in to the query. `param` blocks may only be specified for images that specify the `sql` argument. -| `query` | Query Reference | Optional | A reference to a [query](reference/mod-resources/query) resource that defines the query to run. An `image` may either specify the `query` argument or the `sql` argument, but not both. -| `sql` | String | Optional | An SQL string to provide data for the `image`. An `image` may either specify the `query` argument or the `sql` argument, but not both. -| `src` | String | Optional | Publicly-accessible URL for the image. -| `title` | String | Optional | A plain text [title](/docs/reference/mod-resources/dashboard#title) to display for this image. -| `width` | Number | Optional | The [width](/docs/reference/mod-resources/dashboard#width) as a number of grid units that this item should consume from its parent. - -## Data Structure - -If an image provides a `sql` query, it supports 2 data structures. - -1. A simple structure where column 1's name is the `alt` and column 1's value is the image `src`. -2. A formal data structure where the column names map to properties of the `image`. - -Simple data structure: - -| | -|--------| -| | - -Formal data structure: - -| src | alt | -| ---------------------------------------------- | -------------- | -| https://steampipe.io/images/steampipe-logo.png | Steampipe Logo | - - -## More Examples - - -### Via query - - -```hcl -image { - sql = <<-EOQ - select - avatar_url as "Avatar" - from - github_user - where - login = 'torvalds' - EOQ - width = 2 -} -``` - - -### Dynamic Styling via formal query data structure - - -```hcl -image { - sql = <<-EOQ - select - avatar_url as src, - 'Avatar' as alt - from - github_user - where - login = 'ken' - EOQ - width = 2 -} -``` \ No newline at end of file diff --git a/docs/reference/mod-resources/input.md b/docs/reference/mod-resources/input.md deleted file mode 100644 index 729889d..0000000 --- a/docs/reference/mod-resources/input.md +++ /dev/null @@ -1,396 +0,0 @@ ---- -title: Input -sidebar_label: input ---- - -# input - -> ***Powerpipe is now the recommended way to run dashboards and benchmarks!*** -> Mods still work as normal in Steampipe for now, but they are deprecated and will be removed in a future release: -> - [Steampipe Unbundled →](https://steampipe.io/blog/steampipe-unbundled) -> - [Powerpipe for Steampipe users →](https://powerpipe.io/blog/migrating-from-steampipe) - -Allow user input in a dashboard using common form components such as `text`, `select`, `multiselect`, `combo` and `multicombo`. Data can either be static or derived from a SQL query. - -Dashboard components can depend on the value of an input within the dashboard by referring to `self` e.g. `self.input..value`. This allows you to pass the value of an input as an argument to a query (or any other dashboard element) to create dynamic dashboards! - -Input blocks can be declared as named resources at the top level of a mod, or be declared as named blocks inside a `dashboard` or `container`, or be re-used inside a `dashboard` or `container` by using an `input` with `base = .input.`. - -## Example Usage - - - -
- -```hcl -input "vpc_id" { - title = "VPC" - type = "select" - width = 2 - - sql = <<-EOQ - select - title as label, - vpc_id as value - from - aws_vpc; - EOQ -} -``` - - - - -## Argument Reference - -| Argument | Type | Optional? | Description | -|---------------|---------|-----------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| `args` | Map | Optional| A map of arguments to pass to the query. -| `base` | String | Optional | A reference to a named `input` resource that this `input` should source its definition from. `title`, `sql`, `type`, `options` and `width` can be overridden after sourcing via `base`. | -| `option` | Block | Optional | [option](#option) block to add static values to the input | -| `placeholder` | String | Optional | Placeholder text to display. If a `placeholder` is set for a `combo`, `multicombo`, `select` or `multiselect`, then dependent resources will not run until a selection is made. If no `placeholder` is set, the first item in the list will be selected by default. -| `param` | Block | Optional| A [param](reference/mod-resources/query#param) block that defines the parameters that can be passed in to the query. `param` blocks may only be specified for inputs that specify the `sql` argument. -| `query` | Query Reference | Optional | A reference to a [query](reference/mod-resources/query) resource that defines the query to run. An `input` may either specify the `query` argument or the `sql` argument, but not both. -| `sql` | String | Optional | An SQL string to provide data for the `input`. An `input` may either specify the `query` argument or the `sql` argument, but not both. -| `title` | String | Optional | A plain text [title](/docs/reference/mod-resources/dashboard#title) to display for this input. | -| `type` | String | Optional | The [type of the input](#input-types). Can be `text`, `combo`, `multicombo`, `select` or `multiselect`. | -| `width` | Number | Optional | The [width](/docs/reference/mod-resources/dashboard#width) as a number of grid units that this item should consume from its parent. | - - -## Input Types -| Type | Description -|------|----------------- -| [text](#text-input) | Enter a single line of text -| [select](#select-with-dynamic-options) | Select a single item from a dropdown list -| [multiselect](#multi-select-with-dynamic-options) | Select one or more items from a dropdown list -| [combo](#combo-box) | Select a single item from a dropdown list, or enter a new value -| [multicombo](#multi-select-combo-box) | Select one or more items from a dropdown list, or enter new values - - -## Common Input Properties - -### option - -Add static options to an input. Applies to `select` and `multiselect`. The block name is the value. If a `label` is not specified, the value will be used as the label. - -| Property | Type | Default | Description | -| -------- | ---------| ------- | ----------- | -| `label` | String | If not specified, the value will be used as the label | the display label for this option | - - - -## Data Structure - -The data structure for an `input` will depend on the `type`. - -## select / multiselect / combo / multicombo - -| label | value | tags | -| ------------------- | --------------------- |----------------------------------| -| default | vpc-05657e5bef9676266 | null | -| acme @ 10.84.0.0/16 | vpc-03656e5eef967f366 | { "account_id": "123456789012" } | - -`tags` is an optional JSONB object of key/value pairs. Any tag values will be displayed in the list of available options, along with the selected option(s). This will allow you to identify resources across multi-account queries for example. When a user types to search in the input, the labels and tags will be searched. - -## More Examples - -### Single-select with tags - - - -
- -```hcl -input "instance_arn" { - title = "Select an instance:" - width = 4 - - sql = <<-EOQ - select - title as label, - arn as value, - json_build_object( - 'region', region, - 'instance_id', instance_id - ) as tags - from - aws_ec2_instance - order by - title; - EOQ - } - -``` - - -### Single-select with fixed options - - - - -```hcl - -input "regions" { - title = "Select regions:" - width = 2 - option "us-east-1" {} - option "us-east-2" {} -} - -``` - -### Single-select with fixed options, using labels - - - - -```hcl - -input "vpc_id" { - title = "Select VPC:" - width = 2 - option "vpc-05657e5bef9676266" { - label = "default" - } - option "vpc-03656e5eef967f366" { - label = "acme @ 10.84.0.0/16" - } -} - -``` - -### Multi-select with dynamic options - - - -
- -```hcl - -input "policy_arns" { - title = "Select policies:" - type = "multiselect" - width = 2 - sql = <<-EOQ - select - name as label, - arn as value - from - aws_iam_policy; - EOQ -} - -``` - - -### Select with dynamic options - - - -
- -```hcl -input "vpc_id" { - title = "VPC" - type = "select" - width = 2 - - sql = <<-EOQ - select - title as label, - vpc_id as value - from - aws_vpc; - EOQ -} -``` - - - - -### Text input - - - -
- -```hcl -input "search_string" { - title = "Search String:" - width = 2 - type = "text" - placeholder = "enter a search string" -} - -``` - - - -### Combo box - - - -
- -```hcl -input "cost_center" { - title = "Select a Cost Center:" - type = "combo" - width = 2 - - sql = <<-EOQ - select distinct - tags ->> 'costcenter' as label, - tags ->> 'costcenter' as value - from - aws_tagging_resource - where - tags ->> 'costcenter' is not null; - EOQ -} -``` - -### Multi-select combo box - - - -
- -```hcl -input "cost_centers" { - title = "Select a Cost Center:" - type = "multicombo" - width = 3 - - sql = <<-EOQ - select distinct - tags ->> 'costcenter' as label, - tags ->> 'costcenter' as value - from - aws_tagging_resource - where - tags ->> 'costcenter' is not null; - EOQ -} -``` - - - - - - - - - - - - -### Example dashboard using an input - - - -
- - -```hcl -query "aws_region_input" { - sql = <<-EOQ - select - distinct region as label, - region as value - from - aws_region - order by - region; - EOQ -} - -query "aws_s3_buckets_by_versioning_enabled" { - sql = <<-EOQ - with versioning as ( - select - case when versioning_enabled then 'Enabled' else 'Disabled' end as versioning_status, - region - from - aws_s3_bucket - ) - select - versioning_status, - count(versioning_status) as "Total" - from - versioning - where - region = $1 - group by - versioning_status - EOQ - - param "region" {} -} - - -query "aws_s3_buckets_in_region" { - sql = <<-EOQ - select - name, - versioning_enabled - from - aws_s3_bucket - where - region = $1 - EOQ - - param "region" {} -} - -dashboard "inputs_example_dashboard" { - title = "Inputs Example Dashboard" - - input "region" { - sql = query.aws_region_input.sql - width = 3 - } - - container { - - chart { - title = "AWS Bucket Versioning Status" - type = "pie" - width = 2 - query = query.aws_s3_buckets_by_versioning_enabled - args = { - "region" = self.input.region.value - } - } - - table { - width = 4 - query = query.aws_s3_buckets_in_region - args = { - "region" = self.input.region.value - } - } - - } -} -``` diff --git a/docs/reference/mod-resources/locals.md b/docs/reference/mod-resources/locals.md deleted file mode 100644 index 354ba74..0000000 --- a/docs/reference/mod-resources/locals.md +++ /dev/null @@ -1,49 +0,0 @@ ---- -title: locals -sidebar_label: locals ---- - -# locals - -> ***Powerpipe is now the recommended way to run dashboards and benchmarks!*** -> Mods still work as normal in Steampipe for now, but they are deprecated and will be removed in a future release: -> - [Steampipe Unbundled →](https://steampipe.io/blog/steampipe-unbundled) -> - [Powerpipe for Steampipe users →](https://powerpipe.io/blog/migrating-from-steampipe) - -The `locals` block defines and sets one or more [local variables](mods/mod-variables#local-variables), using standard HCL assignment syntax. The locals are scoped to the mod, and a mod may contain multiple `locals` blocks. Locals may reference other values in the mod, including other local values. - -You can reference local values as `local.` - - - -## Example Usage - -```hcl -locals { - cis_version = "v1.4.0" - plugin_name = "aws" -} - -locals { - cis_v140_common_tags = { - cis = "true" - cis_version = local.cis_version - plugin = local.plugin_name - } -} - -benchmark "cis_v140" { - title = "CIS v1.4.0" - description = "The CIS Amazon Web Services Foundations Benchmark provides prescriptive guidance for configuring security options for a subset of Amazon Web Services with an emphasis on foundational, testable, and architecture agnostic settings." - documentation = file("./cis_v140/docs/cis_overview.md") - children = [ - benchmark.cis_v140_1, - benchmark.cis_v140_2, - benchmark.cis_v140_3, - benchmark.cis_v140_4, - benchmark.cis_v140_5 - ] - tags = local.cis_v140_common_tags -} -``` - diff --git a/docs/reference/mod-resources/mod.md b/docs/reference/mod-resources/mod.md deleted file mode 100644 index 0e37dc4..0000000 --- a/docs/reference/mod-resources/mod.md +++ /dev/null @@ -1,151 +0,0 @@ ---- -title: mod -sidebar_label: mod ---- - - -# mod - -> ***Powerpipe is now the recommended way to run dashboards and benchmarks!*** -> Mods still work as normal in Steampipe for now, but they are deprecated and will be removed in a future release: -> - [Steampipe Unbundled →](https://steampipe.io/blog/steampipe-unbundled) -> - [Powerpipe for Steampipe users →](https://powerpipe.io/blog/migrating-from-steampipe) - -Every mod must contain a `mod.sp` file with a single `mod` block. - -The `mod` block contains metadata for the mod (including metadata used in the hub site and social media), as well as dependency data. A mod author may edit the mod block directly, but Steampipe will **also** edit the file, adding, removing and modifying dependencies in the file when users add and remove mods via the [`steampipe mod` commands](/docs/reference/cli/mod). For this reason, it is recommended that the `mod.sp` *only* contain a `mod` block; do not add other mod resources (`query`, `control`, `dashboard`, etc) to this file. - -The block name (`aws_cis` in the example) is the mod name. Mod names use lower_snake_case. They may contain lowercase chars, numbers or underscores, and must start with a letter. - - -## Example Usage - -```hcl -mod "aws_cis" { - # hub metadata - title = "AWS CIS" - description = "AWS CIS Reporting and remediation" - color = "#FF9900" - documentation = file("./aws_cis_docs.md") - icon = "/images/plugins/turbot/aws.svg" - categories = ["Public Cloud", "AWS"] - - opengraph { - title = "Steampipe Mod for AWS CIS" - description = "CIS reports, queries, and actions for AWS. Open source CLI. No DB required." - } - - require { - steampipe { - min_version = "0.20.0" - } - - plugin "aws" { - min_version = "0.86.0" - } - - plugin "gcp" { - min_version = "0.29.0" - } - - mod "github.com/turbot/steampipe-mod-aws-compliance" { - version = "^0.10" - } - mod "github.com/turbot/steampipe-mod-gcp-compliance" { - version = "*" - } - } -} -``` - -## Argument Reference - -| Name | Type | Required? | Description -|-|-|-|- -| `categories` | List(String) | Optional | A list of labels, used to categorize mods (such as on the Steampipe Hub). -| `color` | String |Optional | A hexadecimal RGB value to use as the color scheme for the mod on hub.steampipe.io. -| `description` | String | Optional | A string containing a short description. -| `documentation` | String (Markdown)| Optional | A markdown string containing a long form description, used as documentation for the mod on hub.steampipe.io. -| `icon` | String | Optional | The url of an icon to use for the mod on hub.steampipe.io. -| `opengraph` | Block | Optional | Block of metadata for use in social media applications that support [Opengraph](#opengraph) metadata. -| `require` | Block | Optional | A block that specifies one or more [mod dependencies](#require). -| `tags` | Map | Optional | A map of key:value metadata for the mod, used to categorize, search, and filter. -| `title` | String | Optional | The display title of the mod. - - -#### opengraph -The `opengraph` block is an optional block of metadata for use in social media applications that support [Opengraph](https://ogp.me/) metadata. - -| Name | Type| Description -|-|-|- -| `description` | String | The opengraph description (`og:description`) of the mod, for use in social media applications. -| `title` | String | The opengraph display title (`og:title`) of the mod, for use in social media applications. - - - -#### require -A mod may contain a `require` block to specify version dependencies for the Steampipe CLI, plugins, and mods. While it is possible to edit this section manually, Steampipe will also modify it (including reordering and removing comments) when you run a `steampipe mod` command to install, update, or uninstall a mod. - -A mod may specify a dependency on the Steampipe CLI. Steampipe will evaluate the dependency when the mod is loaded, and will error if the constraint is not met, but it will not install or upgrade the CLI. A `steampipe` constraint specifies a *minimum version*, and does not support semver syntax: -```hcl -require { - steampipe { - min_version = "0.20.0" - } -} -``` - -A mod may specify a dependency on one or more plugins. Steampipe will evaluate the dependency when the mod is loaded, and will error if the constraint is not met, but it will not install or upgrade the plugin. A `plugin` constraint specifies a *minimum version*, and does not support semver syntax: -```hcl -require { - plugin "aws" { - min_version = "0.24.0" - } -} -``` - -A mod may specify dependencies on other mods. While you can manually edit the `mod` dependencies in the `mod.sp`, they are more commonly managed by Steampipe when you install, update, or uninstall mods via the [steampipe mod commands](/docs/reference/cli/mod). The `version` can be an exact version or a [semver](https://semver.org/) string: - -```hcl -require { - mod "github.com/turbot/steampipe-mod-aws-compliance" { - version = "^0.10" - } - mod "github.com/turbot/steampipe-mod-aws-insights" { - version = "2.0" - } - mod "github.com/turbot/steampipe-mod-gcp-compliance" { - version = "*" - } -} -``` - -You may pass `args` to set variables defined in the dependency mods. You can pass hard-coded values (literals), however it is more common to define variables in your mod that encapsulate the variables and optionality of your dependencies: - - -```hcl -variable "common_dimensions" { - type = list(string) - description = "A list of common dimensions to add to each control." - default = [ "account_id", "region" ] -} - -variable "tag_dimensions" { - type = list(string) - description = "A list of tags to add as dimensions to each control." - default = [] -} - -mod "aws_well_architected" { - require { - mod "github.com/turbot/steampipe-mod-aws-compliance" { - version = "^0.63.0" - args = { - common_dimensions = var.common_dimensions, - tag_dimensions = var.tag_dimensions - } - } - } -} - -``` diff --git a/docs/reference/mod-resources/node.md b/docs/reference/mod-resources/node.md deleted file mode 100644 index 98de6a8..0000000 --- a/docs/reference/mod-resources/node.md +++ /dev/null @@ -1,173 +0,0 @@ ---- -title: Node -sidebar_label: node ---- - -# node - -> ***Powerpipe is now the recommended way to run dashboards and benchmarks!*** -> Mods still work as normal in Steampipe for now, but they are deprecated and will be removed in a future release: -> - [Steampipe Unbundled →](https://steampipe.io/blog/steampipe-unbundled) -> - [Powerpipe for Steampipe users →](https://powerpipe.io/blog/migrating-from-steampipe) - - -The `node` block represents a vertex in a `graph`, `hierarchy` or `flow`. - -Anonymous `node` blocks can be declared inside a `graph`, `hierarchy` or `flow`. They may also may be declared as named resources at the top level of a mod and referenced via `base` from other nodes in a `graph`, `hierarchy` or `flow`. - - -## Example Usage - -```hcl -node { - category = category.plugin_version - - sql = <<-EOQ - select - digest as id, - left(split_part(digest,':',2),12) as title, - json_build_object( - 'digest', digest, - 'created', create_time, - 'updated', update_time - ) as properties - from - steampipe_registry_plugin_version - where - name = 'turbot/ldap' - EOQ -} -``` - - - -## Argument Reference -| Argument | Type | Optional? | Description -|-|-|-|- -| `args` | Map | Optional| A map of arguments to pass to the query. -| `base` | flow Reference | Optional | A reference to a named `node` resource that this `node` should source its definition from. -| `category` | Block | Optional| [category](/docs/reference/mod-resources/category) blocks that specify display options for nodes with that category. -| `depth` | Number | Optional | An integer to set the position of the node in a flow. The layout of the nodes is inferred from the query, however you can force placement with the `depth` argument if you need to override the default behavior. The `depth` argument is optional, and is only used by `flow` resources. -| `param` | Block | Optional| [param](reference/mod-resources/query#param) blocks that defines the parameters that can be passed in to the query. `param` blocks may only be specified when the node is defined as a top-level (mod level), named resource. -| `query` | Query Reference | Optional | A reference to a [query](reference/mod-resources/query) resource that defines the query to run. You must either specify the `query` argument or the `sql` argument, but not both. -| `sql` | String | Optional | An SQL string to provide data for the `node`. You must either specify the `query` argument or the `sql` argument, but not both. -| `title` | String | Optional | A plain text [title](/docs/reference/mod-resources/dashboard#title) to display for this node. - - -## Data Format -Data must be provided in a format where each row represents a *node* (vertex). Significant columns are: - -| Name | Description -|------------|--------------------------------------------------- -| `id` | A unique identifier for the node. `id` is required for nodes. -| `title` | An optional title to display for the node. -| `category` | An optional display category. This must be the name of a `category` declared in the parent `graph`, `flow`, or `hierarchy`. -| `depth` | An integer to set the position of the node in a flow. The layout of the nodes is inferred from the query, however you can force placement with the `depth` column if you need to override the default behavior. The `depth` column is optional, and is only used in `flow` resources. -| `properties`| A jsonb key/value map of properties to display for the node/edge when the user hovers over it. The `properties` column is optional. - -The `category`, `depth`, and `title` may be specified either in the SQL results or in HCL. If both are specified, the value in the SQL result set has precedence. - - - -## More Examples - -### inline node in a graph - -```hcl -dashboard "node_ex_1" { - graph { - - node { - category = category.plugin_version - - sql = <<-EOQ - select - digest as id, - left(split_part(digest,':',2),12) as title, - json_build_object( - 'digest', digest, - 'created', create_time, - 'updated', update_time - ) as properties - from - steampipe_registry_plugin_version - where - name = 'turbot/ldap' - EOQ - } - - } -} -``` - - -### Reusable node with `base` - -```hcl -dashboard "node_ex_2" { - graph { - - node { - base = node.plugin - } - - } -} - -node "plugin"{ - category = category.plugin - - sql = <<-EOQ - select - name as id, - name as title, - json_build_object( - 'name', name, - 'created', create_time, - 'updated', update_time - ) as properties - from - steampipe_registry_plugin - EOQ -} -``` - - -### Reusable node with `base` passing args - -```hcl -dashboard "node_ex_3" { - graph { - - node { - base = node.plugin - args = { - plugin_name = self.input.plugin_name.value - } - } - - } -} - -node "plugin"{ - category = category.plugin - - sql = <<-EOQ - select - name as id, - name as title, - json_build_object( - 'name', name, - 'created', create_time, - 'updated', update_time - ) as properties - from - steampipe_registry_plugin - where - name = $1 - EOQ - - param "plugin_name" {} -} -``` - diff --git a/docs/reference/mod-resources/overview.md b/docs/reference/mod-resources/overview.md deleted file mode 100644 index 59f49f6..0000000 --- a/docs/reference/mod-resources/overview.md +++ /dev/null @@ -1,50 +0,0 @@ ---- -title: Mod Resources -sidebar_label: Mod Resources ---- - -# Mod Resources - -> ***Powerpipe is now the recommended way to run dashboards and benchmarks!*** -> Mods still work as normal in Steampipe for now, but they are deprecated and will be removed in a future release: -> - [Steampipe Unbundled →](https://steampipe.io/blog/steampipe-unbundled) -> - [Powerpipe for Steampipe users →](https://powerpipe.io/blog/migrating-from-steampipe) - -## General - -| Type | Description -|-|- -| [locals](reference/mod-resources/locals) | Locals are internal, module level variables. -| [mod](reference/mod-resources/mod) | The mod block contains metadata, documentation, and dependency data for the mod. -| [query](reference/mod-resources/query) | Queries define common SQL statements that may be used alone, or referenced by arguments in other blocks like reports and actions. -| [variable](reference/mod-resources/variable) | Variables are module level objects that essentially act as parameters for a module. - - -## Benchmarks and Controls - -| Type | Description -|-|- -| [benchmark](reference/mod-resources/benchmark) | Benchmark provides a mechanism for organizing controls into hierarchical structures. -| [control](reference/mod-resources/control) | Controls provide a defined structure and interface for queries that draw a specific conclusion (e.g. 'ok', 'alarm') about each row. - - -## Dashboards - -| Type | Description | Valid children types | Allowed at top-level | -| ----------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------ | -------------------- | -| [dashboard](reference/mod-resources/dashboard) | Compose resources to meet a reporting requirement. Within the `steampipe dashboard` UI, each `dashboard` will be presented as an available item to run. | `chart`
`container`
`control`
`card`
`flow`
`graph`
`hierarchy`
`image`
`input`
`table`
`text`
`with` | Yes | -| [container](reference/mod-resources/container) | Lay out reporting resources within a dashboard. Conceptually similar to a dashboard, except it will not be presented as an available item to run within the steampipe dashboard UI. | `chart`
`container`
`control`
`card`
`flow`
`graph`
`image`
`input`
`table`
`text`
`with` | Yes | -| [card](reference/mod-resources/card) | Display a simple value in different styles e.g. `plain`, `alert` etc. Supports static values, or derived from SQL. | None | Yes | -| [category](reference/mod-resources/category) | Specify display options for `nodes` and `edges` | None | Yes | -| [chart](reference/mod-resources/chart) | Visualize SQL data in a chart e.g. `bar`, `column`, `line`, `pie` etc. | None | Yes | -| [edge](reference/mod-resources/edge) | Display an edge to connect nodes on a`flow`, `graph`, or `hierarchy` | None| Yes | -| [flow](reference/mod-resources/flow) | Visualize flow data using things such as `sankey`. | `node`, `edge` | Yes | -| [graph](reference/mod-resources/graph) | Visualize graph relationships. | `node`, `edge` | Yes | -| [hierarchy](reference/mod-resources/hierarchy) | Visualize hierarchical data using things such as `tree`.| `node`, `edge` | Yes | -| [image](reference/mod-resources/image) | Embed images in reports. Supports static URLs, or can be derived from SQL. | None | Yes | -| [input](reference/mod-resources/input) | Enable dynamic dashboards based on user-provided `input`. | None | Yes | -| [node](reference/mod-resources/node) | Display a vertex (node) on a`flow`, `graph`, or `hierarchy` | None| Yes | -| [table](reference/mod-resources/table) | Display tabular data in a dashboard. | None | Yes | -| [text](reference/mod-resources/text) | Add GitHub-flavoured markdown to a dashboard. | None | Yes | -| [with](reference/mod-resources/with) | Specify additional queries or SQL statements to run **first**, and then pass the query results as arguments to `sql`, `query`, and `node` & `edge` blocks.| None | No | - \ No newline at end of file diff --git a/docs/reference/mod-resources/query.md b/docs/reference/mod-resources/query.md deleted file mode 100644 index 7100fa4..0000000 --- a/docs/reference/mod-resources/query.md +++ /dev/null @@ -1,307 +0,0 @@ ---- -title: query -sidebar_label: query ---- - -# query - -> ***Powerpipe is now the recommended way to run dashboards and benchmarks!*** -> Mods still work as normal in Steampipe for now, but they are deprecated and will be removed in a future release: -> - [Steampipe Unbundled →](https://steampipe.io/blog/steampipe-unbundled) -> - [Powerpipe for Steampipe users →](https://powerpipe.io/blog/migrating-from-steampipe) - -Queries define common SQL statements that may be used alone, or referenced in other blocks like controls and charts. - -Note that a Steampipe `query` is NOT a database resource. It does not create a view, stored procedure, etc. `query` blocks are interpreted by and executed by Steampipe, and are only available from Steampipe, not from 3rd party tools. - -## Example Usage - -```hcl -query "plus_size_instances" { - title = "EC2 Instances xlarge and bigger" - sql = "select * from aws_ec2_instance where instance_type like '%xlarge'" -} -``` - -You can run a query by its fully qualified name in the Steampipe query shell: -```sql -> aws_ec2_reports.query.prohibited_instance_types -``` - -Or in a non-interactive `steampipe query` command: -```bash -$ steampipe query "aws_ec2_reports.query.plus_size_instances" -``` - -## Argument Reference - -| Argument |Type | Required? | Description -|-|-|-|- -| `description` | String | Optional| A description of the query. -| `documentation` | String (Markdown)| Optional | A markdown string containing a long form description, used as documentation for the mod on hub.steampipe.io. -| `param` | Block | Optional| A [param](#param) block that defines the parameters that can be passed in to the query. -| `sql` | String | Required | SQL statement to define the query. -| `tags` | Map | Optional | A map of key:value metadata for the query, used to categorize, search, and filter. The structure is up to the mod author and varies by mod. -| `title` | String | Optional | A display title for the query. - - -#### param -One or more `param` blocks may optionally be used in a query to define parameters that the query accepts. Note that the SQL statement only supports positional arguments (`$1`, `$2`, ...) and that the param blocks are assigned in order -- the first param block describes `$1`, the second describes `$2`, etc. - -| Name | Type| Description -|-|-|- -| `description` | String | A description of the parameter. -| `default` | Any | A value to use if no argument is passed for this parameter when the query is run. - - -```hcl -variable "max_access_key_age" { - type = number -} - -query "old_access_keys" { - sql = <<-EOT - select - access_key_id, - user_name, - create_date, - age(create_date) as age - from - aws_iam_access_key - where - create_date < NOW() - ($1 ::numeric || ' days')::interval; - EOT - - param "max_days" { - default = var.max_access_key_age - description = "The maximum allowed key age, in days." - } -} -``` - -You can run a parameterized query for any named query from `steampipe query`. If the query provides defaults for all the parameters, you can run it without arguments in the same way you would run a query that takes no parameters, and it will run with the default values: - -```sql -query.old_access_keys -``` - -If the query does not provide a default, or you wish to run the query with a different value, you can pass an argument to the query. - -You can pass them by name: -```sql -query.old_access_keys(max_days => 365) -``` - -Or by position: -```sql -query.old_access_keys(365) -``` - -If the parameter takes an array, you can pass an array literal: -```sql -query.bucket_count_for_regions(["us-east-2", "us-east-1"]) -``` - - ---- - -# Query-based Resources - -There are many Steampipe mod elements that execute a query, including `control` and most dashboard visualization elements (`card`,`chart`, `node`, `edge`, `graphs` etc). These resources essentially implement the same interface: - - They have a `sql` argument for specifying a SQL string to execute - - They have a `query` argument for referencing a `query` to execute - - They require the user to set either `sql` or `query`, but both may not be specified. - - They have an optional `param` argument to specify parameters that the `sql` accepts - - They have an optional `args` argument to specify what arguments to pass to the `query` or `sql` - - -## Query v/s SQL - -When using a query-based resource, you **must** specify either the `sql` or `query` argument, but not both. - -The difference between these arguments is somewhat subtle. - -The `sql` argument is a simple *string* that defines a SQL statement to execute. This allows you to inline the statement in the resource definition: - -```hcl -card { - sql = <<-EOQ - select - count(*) as "Buckets" - from - aws_s3_bucket - EOQ - - width = 2 -} -``` - -The `query` argument is a *reference to a named `query` resource* to run. This allows you to reuse a query from multiple other resources: - -```hcl -card { - width = 2 - query = query.bucket_count -} - -query "bucket_count" { - sql = <<-EOQ - select - count(*) as "Buckets" - from - aws_s3_bucket - EOQ - -} -``` - - -## Params v/s Args - -Query-based resources allow you to specify parameters that the `sql` accepts and to specify what arguments to pass to the `query` or `sql`. The difference between `param` and `args` is a common source of confusion. - -A `param` block is used to define a parameter *that the `sql` accepts* - It is a definition of what you can pass. The `args` block is used to specify what *values to pass to the `query` or `sql`* - these are the actual values to use when running the query. - -`param` blocks are only used when the `sql` argument is specified and the SQL statement includes parameters (`$1`, `$2`): - -```hcl -card "bucket_count_for_region" { - sql = <<-EOQ - select - count(*) as "Buckets" - from - aws_s3_bucket - where - region = $1 - EOQ - - param "region" { - default = "us-east-1" - } - - width = 2 -} -``` - -***You can only specify `param` blocks for resources that are defined as top-level named resources in your mod.*** It would not make sense to specify a `param` block for an anonymous resource that is defined in dashboard, since you cannot reference it anyway. - -The `args` argument is used to pass values to a query at run time. If the `query` resource has parameters defined, then the `args` argument is used to pass values to the query: - -```hcl -dashboard "s3_dashboard" { - card { - width = 2 - query = query.bucket_count_for_region - - args = { - region = "us-east-1" - } - } -} - -query "bucket_count_for_region" { - sql = <<-EOQ - select - count(*) as "Buckets" - from - aws_s3_bucket - where - region = $1 - EOQ - - param "region" {} -} -``` - -Note that most query-based resources can be defined as top-level resources and reused with `base`, and you can pass arguments to them in the same manner: - -```hcl -dashboard "s3_buckets" { - - card { - base = card.bucket_count_for_region - args = { - region = "us-east-1" - } - } - - card { - base = card.bucket_count_for_region - args = { - region = "us-east-2" - } - } -} - -card "bucket_count_for_region" { - sql = <<-EOQ - select - count(*) as "Buckets" - from - aws_s3_bucket - where - region = $1 - EOQ - - param "region" { - default = "us-east-1" - } - - width = 2 -} -``` - -While `param` blocks are ***recommended*** when SQL statements define parameters, they are not required -- you wont be able to pass arguments by name, but you can still pass them positionally using the array format of the `args` argument: - -```hcl -dashboard "s3_dashboard" { - card { - width = 2 - query = query.bucket_count_for_region - - args = ["us-east-1"] - } -} - -query "bucket_count_for_region" { - sql = <<-EOQ - select - count(*) as "Buckets" - from - aws_s3_bucket - where - region = $1 - EOQ -} -``` - - -## Running from the Steampipe CLI - -You can run top-level, named query-based resources from `steampipe query` by name, in the same way that you run named queries. - -```sql -card.bucket_count_for_region -``` - -If the query does not provide a default, or you wish to run the query with a different value, you can pass an argument to the query. - -You can pass them by name: -```sql - card.bucket_count_for_region(region => "us-east-2") -``` - -Or by position: -```sql -card.bucket_count_for_region("us-east-2") -``` - -If the parameter takes an array, you can pass an array literal: -```sql -card.bucket_count_for_regions(["us-east-2", "us-east-1"]) -``` diff --git a/docs/reference/mod-resources/table.md b/docs/reference/mod-resources/table.md deleted file mode 100644 index 592b8b2..0000000 --- a/docs/reference/mod-resources/table.md +++ /dev/null @@ -1,197 +0,0 @@ ---- -title: Table -sidebar_label: table ---- - -# table - -> ***Powerpipe is now the recommended way to run dashboards and benchmarks!*** -> Mods still work as normal in Steampipe for now, but they are deprecated and will be removed in a future release: -> - [Steampipe Unbundled →](https://steampipe.io/blog/steampipe-unbundled) -> - [Powerpipe for Steampipe users →](https://powerpipe.io/blog/migrating-from-steampipe) - -A table displays query results in a dashboard. - -Table blocks can be declared as named resources at the top level of a mod, or be declared as anonymous blocks inside a `dashboard` or `container`, or be re-used inside a `dashboard` or `container` by using a `table` with `base = .table.`. - - -## Example Usage - - -```hcl -table { - title = "S3 Buckets" - width = 4 - - sql = <<-EOQ - select - title, - arn, - region - from - aws_s3_bucket - order by - title - EOQ -} -``` - - - - - -## Argument Reference -| Argument | Type | Optional? | Description | -|----------|-------------------|-----------|---------------------------------------------------------------------------------------------------------------------------------------------------------------| -| `args` | Map | Optional| A map of arguments to pass to the query. -| `base` | Table Reference | Optional | A reference to a named `table` resource that this `table` should source its definition from. `title` and `width` can be overridden after sourcing via `base`. | -| `column` | String | Optional | A named block matching the name of the column you wish to configure. See [column](#column). | -| `param` | Block | Optional| A [param](reference/mod-resources/query#param) block that defines the parameters that can be passed in to the query. `param` blocks may only be specified for tables that specify the `sql` argument. -| `query` | Query Reference | Optional | A reference to a [query](reference/mod-resources/query) resource that defines the query to run. A `table` may either specify the `query` argument or the `sql` argument, but not both. -| `sql` | String | Optional | An SQL string to provide data for the `table`. A `table` may either specify the `query` argument or the `sql` argument, but not both. -| `title` | String | Optional | A plain text [title](/docs/reference/mod-resources/dashboard#title) to display for this table. | -| `type` | String | Optional | The type of the table. Can be `table` (default) or `line`. `line` view transposes each row into a key/value pair (column name/column value) item view | -| `width` | Number | Optional | The [width](/docs/reference/mod-resources/dashboard#width) as a number of grid units that this item should consume from its parent. | - - -## Table Properties - -### column - -| Property | Type | Default | Values | -|-----------| ------ |---------|---------------------------------------------------------------------------------------------------------------| -| `display` | string | `all` | `all` (default) will show the column at all breakpoints. `none` will never show the column. | -| `wrap` | string | `none` | `all` will wrap the column contents at all breakpoints. `none` (default) will never wrap the column contents. | -| `href` | string | | A url that the column should link to. The `href` may use a [jq template](#jq-templates) to dynamically generate the link for each row. | - -#### jq Templates -The `href` argument allows you to specify a [jq](https://stedolan.github.io/jq/) template to dynamically generate a hyperlink from the data in the row. To use a jq template, enclose the jq in double curly braces (`{{ }}`). - -Steampipe will pass each row of data to jq in the same format that is returned by [steampipe query json mode output](reference/dot-commands/output), where the keys are the column names and the values are the data for that row. - -For example, this query: -```sql -select - instance_id, - region, - sg->>'GroupId' as security_group -from - aws_ec2_instance, - jsonb_array_elements(security_groups) as sg - -``` - -will present rows to the jq template in this format: -```json -{ - "instance_id": "i-03d11d111b1407bbc", - "region": "us-east-2", - "security_group": "sg-01ee40ea54e0fa089" - } -``` - -which you can then use in a jq template in the `href` argument: - -```hcl -table { - title = "Attached Security Groups" - width = 4 - sql = <<-EOQ - select - instance_id, - region, - sg->>'GroupId' as security_group - from - aws_ec2_instance, - jsonb_array_elements(security_groups) as sg - EOQ - - column "security_group" { - href = "${dashboard.aws_vpc_security_group_detail.url_path}?input.security_group_id={{.'security_group' | @uri}}" - } -} -``` - -Refer to [JQ Escaping & Interpolation ](/docs/reference/mod-resources/dashboard#jq-escaping--interpolation) for more advanced examples. -## More Examples - - -### Table with options for columns - - -```hcl -table { - title = "S3 Buckets" - - width = 3 - sql = <<-EOQ - select - title, - arn, - region - from - aws_s3_bucket - where name like '%vandelay%' - order by - title - EOQ - - column "region" { - display = "none" - } - - column "arn" { - wrap = "all" - } -} -``` - - -### Table in line mode - - -```hcl -table { - type = "line" - title = "S3 Buckets" - width = 4 - - sql = <<-EOQ - select - title, - arn, - region - from - aws_s3_bucket - where - name = 'vandelay-industries-elaines-bucket' - order by - title - EOQ -} -``` - - - -### Table with links - - -```hcl -table { - title = "Attached Security Groups" - width = 4 - sql = <<-EOQ - select - instance_id, - region, - sg->>'GroupId' as security_group - from - aws_ec2_instance, - jsonb_array_elements(security_groups) as sg - EOQ - - column "security_group" { - href = "${dashboard.aws_vpc_security_group_detail.url_path}?input.security_group_id={{.'security_group' | @uri}}" - } -} -``` diff --git a/docs/reference/mod-resources/text.md b/docs/reference/mod-resources/text.md deleted file mode 100644 index 2d5a117..0000000 --- a/docs/reference/mod-resources/text.md +++ /dev/null @@ -1,60 +0,0 @@ ---- -title: Text -sidebar_label: text ---- - -# text - -> ***Powerpipe is now the recommended way to run dashboards and benchmarks!*** -> Mods still work as normal in Steampipe for now, but they are deprecated and will be removed in a future release: -> - [Steampipe Unbundled →](https://steampipe.io/blog/steampipe-unbundled) -> - [Powerpipe for Steampipe users →](https://powerpipe.io/blog/migrating-from-steampipe) - -Display either rendered `markdown` ([GitHub Flavored Markdown](https://github.github.com/gfm/)) or `raw` text with no interpretation of markup. - -Text blocks can be declared as named resources at the top level of a mod, or be declared as anonymous blocks inside a `dashboard` or `container`, or be re-used inside a `dashboard` or `container` by using a `text` with `base = .text.`. - - - -## Example Usage - - - -```hcl - -text { - width = 2 - value = <<-EOM - # I am some markdown text. - - *I* respect ***markdown***. - - EOM -} -``` - - -## Argument Reference -| Argument | Type | Optional? | Description -|-|-|-|- -| `base` | Text Reference | Optional | A reference to a named `text` resource that this `text` should source its definition from. `title` and `width` can be overridden after sourcing via `base`. -| `title` | String | Optional | A plain text [title](/docs/reference/mod-resources/dashboard#title) to display for this text. -| `type` | String | Optional | `markdown` (default) or `raw`. -| `value` | String | Optional | The `markdown` or `html` string to use. Can also be sourced using the HCL `file` function. -| `width` | Number | Optional | The [width](/docs/reference/mod-resources/dashboard#width) as a number of grid units that this item should consume from its parent. - - - -### More Examples - - ### Plain Text - - - -```hcl -text { - width = 2 - type = "raw" - value = "

I am a HTML title, but I'll be displayed as-is

" -} -``` \ No newline at end of file diff --git a/docs/reference/mod-resources/variable.md b/docs/reference/mod-resources/variable.md deleted file mode 100644 index ffaacf2..0000000 --- a/docs/reference/mod-resources/variable.md +++ /dev/null @@ -1,60 +0,0 @@ ---- -title: variable -sidebar_label: variable ---- - - -# variable - -> ***Powerpipe is now the recommended way to run dashboards and benchmarks!*** -> Mods still work as normal in Steampipe for now, but they are deprecated and will be removed in a future release: -> - [Steampipe Unbundled →](https://steampipe.io/blog/steampipe-unbundled) -> - [Powerpipe for Steampipe users →](https://powerpipe.io/blog/migrating-from-steampipe) - -[Variables](mods/mod-variables#input-variables), are module-level objects that essentially act as parameters for the module. When running `steampipe check`, you can pass values on the command line, from a `.spvars` file, or from environment variables, and you will be prompted for any variables that have no values. - -You can reference variable values as `var.` - - - -## Example Usage -```hcl -variable "instance_state" { - type = string - default = "stopped" -} - -query "instances_in_state" { - sql = "select instance_id, instance_state from aws_ec2_instance where instance_state = $1;" - param "find_state" { - default = var.instance_state - } -} -``` - -## Argument Reference -| Argument | Type | Optional? | Description -|-|-|-|- -| `default` | Any |Optional| A default value. If no value is passed, the user is not prompted and the default is used. -| `description` | String| Optional| A description of the variable. This text is included when tne user is prompted for a variable's value. -| `type` | String | Optional | The [variable type](#variable-types). This may be a simple type or a collection. - - - -## Variable Types -Variables may be simple types: -- `string` -- `number` -- `bool` - -Variables may also be collection types: -- `list()` -- `set()` -- `map()` -- `object({ = , ... })` -- `tuple([, ...])` - -The keyword `any` may be used to indicate that any type is acceptable diff --git a/docs/reference/mod-resources/with.md b/docs/reference/mod-resources/with.md deleted file mode 100644 index 53ea2e8..0000000 --- a/docs/reference/mod-resources/with.md +++ /dev/null @@ -1,142 +0,0 @@ ---- -title: With -sidebar_label: with ---- - -# with - -> ***Powerpipe is now the recommended way to run dashboards and benchmarks!*** -> Mods still work as normal in Steampipe for now, but they are deprecated and will be removed in a future release: -> - [Steampipe Unbundled →](https://steampipe.io/blog/steampipe-unbundled) -> - [Powerpipe for Steampipe users →](https://powerpipe.io/blog/migrating-from-steampipe) - -Some resources may also include `with` blocks. Similar to a `with` clause in a Postgres CTE, the `with` block allows you to specify additional queries or SQL statements to run **first**, and then pass the query results as arguments to `sql`, `query`, and `node` & `edge` blocks. - -`with` is not a top-level named resource in its own right - it is ONLY a block within other resources. - -You can only specify `with` blocks on `dashboard`, `graph`, `hierarchy`, and `flow`, and only when they are defined as **top-level named resources** in your mod. The results of the `with` query can be referenced only within the resource in which it is defined (including any sub-blocks). - -## Example Usage -```hcl -dashboard "with_ex2" { - - input "vpc" { - sql = <<-EOQ - select - title as label, - vpc_id as value - from aws_vpc - EOQ - } - - with "subnets" { - sql = <<-EOQ - select - subnet_id - from - aws_vpc_subnet - where - vpc_id = $1 - EOQ - - args = [self.input.vpc.value] - } - - graph { - - node "subnet" { - sql = <<-EOQ - select - subnet_id as id, - title, - json_build_object( - 'id', subnet_id, - 'region', region, - 'account id', account_id - ) as properties - from - aws_vpc_subnet - where - subnet_id = any($1) - EOQ - - args = [with.subnets.rows[*].subnet_id] - } - - node "vpc"{ - sql = <<-EOQ - select - vpc_id as id, - title, - json_build_object( - 'id', vpc_id, - 'region', region, - 'account id', account_id - ) as properties - from - aws_vpc - where - vpc_id = $1 - EOQ - - args = [self.input.vpc.value] - } - - edge { - sql = <<-EOQ - select - vpc_id as from_id, - subnet_id as to_id - from - aws_vpc_subnet - where - vpc_id = $1 - EOQ - - args = [self.input.vpc.value] - } - - } -} -``` - -## Argument Reference -| Argument | Type | Optional? | Description -|-|-|-|- -| `args` | Map | Optional| A map of arguments to pass to the query. -| `query` | Query Reference | Optional | A reference to a [query](reference/mod-resources/query) resource that defines the query to run. You must either specify the `query` argument or the `sql` argument, but not both. -| `sql` | String | Optional | An SQL string to provide data for the `edge`. You must either specify the `query` argument or the `sql` argument, but not both. - - - -## Referencing `with` results -`with` blocks are scoped to the top-level resource in which they are defined. You can reference the results of a `with` block in any sub-block of that resource. - -`with` blocks can only be added to **top-level named resources**. You cannot add `with` blocks to **top-level anonymous resources** or **sub-resources**. - -You can reference the `with` query results as `with..rows`. - -For example, given the following `with` block: -```h -with "stuff1" { - sql = "select a, b from table" -} -``` - -`with..rows` is a list, and you can index it to get a single row. Each row, in turn, contains all the columns, so you can get a single column of a single row: -```h -with.stuff1.rows[0].a -``` - -If you [splat](https://developer.hashicorp.com/terraform/language/expressions/splat) the row, then you can get an array of a single column from all rows. This would be passed to sql as an array: -```h -with.stuff1.rows[*].a -``` -- if `a` is a scalar value, then `with.stuff1.rows[*].a` is an array of scalar values -- if `a` is an array or jsonb array, then `with.stuff1.rows[*].a` is an array of arrays. - -At this time, you cannot pass an entire set (`with..rows`) to a query - you may pass either a single value (`with..rows[0].column`) or an array from a single column (`with..rows[*].column`). - diff --git a/docs/reference/overview.md b/docs/reference/overview.md index 5ca4241..a3ede7f 100644 --- a/docs/reference/overview.md +++ b/docs/reference/overview.md @@ -10,5 +10,4 @@ When all else fails, read the manual... - **[Command line reference →](reference/cli/overview)** - **[Config File reference →](reference/config-files/overview)** - **[Meta-Commands reference →](reference/dot-commands/overview)** -- **[Mod Resource reference →](reference/mod-resources/overview)** - **[Environment Variables reference →](reference/env-vars/overview)**