Skip to content

Commit

Permalink
fix links in FAQ
Browse files Browse the repository at this point in the history
  • Loading branch information
johnsmyth committed Oct 4, 2024
1 parent 1b56af7 commit 9e6302e
Showing 1 changed file with 23 additions and 19 deletions.
42 changes: 23 additions & 19 deletions docs/faq/overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,8 +22,8 @@ sidebar_label: FAQ

### What kinds of data sources can Steampipe query?

Steampipe's extensible [plugin](https://steampipe.io/docs/managing/plugins) model allows it so support a wide range of source data, including:
- Cloud providers like [AWS](https://hub.steampipe.io/plugins/turbot/aws), [Azure](https://hub.steampipe.io/plugins/turbot/azure), [GCP](https://hub.steampipe.io/plugins/turbot/gcp), [Cloudflare](https://hub.steampipe.io/plugins/turbot/cloudflare), [Alibaba Cloud](https://hub.steampipe.io/plugins/turbot/alicloud), [IBM CLoud](https://hub.steampipe.io/plugins/turbot/ibm), and [Oracle Cloud](https://hub.steampipe.io/plugins/turbot/oci).
Steampipe's extensible [plugin](/docs/managing/plugins) model allows it so support a wide range of source data, including:
- Cloud providers like [AWS](https://hub.steampipe.io/plugins/turbot/aws), [Azure](https://hub.steampipe.io/plugins/turbot/azure), [GCP](https://hub.steampipe.io/plugins/turbot/gcp), [Cloudflare](https://hub.steampipe.io/plugins/turbot/cloudflare), [Alibaba Cloud](https://hub.steampipe.io/plugins/turbot/alicloud), [IBM Cloud](https://hub.steampipe.io/plugins/turbot/ibm), and [Oracle Cloud](https://hub.steampipe.io/plugins/turbot/oci).
- Cloud-based services like[GitHub](https://hub.steampipe.io/plugins/turbot/github), [Zoom](https://hub.steampipe.io/plugins/turbot/zoom), [Okta](https://hub.steampipe.io/plugins/turbot/okta), [Slack](https://hub.steampipe.io/plugins/turbot/slack), [Salesforce](https://hub.steampipe.io/plugins/turbot/salesforce), and [ServiceNow](https://hub.steampipe.io/plugins/turbot/servicenow).
- Structured files like [CSV](https://hub.steampipe.io/plugins/turbot/csv), [YML](https://hub.steampipe.io/plugins/turbot/config) and [Terraform](https://hub.steampipe.io/plugins/turbot/terraform).
- Ad hoc investigation of [network services](https://hub.steampipe.io/plugins/turbot/net) like DNS & HTTP. You can even run [arbitrary commands]((https://hub.steampipe.io/plugins/turbot/exec)) on local or remote systems and query the output.
Expand All @@ -37,12 +37,12 @@ No. Plugins make API calls, results flow into Postgres as ephemeral tables that

### Can I use `psql`, `pgadmin`, or another client with Steampipe?

Yes. Steampipe exposes a [Postgres endpoint](https://steampipe.io/docs/query/third-party) that any Postgres client can connect to. When you start the Steampipe [service](https://steampipe.io/docs/managing/service), Steampipe will print the connection string to the console. You can also run `steampipe service status` to see the connection string.
Yes. Steampipe exposes a [Postgres endpoint](/docs/query/third-party) that any Postgres client can connect to. When you start the Steampipe [service](/docs/managing/service), Steampipe will print the connection string to the console. You can also run `steampipe service status` to see the connection string.


### Can I export query results as CSV, JSON, etc?

Yes. You can run [steampipe query](https://steampipe.io/docs/reference/cli/query) with the `--output` argument to capture results in CSV or JSON format:
Yes. You can run [steampipe query](/docs/reference/cli/query) with the `--output` argument to capture results in CSV or JSON format:
```bash
steampipe query --output json "select * from aws_account"
```
Expand All @@ -60,11 +60,11 @@ No. Steampipe is optimized for read-only query. However, it works closely with [

### How do I know what tables and columns are available to query?

In the Steampipe CLI you can use [.inspect](https://steampipe.io/docs/reference/dot-commands/inspect) to list tables by plugin name, e.g. `.inspect aws` to produce a selectable list of tables. When you select one, e.g. `.inspect aws_s3_bucket` you'll see the schema for the table. You can see the same information on the [Steampipe Hub](https://hub.steampipe.io/plugins/turbot/aws/tables/aws_s3_bucket#inspect).
In the Steampipe CLI you can use [.inspect](/docs/reference/dot-commands/inspect) to list tables by plugin name, e.g. `.inspect aws` to produce a selectable list of tables. When you select one, e.g. `.inspect aws_s3_bucket` you'll see the schema for the table. You can see the same information on the [Steampipe Hub](https://hub.steampipe.io/plugins/turbot/aws/tables/aws_s3_bucket#inspect).

### Can I query more than one AWS account / Azure subscription / GCP project?

Yes. You can create an [aggregator](https://steampipe.io/docs/managing/connections#using-aggregators). This works for multiple connections of the same type, including AWS/Azure/GCP as well as all other plugins.
Yes. You can create an [aggregator](/docs/managing/connections#using-aggregators). This works for multiple connections of the same type, including AWS/Azure/GCP as well as all other plugins.

It's common practice to use a script to generate the `.spc` files for an organization. See, for example, [https://github.com/happy240/steampipe-conn-generator-for-aws-organization](https://github.com/happy240/steampipe-conn-generator-for-aws-organization).

Expand All @@ -74,10 +74,10 @@ Turbot Pipes provides [integrations](https://turbot.com/pipes/docs/integrations)

### How well does Steampipe perform when querying multiple connections?

Multiple connections are queried in parallel, subject to plugin-specific [rate-limiting mechanisms](https://steampipe.io/docs/guides/limiter). Recent data is served from cache. Connection-level qualifiers, like AWS `account_id`, can reduce the number of connections queried.
Multiple connections are queried in parallel, subject to plugin-specific [rate-limiting mechanisms](/docs/guides/limiter). Recent data is served from cache. Connection-level qualifiers, like AWS `account_id`, can reduce the number of connections queried.

### How does Steampipe handle rate-limiting?
Generally, Plugins are responsible for handling rate limiting because the details are service specific. Plugins should typically recognize when they are being rate limited and backoff and retry either using their native Go SDK, the basic rate-limiting provided by the [plugin SDK](https://github.com/turbot/steampipe-plugin-sdk), or adding [limiters](https://steampipe.io/docs/guides/limiter) compiled in the plugin . You can also define your own custom [limiters](https://steampipe.io/docs/guides/limiter) in [configuration files](https://steampipe.io/docs/reference/config-files/overview).
Generally, Plugins are responsible for handling rate limiting because the details are service specific. Plugins should typically recognize when they are being rate limited and backoff and retry either using their native Go SDK, the basic rate-limiting provided by the [plugin SDK](https://github.com/turbot/steampipe-plugin-sdk), or adding [limiters](/docs/guides/limiter) compiled in the plugin . You can also define your own custom [limiters](/docs/guides/limiter) in [configuration files](/docs/reference/config-files/overview).

### How can I control the amount of memory used by Steampipe and plugins?

Expand All @@ -99,29 +99,29 @@ Alternatively, you can set a default memory limit for all plugin processes using

### Can plugin X have a table for Y?

If the plugin lacks a table you need, file a feature request (GitHub issue) for a new table in the applicable plugin repo, e.g. `github.com/turbot/steampipe-plugin-{pluginName}/issues`. Of course we welcome contributions! The following [guide](https://steampipe.io/docs/develop/writing-your-first-table) shows you how to write your first table.
If the plugin lacks a table you need, file a feature request (GitHub issue) for a new table in the applicable plugin repo, e.g. `github.com/turbot/steampipe-plugin-{pluginName}/issues`. Of course we welcome contributions! The following [guide](/docs/develop/writing-your-first-table) shows you how to write your first table.

### Does Steampipe have a plugin for X?

If you have an idea for a new plugin, file a [feature request](https://github.com/turbot/steampipe/issues/) (GitHub issue) with the label 'plugin suggestions'. We welcome code contributions as well. If you want to write a plugin, our [guide](https://steampipe.io/docs/develop/writing_plugins/overview) will help you get started.
If you have an idea for a new plugin, file a [feature request](https://github.com/turbot/steampipe/issues/) (GitHub issue) with the label 'plugin suggestions'. We welcome code contributions as well. If you want to write a plugin, our [guide](/docs/develop/writing_plugins/overview) will help you get started.

### How can I dynamically create Steampipe connections?

All connections are specified in [~/.steampipe/config/*.spc](https://steampipe.io/docs/reference/config-files/overview) files. Steampipe watches those files and reacts to changes, so if you build those files dynamically you can create connections dynamically.
All connections are specified in [~/.steampipe/config/*.spc](/docs/reference/config-files/overview) files. Steampipe watches those files and reacts to changes, so if you build those files dynamically you can create connections dynamically.

### Can I create and use regular Postgres tables?

Yes. Each Steampipe plugin defines its own foreign-table schema, but you can create native Postgres tables and views in the public schema.

### Can I use Steampipe plugins with my own database?

Yes. Most plugins support native [Postgres FDWs](https://steampipe.io/docs/steampipe_postgres/overview) and [SQLite Extensions](https://steampipe.io/docs/steampipe_sqlite/overview). Find the details for a plugin in its Steampipe Hub documentation, e.g. the AWS plugin [for Postgres](https://hub.steampipe.io/plugins/turbot/github#postgres-fdw) and for [SQLite](https://hub.steampipe.io/plugins/turbot/github#https://hub.steampipe.io/plugins/turbot/github#sqlite-extension).
Yes. Most plugins support native [Postgres FDWs](/docs/steampipe_postgres/overview) and [SQLite Extensions](/docs/steampipe_sqlite/overview). Find the details for a plugin in its Steampipe Hub documentation, e.g. the AWS plugin [for Postgres](https://hub.steampipe.io/plugins/turbot/github#postgres-fdw) and for [SQLite](https://hub.steampipe.io/plugins/turbot/github#https://hub.steampipe.io/plugins/turbot/github#sqlite-extension).

## Deployment

### Can I run Steampipe in a CI/CD pipeline or cloud shell?

Yes. Steampipe deploys as a single binary, it's easy to install and use in any [CI/CD pipeline or cloud shell](https://steampipe.io/docs/integrations/overview).
Yes, it's easy to install and use Steampipe in any [CI/CD pipeline or cloud shell](/docs/integrations/overview).


### Where is the Dockerfile or container example?
Expand All @@ -139,34 +139,38 @@ We welcome users to create and share open-source container definitions for Steam

### My query resulted in an error stating that is `missing 1 required qual`. What does that mean?

The error indicates that you must add a `where =` (or `join...on`) clause for the specified column to your query. The Steampipe database doesn't store data, it makes API calls to get data dynamically. There are times when listing ALL the elements represented by a table is impossible or prohibitively slow. In such cases, a table may [require you to specify a qualifier](http://localhost:3000/docs/sql/tips#some-tables-require-a-where-or-join-clause) in a where = (or join...on) clause.
The error indicates that you must add a `where =` (or `join...on`) clause for the specified column to your query. The Steampipe database doesn't store data, it makes API calls to get data dynamically. There are times when listing ALL the elements represented by a table is impossible or prohibitively slow. In such cases, a table may [require you to specify an equals qualifier](/docs/sql/tips#some-tables-require-a-where-or-join-clause) in a `where` or `join` clause.


### How can I know what API calls a plugin makes?
Steampipe plugins are open source, and you can inspect the code to see what calls it is making.

Some plugins (like the AWS plugin) provide information about the APIs being called using [function tags](https://steampipe.io/docs/guides/limiter#function-tags) that can be inspected by running Steampipe in [diagnostic mode](https://steampipe.io/docs/guides/limiter#exploring--troubleshooting-with-diagnostic-mode).
Some plugins (like the AWS plugin) provide information about the APIs being called using [function tags](/docs/guides/limiter#function-tags) that can be inspected by running Steampipe in [diagnostic mode](/docs/guides/limiter#exploring--troubleshooting-with-diagnostic-mode).



### Can I disable the Steampipe cache?

Yes. To disable caching at the server level, you can set the cache option to false in `~/.steampipe/config/default.spc`:
Yes. [Caching](/docs/guides/caching) significantly improves query performance and reduces API calls to external systems. It is enabled by default, but you can disable it either for the server or for a given client session.

To disable caching at the server level, you can set the cache option to false in `~/.steampipe/config/default.spc`:

```
options "database" {
cache = false
}
```

If you want to disable caching for a client session, you can do this in the CLI: `.cache off`.
Alternatively, set the [STEAMPIPE_CACHE](/docs/reference/env-vars/steampipe_cache) environment variable to `false` before starting Steampipe.

Within an interactive query session, you can disable caching for the client session with the [`.cache off` meta-command](/docs/reference/dot-commands/cache).


Or run Steampipe like so: `export STEAMPIPE_CACHE=false steampipe query`.

### A plugin isn't doing what I expect, how can I debug?

Steampipe writes plugin logs to `~/steampipe/logs/plugin-YYYY-MM-DD.log`. By default, these logs are written at `warn` level.
You can change the log level with the [STEAMPIPE_LOG_LEVEL](https://steampipe.io/docs/reference/env-vars/steampipe_log) environment variable:
You can change the log level with the [STEAMPIPE_LOG_LEVEL](/docs/reference/env-vars/steampipe_log) environment variable:

```
export STEAMPIPE_LOG_LEVEL=TRACE
Expand Down

0 comments on commit 9e6302e

Please sign in to comment.