Skip to content

Commit

Permalink
add docs for pipeling logs - steampipe db logs, powerpipe activity, f…
Browse files Browse the repository at this point in the history
…lowpipe executions
  • Loading branch information
johnsmyth committed Oct 24, 2024
1 parent 7687f1c commit 7fdc712
Show file tree
Hide file tree
Showing 15 changed files with 68 additions and 9 deletions.
6 changes: 4 additions & 2 deletions docs/sidebar.json
Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,8 @@
"items": [
"using/powerpipe/dashboards",
"using/powerpipe/snapshots",
"using/powerpipe/mods"
"using/powerpipe/mods",
"using/powerpipe/activity"
]
},
{
Expand All @@ -78,7 +79,8 @@
"items": [
"using/flowpipe/pipelines",
"using/flowpipe/triggers",
"using/flowpipe/mods"
"using/flowpipe/mods",
"using/flowpipe/executions"
]
}
]
Expand Down
22 changes: 22 additions & 0 deletions docs/using/flowpipe/executions.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
---
title: Executions
sidebar_label: Executions
---

# Executions

The **Executions** tab provides visibility into the Flowpipe pipeline and triggers executions. You can view the status of running and completed processes and view the logs.

To view your activity, navigate to your workspace. From the **Pipes** tab, select **Flowpipe**, then go to the **Executions** tab.

![](/images/docs/pipes/flowpipe/flowpipe_executions_list.png)

You can filter the list of processes using a query filter. Click the **Query** dropdown to select a pre-defined filter, or enter your own using the [Pipes query filter syntax](/pipes/docs/reference/query-filter#syntax).


![](/images/docs/pipes/flowpipe/flowpipe_executions_filtered.png)


Click on a process to view the detail. The process detail displays the status, which user initiated it the full detailed logs, with expandable and copyable data where available.

![](/images/docs/pipes/flowpipe/flowpipe_executions_detail.png)
22 changes: 22 additions & 0 deletions docs/using/powerpipe/activity.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
---
title: Activity
sidebar_label: Activity
---

# Activity

The Powerpipe **Activity** tab provides visibility into the Powerpipe scheduled snapshot processes. You can view the status of running and completed processes and view the logs.

To view your activity, navigate to your workspace. From the **Pipes** tab, select **Powerpipe**, then go to the **Activity** tab.

![](/images/docs/pipes/powerpipe/powerpipe_activity.png)

You can filter the list of processes using a query filter. Click the **Query** dropdown to select a pre-defined filter, or enter your own using the [Pipes query filter syntax](/pipes/docs/reference/query-filter#syntax).


![](/images/docs/pipes/powerpipe/powerpipe_activity_filtered.png)


Click on a process to view the detail. The process detail displays the status, which user initiated it the full detailed logs, with expandable and copyable data where available.

![](/images/docs/pipes/powerpipe/powerpipe_process_log.png)
2 changes: 1 addition & 1 deletion docs/using/powerpipe/dashboards.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ workspace, the [Dashboards](https://powerpipe.io/docs/run/dashboard) and
filter by name / tag etc.


Navigate to your workspace, then the the **Pipes** tab, and choose **Dashbaords**.
Navigate to your workspace, then from the **Pipes** tab choose **Powerpipe**.

![](/images/docs/pipes/gs_dashboard_list.png)

Expand Down
4 changes: 4 additions & 0 deletions docs/using/powerpipe/snapshots.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,10 @@ The **Snapshots** tab lists all the snapshots in the workspace, including the sn

![](/images/docs/pipes/powerpipe/powerpipe_snapshots_list.png)


You can filter the list of snapshots using a query filter. Click the **Query** dropdown to select a pre-defined filter, or enter your own using the [Pipes query filter syntax](/pipes/docs/reference/query-filter#syntax).


Click on a snapshot title to [view or edit it](#editing-snapshots).


Expand Down
6 changes: 5 additions & 1 deletion docs/using/steampipe/aggregators.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,11 @@ To manage the aggregators for your workspace, navigate to your workspace, then s

![](/images/docs/pipes/steampipe/steampipe_aggregators_list.png)

You will see a list of aggregators for the workspace. Click on an aggregator to view it. The **Connections** tab will list the connections that are currently included in the aggregator. You can change the handle or the included connections of the aggregator from the **Settings** tab.
You will see a list of aggregators for the workspace.

You can filter the list of aggregators using a query filter. Click the **Query** dropdown to select a pre-defined filter, or enter your own using the [Pipes query filter syntax](/pipes/docs/reference/query-filter#syntax).

Click on an aggregator to view it. The **Connections** tab will list the connections that are currently included in the aggregator. You can change the handle or the included connections of the aggregator from the **Settings** tab.

<img src="/images/docs/pipes/cloud-connections-edit-aggregator.png" width="400pt"/>
<br />
Expand Down
11 changes: 6 additions & 5 deletions docs/using/steampipe/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,6 @@ You can accelerate your queries with a Datatank. With Datatank, you can proactiv




## Steampipe Database Search Path

Your workspace includes a Steampipe database, and this database may include many schemas. When querying the database, any unqualified table names are resolved using the schema [search path](https://steampipe.io/docs/guides/search-path). Mods are written using unqualified names, which allows you to target specific [datatanks](/pipes/docs/using/steampipe/datatank), [connections](/pipes/docs/using/steampipe/connections), and [aggregators](/pipes/docs/using/steampipe/aggregators). Pipes is opinionated on the search path, because *usually* you want to use a table from the datatank if it exists, otherwise an aggregator if one exists, otherwise a connection.
Expand Down Expand Up @@ -74,7 +73,9 @@ You can also **Wake Up** (enable), **Sleep** (disable) or **Reboot** your worksp
| `deleted` | The workspace has been deleted.


<!--
| `pausing` | The workspace database is running, but all pipelines and datatanks are being paused.
| `paused` ? | [Future??] The workspace database is running, but all pipelines and datatanks are paused.
-->
## Query Log

Turbot Pipes provides a log of the queries that have been run against your
Steampipe workspace database. You can view the query logs from the Steampipe **Advanced** tab for your workspace. Navigate to your workspace, and from the **Pipes** tab select **Steampipe**. From the Steampipe **Advanced** tab, select **Query Log** from the left hand menu.

![](/images/docs/pipes/steampipe/steampipe_query_log.png)
4 changes: 4 additions & 0 deletions docs/using/steampipe/snapshots.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,10 @@ The **Snapshots** tab lists all the query snapshots in the workspace, including

![](/images/docs/pipes/steampipe/steampipe_snapshot_list.png)


You can filter the list of snapshots using a query filter. Click the **Query** dropdown to select a pre-defined filter, or enter your own using the [Pipes query filter syntax](/pipes/docs/reference/query-filter#syntax).


Click on a snapshot title to [view or edit it](#editing-snapshots).


Expand Down
Binary file added images/flowpipe/flowpipe_executions_detail.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/flowpipe/flowpipe_executions_filtered.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/flowpipe/flowpipe_executions_list.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/powerpipe/powerpipe_activity.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/powerpipe/powerpipe_activity_filtered.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/powerpipe/powerpipe_process_log.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/steampipe/steampipe_query_log.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

0 comments on commit 7fdc712

Please sign in to comment.