doc/user: add considerations to the Grafana/Datadog guides (#23310)

This PR adds the same considerations applied [to the demos repository](MaterializeInc/demos#95) for the monitoring guides.
MaterializeInc · Nov 20, 2023 · d51d600 · d51d600
1 parent 81d575a
commit d51d600
Show file tree

Hide file tree

Showing 2 changed files with 21 additions and 4 deletions.
diff --git a/doc/user/content/manage/monitor/datadog.md b/doc/user/content/manage/monitor/datadog.md
@@ -72,8 +72,7 @@ which has been tried and tested in production environments.
      ...
    ```
 
-1. Then, configure the `queries` that the Prometheus SQL Exporter should run at
-   the specified `interval` to export metrics from Materialize.
+1. Then, configure the `queries` that the Prometheus SQL Exporter should run at the specified `interval`. Take [these considerations](#considerations) into account when exporting metrics from Materialize.
 
    ```yaml
     ...
@@ -157,3 +156,13 @@ defined in the sample `config.yml`.
     <br>
 
     <img width="1728" alt="Template Datadog monitoring dashboard" src="https://user-images.githubusercontent.com/11491779/216036715-9a4b4db7-8f93-4b6a-ac21-f7eb5a01d151.png">
+
+## Considerations
+
+Before adding a custom query, make sure to consider the following:
+
+1. The label set cannot repeat across rows within the results of the same query.
+2. Columns must not contain `NULL` values.
+3. Value columns must be of type `float`.
+4. The Datadog agent is subject to a limit of 2000 metrics.
+5. Queries can impact cluster performance.
diff --git a/doc/user/content/manage/monitor/grafana.md b/doc/user/content/manage/monitor/grafana.md
@@ -67,8 +67,7 @@ which has been tried and tested in production environments.
      ...
    ```
 
-3. Then, configure the `queries` that the Prometheus SQL Exporter should run at
-   the specified `interval` to export metrics from Materialize.
+3. Then, configure the `queries` that the Prometheus SQL Exporter should run at the specified `interval`. Take [these considerations](#considerations) into account when exporting metrics from Materialize.
 
    ```yaml
     ...
@@ -192,3 +191,12 @@ defined in the sample `config.yml`.
     <br>
 
     <img width="1728" alt="Template Grafana monitoring dashboard" src="https://github.com/joacoc/materialize/assets/11491779/500fdc03-546c-4f56-b2c3-dc4e92e04328">
+
+## Considerations
+
+Before adding a custom query, make sure to consider the following:
+
+1. The label set cannot repeat across rows within the results of the same query.
+2. Columns must not contain `NULL` values.
+3. Value columns must be of type `float`.
+4. Queries can impact cluster performance.