Allow tailing directory instead of specific file

[justinas-b]

It would be very handy to have functionality which would allow to tail whole directory, instead of file which has to be explicitly provided. For example:

annotations:
  tailing-sidecar: tail:logs:/var/log/audit/**/*;

This functionality does not work at the moment and i must provide exact file name which will be tailed:

annotations:
  tailing-sidecar: tail1:logs:/var/log/audit/main_audit.log;

[kasia-kujawa]

Now it is possible to specify directory instead of file but outputs from all files in the directory are redirected to stdout of one tailing sidecar container, for example:
when tailing sidecar operator is deployed in the cluster and pod has following specification

apiVersion: v1
kind: Pod
metadata:
  name: pod-with-annotations
  namespace: tailing-sidecar-system
  annotations:
    tailing-sidecar: varlog:/var/log/*
spec:
  containers:
  - name: count
    image: busybox
    args:
    - /bin/sh
    - -c
    - >
      i=0;
      while true;
      do
        echo "example0: $i $(date)" >> /var/log/example0.log;
        echo "example1: $i $(date)" >> /var/log/example1.log;
        echo "example2: $i $(date)" >> /var/log/example2.log;
        i=$((i+1));
        sleep 1;
      done
    volumeMounts:
    - name: varlog
      mountPath: /var/log
  volumes:
  - name: varlog
    emptyDir: {}

then tailing sidecar is added to the Pod:

$ kubectl get pods -n tailing-sidecar-system pod-with-annotations
NAME                   READY   STATUS    RESTARTS   AGE
pod-with-annotations   2/2     Running   0          4m17s

and logs from tailing sidecar container has this form:

$ kubectl logs -n tailing-sidecar-system pod-with-annotations tailing-sidecar-0 --tail 21
example0: 307 Tue Feb 15 12:09:39 UTC 2022
example1: 307 Tue Feb 15 12:09:39 UTC 2022
example2: 307 Tue Feb 15 12:09:39 UTC 2022
example0: 308 Tue Feb 15 12:09:40 UTC 2022
example1: 308 Tue Feb 15 12:09:40 UTC 2022
example2: 308 Tue Feb 15 12:09:40 UTC 2022
example0: 309 Tue Feb 15 12:09:41 UTC 2022
example1: 309 Tue Feb 15 12:09:41 UTC 2022
example2: 309 Tue Feb 15 12:09:41 UTC 2022
example0: 310 Tue Feb 15 12:09:42 UTC 2022
example1: 310 Tue Feb 15 12:09:42 UTC 2022
example2: 310 Tue Feb 15 12:09:42 UTC 2022
example0: 311 Tue Feb 15 12:09:43 UTC 2022
example1: 311 Tue Feb 15 12:09:43 UTC 2022
example2: 311 Tue Feb 15 12:09:43 UTC 2022
example0: 312 Tue Feb 15 12:09:44 UTC 2022
example1: 312 Tue Feb 15 12:09:44 UTC 2022
example2: 312 Tue Feb 15 12:09:44 UTC 2022
example0: 313 Tue Feb 15 12:09:45 UTC 2022
example1: 313 Tue Feb 15 12:09:45 UTC 2022
example2: 313 Tue Feb 15 12:09:45 UTC 2022

I have a doubt if at this moment it is possible to make the solution to read files from directory and redirect them to stdout of different containers as Fluent Bit with out_gstdout plugin is used to read log files and write records to the stdout.
More information about sidecar container and Fluent Bit configuration can be found here: https://github.com/SumoLogic/tailing-sidecar/tree/main/sidecar

[justinas-b]

Hey @kkujawa-sumo ,

My folder structure looks something like:

/var/log/audit/20220215/file1.log
/var/log/audit/20220214/file2.log
/var/log/audit/main_audit.log

If the directory used in annotation contains other directories, it seems this does not work. Only root folder is parsed.
Meaning that only main_audit.log will be picked up while file1.log and file2.log will be skipped

[kasia-kujawa]

For nested structure of directories, you can use comma separated list of directories:

apiVersion: v1
kind: Pod
metadata:
  name: pod-with-annotations
  namespace: tailing-sidecar-system
  annotations:
    tailing-sidecar: varlog:/var/log/*/*,/var/log/*
spec:
  containers:
  - name: test
    image: busybox
    args:
    - /bin/sh
    - -c
    - >
      i=0;
      mkdir /var/log/test/;
      while true;
      do
        echo "example0: $i $(date)" >> /var/log/example0.log;
        echo "example1: $i $(date)" >> /var/log/test/example1.log;
        echo "example2: $i $(date)" >> /var/log/test/example2.log;
        i=$((i+1));
        sleep 1;
      done
    volumeMounts:
    - name: varlog
      mountPath: /var/log
  volumes:
  - name: varlog
    emptyDir: {}

$ kubectl logs -n tailing-sidecar-system pod-with-annotations  tailing-sidecar-0  --tail 6
example0: 142 Thu Feb 17 16:17:05 UTC 2022
example1: 142 Thu Feb 17 16:17:05 UTC 2022
example2: 142 Thu Feb 17 16:17:05 UTC 2022
example0: 143 Thu Feb 17 16:17:06 UTC 2022
example1: 143 Thu Feb 17 16:17:06 UTC 2022
example2: 143 Thu Feb 17 16:17:06 UTC 2022

the path in configuration can be set to any format accepted by Fluent Bit Tail plugin, please see also documentation of Path parameter in https://docs.fluentbit.io/manual/pipeline/inputs/tail/