Syslog ingest pipeline causes Null Pointer Exception

[leonschmidt99]

Version

2.4.110

Installation Method

Security Onion ISO image

Description

other (please provide detail below)

Installation Type

Standalone

Location

on-prem with Internet access

Hardware Specs

Exceeds minimum requirements

CPU

12

RAM

48G

Storage for /

72G

Storage for /nsm

137G

Network Traffic Collection

tap

Network Traffic Speeds

Less than 1Gbps

Status

Yes, all services on all nodes are running OK

Salt Status

No, there are no failures

Logs

No, there are no additional clues

Detail

SO 2.4.110, no changes made to ingest pipelines

While testing syslog data ingestion, I encountered a Null Pointer Exception in the Syslog pipeline. This seems to be caused by a missing Null check in the 'global@custom' pipeline which is called by the 'common' pipeline which is called by the 'syslog' pipeline.

To reproduce, go to Kibana > Stack Management > Ingest Pipelines > search for and edit the "syslog" pipeline > "Test pipeline: Add documents" > Entirely copy and paste this test data and run:

[
  {
  "_source": {
    "message": "<134>1 2023-03-01T16:16:18.807431+00:00 testseconion foo: bar"
    }
  }
]

Expected behavior:
The syslog message can be ingested and parsed (at least partially).

Actual behavior:
Null pointer exception with this error message:

{
  "docs": [
    {
      "error": {
        "root_cause": [
          {
            "type": "script_exception",
            "reason": "runtime error",
            "script_stack": [
              "ctx.tags.0 == 'import'",
              "        ^---- HERE"
            ],
            "script": "ctx.tags.0 == 'import'",
            "lang": "painless",
            "position": {
              "offset": 8,
              "start": 0,
              "end": 22
            }
          }
        ],
        "type": "script_exception",
        "reason": "runtime error",
        "script_stack": [
          "ctx.tags.0 == 'import'",
          "        ^---- HERE"
        ],
        "script": "ctx.tags.0 == 'import'",
        "lang": "painless",
        "position": {
          "offset": 8,
          "start": 0,
          "end": 22
        },
        "caused_by": {
          "type": "null_pointer_exception",
          "reason": "cannot access method/field [0] from a null def reference"
        }
      }
    }
  ]
}

Fix:
I took this part of the 'global@custom' pipeline:

  {
    "set": {
      "if": "ctx.tags.0 == 'import'",
      "override": true,
      "field": "data_stream.dataset",
      "value": "import"
    }
  },
  {
    "set": {
      "if": "ctx.tags.0 == 'import'",
      "override": true,
      "field": "data_stream.namespace",
      "value": "so"
    }
  },

and replaced it with

  {
    "set": {
      "if": "ctx.tags?.0 == 'import'",
      "override": true,
      "field": "data_stream.dataset",
      "value": "import"
    }
  },
  {
    "set": {
      "if": "ctx.tags?.0 == 'import'",
      "override": true,
      "field": "data_stream.namespace",
      "value": "so"
    }
  },

i.e., I added a null check for ctx.tags. After this, the above example does not cause any errors and is ingested by the syslog parser.

Let me know if you have any questions or require further information.

Thank you and kind regards.

Guidelines

• I have read the discussion guidelines at Read before posting! #1720 and assert that I have followed the guidelines.

[reyesj2]

Thanks for the suggestion! Went ahead and created #14117 for this issue