Skip to content

Tuning SELKS

Jump to bottom
Peter Manev edited this page Oct 25, 2016 · 50 revisions

Tuning, configuration and considerations

Each major component can be independently upgraded of the others(unless there is inter dependency of course for some version).

Tuning Suricata and Elasticsearch is a process and is highly dependent on a number of things and you can find some suggestions further down in this document.

Please carefully consider and test HW needs before putting into production.

SELKS is based on Debian live.

Suricata

Suricata is build with dropping privileges capability. Suricata 3.0RC1 at the moment of this writing - example below) is installed and configured with the following Suricata Configuration (info produced by suricata --build-info command):

root@SELKS:~# suricata --build-info
This is Suricata version 3.1dev (rev ec60208)
Features: NFQ PCAP_SET_BUFF LIBPCAP_VERSION_MAJOR=1 AF_PACKET HAVE_PACKET_FANOUT LIBCAP_NG LIBNET1.1  HAVE_HTP_URI_NORMALIZE_HOOK HAVE_NSS HAVE_LUA HAVE_LUAJIT HAVE_LIBJANSSON TLS
SIMD support: none
Atomic intrisics: 1 2 4 8 byte(s)
64-bits, Little-endian architecture
GCC version 4.9.2, C version 199901
compiled with _FORTIFY_SOURCE=2
L1 cache line size (CLS)=64
thread local storage method: __thread
compiled with LibHTP v0.5.21, linked against LibHTP v0.5.21

Suricata Configuration:
 AF_PACKET support:                       yes
 PF_RING support:                         no
 NFQueue support:                         yes
 NFLOG support:                           no
 IPFW support:                            no
 Netmap support:                          no
 DAG enabled:                             no
 Napatech enabled:                        no

 Unix socket enabled:                     yes
 Detection enabled:                       yes

 libnss support:                          yes
 libnspr support:                         yes
 libjansson support:                      yes
 hiredis support:                         no
 Prelude support:                         no
 PCRE jit:                                no, libpcre 8.35 blacklisted
 LUA support:                             yes, through luajit
 libluajit:                               yes
 libgeoip:                                yes
 Non-bundled htp:                         yes
 Old barnyard2 support:                   no
 CUDA enabled:                            no
 Hyperscan support:                       no
 Libnet support:                          yes

 Suricatasc install:                      yes

 Profiling enabled:                       no
 Profiling locks enabled:                 no

Development settings:
 Coccinelle / spatch:                     no
 Unit tests enabled:                      no
 Debug output enabled:                    no
 Debug validation enabled:                no

Generic build parameters:
 Installation prefix:                     /usr
 Configuration directory:                 /etc/suricata/
 Log directory:                           /var/log/suricata/

 --prefix                                 /usr
 --sysconfdir                             /etc
 --localstatedir                          /var

 Host:                                    x86_64-pc-linux-gnu
 Compiler:                                gcc (exec name) / gcc (real)
 GCC Protect enabled:                     yes
 GCC march native enabled:                no
 GCC Profile enabled:                     no
 Position Independent Executable enabled: yes
 CFLAGS                                   -g -O2 -fstack-protector-strong -Wformat -Werror=format-security
 PCAP_CFLAGS                               -I/usr/include
 SECCFLAGS                                -fstack-protector -D_FORTIFY_SOURCE=2 -Wformat -Wformat-security

Suricata Tuning Considerations

There are a number of things that do come into consideration and do require copious research when tuning your Suricata IDS/IPS - traffic type, size, pps, avg packet size, number/type of rules enabled, runmode/type, HW used/available, NIC and so on.

Some quick recommendations for better performance on high speed traffic you could consider are adjusting the below values(in /etc/suricata/suricata.yaml):

  • increase the default stream: memcap: and stream: reassembly: memcap: values
  • decrease the default flow-timeouts: (at least for TCP)
  • set checksum-validation: no
  • increase max-pending-packets value
  • confirm mpm-algo: ac-ks
  • change to high profile in -> detect-engine: profile: high
  • change mpm context to full -> detect-engine: sgh-mpm-context: full
  • increase the request-body-limit: and response-body-limit: values

and further more:

For a more thorough reading you could use those articles below:

plus for more information on what and how you are highly advised to refer to the Suricata's wiki page:

Elasticsearch and Logstash

Elasticsearch and Logstash use the default configuration settings and directories with only the following changes/additions for Elasticsearch in /etc/elasticsearch/elasticsearch.yml

# Prevent swapping
# as explained here -
# http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/setup-configuration.html
bootstrap.mlockall: true

#Disable multicast discovery
discovery.zen.ping.multicast.enabled: false

# Incease Thread poools #
threadpool:
    search:
        type: fixed
        size: 10000

# Increase memory for indexing
indices.memory.index_buffer_size: 25%

# Increase the refresh interval
index.refresh_interval: 20s

# Limit fielddata cache
indices.fielddata.cache.size: 35%

# as per
# https://www.elastic.co/guide/en/elasticsearch/guide/current/heap-sizing.html
# to help facilitate having everything on the same machine
cluster.routing.allocation.same_shard.host: true

# SELKS is all-in-one distro so we disable replicas
index.number_of_replicas: 0

# avoid issues with IP changes
network.bind_host: 127.0.0.1

Elasticsearch Tuning Considerations

Elasticsearch is capable of digesting of tens of thousands of logs per second and subsequently giving you the power of sorting, searching and filtering of billions of log entries seamlessly with the right set up/tuning. All that with native JSON log importing - a standard output of Suricata IDS/IPS.

When all JSON outputs are enabled in Suricata's /etc/suricata/suricta.yaml on a high speed traffic - that could easily result in hundreds of millions of events per day on a high speed network IDS/IPS-ing and the SELKS default configuration must be adjusted accordingly.

Some quick recommendations for better performance could be to increase the heap size and enable mem locking in /etc/default/elasticsearch (add to the bottom of the file) for example:

ES_HEAP_SIZE=8g
MAX_LOCKED_MEMORY=unlimited

save it and restart systemctl restart elasticsearch

If you have the memory - optimal mem config settings for a one node elasticsearch in SELKS configuration could be:

ES_HEAP_SIZE=30g
MAX_LOCKED_MEMORY=unlimited

NOTE: Please note you will need around 60G RAM for that set up for Elasticsearch - optimal performance.

There is much MUCH more to tuning Elasticsearch than just changing those values. Elasticsearch is a powerful and wonderful Open Source application and much more is needed to be able to understand it and tune it properly. Here are some basic recommendation links:

NOTE: Elasticsearch recommends using Oracle JAVA

In general you can use Elasticsearch to centralize all your logging needs from different systems across your organization and with that in mind - to get actually not only Suricata IDS/IPS data events but also correlate those to other systems logs as well - the pinnacle of it all.

Logstash Tuning Considerations

Below are some links to the official guide for configuration and performance tuning for Logstash :

Clone this wiki locally