In this article, we’ll explore the use of a SIEM for it’s most optimal utility. To get the best value out of a SIEM, we need to ensure the logs ingested are high-value logs (explained below)
SIEM: A SIEM (Security Information and Event Management) takes in logs, assesses them against threat feeds and logic, and makes a determination about the status of those logs. SIEMs are generally expensive with ingestion costs being the primary cost factor. Ie, the greater the volume of ingested logs, the greater the cost. The storage (retention) cost is another factor, but retention is usually a minor part of the total cost.
Log Storage Solution: We must distinguish between a SIEM and a log storage solution. A log storage solution takes in logs and stores them in a searchable format. Such a system is great for long term and low cost storage. Such systems are often free if self-hosted, and may include an add-on to enhance capabilities to be more SIEM-like. A log storage solution typically (and ideally would) provide an analysis interface. Think Elasticsearch and Kibana. While it may have features for generating alerts for conditions (too many of X in a given time window), this is not its primary role.
Returning to the opening statement, to get the best value from a SIEM, we must ensure only high-value logs are ingested. A high-value log is one that could reasonably be expected to either directly or with additional information indicate an incident has occurred, or will occur. There are various options to achieve this. One is to only send logs from the source systems that match that criteria. Another is to filter the logs before that get to the SIEM. But allowing low-value logs from reaching the point of ingestion into a SIEM results in wasted money and a greater haystack to search for the needles.
The following diagram shows the flow of logs from the source, to an aggregator, and finally to either the SIEM or the log storage solution.
The above diagram shows that logs can either go to the SIEM or to the log storage solution. The aggregator decides which logs are high-value and which are low-value, and routes them accordingly. It could be that all logs go to the log storage solution, or just what doesn’t go to the SIEM. The most important consideration is that the SIEM only receive high-value logs.
The idea of a single pane view of logs is enticing and highly desirable. The cost is not. The solution is to use two solutions as indicated throughout this article. One solution for incident management, and the other for audits and troubleshooting.