Hello,
From a reporting perspective, I have apache logs in a company standard format. Due to load balancing configuration, we have 3 possible fields where the source ip is noted.
These fields are as follows:
clientip (standard source IP field)
X_FORWARDED_IP (x-forwarded-for http header)
ns_client_ip (load balancer's view of source ip)
Since all 3 fields exist in sourcetype=access_combined (apache) logs, how do I coalesce the fields to "src" to make it CIM compliant?
I will mention that Apache logs a hyphen "-" for null field values for the above too.
Thanks for your help.
Inside props.conf do this:
EVAL-src = case((clientip != "-"), clientip, (X_FORWARDED_IP != "-"), X_FORWARDED_IP, (ns_client_ip != "-"), ns_client_ip)
Inside props.conf do this:
EVAL-src = case((clientip != "-"), clientip, (X_FORWARDED_IP != "-"), X_FORWARDED_IP, (ns_client_ip != "-"), ns_client_ip)
Thanks! Does this go on both indexer and search head?
this is a search time settings so will have no effect on a indexer (but should be in a TA which will be deployed on both SH and IDX)
Looks like the regex is being changed when I post, here is a working one
\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}
Try this regex as a search time extraction.
The Regex is fine for IP addresses, but there's no if logic to assess which IP to use since its common to have both clientip and X_FORWARDED_IP present.
Can you put an example which may make it easier to understand 🙂
Is there a time where all fields will be the same?
The X_FORWARDED_IP commonly matches clientip or SOURCE_IP, but depending on the load balancer configuration either clientip is "-" or SOURCE_IP is "-", they never match.
Regex is your friend .
Create a search time field extraction for the following
\d{1,3}.\d{1,3}.\d{1,3}.\d{1,3}
This will grab the IP address. click extract new fields and then I prefer to write My own regex.