From a reporting perspective, I have apache logs in a company standard format. Due to load balancing configuration, we have 3 possible fields where the source ip is noted.
These fields are as follows:
clientip (standard source IP field)
XFORWARDEDIP (x-forwarded-for http header)
nsclientip (load balancer's view of source ip)
Since all 3 fields exist in sourcetype=access_combined (apache) logs, how do I coalesce the fields to "src" to make it CIM compliant?
I will mention that Apache logs a hyphen "-" for null field values for the above too.
Thanks for your help.
The XFORWARDEDIP commonly matches clientip or SOURCEIP, but depending on the load balancer configuration either clientip is "-" or SOURCEIP is "-", they never match.
Regex is your friend .
Create a search time field extraction for the following
This will grab the IP address. click extract new fields and then I prefer to write My own regex.
Looks like the regex is being changed when I post, here is a working one
Try this regex as a search time extraction.
The Regex is fine for IP addresses, but there's no if logic to assess which IP to use since its common to have both clientip and XFORWARDEDIP present.
Inside props.conf do this:
EVAL-src = case((clientip != "-"), clientip, (X_FORWARDED_IP != "-"), X_FORWARDED_IP, (ns_client_ip != "-"), ns_client_ip)
this is a search time settings so will have no effect on a indexer (but should be in a TA which will be deployed on both SH and IDX)