Getting Data In

Props.conf, transforms.conf and Index Time field extractions not working in production environment

barora21
Engager

We are logging one application deployed in Kubernetes and ingesting its tomcat localhost access logs in Splunk via HEC (HF).

I've pushed the props.conf and transforms.conf on HF as well as on indexers through Indexer Master but the problem is that extractions as well as transforms are not working et all.

Sample log:

10.1.0.225 - - [12/Mar/2021:13:39:51 +0000] "PUT /outlookaddin/v1/edap/sessions HTTP/1.1" 200 25

===============================

props.conf

[tomcat:localhost]
NO_BINARY_CHECK = true
category = Custom
disabled = false
pulldown_type = true
ANNOTATE_PUNCT = false
SHOULD_LINEMERGE = false
TIME_PREFIX=\[
TIME_FORMAT = %d/%b/%Y:%H:%M:%S %z
LINE_BREAKER = ([\r\n]+)\d+\.\d+.\d+\.\d+
TRUNCATE = 0
EXTRACT-access =^(?P<ip>[^\s]+)\s(?P<indent>(-|\w+))\s(?P<user>(-|\w+))\s\[(?<req_time>[^\]]+)\]\s\"(?P<method>\w+)\s(?P<request_uri>[\S]+)\s(?P<protocol>[^\"]+)\"\s(?P<status>\d{3})\s(?P<bytes_sent>(?:\d+|-))
FIELDALIAS-bytes_in = bytes_sent AS bytes_in
FIELDALIAS-http_method = method AS http_method
FIELDALIAS-uri_query = request_uri AS uri_query
FIELDALIAS-ip = ip AS src
EVAL-bytes_in = if(bytes_in=="-", 0, bytes_in)
EVAL-bytes_sent = if(bytes_sent=="-", 0, bytes_sent)
EVAL-vendor_product = "Apache Tomcat"
EVAL-product_family = "Apache Foundation Software"
EVAL-bytes = coalesce(bytes_in, 0)+coalesce(bytes_out, 0)
FIELDALIAS-response_code = status AS response_code
TRANSFORMS-anonymize=token-anonymizer

==========================================

transforms.conf

[token-anonymizer]
REGEX = (?m)^(.*accessToken\=).+(tokenType.*refreshToken=).+(expiresInSeconds.*username\=)\w+(.+ParamKey-dimensions-ParamKey\-).*(ParamKey.+)
FORMAT = $1######&$2######&$3######&$4#######$5
DEST_KEY = _raw

barora21_0-1615561252767.png

 

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Did the extractions work in the Dev environment?  Are you searching in Verbose Mode?  Did you restart the indexers and HFs?  The props.conf file must also be installed on the search head because there are search-time fields in it.

---
If this reply helps you, Karma would be appreciated.
0 Karma
Get Updates on the Splunk Community!

Building Reliable Asset and Identity Frameworks in Splunk ES

 Accurate asset and identity resolution is the backbone of security operations. Without it, alerts are ...

Cloud Monitoring Console - Unlocking Greater Visibility in SVC Usage Reporting

For Splunk Cloud customers, understanding and optimizing Splunk Virtual Compute (SVC) usage and resource ...

Automatic Discovery Part 3: Practical Use Cases

If you’ve enabled Automatic Discovery in your install of the Splunk Distribution of the OpenTelemetry ...