Getting Data In

Props.conf, transforms.conf and Index Time field extractions not working in production environment

barora21
Engager

We are logging one application deployed in Kubernetes and ingesting its tomcat localhost access logs in Splunk via HEC (HF).

I've pushed the props.conf and transforms.conf on HF as well as on indexers through Indexer Master but the problem is that extractions as well as transforms are not working et all.

Sample log:

10.1.0.225 - - [12/Mar/2021:13:39:51 +0000] "PUT /outlookaddin/v1/edap/sessions HTTP/1.1" 200 25

===============================

props.conf

[tomcat:localhost]
NO_BINARY_CHECK = true
category = Custom
disabled = false
pulldown_type = true
ANNOTATE_PUNCT = false
SHOULD_LINEMERGE = false
TIME_PREFIX=\[
TIME_FORMAT = %d/%b/%Y:%H:%M:%S %z
LINE_BREAKER = ([\r\n]+)\d+\.\d+.\d+\.\d+
TRUNCATE = 0
EXTRACT-access =^(?P<ip>[^\s]+)\s(?P<indent>(-|\w+))\s(?P<user>(-|\w+))\s\[(?<req_time>[^\]]+)\]\s\"(?P<method>\w+)\s(?P<request_uri>[\S]+)\s(?P<protocol>[^\"]+)\"\s(?P<status>\d{3})\s(?P<bytes_sent>(?:\d+|-))
FIELDALIAS-bytes_in = bytes_sent AS bytes_in
FIELDALIAS-http_method = method AS http_method
FIELDALIAS-uri_query = request_uri AS uri_query
FIELDALIAS-ip = ip AS src
EVAL-bytes_in = if(bytes_in=="-", 0, bytes_in)
EVAL-bytes_sent = if(bytes_sent=="-", 0, bytes_sent)
EVAL-vendor_product = "Apache Tomcat"
EVAL-product_family = "Apache Foundation Software"
EVAL-bytes = coalesce(bytes_in, 0)+coalesce(bytes_out, 0)
FIELDALIAS-response_code = status AS response_code
TRANSFORMS-anonymize=token-anonymizer

==========================================

transforms.conf

[token-anonymizer]
REGEX = (?m)^(.*accessToken\=).+(tokenType.*refreshToken=).+(expiresInSeconds.*username\=)\w+(.+ParamKey-dimensions-ParamKey\-).*(ParamKey.+)
FORMAT = $1######&$2######&$3######&$4#######$5
DEST_KEY = _raw

barora21_0-1615561252767.png

 

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Did the extractions work in the Dev environment?  Are you searching in Verbose Mode?  Did you restart the indexers and HFs?  The props.conf file must also be installed on the search head because there are search-time fields in it.

---
If this reply helps you, Karma would be appreciated.
0 Karma
Get Updates on the Splunk Community!

Enterprise Security Content Update (ESCU) | New Releases

In December, the Splunk Threat Research Team had 1 release of new security content via the Enterprise Security ...

Why am I not seeing the finding in Splunk Enterprise Security Analyst Queue?

(This is the first of a series of 2 blogs). Splunk Enterprise Security is a fantastic tool that offers robust ...

Index This | What are the 12 Days of Splunk-mas?

December 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...