Hello Splunk Community, We have two types of logs being forwarded to splunk a simple .log file and json logs that are being forwarded to splunk.   I am only interested in one of the objects which has key-value pairs.  In this example I am only interested in the log object.   JSON LOG   { [-] kubernetes: { [+] } log: 2020-06-24T13:23:12.8735410Z CI=4fomit248-2e46-4omit9-8019-838cdac1a4a4 L=INFO This is some log message here HRM=GET HRU=http://00.00.000.00:80/bar/v1/foo IP=::ffff:00.00.000.000 AV=? HSC=200 ET=1 stream: stdout time: 2020-06-24T13:23:12.873853339Z }   In the log object I want the fields to be extracted as followed: Key Value CI 4fomit248-2e46-4omit9-8019-838cdac1a4a4 L INFO IP ::ffff:00.00.000.000 <THIS WOULD BE THE LOG MESSAGE NOT A KEY> This is some log message here   I understand how to parse fields from the spath output using regex. However I would prefer this is is parsed at index time. Our other set of logs are exactly what is in the log object they are not in JSON format and splunk picks up the fields just fine. Log from .log file   2020-06-24 06:41:31.195 ST=C5D17Domitted72738B0D136DA9 CI=b1d0b050-omitted-46d2-omitted-80a61dfadf7d L=INFO Some log message here HRM=GET SN=FOO MN=Get HRU=http://foo.omit/bar/v2/foobar IP=00.00.000.000 ET=31 HSC=200 FOWCF=4   Is it possible to extract the log object at index time and turn it into its own log where the key value pairs are extracted as fields? I also read this blog post is this the best approach? Eureka! Extracting key-value pairs from JSON fields  ... View more

I am trying to create a new field called collection which is extracted from the existing source field. I am able to extract the field during an adhoc search, but want to create it using the field transformations without having to generate a regex during each search. The source field value is just a path (ex: source=D:\Logs\SomeCollectionName_SomeDomain\SomeDomain.log) and I am extracting part of that path to aggregate by collection. I want to specifically want to target the source field to regex not _raw. BEFORE Extraction: source=D:\Logs\SomeCollectionName_SomeDomain\SomeDomain.log AFTER Extraction: collection=SomeCollectionName I have created a field transformation called testcollection and nothing is being extracted at search time. Here are my settings for the testcollection field transformations (permissions are set for everyone to be able to read in the search app) Maybe I have a misunderstanding of Field Transformations and should be using field extractions any guidance would be helpful. I am just using the default formatting, but maybe that is incorrect. Note: I have gone through the documentation for field transformations and field extracts. I understand how to extract new fields during a search, but I want this new field to be available to all of the users in our account. ... View more