Getting Data In

search time extraction using transforms.conf SOURCE_KEY fields:mutliple,fields

EirikAskheim
Engager

I'm trying to make a field extraction with transforms.conf. I have a stanza in props.conf for the source.

[source::/path/to/file]
REPORT-some-name = extraction_rule_1, extraction_rule_2...

and in transforms.conf:

[extraction_rule_1]
REGEX = code_1 (?<special_field_1>[^,]*),(?<special_field_2>[^,]*)
SOURCE_KEY = fields:code,field_with_commasepareted_data

I do not get any "special_field_N" in my result.

Also tried to make a calulated field with code and field_with_commasepareted_data. This field works, but the extraction does not run on the calculated field for some reason.

The reason i try to do this is that I have around 200 codes that have different types of data in the field_with_commasepareted_data field.

sshelly_splunk
Splunk Employee
Splunk Employee

Try this:
In props.conf:
TRANSFORMS-set = myfieldextract
In transforms.conf
[my_fieldextract]
REGEX = "(\w+)":"([^"]+)
FORMAT = $1::$2
That will extract all fields/values for you (i think:))

0 Karma

EirikAskheim
Engager

From documentation:

SOURCE_KEY = <string>
* NOTE: This attribute is valid for both index-time and search-time field
  extractions.
* Optional. Defines the KEY that Splunk applies the REGEX to.
* For search time extractions, you can use this attribute to extract one or
  more values from the values of another field. You can use any field that
  is available at the time of the execution of this field extraction
* For index-time extractions use the KEYs described at the bottom of this
  file.
  * KEYs are case-sensitive, and should be used exactly as they appear in
    the KEYs list at the bottom of this file. (For example, you would say
    SOURCE_KEY = MetaData:Host, *not* SOURCE_KEY = metadata:host .)
* If <string> starts with "field:" or "fields:" the meaning is changed.
  Instead of looking up a KEY, it instead looks up an already indexed field.
  For example, if a CSV field name "price" was indexed then
  "SOURCE_KEY = field:price" causes the REGEX to match against the contents
  of that field.  It's also possible to list multiple fields here with
  "SOURCE_KEY = fields:name1,name2,name3" which causes MATCH to be run
  against a string comprising of all three values, separated by space
  characters.
* SOURCE_KEY is typically used in conjunction with REPEAT_MATCH in
  index-time field transforms.
* Defaults to _raw, which means it is applied to the raw, unprocessed text
  of all events.

Sample data:

{"timestamp":"2016-10-28T15:22:43.915+02:00","log_level":"INFO","src_ip":"1.2.3.4","app_name":"SOME-APP","ssn":"1234567890123","session_id":"4E1A043B.C25E.58134D14","agreement_id":"1234567890123","frontend_host":"www.sparebank1.no","market":"pm","thread":"qtp683287027-4312126","request_id":"REQID-7777deb3a68f","log_type":"audit","audit_code":"pm006","fields":"9876543212,Name of creditor","status":"ok"}

"fields" is different for every "audit_code"

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Do you have any sample data you can share?

---
If this reply helps you, Karma would be appreciated.
0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Modernize your Splunk Apps – Introducing Python 3.13 in Splunk

We are excited to announce that the upcoming releases of Splunk Enterprise 10.2.x and Splunk Cloud Platform ...

Step into “Hunt the Insider: An Splunk ES Premier Mystery” to catch a cybercriminal ...

After a whole week of being on call, you fell asleep on your keyboard, and you hit a sequence of buttons that ...

SplunkTrust Application Period is Officially OPEN!

It's that time, folks! The application/nomination period for the 2026-2027 SplunkTrust is officially open. If ...