Splunk Search

Index time field extraction/re-write

Adrian
Path Finder

I currently have a custom sourcetype=vuln_scan that looks like this:

response_datetime="2014-01-24 06:41:22" scan_date="2014-01-24 06:41:22" org_id=AB5X1896 scan_id=1H6785E host_id=522ZB769 ip=190.1.19.15 testid=2533 vuln_type="FTP servers" vuln_risk=8 vuln_name="HP/UX FTPd Negative REST Buffer Overflow" port=21 protocol=tcp results= 

Our goal is to modify the automatic field extractions that occur due to the "=" sign with another field name. For instance ip=190.1.9.15 is automatically extracted giving us a field name "ip" with a value of "190.1.9.15". We would like to map to the common information model (CIM) using the field name "dest" instead of "ip" at index time, not at search time. How would we go about reaching this objective?

Tags (3)
0 Karma

somesoni2
Revered Legend

One of the easier options would be to configure field alias for the already extracted fields.

Steps for creating it from Splunk Web: (for the example you provided)
1. Go to Manager-> Fields -> Field Aliases
2. Click on New, Select destination app
3. Provide Name as dest
4. Select sourcetype as vuln_scan
5. In the field aliases section, first textbox - put "ip", second textbox - put "dest"
6. you can add more field aliases for the same source type by clicking add another field and repeating step 5.
7. Click on save once done. You search time field aliases will be available during search (Dont forget to set the appropriate sharing permission)

Steps for props.conf change: here

Update

If you want splunk to stop auto extracting field with key=value format, you can add "KV_MODE=none" in your props.conf under the sourcetype vuln_scan. Note that Splunk will not extract any field now and you would have to write your own custom field extraction for all the fields.

0 Karma

Adrian
Path Finder

Ok, I will give it a test run and let you know how it turns out. Thanks again for the pointers @Ayn and @somesoni2

0 Karma

Ayn
Legend

I can guarantee you that is not the case. Actually Splunk strongly suggest against index-time field extractions in the ES docs (as well as the docs for the core Splunk product).

Adrian
Path Finder

I am not positive but I think we require index time to populate the dashboards for Splunk App for ES if I am not mistaken, which could definitely be the case. If this is not the case then field aliasing would work fine.

0 Karma

Ayn
Legend

Why do you require index-time? It's almost always a bad idea.

0 Karma

Adrian
Path Finder

Isn't Field Aliasing at search time? We require index time field extraction/re-write.

0 Karma
Did you miss .conf21 Virtual?

Good news! The event's keynotes and many of its breakout sessions are now available online, and still totally FREE!