Splunk Search
Highlighted

Index time field extraction/re-write

Path Finder

I currently have a custom sourcetype=vuln_scan that looks like this:

response_datetime="2014-01-24 06:41:22" scan_date="2014-01-24 06:41:22" org_id=AB5X1896 scan_id=1H6785E host_id=522ZB769 ip=190.1.19.15 testid=2533 vuln_type="FTP servers" vuln_risk=8 vuln_name="HP/UX FTPd Negative REST Buffer Overflow" port=21 protocol=tcp results= 

Our goal is to modify the automatic field extractions that occur due to the "=" sign with another field name. For instance ip=190.1.9.15 is automatically extracted giving us a field name "ip" with a value of "190.1.9.15". We would like to map to the common information model (CIM) using the field name "dest" instead of "ip" at index time, not at search time. How would we go about reaching this objective?

Tags (3)
0 Karma
Highlighted

Re: Index time field extraction/re-write

SplunkTrust
SplunkTrust

One of the easier options would be to configure field alias for the already extracted fields.

Steps for creating it from Splunk Web: (for the example you provided)
1. Go to Manager-> Fields -> Field Aliases
2. Click on New, Select destination app
3. Provide Name as dest
4. Select sourcetype as vuln_scan
5. In the field aliases section, first textbox - put "ip", second textbox - put "dest"
6. you can add more field aliases for the same source type by clicking add another field and repeating step 5.
7. Click on save once done. You search time field aliases will be available during search (Dont forget to set the appropriate sharing permission)

Steps for props.conf change: here

Update

If you want splunk to stop auto extracting field with key=value format, you can add "KVMODE=none" in your props.conf under the sourcetype vulnscan. Note that Splunk will not extract any field now and you would have to write your own custom field extraction for all the fields.

0 Karma
Highlighted

Re: Index time field extraction/re-write

Path Finder

Isn't Field Aliasing at search time? We require index time field extraction/re-write.

0 Karma
Highlighted

Re: Index time field extraction/re-write

Legend

Why do you require index-time? It's almost always a bad idea.

0 Karma
Highlighted

Re: Index time field extraction/re-write

Path Finder

I am not positive but I think we require index time to populate the dashboards for Splunk App for ES if I am not mistaken, which could definitely be the case. If this is not the case then field aliasing would work fine.

0 Karma
Highlighted

Re: Index time field extraction/re-write

Legend

I can guarantee you that is not the case. Actually Splunk strongly suggest against index-time field extractions in the ES docs (as well as the docs for the core Splunk product).

Highlighted

Re: Index time field extraction/re-write

Path Finder

Ok, I will give it a test run and let you know how it turns out. Thanks again for the pointers @Ayn and @somesoni2

0 Karma