Index time field extraction/re-write

[Adrian]

I currently have a custom sourcetype=vuln_scan that looks like this:

response_datetime="2014-01-24 06:41:22" scan_date="2014-01-24 06:41:22" org_id=AB5X1896 scan_id=1H6785E host_id=522ZB769 ip=190.1.19.15 testid=2533 vuln_type="FTP servers" vuln_risk=8 vuln_name="HP/UX FTPd Negative REST Buffer Overflow" port=21 protocol=tcp results= 

Our goal is to modify the automatic field extractions that occur due to the "=" sign with another field name. For instance ip=190.1.9.15 is automatically extracted giving us a field name "ip" with a value of "190.1.9.15". We would like to map to the common information model (CIM) using the field name "dest" instead of "ip" at index time, not at search time. How would we go about reaching this objective?

[somesoni2]

One of the easier options would be to configure field alias for the already extracted fields.

Steps for creating it from Splunk Web: (for the example you provided)
1. Go to Manager-> Fields -> Field Aliases
2. Click on New, Select destination app
3. Provide Name as dest
4. Select sourcetype as vuln_scan
5. In the field aliases section, first textbox - put "ip", second textbox - put "dest"
6. you can add more field aliases for the same source type by clicking add another field and repeating step 5.
7. Click on save once done. You search time field aliases will be available during search (Dont forget to set the appropriate sharing permission)

Steps for props.conf change: here

Update

If you want splunk to stop auto extracting field with key=value format, you can add "KV_MODE=none" in your props.conf under the sourcetype vuln_scan. Note that Splunk will not extract any field now and you would have to write your own custom field extraction for all the fields.

[Adrian]

Ok, I will give it a test run and let you know how it turns out. Thanks again for the pointers @Ayn and @somesoni2

[Ayn]

I can guarantee you that is not the case. Actually Splunk strongly suggest against index-time field extractions in the ES docs (as well as the docs for the core Splunk product).

[Adrian]

I am not positive but I think we require index time to populate the dashboards for Splunk App for ES if I am not mistaken, which could definitely be the case. If this is not the case then field aliasing would work fine.

[Ayn]

Why do you require index-time? It's almost always a bad idea.

[Adrian]

Isn't Field Aliasing at search time? We require index time field extraction/re-write.