Oneshot and Inputs.conf

[mgherman]

Hi,

I recently had to go back and add some historical data that was generated prior to splunk being installed on some servers.

To do this I used "./splunk add oneshot /var/log/nginx/error.log.1" on a server that is functioning as a Forwarder.

The data has been indexed and is available on Splunk server, however it appears that 2 additional indexed fields that are defined in $SPLUNK/etc/system/local/inputs.conf have not been added.

Data that has been generated and indexed in realtime has these fields defined.

==inputs.conf==

[default]
host = hostname.example.com
_meta = role::examplerole farm::examplefarm

Is this the correct behaviour and how can I define those indexed fields when using oneshot in future?

Regards,

Michael Herman

[Lowell · Accepted solution]

I think you would have to rewrite these explicit index field additions as a index-time transformer associated with the sourcetype that these events get assigned. From there it's a simple matter of making sure oneshot gets the sourcetype right; which can be down with a [source:....] pattern or a number of other ways.

The oneshot command would have no way of knowing that you have an inputs.conf entry to pick up a similarity named file; therefore it would have no way of knowing that you want to add some indexed fields to that input. Does that make sense? My understanding is that oneshot is like temporarily adding a completely new inputs.conf entry which therefore wouldn't have have a "_meta" key.

[Genti]

Im not sure where you have found the above option, it might work, but is definitely not documented..

Anyhow, a while back i submitted a bug about .csv files and oneshot not respecting the props/transforms but i have not tried it with regular log files (note csv use headers)

Question is: Does this have to happen at index time? Can you not use props/transforms in order to make a search time field extraction instead? That would be a lot cheaper (resource wise) for the indexer and faster. Also, you would not have to worry about if it works with oneshot or not.

Check the docs for more info on search time field extractions