- Splunk Answers
- :
- Splunk Administration
- :
- Getting Data In
- :
- Splunk on-demand indexing of a file
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark Topic
- Subscribe to Topic
- Mute Topic
- Printer Friendly Page
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Splunk on-demand indexing of a file
I currently have a FIX log file which generates HUGE amounts of data every day. With my current license its impossible to index all that data.
Is it possible to get parts of the log file on-demand and then index and query them as and when needed? Is it possible to achieve this via a script invoked from a Splunk query?
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
or basically any way to directly query data without indexing it in Splunk?
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Also, see routing and filtering data, specifically to a null queue: http://docs.splunk.com/Documentation/Splunk/latest/Deploy/Routeandfilterdatad
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Yes, you can use scripted input (http://docs.splunk.com/Documentation/Splunk/5.0.1/AdvancedDev/ScriptSetup).
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I would say no to your question - there might in certain situations be some kind of hack that makes you do this but in essence this is not how Splunk was meant to work.
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
"Currently, it is not supported by Splunk"-- Not sure whether this command works anymore. any ideas guys?
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Possibly, but only if the data to index is located on the Splunk search head (or is reachable via a script on the search head). See http://docs.splunk.com/Documentation/Splunk/latest/SearchReference/Runshellscript . This will run a script on the search head.
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
ok sounds good in terms of permissions but as you mentioned this can be scheduled to run via cron or task scheduler. What I need is to run it on demand anytime and gat her data only from a particular time period(which I think can be done via arguements to the script). Bu thow to invoke the script on demand rather than on a schedule?
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Scripted inputs run under the context of the Splunk user. If that doesn't work for your specific needs, you could create a script that is run via Windows task scheduler (which will let you select the user to run the script as). The script just needs to do what ever parsing of the original log file you want, and then output it to a new file that you can have Splunk monitor/index.
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
scripted input don't think would fit the case as the script should be invokable via non admin users as well, which I am not sure can be done
Introducing Splunk Enterprise 9.2
Adoption of RUM and APM at Splunk
Routing logs with Splunk OTel Collector for Kubernetes
