Getting Data In

How to send dbquery results to an indexer from a forwarder?

srinathd
Contributor

how to send dbquery results to indexer for indexing from forwarder.

 |dbquery dtParts limit=300000 "SELECT * from dtPart1 WITH (NOLOCK) where Id > 12 order by Id desc;"
0 Karma

lguinn2
Legend

The forwarder can only send data that is specified in inputs.conf to the indexer. Therefore, you have several choices:

1 - write a script that runs the query and sends the query results to stdout. Use the script as a "scripted input". Splunk will run the script periodically and forward the output of the script.

2 - write a search that runs the query and saves the output in a file. Run the search as a scheduled search. Set up a "monitor input" to read the saved results of the search.

3 - write a script that runs the query and appends the results to a file. I strongly suggest that the script prepend the query results with a full timestamp. You can have Splunk run the query as a "scripted input" - or you can run it from any batch job scheduler provided by your OS (such as cron). Set up a monitor input to read the resulting file. Set up a file rotation scheme to periodically remove/reset the file.

Personally, I prefer option #3, as it leaves behind a record of each script execution.

esix_splunk
Splunk Employee
Splunk Employee

3 is ideal in this.

Another option, although a bit more convoluted, is to use option 2. Schedule a search to run, then output that to a csv file (append? overwrite?) You can then run an input to monitor that csv file for changes, and have that transformed, indexed etc.

0 Karma
Get Updates on the Splunk Community!

Technical Workshop Series: Splunk Data Management and SPL2 | Register here!

Hey, Splunk Community! Ready to take your data management skills to the next level? Join us for a 3-part ...

Spotting Financial Fraud in the Haystack: A Guide to Behavioral Analytics with Splunk

In today's digital financial ecosystem, security teams face an unprecedented challenge. The sheer volume of ...

Solve Problems Faster with New, Smarter AI and Integrations in Splunk Observability

Solve Problems Faster with New, Smarter AI and Integrations in Splunk Observability As businesses scale ...