topic Re: Data upload to spool is truncated in Splunk Dev

Data upload to spool is truncated

philip_w — Mon, 11 Sep 2017 23:33:17 GMT

I'm using powershell to get a web page in order to keep track my service status.
I tested my script which can write the whole page into local file without problem.
Then I changed to write it to $SPLUNK_HOME/var/spool/splunk

However, I found from Splunk search it always only captured the first few lines in HTML before the

Can anyone tell there's any setting affecting spool indexing behavior?

Thanks!!

Re: Data upload to spool is truncated

lfedak_splunk — Mon, 11 Sep 2017 23:51:31 GMT

Hey @philip_w, did a portion of your post get cut off? This part: "However, I found from Splunk search it always only captured the first few lines in HTML before the" You can edit your post by pressing the gear icon to the top right of the post.

Re: Data upload to spool is truncated

woodcock — Tue, 12 Sep 2017 00:13:54 GMT

Why would you ever write to $SPLUNK_HOME at all, especially var? Please point us to splunk docs that describes the way you are using this directory (which so far as I know is for internal use regarding primarily summary indexing).

Re: Data upload to spool is truncated

philip_w — Tue, 12 Sep 2017 00:19:19 GMT

I thought writing file to spool is the easiest way if I don't want to keep the file after indexing. Ok, seems I shouldn't use without good knowledge.

There is another story about powershell... I initially wanted to get the page through stdin/out. I failed to, so I wrote the html content into file first

Re: Data upload to spool is truncated

woodcock — Tue, 12 Sep 2017 00:21:12 GMT

Maybe it is a thing now. Show the the docs page.

Re: Data upload to spool is truncated

woodcock — Tue, 12 Sep 2017 00:22:34 GMT

If you need to blast a few files into splunk using a script, then just use add oneshot:

https://docs.splunk.com/Documentation/SplunkCloud/latest/Data/MonitorfilesanddirectoriesusingtheCLI

Re: Data upload to spool is truncated

philip_w — Tue, 12 Sep 2017 00:29:41 GMT

I should go for [batch://] indeed.

Thank you for your advice!

Re: Data upload to spool is truncated

woodcock — Tue, 12 Sep 2017 00:33:28 GMT

Yes, that will delete after sending, if you configure it properly.