Splunk Enterprise

Updating data added to Splunk

Explorer

I used ./splunk add oneshot “/your/log/file/myfile.log” –sourcetype myfile to add data to my instance of Splunk Light successfully. I followed steps from this blog post http://blogs.splunk.com/2014/03/21/search-command-coalesce/ to do it. Here is the content of the file:

Thu Mar 6 11:33:49 EST 2014 src_ip=1.1.1.1
Thu Mar 6 11:33:45 EST 2014 sourceip=8.1.2.3
Thu Mar 6 11:33:48 EST 2014 source_ip=1.1.1.0
Thu Mar 6 11:33:47 EST 2014 sip=1.1.1.199
Thu Mar 6 11:33:46 EST 2014 ip=
Thu Mar 6 11:33:46 EST 2014 ip=22.22.22.22

However when I made change to that file (added another line to it)

Thu Mar 7 11:33:46 EST 2014 ip=22.22.22.22

Splunk did not reflect the additional event. The same happens when I add a source using the UI, in Add Data>Upload files from your computer. Consequent changes to the file are not reflected in Splunk. Only the data that existed at the time of import is available.
What is the way to make Splunk to include later additions of content to the local file? I understand that this is a contrived example (adding an extra line manually is not what happens in real practice). Thank you.

Tags (1)
0 Karma

Splunk Employee
Splunk Employee

oneshot is exactly that: a one-shot upload. You want to use monitor. See Monitor files and directories in the Splunk Enterprise Getting Data In manual.

0 Karma

Explorer

Unfortunately I tried it without quotes, with double quotes and with single quotes with the same outcome:
Parameters must be in the form '-parameter value'

0 Karma

Splunk Employee
Splunk Employee

Hi,

In our discussion we found a conflation in the doc page. It doesn't say that you can use either

./splunk monitor -source <source> 

or

./splunk monitor <source>

I've updated the page to say that you can use either of those, but I've changed the examples to remove the -source argument since you will hardly ever use it unless you specifically want to. Apologies for any confusion.

Explorer

Thank you. I tried this: ./splunk add index -name "newindex"

which returned this: Index "newindex" added

then added monitor:

./splunk add monitor "/Users/myuser/Desktop/path/mylogs/firewall.log" -index newindex

which returned this, which seems to indicate success:

Added monitor of /Users/myuser/Desktop/path/mylogs/firewall.log

But the new sourcetype is not created. When I used add oneshot a new sourcetype was added. But add monitor command did not. Would you please let me know what else I am missing?

0 Karma

Splunk Employee
Splunk Employee

It should be there. Check the time range of your search to make sure that it includes the interval when you updated the file.

0 Karma

Explorer

I used the content of the original file that is in my post. It has time intervals. I don't even see it as a sourcetype. I have not even tested updating it since it is not showing in Spunk. I also used
-sourcetype firewall with the add monitor command but still no sourcetype is created.

0 Karma

Splunk Employee
Splunk Employee

Sorry!

./splunk add monitor -source /Users/myuser/path/mylogs/firewall.log -index newindex

or

./splunk add monitor "/Users/myuser/path/mylogs/firewall.log" -index newindex

I mashed up the two, my apologies.

0 Karma

Explorer

Thank you @ChrisG When I try this command:

./splunk add monitor source "/Users/myuser/path/mylogs/firewall.log" -index newindex 

I get this error:

Parameters must be in the form '-parameter value'
0 Karma

Splunk Employee
Splunk Employee

Try it without quotes:

./splunk add monitor source /Users/myuser/path/mylogs/firewall.log -index newindex

0 Karma