I am trying to index new data and it is not happening.
I am indexing a single log file that is being written to by the server when ever new events are added.
I put this statement into the MSIADDED inputs on the universal forwarder because that is where my current input live.
This is what I added.
[Monitor://D:\Software\Waratek\HR-Config\HR.log] disabled = 0 sourcetype = waratek index = main
This is sample of the file.
2018-05-02 11:02:09,851 CEF:0|ARMR:CWE-114: Process Control|CWE-114: Process Control|1.0|Process Forking - 02|Load Rule|Low|outcome=success 2018-05-02 11:02:13,252 CEF:0|ARMR:CWE-114: Process Control|CWE-114: Process Control|1.0|Process Forking - 02|Link Rule|Low|outcome=success 2018-05-02 11:02:13,263 CEF:0|ARMR:CWE-114: Process Control|CWE-114: Process Control|1.0|Process Forking - 03|Load Rule|Low|outcome=success 2018-05-02 11:02:14,135 CEF:0|ARMR:CWE-114: Process Control|CWE-114: Process Control|1.0|Process Forking - 03|Link Rule|Low|outcome=success
I can see the sourcetype show up in data summary; however, when I search for the data there is nothing there. Any suggestions here?
In data summary, does the sourcetype shows any count? The events seems to be from May 2nd, does your time range large enough to include this? Does your user role has access to read data from index main?
It will show 64 lines. I did not count them specifically but it looks right.
I have been putting my searches to All Time searches.
I am an admin but I also just verified my role. I have default admin and rights to all non internal and internal indexes.
I have done multiple attempts at the input.conf file (tried it on a different server too). They all show similar issues. I just deleted my fishbucket on the forwarder again and restarted the service. But this has not made a difference in the past. I don't have anything to normalize the data yet but I can't see it soooooo
You could try this:
| tstats prestats=t count where sourcetype=waratek AND index=* by _time index | timechart count by index
Set the search range to include events from 10 years ago until 10 years in the future, just in case some strange timestamp recognition happens.
The timechart visualization should also show you the time range in which those events are, that might give you a hint what went wrong (e.g. wrong timestamp recognition = events in the future).