topic syslog indexing in Getting Data In

syslog indexing

riqbal — Mon, 21 May 2018 08:13:36 GMT

However at this point I am getting logs from syslog data source and they are saved at /central/$hostname$/gateway.log
I install the UF on syslog server and below is my inputs.conf file.

[root@sysxx ~]# cat /opt/splunkforwarder/etc/system/local/inputs.conf

[default]

[monitor:///cental/gateway/]
index = sophos
sourcetype = sophos:utm:firewall
disabled = 0

All my logs are going to main index.
If I move index and sourcetype parameter above to [monitor:///cental/gateway/] then I can see the logs under index=sophos.

how can I solve this.

in future I will have logs from more data sources and I want to index them under different index name.

Re: syslog indexing

mayurr98 — Mon, 21 May 2018 09:46:12 GMT

I did not understand your question. If you put index = sophos sourcetype = sophos:utm:firewall data will go to sophos otherwise it will go to default index called main.

Re: syslog indexing

riqbal — Tue, 29 Sep 2020 19:39:20 GMT

let me explain in more detail:
1- I have one syslog server where all the network devices sending logs and that logs are saving at
/central/$hostname$/$hostname$.log
2- I install UF on that syslog server and configure it to send logs to HF.
3- with this config(as shown above), all logs are going to main index.

Interestingly, when I define index on top(before [monitor:///cental/gateway/]), the logs are getting saved in index=Sophos.

=========================================

I just experiment this on my workstation. my workstation is also sending logs to splunk.
below is my input.conf file.

[WinEventLog://Microsoft-Windows-Sysmon/Operational]
index = os_xx1
disabled = 0

with that, the logs are not getting saved in index = os_xx1.

BUT WHEN I CHAGNE props.conf and transforms.conf, the logs are going to right index.
below is props.conf and transforms.conf:
props.conf
[WinEventLog:Security]
TRANSFORMS-Windows = windows_security

[WinEventLog://Microsoft-Windows-Sysmon/Operational]

TRANSFORMS-Windows = windows_sysmon

transforms.conf
[windows_security]
REGEX = (.*)
FORMAT = os_xx1
WRITE_META = true

[windows_sysmon]
REGEX = (.*)
FORMAT = os_xx1
WRITE_META = true

========================================================

Re: syslog indexing

riqbal — Mon, 21 May 2018 12:51:02 GMT

referring "https://answers.splunk.com/answers/468907/is-it-possible-to-have-separate-indexes-within-a-s.html"
I think i am not referring correct REGEX value.

Re: syslog indexing

muebel — Mon, 21 May 2018 14:14:23 GMT

Hi riqbal, I think part of the issue might be related to some additional config you aren't aware of. Try running btool to get an view of all the inputs config. A usage example is:

/opt/splunk/bin/splunk btool --debug inputs list

This will give you a rolled up view of all inputs config.

Additionally, you can look at inputs config from Splunk with this app I made for this purpose : https://splunkbase.splunk.com/app/3923/

Although it seems less likely, there could also be some props.conf config causing issues (rewriting the index config), but I think doing a thorough examination of the inputs config at each step will be the most helpful thing to do.

Please let me know if this helps!