How to see www* as host from secure.log and access.log ?

[princemanto2580]

Hello Splunkers,

I am forwarding logs from Universal Forwarder, to a Search Peer (Standalone Inderxer) and doing the search from a standalone Search Head. I have done as far from my understanding. How can I see access.log and secure.log from host www1 -www9.

Below is the inputs.conf of my UF: (log path:- /opt/logs/www1 - www9)

[default]
host = UF-01-248

[monitor:///opt/log/www*/secure.log]
disabled = 0
host_segment = 5
sourcetype = secure.log
index = main

[monitor:///opt/log/www*/access.log]
disabled = 0
host_segment = 9
sourcetype = access.log
index = web

[pjvarjani]

Try this:

[monitor:///opt/log/www*/secure.log]
disabled = 0
host_segment = 3
sourcetype = secure.log
index = main

[monitor:///opt/log/www*/access.log]
disabled = 0
host_segment = 3
sourcetype = access.log
index = web

Let me know if that doesn't work.

[princemanto2580]

[princemanto2580]

Hi Pankaj,

I followed this method to remove the events and reindex the same logs.

1. Used |delete to delete all the events on Search Head
2. On each Indexer use ./splunk stop and then ./splunk clean eventdata -index _fishbucket
3. On a Universal-Forwarder rm -rf /opt/splunkforwarder/var/lib/splunk/fishbucket/*
4. Put the stanza as u mention on Deployment Server and done ./splunk reload deploy-server to reflect it on UF.

[monitor:///opt/log/www*/secure.log]
disabled = 0
host_segment = 3
sourcetype = secure.log
index = main

[monitor:///opt/log/www*/access.log]
disabled = 0
host_segment = 3
sourcetype = access.log
index = web

1. On Indexer done ./splunk start

[princemanto2580]

Sorry, I tried it earlier but didn't work.

[pjvarjani]

I tried this in my environment and its working perfectly

[monitor:///opt/log/www*/access.log]
index = web
host_segment = 3

[monitor:///opt/log/www*/secure.log]
host_segment = 3

Can you clear the fishbucket and try indexing the data again?

Thanks,
Pankaj

[nkwong_splunk · Accepted solution]

You can use the host_segment attribute to choose any segment of the monitored path to be the host value. For example, a host_segment=3 setting should pick up the "www*" value from your above monitored path. Also, you can use regular expression with the host_regex attribute for more advanced ways to dynamically set the host value.

Here is the documentation and examples on how to dynamically setup the host value.
https://docs.splunk.com/Documentation/Splunk/6.5.1/Data/Setadefaulthostforaninput#Dynamically_set_th...

[princemanto2580]

During search when I putting index=web, it shows all individual host for access.log. But from Welcome screen, I can not see sourcetype as access.log.

[princemanto2580]

Thanks for the update, but I achieved 50% as per my requirement. As I would like to send this access.log into index = web.

Below changes, will work ?

[monitor:///opt/log/]
disabled = 0
host_segment = 3

[monitor:///opt/log/]
disabled = 0
host_segment = 3
sourcetype = access.log
index = web

[esix_splunk]

Yes they will.

[somesoni2]

Try setting host_segment (which is basically on what level the host is available in file path/source) to 3 for both. Seems like 3rd portion of the path is what you want as host.

In /opt/log/www*/ : opt-1st, log-2nd, www*-3rd

[princemanto2580]

Thanks for reviewing my post. You mean to say like below,

[default]
host = UF-01-248

[monitor:///opt/log/www*/secure*]
disabled = 0
host_segment = 5
sourcetype = secure.log
index = main

[monitor:///opt/log/www*/access*]
disabled = 0
host_segment = 9
sourcetype = access.log
index = web

My requirement is to see www1, www2 etc as individual host from Search Head with individual access.log or secure.log