topic Re: How to see www* as host from secure.log and access.log ? in Getting Data In

How to see www* as host from secure.log and access.log ?

princemanto2580 — Thu, 15 Dec 2016 15:52:50 GMT

Hello Splunkers,

I am forwarding logs from Universal Forwarder, to a Search Peer (Standalone Inderxer) and doing the search from a standalone Search Head. I have done as far from my understanding. How can I see access.log and secure.log from host www1 -www9.

Below is the inputs.conf of my UF: (log path:- /opt/logs/www1 - www9)

[default]
host = UF-01-248

[monitor:///opt/log/www*/secure.log]
disabled = 0
host_segment = 5
sourcetype = secure.log
index = main

[monitor:///opt/log/www*/access.log]
disabled = 0
host_segment = 9
sourcetype = access.log
index = web

Re: How to see www* as host from secure.log and access.log ?

somesoni2 — Thu, 15 Dec 2016 16:53:40 GMT

Try setting host_segment (which is basically on what level the host is available in file path/source) to 3 for both. Seems like 3rd portion of the path is what you want as host.

In /opt/log/www*/ : opt-1st, log-2nd, www*-3rd

Re: How to see www* as host from secure.log and access.log ?

princemanto2580 — Tue, 29 Sep 2020 12:08:11 GMT

Thanks for reviewing my post. You mean to say like below,

[default]
host = UF-01-248

[monitor:///opt/log/www*/secure*]
disabled = 0
host_segment = 5
sourcetype = secure.log
index = main

[monitor:///opt/log/www*/access*]
disabled = 0
host_segment = 9
sourcetype = access.log
index = web

My requirement is to see www1, www2 etc as individual host from Search Head with individual access.log or secure.log

Re: How to see www* as host from secure.log and access.log ?

nkwong_splunk — Tue, 20 Dec 2016 00:32:04 GMT

You can use the host_segment attribute to choose any segment of the monitored path to be the host value. For example, a host_segment=3 setting should pick up the "www*" value from your above monitored path. Also, you can use regular expression with the host_regex attribute for more advanced ways to dynamically set the host value.

Here is the documentation and examples on how to dynamically setup the host value.
https://docs.splunk.com/Documentation/Splunk/6.5.1/Data/Setadefaulthostforaninput#Dynamically_set_the_default_host_value

Re: How to see www* as host from secure.log and access.log ?

princemanto2580 — Tue, 20 Dec 2016 06:04:14 GMT

Thanks for the update, but I achieved 50% as per my requirement. As I would like to send this access.log into index = web.

Below changes, will work ?

[monitor:///opt/log/]
disabled = 0
host_segment = 3

[monitor:///opt/log/]
disabled = 0
host_segment = 3
sourcetype = access.log
index = web

Re: How to see www* as host from secure.log and access.log ?

pjvarjani — Tue, 20 Dec 2016 10:04:39 GMT

Try this:

[monitor:///opt/log/www*/secure.log]
disabled = 0
host_segment = 3
sourcetype = secure.log
index = main

[monitor:///opt/log/www*/access.log]
disabled = 0
host_segment = 3
sourcetype = access.log
index = web

Let me know if that doesn't work.

Re: How to see www* as host from secure.log and access.log ?

princemanto2580 — Tue, 20 Dec 2016 11:34:31 GMT

Sorry, I tried it earlier but didn't work.

Re: How to see www* as host from secure.log and access.log ?

pjvarjani — Tue, 20 Dec 2016 11:46:15 GMT

I tried this in my environment and its working perfectly

[monitor:///opt/log/www*/access.log]
index = web
host_segment = 3

[monitor:///opt/log/www*/secure.log]
host_segment = 3

Can you clear the fishbucket and try indexing the data again?

Thanks,
Pankaj

Re: How to see www* as host from secure.log and access.log ?

princemanto2580 — Tue, 20 Dec 2016 12:17:01 GMT

Hi Pankaj,

I followed this method to remove the events and reindex the same logs.

Used |delete to delete all the events on Search Head
On each Indexer use ./splunk stop and then ./splunk clean eventdata -index _fishbucket
On a Universal-Forwarder rm -rf /opt/splunkforwarder/var/lib/splunk/fishbucket/*
Put the stanza as u mention on Deployment Server and done ./splunk reload deploy-server to reflect it on UF.

[monitor:///opt/log/www*/secure.log]
disabled = 0
host_segment = 3
sourcetype = secure.log
index = main

[monitor:///opt/log/www*/access.log]
disabled = 0
host_segment = 3
sourcetype = access.log
index = web

On Indexer done ./splunk start

Re: How to see www* as host from secure.log and access.log ?

princemanto2580 — Tue, 20 Dec 2016 13:56:18 GMT

Re: How to see www* as host from secure.log and access.log ?

princemanto2580 — Thu, 22 Dec 2016 06:08:17 GMT

During search when I putting index=web, it shows all individual host for access.log. But from Welcome screen, I can not see sourcetype as access.log.

Re: How to see www* as host from secure.log and access.log ?

esix_splunk — Thu, 22 Dec 2016 09:26:32 GMT

Yes they will.