Getting Data In

How to see www* as host from secure.log and access.log ?

princemanto2580
Path Finder

Hello Splunkers,

I am forwarding logs from Universal Forwarder, to a Search Peer (Standalone Inderxer) and doing the search from a standalone Search Head. I have done as far from my understanding. How can I see access.log and secure.log from host www1 -www9.

Below is the inputs.conf of my UF: (log path:- /opt/logs/www1 - www9)

[default]
host = UF-01-248

[monitor:///opt/log/www*/secure.log]
disabled = 0
host_segment = 5
sourcetype = secure.log
index = main

[monitor:///opt/log/www*/access.log]
disabled = 0
host_segment = 9
sourcetype = access.log
index = web
0 Karma
1 Solution

nkwong_splunk
Splunk Employee
Splunk Employee

You can use the host_segment attribute to choose any segment of the monitored path to be the host value. For example, a host_segment=3 setting should pick up the "www*" value from your above monitored path. Also, you can use regular expression with the host_regex attribute for more advanced ways to dynamically set the host value.

Here is the documentation and examples on how to dynamically setup the host value.
https://docs.splunk.com/Documentation/Splunk/6.5.1/Data/Setadefaulthostforaninput#Dynamically_set_th...

View solution in original post

0 Karma

pjvarjani
Path Finder

Try this:

[monitor:///opt/log/www*/secure.log]
disabled = 0
host_segment = 3
sourcetype = secure.log
index = main

[monitor:///opt/log/www*/access.log]
disabled = 0
host_segment = 3
sourcetype = access.log
index = web

Let me know if that doesn't work.

0 Karma

princemanto2580
Path Finder

alt text

alt text

0 Karma

princemanto2580
Path Finder

Hi Pankaj,

I followed this method to remove the events and reindex the same logs.

  1. Used |delete to delete all the events on Search Head
  2. On each Indexer use ./splunk stop and then ./splunk clean eventdata -index _fishbucket
  3. On a Universal-Forwarder rm -rf /opt/splunkforwarder/var/lib/splunk/fishbucket/*
  4. Put the stanza as u mention on Deployment Server and done ./splunk reload deploy-server to reflect it on UF.

[monitor:///opt/log/www*/secure.log]
disabled = 0
host_segment = 3
sourcetype = secure.log
index = main

[monitor:///opt/log/www*/access.log]
disabled = 0
host_segment = 3
sourcetype = access.log
index = web

  1. On Indexer done ./splunk start
0 Karma

princemanto2580
Path Finder

Sorry, I tried it earlier but didn't work.

0 Karma

pjvarjani
Path Finder

I tried this in my environment and its working perfectly

[monitor:///opt/log/www*/access.log]
index = web
host_segment = 3

[monitor:///opt/log/www*/secure.log]
host_segment = 3

Can you clear the fishbucket and try indexing the data again?

Thanks,
Pankaj

0 Karma

nkwong_splunk
Splunk Employee
Splunk Employee

You can use the host_segment attribute to choose any segment of the monitored path to be the host value. For example, a host_segment=3 setting should pick up the "www*" value from your above monitored path. Also, you can use regular expression with the host_regex attribute for more advanced ways to dynamically set the host value.

Here is the documentation and examples on how to dynamically setup the host value.
https://docs.splunk.com/Documentation/Splunk/6.5.1/Data/Setadefaulthostforaninput#Dynamically_set_th...

0 Karma

princemanto2580
Path Finder

During search when I putting index=web, it shows all individual host for access.log. But from Welcome screen, I can not see sourcetype as access.log.

0 Karma

princemanto2580
Path Finder

Thanks for the update, but I achieved 50% as per my requirement. As I would like to send this access.log into index = web.

Below changes, will work ?

[monitor:///opt/log/]
disabled = 0
host_segment = 3

[monitor:///opt/log/]
disabled = 0
host_segment = 3
sourcetype = access.log
index = web

0 Karma

esix_splunk
Splunk Employee
Splunk Employee

Yes they will.

0 Karma

somesoni2
SplunkTrust
SplunkTrust

Try setting host_segment (which is basically on what level the host is available in file path/source) to 3 for both. Seems like 3rd portion of the path is what you want as host.

In /opt/log/www*/ : opt-1st, log-2nd, www*-3rd
0 Karma

princemanto2580
Path Finder

Thanks for reviewing my post. You mean to say like below,

[default]
host = UF-01-248

[monitor:///opt/log/www*/secure*]
disabled = 0
host_segment = 5
sourcetype = secure.log
index = main

[monitor:///opt/log/www*/access*]
disabled = 0
host_segment = 9
sourcetype = access.log
index = web

My requirement is to see www1, www2 etc as individual host from Search Head with individual access.log or secure.log

0 Karma
Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...