topic Re: Changing the sourcetype, not working in Getting Data In

Changing the sourcetype, not working

jravida — Thu, 08 May 2014 18:08:03 GMT

Hi folks,

I'm trying to figure out how to change the sourcetype of a log source I have started ingesting.

I installed a universal forwarder on the remote server, and told it to watch a folder for logs and send them to the universal forwarder on my distributed deployment. I named the index and sourcetype to "weblogs". I could only see the data when I changed the inputs.conf on the *remote server to state that (unintuitive considering it was the remote server, but it worked).

I wanted to change the sourcetype to "iis" instead of "weblogs" now that I have it ingesting and searching on that sourcetype. But when I change the sourcetype to "iis" on the remote server's inputs.conf, I don't get anything when I search on sourcetype=iis.

Is there some conf file I am missing? Do I change it on the universal forwarder or index server? Thanks in advance!

Re: Changing the sourcetype, not working

somesoni2 — Thu, 08 May 2014 18:53:25 GMT

Did you create a configuration for iis sourcetype in props.conf (your must have a stanza for weblogs, you can rename it to iis, update inputs.conf and restart)

Re: Changing the sourcetype, not working

jravida — Thu, 08 May 2014 19:23:38 GMT

I did this on the remote server's inputs.conf. I set it for index and sourcetype=weblogs. But when I change the sourcetype to "iis", and leave the index as weblogs, and restart the forwarder, I get no logs.

Which server's props.conf needs to be changed?

Re: Changing the sourcetype, not working

lguinn2 — Thu, 08 May 2014 21:14:45 GMT

So your configuration is (UF = universal forwarder)

remote system UF -> distributed deployment UF -> indexer

And on the remote system, you have an inputs.conf that looks like this

[monitor:///pathtologs]
index=weblogs
sourcetype=weblogs

I assume that you also have an outputs.conf that directs the remote system to forward to the distributed deployment UF. And on the distributed deployment UF, you have an inputs.conf that listens for the data from the remote system and outputs.conf that sends to the indexer. None of these config files need to say anything about the index or sourcetype. And you do not need a props.conf on any of the forwarders to collect the data.

However, on the indexer, you must have an indexes.conf that defines the weblogs index. (And of course an inputs.conf that listens for the forwarded data.)

IF you want to change the sourcetype on the remote system, you need to think about these things:

What about the data that is already indexed? Changing the sourcetype on the remote UF will not change anything in the data that is already indexed.
What index should the new data go to? If you simply change the sourcetype, the data will continue to go to the weblogs index. So you will need to search index=weblogs to find it. If you want to create a new index (iis?) for the data, you must define the new index first, then change the inputs.conf on the remote UF.
You could do a "sourcetype rename" to change the sourcetype from weblogs to iis for the data that has already been indexed. You would do this on the indexer.

Re: Changing the sourcetype, not working

jravida — Fri, 09 May 2014 20:22:47 GMT

Thanks for the well written response. It made it clear what I was trying to say in my brain.

I started with a new index, as this data is new and I'm not doing anything with it yet. I set the sourcetype and index both for "iis". Just changing the sourcetype but leaving the old index wasn't working. Now that the sourcetype is correct, Splunk is extracting the fields beautifully. Thanks again!