topic Re: Basic Search with Sourcetype Filter Issue in Splunk Search

Basic Search with Sourcetype Filter Issue

markwymer — Tue, 29 Sep 2020 12:13:53 GMT

Hi all,

I'm not sure whether this is a bug or a 'holiday hangover'!

I used props.conf and transform.conf to re-sourcetype a specific message to a new sourcetype ( i will, also, be changing the index as well when I am satisfied!). The events show up in a search e.g.
index=ba.com.logs DOB_RESULTS
(DOB_RESULTS is the same string that I use in my regex in the transforms.conf)

There are 270 results returned and when I check the 'Sourcetype' field on the left it does show that all 270 events are now in the new Sourcetype - ba.com:authentication:dob

However, when I click on the new sourcetype to add it to my search string it returns zero results?

Throwing in a few wildcard's returns the correct results though....
index=ba.com.logs DOB_RESULTS sourcetype="ba.com:authentication*"
or
index=ba.com.logs DOB_RESULTS sourcetype="*authentication:dob"

Am I missing a trick here or is this a bug?

Cheers and Happy New Year to you all.
Mark

Re: Basic Search with Sourcetype Filter Issue

rjthibod — Thu, 29 Dec 2016 16:30:52 GMT

Am I not 100% that having two colons in a metadata field like sourcetype is a good thing. Reading the segmenters.conf spec (https://docs.splunk.com/Documentation/Splunk/latest/Admin/Segmentersconf), colon is a minor breaker and how splunk indexes up to the first minor breaker seems to suggest that it might be your problem. I am no expert on this aspect of Splunk, but it may be something to consider by replacing the second colon with an underscore.

Do you have anywhere else in your setup that has sourcetypes with two colons and this problem does not occur? If so, then you can probably ignore my first comments.

Re: Basic Search with Sourcetype Filter Issue

somesoni2 — Thu, 29 Dec 2016 16:34:51 GMT

How are you overriding/renaming the sourcetype? Also, can you try this and see if this works (run in smart mode)?

index=ba.com.logs DOB_RESULTS | search sourcetype="*authentication:dob"

Re: Basic Search with Sourcetype Filter Issue

markwymer — Fri, 30 Dec 2016 09:50:04 GMT

Hi there, thanks for your response / suggestion. Doing the 'second' search made no difference.

I'm doing the sourcetype override in the transforms.conf file. I 99.999% certain that is working correctly as I the initial search (without the sourcetype filter) works fine and the sourcetype field (in the list of 'Interesting Fields') shows the name correctly.

Re: Basic Search with Sourcetype Filter Issue

markwymer — Fri, 30 Dec 2016 10:13:47 GMT

Hi there, thanks for your response.

I haven't read that particular document, thanks for the link. I did, however find this old Blog post - http://blogs.splunk.com/2012/08/10/sourcetypes-what%E2%80%99s-in-name/ - that seems to suggest that is is possible.

I just did a search of the various props files that come with the Enterprise Security app and there are loads of pre-installed apps that use multiple colons in their sourcetype names. However, I can't find anything (except my own) that use a combination of periods and colons. I'll test that.

Thanks,
Mark.

Re: Basic Search with Sourcetype Filter Issue

markwymer — Mon, 09 Jan 2017 12:38:44 GMT

Hi,

I did some more testing and noticed a typo in my transforms.conf.

I had missed off the 'sourcetype::' directive. e.g.

FORMAT = ba.com:authentication:dob

rather than

FORMAT = sourcetype::ba.com:authentication:dob

Strangley, Splunk still recognised ba.com:authentication:dob as a sourcetype but I just couldn't use it as a search filter!

Thanks for you responses,
Mark.