Splunk Search

Why does tstats returns events by sourcetype, but search doesn't?

jeremyhagand61
Communicator

I have been using tstats to get event counts by day per sourcetype, but when I search for events in some of the identified sourcetypes search returns no results. I am a Splunk admin and have access to All Indexes. Here are the searches I have run:

 

| tstats count where index=myindex groupby sourcetype,_time

 

One of the sourcetype returned was novell_groupwise (which was quite a surprise to me), but when I search

 

index=myindex sourcetype=novell_groupwise

 

on a day that tstats indicated there was events on, nothing is returned.

Can anyone explain this discrepancy?

Labels (1)
0 Karma
1 Solution

VatsalJagani
Champion

@jeremyhagand61 - One of the reasons could be the sourcetype is using rename attribute from props.conf for search time sourcetype renaming.

If you use rename attribute then when you search with "| search" then you need to specify the newly renamed sourcetype.

Vs tstats read indexed fields/data only so it does not able to see renamed sourcetype (because rename attribute is search time.)

(On the side note make sure you are running the search in the same timerange.)

--------
I hope this helps, if it does consider accepting/upvoting!!!

View solution in original post

msquicc
Explorer

I've been struggling with the sourcetype renaming and tstats for some time now. It doesn't honor the rename like normal searches, and it doesn't offer you a _sourcetype field.  In a normal search, _sourcetype contains the old sourcetype name:

index=* sourcetype=wineventlog 
| eval old_sourcetype = _sourcetype
| top sourcetype, old_sourcetype

 

I finally figured out an elegant way around it, though!  

| tstats count where index=* sourcetype=wineventlog* by index, sourcetype

```Update Renamed Sourcetypes```
| join type=left sourcetype
    [| rest /services/data/props/sourcetype-rename 
    | eval sourcetype = title
    | eval renamed_sourcetype = value
    | fields sourcetype, renamed_sourcetype] 
| eval old_sourcetype = sourcetype
| eval sourcetype = coalesce(renamed_sourcetype, sourcetype) 
| eval sourcetype_renamed_from = if(isnotnull(renamed_sourcetype),old_sourcetype,null())
| table index, sourcetype, count, sourcetype_renamed_from


Sometimes, I prefer to lowercase index and sourcetype in the main and joined query.  

Also, I'm not sure what rights the rest command requires.  If that's an issue you could always do the same with a lookup. 

I don't know if this will help the OP, but I do hope it benefits others that may not use tstats so much because of this issue.  

VatsalJagani
Champion

@msquicc - I have not tested the query myself, but I really like the idea!!!

0 Karma

VatsalJagani
Champion

@jeremyhagand61 - One of the reasons could be the sourcetype is using rename attribute from props.conf for search time sourcetype renaming.

If you use rename attribute then when you search with "| search" then you need to specify the newly renamed sourcetype.

Vs tstats read indexed fields/data only so it does not able to see renamed sourcetype (because rename attribute is search time.)

(On the side note make sure you are running the search in the same timerange.)

--------
I hope this helps, if it does consider accepting/upvoting!!!

Get Updates on the Splunk Community!

Improve Your Security Posture

Watch NowImprove Your Security PostureCustomers are at the center of everything we do at Splunk and security ...

Maximize the Value from Microsoft Defender with Splunk

 Watch NowJoin Splunk and Sens Consulting for this Security Edition Tech TalkWho should attend:  Security ...

This Week's Community Digest - Splunk Community Happenings [6.27.22]

Get the latest news and updates from the Splunk Community here! News From Splunk Answers ✍️ Splunk Answers is ...