group source types

[athorat]

Hi

I am getting the list of sourcetype and its usage in GB.
index=_internal source=*metrics.log | eval GB=kb/(1024*1024) | search group="per_sourcetype_thruput" | timechart span=1d sum(GB) by series limit=0 | sort -sourcetype

Want to group/categorize certain source types , like those with ecsweb:prod:E1, ecsweb:prod:E2 and ecsweb:prod:E3 in Category Prod.

Tried using the following query which does not work.
index=_internal source=metrics.log | eval GB=kb/(1024*1024) | search group="per_sourcetype_thruput" | timechart span=1d sum(GB) by series limit=0 | sort +sourcetype| eval series =if(series ="ecsweb:", "PROD", series)

[athorat]

I am not sure about rex because the soruce types are being captured only challenge is to bucket them by name/similar names types into specific categories.

[mmccul]

That's the point. You're capturing the sourcetypes into a field. A transform to define a new field with the reduced portion allows you to clump them according to the pattern you identified into a new field. rex field=sourcetype "^[^:]+:(?<newsrctyp>.*)" would strip out the portion up to and including the first colon for you and place into the field newsrctyp, which is now grouped.

[athorat]

Hey So the sources are coming in a field called series

index=_internal source=*metrics.log | eval GB=kb/(1024*1024) | search group="per_sourcetype_thruput" | timechart span=1d sum(GB) by series limit=0 | sort -sourcetype|

so the source type here should be Series? Tried both, no luck.
rex field=sourcetype "^[^:]+:(?<newsrctyp>.*)"

Correct me if I am wrong.

Thanks,
Anil.

[mmccul]

Run a rex to create a new field based on the sourcetype to capture the portion of the sourcetype you care about. Use that new transformed sourcetype field instead.

[athorat]

Filtering by sourcetype=ecsweb:prod:E* does return any results.
So the goal is, when the first query returns the results. The source types should be categorised into buckets.
like ECSweb:PROD :to PROD
ECSweb:PERF: to PERF.

[esix_splunk]

You can try this, again this shouldnt be in your _internal index...

index=_internal source=metrics.log group="per_sourcetype_thruput" | rex field=sourcetype "ecsweb:(?<newsrc>.*)" | eval GB=kb/(1024*1024) | timechart span=1d sum(GB) by newsrc | sort -newsrc

[athorat]

Throws an error as it has by series and by newsrc

Didnt get you this shouldnt be in your _internal index...???

[esix_splunk]

Updated search, please try.

[esix_splunk]

Something like this? However this sourcetype shouldnt be in _internal...

index=_internal source=metrics.log group="per_sourcetype_thruput" sourcetype=ecsweb:prod:E* | eval GB=kb/(1024*1024) | timechart span=1d sum(GB) by series limit=0 | sort +sourcetype