topic retrieve indexes from splunk.log in Getting Data In

retrieve indexes from splunk.log

tay — Mon, 25 Mar 2024 14:06:34 GMT

Hello splunk community,

I have this query but I would also like to retrieve the index to which the sourcetype belongs

index=_internal splunk_server=* source=*splunkd.log* sourcetype=splunkd (component=AggregatorMiningProcessor OR component=LineBreakingProcessor OR component=DateParserVerbose OR component=MetricSchemaProcessor OR component=MetricsProcessor) (log_level=WARN OR log_level=ERROR OR log_level=FATAL) | rex field=event_message "\d*\|(?<st>[\w\d:-]*)\|\d*" | eval data_sourcetype=coalesce(data_sourcetype, st) | rename data_sourcetype as sourcetype | table sourcetype event_message component thread_name _time _raw | stats first(event_message) as event_message by sourcetype component

any ideas ?
thx in advance

Re: retrieve indexes from splunk.log

richgalloway — Mon, 25 Mar 2024 13:48:39 GMT

You can't retrieve the index from the log if it isn't there, which is the case for these events. You'll have to search for the index by sourcetype.

| tstats count where index=* sourcetype=data_sourcetype | fields - count

Re: retrieve indexes from splunk.log

tay — Mon, 25 Mar 2024 13:52:38 GMT

This is precisely my problem, I have to start from this command and therefore retrieve the index elsewhere... but then what happens if the indexes have sourcetype names in common?

Re: retrieve indexes from splunk.log

richgalloway — Mon, 25 Mar 2024 13:57:34 GMT

Nothing "happens". It's legitimate for a sourcetype to be present in more than one index. It may complicate your query, though.

Re: retrieve indexes from splunk.log

tay — Mon, 25 Mar 2024 14:16:39 GMT

so if I do a "join" with your query, the correct index will be associated with the sourcetype?

Re: retrieve indexes from splunk.log

richgalloway — Mon, 25 Mar 2024 16:59:59 GMT

I try to avoid join where possible, but I can't make this query work without it. See if this helps you.

index=_internal splunk_server=* source=*splunkd.log* sourcetype=splunkd (component=AggregatorMiningProcessor OR component=LineBreakingProcessor OR component=DateParserVerbose OR component=MetricSchemaProcessor OR component=MetricsProcessor) (log_level=WARN OR log_level=ERROR OR log_level=FATAL) | rex field=event_message "\d*\|(?<st>[\w\d:-]*)\|\d*" | eval data_sourcetype=coalesce(data_sourcetype, st) | rename data_sourcetype as sourcetype | fields sourcetype event_message component | join sourcetype [| tstats count where index=* by sourcetype, index | fields - count ] | table sourcetype component event_message index

Re: retrieve indexes from splunk.log

PickleRick — Mon, 25 Mar 2024 20:43:25 GMT

Probably append with some stats values() would do the trick similarily to join.

Re: retrieve indexes from splunk.log

tay — Tue, 26 Mar 2024 08:07:23 GMT

Thanks, even if the query consumes a lot, but it works