Getting data from referenced sources if no data found in selected source?

boxmetal — Mon, 13 Mar 2023 10:09:31 GMT

Hi all,

I want to get data from an xml file from a selected source ( eg: Source_A, Source_B, ...). When there is no data found in xml file, is it possible to get data from the referenced xml sources and pick the latest one to display the data? The referenced sources are in a text file in the same location of selected source.

The structure of folders look like this:

D:\datasource\<source_name>\release.xml
D:\datasource\<source_name>\referenced_sources.txt

The referenced_source text file contains values seperated by commas. Example:

Source_A,Source_B

And my current SPL to retrieve data is:

index=sample_index source=*$selected_source$* source="*.xml"

Thanks in advance

Re: Getting data from referenced sources if no data found in selected source?

yuanliu — Sun, 19 Mar 2023 07:54:01 GMT

After reading the question again and again, I think I get the gist of the setup: the key is the differentiation between "selected source" and "referenced source". (The repeated use of Source_A and Source_B in both contexts makes the question extremely confusing.)

But it is still unclear what "data" means in each of the word's four occurrences, and what "no data found" or "get data" really mean. Let me make the following assumptions:

"data" simply means some value of a given field, or values of select fields.
"no data found" mean that given field has no value in the "selected source". Whether the field appeared in the source as null value or the field name doesn't appear in that source is unimportant.

For simplicity, I will be looking for a field named 'data'.

index=sample_index data=* (source="D:\datasource\*\release.xml" OR souce=D:\datasource\$selected_source$\referenced_source.txt) | eval referenced_source = if(source == "D:\datasource\$selected_source$\referenced_source.txt", split(_raw, ","), null()) | eval referenced_source = mvmap(referenced_source, "D:\datasource\\" . referenced_source . "\release.xml") | where source == "D:\datasource\$selected_source$\release.xml" OR source == referenced_source | eval selected_data = if(source == "D:\datasource\$selected_source$\release.xml", data, null()) | stats latest(data) as latest_data values(selected_data) as selected_data | eval data = coalesce(selected_data, latest_data)

So, it is possible to do what you wanted if the assumptions are correct. But it is not going to be particularly efficient because all sources have to be retrieved.

topic Re: Getting data from referenced sources if no data found in selected source? in Splunk Search

Getting data from referenced sources if no data found in selected source?

Re: Getting data from referenced sources if no data found in selected source?