Getting Data In

Optimizing query designed to retrieve source and field names only

jsam019
New Member

Using the REST api, I am currently retrieving a set of events from Splunk and extracting all of the field names and log sources, simultaneously building a map of log sources and fields belonging to them. Is there any way that I can retrieve this data with a minimal payload? For example, if I pull back 1 record that is from LogSource1 and has Property1 equal to [some really long string], I really don't want that whole string back. I just need to consume LogSource1 and Property1. I'm open to any ideas.

0 Karma

to4kawa
SplunkTrust
SplunkTrust
source="A" |table * | foreach * [ eval <<FIELD>>="sourceA" ]  |append [ search source="B" | table * | foreach * [ eval <<FIELD>>="sourceB" ]  ] |stats values(*) as * | transpose 0 | where mvcount('row 1')=1

This query shows the fields from only one source. How about this?

0 Karma

woodcock
Esteemed Legend

You are making this impossible. You need to back all the way and explain the problem FULLY and clearly.

0 Karma

woodcock
Esteemed Legend

Where is Splunk in this? The source of the data? The destination of the data? You have told us almost nothing. You need to try again and give ALL the details.

0 Karma

jsam019
New Member

...it is the splunk REST api, sir. That is where the events are located.

0 Karma

woodcock
Esteemed Legend

That is not all the details; that is just one.

0 Karma

jsam019
New Member

I'm looking for suggestions to optimally retrieve event data via splunk's API aside from loading the entire event. I currently send basic SPL queries with a time range and pull out the fields and sources I see.. that results in gigantic payloads which I extract only those 2 pieces of data. I'm not sure what else needs to be clarified. I know about the field summary option, but that doesn't give me the log sources used for each field.

0 Karma

woodcock
Esteemed Legend

You can always end your SPL with | table Just the fields I need.

0 Karma

jsam019
New Member

The issue is that I don't know what fields are available since we have several log sources.

0 Karma

to4kawa
SplunkTrust
SplunkTrust
your search
| fieldsummary

try this and check your fields.

0 Karma

jsam019
New Member

@to4kawa I've alluded to that already. The issue is that it won't indicate which sources contained the fields.

0 Karma

to4kawa
SplunkTrust
SplunkTrust

after searching, select source from left side extract fields
and then, check your fields again.

0 Karma

jsam019
New Member

Thanks @to4kawa - I'm not sure what the SPL looks like for this but I'll try to play around with this. In the end, I want to be able to tell senior mgmt "here are the 10 fields we have, and these 2 are from source 1 while these 2 come from source 2" for today" so this seems to be closer to what I'm looking for.

0 Karma
.conf21 CFS Extended through 5/20!

Don't miss your chance
to share your Splunk
wisdom in-person or
virtually at .conf21!

Call for Speakers has
been extended through
Thursday, 5/20!