Show all fields for a list of sources and show 1 for fields that have values, 0 for the fields that have no value

[lost_alex]

Dear community,

I am using this community since years, so far I've found everything I needed. Now I am stuck!!!

I am trying the following:

I want to list all the index'es fields so when I build a query, to know immediately if a specific source has that field. Second part is easy. Once I have the list I know what I need to do. So, basically, I need something like this:

Fields	index1	index2	index3	indexn
field1	1	1	0	1
field2	0	0	1	1
fieldn	1	1	1	1

where 0 is when the field doesn't exist, 1 there is at least one value in the specific field.

My search looks like:

index IN

(

index 1

index2

indexn

)

| stats count(*) as * by index

| transpose column_name=Field header_field=index

|outputlookup whateverfile.csv

The problem with this search is that it takes ages, I don't need a full count. I just need to count the first value it gets and stop and then move on. In this way I will have a count of 0 if the field doesn't exist, 1 if exists.

Any ideas?

[isoutamo]

Hi

at least I don't know any other way than go through all events on index to see if the last event has different set of fields than previous events. This leads in logical level that you cannot stop processing that index when you have found some fields which already exists in another index or in earlier events.

If you want to get this quicker then probably you should create e.g. summary index where your are counting this in regular base?

r. Ismo

[lost_alex]

Thank you very much for your response. My idea is to cover situations where I build a query, I use some fields with conditions (as example: reputation=5) and looking to multiple sources. The problem is that not all sources have field reputation and by simply adding the sources in a macro, for example, it will give me the false fact that the source is covered by that specific scenario. 

Since I am using fields that should exist in every event, if there are no values within 7 days, it means that field doesn't exist for that source.

One option would be to be specific about fields I am interested rather than making a list of 700 fields where I would need only 3 but the idea was to create a list of these 700 fields for 50 sources and check for each use case if the used fields are in the list and for which source.