Search values in the field of list type

[IVV]

Hello!
I have such events:
1: name="Alex" groups="['staff', 'manager', 'top']"
2: name="Paul" groups="['staff', 'admin']"
3: name="Bob" groups="['outstaffing', 'designer']

I want to search for values in groups.
All staff: "staff" in groups
All admins: "admin" in groups
etc.

Is there any good way to do it? I can change "groups" field format if it is necessary.

[IVV · Accepted solution]

Find another solution. You can use makemv. But before I need to change "groups" field format to groups="staff,manager,top". Then we can do makemv delim="," groups | where groups="staff".

[David]

Nice alternative solution, particularly if you've got the option to modify the source files. Don't forget to add that config to your fields.conf using the TOKENIZER setting so that you don't have to do in line regex, and anyone else can easily search for group=staff without having to put stars around it.

[David]

If you extract the groups individually (e.g., something like:

rex field=groups max_matches=100 "'(?<group>.*?)'"

) then you can do a | stats values(name) as name by group

Does that seem like what you're looking for?

[IVV]

Yeah, thank you. It works. But how can I speed up search? I have millions of events and it isn't fast enough. Should I save groups in another way?

[David]

Accelerated data models are a great approach for a problem like this. Make sure to add that field extraction to your Conf files (props and transforms, with MV_ADD=1; no one likes having to remember inline regex) and then leverage an accelerated data model + pivot.

I have one use case where manually doing a search took 15 minutes for the last 4 hours. With an accelerated data model, I was able to run over 30 days of data in under 12 seconds.

If you don't want to go that approach, you can always use report acceleration, which will do a similar thing but slightly easier.