topic Re: Dealing with Lists in Splunk Search

Dealing with Lists

richtate — Mon, 09 Aug 2021 19:06:27 GMT

I have network logs that show various network device communication that are in an index in Splunk. I have another index that has information about the devices that I need to report on. But I'm having issues because the network logs are summarizing the network activity and showing all the devices with the same activity, as seen below:

How can I get the individual information about the devices and/or how can I enumerate the information above. If I send to a table, the device_ids will be blank, even if there is only one device in the list.

Re: Dealing with Lists

richgalloway — Tue, 10 Aug 2021 00:04:03 GMT

Break the list up into separate events by using mvexpand device_ids.

Re: Dealing with Lists

ITWhisperer — Tue, 10 Aug 2021 07:03:44 GMT

your first search | rename device_ids as device_id | append [your second search | rename device_field as device_id] | stats values(*) as * by device_id

Re: Dealing with Lists

richtate — Tue, 10 Aug 2021 16:31:36 GMT

I attempted to use mxexpand but the same result on search was returned. Only 1 event exactly as shown above. How will this command create new events with the same data, only an individual device_id? I tried using a table, but still nothing:

index="test" sourcetype="network"
| mvexpand device_ids
| table device_ids description devices lastSeen score type

Sorry, I'm a n00b

Re: Dealing with Lists

richtate — Tue, 10 Aug 2021 16:33:14 GMT

device_ids still come in as a list and doesn't expand to find the device from the second list. What am I doing wrong?

Re: Dealing with Lists

ITWhisperer — Tue, 10 Aug 2021 16:38:21 GMT

Can you share the raw event rather than an image of the formatted event?

Re: Dealing with Lists

richtate — Tue, 10 Aug 2021 17:50:53 GMT

The base index of assets is:

index=asset sourcetype=Network:Asset
| table id ipAddress macAddress name type tags

The index with network activity is:

index=test sourcetype=network | table description device_ids devices lastSeen score type

I'm trying to merge the data together with the asset information so I know what and where the activity is. I'd like a table like this:

id ipAddress name description lastSeen score type macAddress tags

I could join the indexes to get what I'm looking for using device ids as you suggested. But the issue I'm having is the second SPL with the list of ids in the device_ids field. I don't know how to enumerate them to merge the asset index with the test index.

Re: Dealing with Lists

ITWhisperer — Tue, 10 Aug 2021 22:09:45 GMT

I can't see how we can help you without you sharing the events you are dealing with because we have suggested things based on our assumptions, which you say don't give you what you want, so our assumptions are probably incorrect. It is fine for you to not share, but I can't see how we can help you further in that case.

Re: Dealing with Lists

richtate — Tue, 10 Aug 2021 22:50:08 GMT

Here is the table output of the first SPL

Re: Dealing with Lists

richtate — Tue, 10 Aug 2021 22:55:53 GMT

Here is the second SPL in table format, you can see the device_ids are blank but the affectedDevicesCount show there are over 1300 ids in the device_ids field. That is why I posted the _raw unformated data to open because I could expand the list given for the device_ids field.

Re: Dealing with Lists

ITWhisperer — Wed, 11 Aug 2021 07:05:45 GMT

It looks like the field is not being extracted. In order to help you extract it in the search, we need to see the raw data from the events. You haven't posted the raw event - rerun the search in verbose mode, click on the events tab, expand one (or more) of the event (if it has more lines to show) and copy the raw data from the event.

Re: Dealing with Lists

richtate — Wed, 11 Aug 2021 15:29:41 GMT

Sorry, I'm a n00b. Here is the first event in the previous screenshot. You can see the device_ids are returned, but not shown on the table. I need to take each device_id and return an event for each id, that way I can join with the other index for network asset information. As you can see now, I don't know what asset it transmitting the packets.

Re: Dealing with Lists

ITWhisperer — Wed, 11 Aug 2021 15:44:45 GMT

| spath device_ids{} output=device_ids | mvexpand device_ids

Re: Dealing with Lists

richtate — Wed, 11 Aug 2021 16:16:31 GMT

Thank you, that is exactly what I was looking for.