topic Re: How to group fields into a single line inside a multiline data? in Splunk Search

How to group fields into a single line inside a multiline data?

tha_ghost99 — Mon, 28 Nov 2022 17:35:25 GMT

my subject may not be worded correctly 😞 but i need some help.

i have the below raw data, and i would like to group them together into its own line for reporting.

Redundancy group: 0 , Failover count: 0
node0 200 primary no no None
node1 2 secondary no no None

Redundancy group: 1 , Failover count: 0
node0 200 primary no no None
node1 20 secondary no no None

Redundancy group: 2 , Failover count: 0
node0 200 primary no no None
node1 2 secondary no no None

how can i have the output with the following:
would like to group them based on Redundancy Group # and Node #

Redundancy group: 0 , Failover count: 0,node0 200 primary no no None
Redundancy group: 0 , Failover count: 0,node1 2 secondary no no None
Redundancy group: 1 , Failover count: 0,node0 200 primary no no None
Redundancy group: 1 , Failover count: 0,node1 2 secondary no no None
Redundancy group: 2 , Failover count: 0,node0 200 primary no no None
Redundancy group: 2 , Failover count: 0,node1 2 secondary no no None

Re: How to group fields into a single line inside a multiline data?

richgalloway — Mon, 28 Nov 2022 20:06:36 GMT

How would Splunk know to which Redundancy group any particular node belongs? There appears to be nothing in a "node" event that connects it to a Redundancy group event.

Re: How to group fields into a single line inside a multiline data?

tha_ghost99 — Mon, 28 Nov 2022 19:01:08 GMT

hi @richgalloway that is correct.

its basically just node0 and node1 because its below redundancy group0 for example.

so each redudancy group 1,2,3 has their own Node0 and Node1 output. would simply want to group them in 1 line.

hope it makes sense.

Re: How to group fields into a single line inside a multiline data?

richgalloway — Mon, 28 Nov 2022 21:44:42 GMT

Here's a run-anywhere query that demonstrates an ugly way to accomplish it.

| makeresults | eval data="Redundancy group: 0 , Failover count: 0 node0 200 primary no no None node1 2 secondary no no None Redundancy group: 1 , Failover count: 0 node0 200 primary no no None node1 20 secondary no no None Redundancy group: 2 , Failover count: 0 node0 200 primary no no None node1 2 secondary no no None" | eval data=split(data," ") | mvexpand data | eval _raw=data | fields - data ```Above just sets up demo data. Delete IRL``` ```Extract events into fields``` | rex "(?<group>Redundancy.*)" | rex "(?<node>node.*)" ```Populate empty group fields``` | filldown group ```Discard empty node fields``` | where isnotnull(node) ```Combine fields into one``` | eval result=group . "," . node | table result

Re: How to group fields into a single line inside a multiline data?

yuanliu — Tue, 29 Nov 2022 07:30:33 GMT

When the consideration is merely presentation, I like to only use string and not be concerned about fields. Here is my alternative. Assuming that each redundancy group is its own event:

| eval data = split(_raw, " ") | eval node = mvrange(1,3) | eval data = mvmap(node, mvindex(data, 0) . "," . mvindex(data, node)) | mvexpand data

If each event in raw data actually combines all redundancy groups with a blank line (it is critical to explain these details in the question), add another split and mvexpand on top.

Either way, you get

data

Redundancy group: 0 , Failover count: 0,node0 200 primary no no None

Redundancy group: 0 , Failover count: 0,node1 2 secondary no no None

Redundancy group: 1 , Failover count: 0,node0 200 primary no no None

Redundancy group: 1 , Failover count: 0,node1 20 secondary no no None

Redundancy group: 2 , Failover count: 0,node0 200 primary no no None

Redundancy group: 2 , Failover count: 0,node1 2 secondary no no None

This is a run-anywhere code to verify (assuming raw event combines groups)

| makeresults | eval _raw = "Redundancy group: 0 , Failover count: 0 node0 200 primary no no None node1 2 secondary no no None Redundancy group: 1 , Failover count: 0 node0 200 primary no no None node1 20 secondary no no None Redundancy group: 2 , Failover count: 0 node0 200 primary no no None node1 2 secondary no no None" ``` data emulation above, assuming combined group data ``` | eval data = split(_raw, " ") | mvexpand data ``` split groups into own events ``` | eval data = split(data, " ") ``` take advantage of fixed line order ``` | eval node = mvrange(1,3) | eval data = mvmap(node, mvindex(data, 0) . "," . mvindex(data, node)) ``` compose display lines ``` | mvexpand data ``` optional - make each line its own row ``` | fields - _* node

Re: How to group fields into a single line inside a multiline data?

tha_ghost99 — Tue, 29 Nov 2022 13:17:07 GMT

Hi @richgalloway ,

when i copy your code directly it works well, and is exactly what i want.

but when i use the actual data i got it working till the last '| eval result=group . "," . node'. it does not seem to provide any output as shown below in RESULT. .

im thinking maybe cause it has spaces in between the original one?

here is the output of the result.

splunk i think reformats and removes some of the tabs/spaces in between the original.

group node result

Redundancy group: 0 , Failover count: 0

Redundancy group: 1 , Failover count: 2

Redundancy group: 2 , Failover count: 2

node0 200 primary no no None

node1 2 secondary no no None

node0 200 primary no no None

node1 20 secondary no no None

node0 200 primary no no None

node1 2 secondary no no None

Re: How to group fields into a single line inside a multiline data?

richgalloway — Tue, 29 Nov 2022 13:55:58 GMT

The difference in our queries is yours is trying to work with multi-value fields. MV fields don't behave the same as single-value fields, which is why you get different results.

Can you please provide a more realistic example of the data (sanitized, of course) so we can offer something that might work?

Re: How to group fields into a single line inside a multiline data?

tha_ghost99 — Tue, 29 Nov 2022 13:54:57 GMT

@richgalloway

hi rich, im sorry, i tried to use exact code you gave and it didnt work, so i kinda tried some other stuff. as far as the data. it is what i sent. attach is the image.

but when i try to use yours which looks similar/identical to the one here its not. i am thinking its something to do with the formatting of splunk when i paste the output here.

Re: How to group fields into a single line inside a multiline data?

richgalloway — Tue, 29 Nov 2022 14:22:37 GMT

I can't do anything with a screenshot other than see you have a lot of information in one multi-value field.

Let's step back and look at how this result was produced. Perhaps if another approach is used we can get the desired result.

Please share the raw data, the current SPL, and the desired output.

Re: How to group fields into a single line inside a multiline data?

tha_ghost99 — Tue, 29 Nov 2022 15:50:03 GMT

@richgalloway

here you go.

Nov 27 13:36:45
Monitor Failure codes:
CS Cold Sync monitoring FL Fabric Connection monitoring
GR GRES monitoring HW Hardware monitoring
IF Interface monitoring IP IP monitoring
LB Loopback monitoring MB Mbuf monitoring
NH Nexthop monitoring NP NPC monitoring
SP SPU monitoring SM Schedule monitoring
CF Config Sync monitoring RE Relinquish monitoring
IS IRQ storm

Cluster ID: 1
Node Priority Status Preempt Manual Monitor-failures

Redundancy group: 0 , Failover count: 0
node0 200 primary no no None
node1 2 secondary no no None

Redundancy group: 1 , Failover count: 2
node0 200 primary no no None
node1 20 secondary no no None

Redundancy group: 2 , Failover count: 2
node0 200 primary no no None
node1 2 secondary no no None

{primary:node0}

Re: How to group fields into a single line inside a multiline data?

richgalloway — Tue, 29 Nov 2022 16:03:51 GMT

What about the current SPL and the desired output?

Re: How to group fields into a single line inside a multiline data?

tha_ghost99 — Tue, 29 Nov 2022 16:56:07 GMT

the SPL is a mixture of your suggested but i just made a few modification (which is probably all wrong) lol

desired result is still the original 🙂 desired result.

Re: How to group fields into a single line inside a multiline data?

yuanliu — Tue, 29 Nov 2022 17:34:37 GMT

If you haven't extracted relevant segments into its own field, do it first. Then manipulate strings.

| eval data = mvindex(split(_raw, " "), 2, -2) ``` retrieve segments related to redundancy group ``` | mvexpand data ``` treat them as own events ``` | eval data = split(data, " ") ``` take advantage of fixed line order ``` | eval node = mvrange(1,3) | eval data = mvmap(node, mvindex(data, 0) . "," . mvindex(data, node)) ``` compose display lines ``` | mvexpand data ``` optional - make each line its own row ```

Again, I used this for emulation:

| makeresults | eval _raw = "Nov 27 13:36:45 Monitor Failure codes: CS Cold Sync monitoring FL Fabric Connection monitoring GR GRES monitoring HW Hardware monitoring IF Interface monitoring IP IP monitoring LB Loopback monitoring MB Mbuf monitoring NH Nexthop monitoring NP NPC monitoring SP SPU monitoring SM Schedule monitoring CF Config Sync monitoring RE Relinquish monitoring IS IRQ storm Cluster ID: 1 Node Priority Status Preempt Manual Monitor-failures Redundancy group: 0 , Failover count: 0 node0 200 primary no no None node1 2 secondary no no None Redundancy group: 1 , Failover count: 2 node0 200 primary no no None node1 20 secondary no no None Redundancy group: 2 , Failover count: 2 node0 200 primary no no None node1 2 secondary no no None {primary:node0}" | fields - _time ``` data emulation above= ```

You can verify that the output is exactly what you prescribed when you combine this with the above.