How to group fields into a single line inside a multiline data?

[tha_ghost99]

my subject may not be worded correctly 😞 but i need some help.

i have the below raw data, and i would like to group them together into its own line for reporting.

Redundancy group: 0 , Failover count: 0
node0 200 primary no no None
node1 2 secondary no no None

Redundancy group: 1 , Failover count: 0
node0 200 primary no no None
node1 20 secondary no no None

Redundancy group: 2 , Failover count: 0
node0 200 primary no no None
node1 2 secondary no no None

how can i have the output with the following:
would like to group them based on Redundancy Group # and Node #

Redundancy group: 0 , Failover count: 0,node0 200 primary no no None
Redundancy group: 0 , Failover count: 0,node1 2 secondary no no None
Redundancy group: 1 , Failover count: 0,node0 200 primary no no None
Redundancy group: 1 , Failover count: 0,node1 2 secondary no no None
Redundancy group: 2 , Failover count: 0,node0 200 primary no no None
Redundancy group: 2 , Failover count: 0,node1 2 secondary no no None

[yuanliu]

When the consideration is merely presentation, I like to only use string and not be concerned about fields.  Here is my alternative.  Assuming that each redundancy group is its own event:

| eval data = split(_raw, "
")
| eval node = mvrange(1,3)
| eval data = mvmap(node, mvindex(data, 0) . "," . mvindex(data, node))
| mvexpand data

If each event in raw data actually combines all redundancy groups with a blank line (it is critical to explain these details in the question), add another split and mvexpand on top.

| eval data = split(_raw, "

")
| mvexpand data
| eval data = split(data, "
")
| eval node = mvrange(1,3)
| eval data = mvmap(node, mvindex(data, 0) . "," . mvindex(data, node))
| mvexpand data

Either way, you get

data

Redundancy group: 0 , Failover count: 0,node0 200 primary no no None

Redundancy group: 0 , Failover count: 0,node1 2 secondary no no None

Redundancy group: 1 , Failover count: 0,node0 200 primary no no None

Redundancy group: 1 , Failover count: 0,node1 20 secondary no no None

Redundancy group: 2 , Failover count: 0,node0 200 primary no no None

Redundancy group: 2 , Failover count: 0,node1 2 secondary no no None

This is a run-anywhere code to verify (assuming raw event combines groups)

| makeresults
| eval _raw = "Redundancy group: 0 , Failover count: 0
node0 200 primary no no None
node1 2 secondary no no None

Redundancy group: 1 , Failover count: 0
node0 200 primary no no None
node1 20 secondary no no None

Redundancy group: 2 , Failover count: 0
node0 200 primary no no None
node1 2 secondary no no None"
``` data emulation above, assuming combined group data ```
| eval data = split(_raw, "

")
| mvexpand data ``` split groups into own events ```
| eval data = split(data, "
") ``` take advantage of fixed line order ```
| eval node = mvrange(1,3)
| eval data = mvmap(node, mvindex(data, 0) . "," . mvindex(data, node)) ``` compose display lines ```
| mvexpand data ``` optional - make each line its own row ```
| fields - _* node

[richgalloway]

Here's a run-anywhere query that demonstrates an ugly way to accomplish it.

| makeresults | eval data="Redundancy group: 0 , Failover count: 0
node0 200 primary no no None
node1 2 secondary no no None
Redundancy group: 1 , Failover count: 0
node0 200 primary no no None
node1 20 secondary no no None
Redundancy group: 2 , Failover count: 0
node0 200 primary no no None
node1 2 secondary no no None" | eval data=split(data,"
") | mvexpand data | eval _raw=data | fields - data
```Above just sets up demo data.  Delete IRL```
```Extract events into fields```
| rex "(?<group>Redundancy.*)"
| rex "(?<node>node.*)"
```Populate empty group fields```
| filldown group
```Discard empty node fields```
| where isnotnull(node)
```Combine fields into one```
| eval result=group . "," . node
| table result

[tha_ghost99]

Hi @richgalloway ,

when i copy your code directly it works well, and is exactly what i want.

but when i use the actual data i got it working till the last '| eval result=group . "," . node'. it does not seem to provide any output as shown below in RESULT. .

im thinking maybe cause it has spaces in between the original one?

here is the output of the result.

splunk i think reformats and removes some of the tabs/spaces in between the original.

| rex field=Redundancy_group mode=sed "s/\n\n/\n/g"
| rex field=Redundancy_group max_match=0 "(?<group>Redundancy.*)"
| rex field=Redundancy_group max_match=0 "(?<node>node.*)"
| filldown group
| where isnotnull(node)
| eval result=group . "," . group
|table group node result

group node result

Redundancy group: 0 , Failover count: 0 Redundancy group: 1 , Failover count: 2 Redundancy group: 2 , Failover count: 2

node0 200 primary no no None node1 2 secondary no no None node0 200 primary no no None node1 20 secondary no no None node0 200 primary no no None node1 2 secondary no no None

[richgalloway]

The difference in our queries is yours is trying to work with multi-value fields.  MV fields don't behave the same as single-value fields, which is why you get different results.

Can you please provide a more realistic example of the data (sanitized, of course) so we can offer something that might work?

[tha_ghost99]

@richgalloway 

here you go.

Nov 27 13:36:45
Monitor Failure codes:
CS Cold Sync monitoring FL Fabric Connection monitoring
GR GRES monitoring HW Hardware monitoring
IF Interface monitoring IP IP monitoring
LB Loopback monitoring MB Mbuf monitoring
NH Nexthop monitoring NP NPC monitoring
SP SPU monitoring SM Schedule monitoring
CF Config Sync monitoring RE Relinquish monitoring
IS IRQ storm

Cluster ID: 1
Node Priority Status Preempt Manual Monitor-failures

Redundancy group: 0 , Failover count: 0
node0 200 primary no no None
node1 2 secondary no no None

Redundancy group: 1 , Failover count: 2
node0 200 primary no no None
node1 20 secondary no no None

Redundancy group: 2 , Failover count: 2
node0 200 primary no no None
node1 2 secondary no no None

{primary:node0}

[yuanliu · Accepted solution]

If you haven't extracted relevant segments into its own field, do it first.  Then manipulate strings.

| eval data = mvindex(split(_raw, "

"), 2, -2) ``` retrieve segments related to redundancy group ```
| mvexpand data ``` treat them as own events ```
| eval data = split(data, "
") ``` take advantage of fixed line order ```
| eval node = mvrange(1,3)
| eval data = mvmap(node, mvindex(data, 0) . "," . mvindex(data, node)) ``` compose display lines ```
| mvexpand data ``` optional - make each line its own row ```

 Again, I used this for emulation:

| makeresults
| eval _raw = "Nov 27 13:36:45
Monitor Failure codes:
CS Cold Sync monitoring FL Fabric Connection monitoring
GR GRES monitoring HW Hardware monitoring
IF Interface monitoring IP IP monitoring
LB Loopback monitoring MB Mbuf monitoring
NH Nexthop monitoring NP NPC monitoring
SP SPU monitoring SM Schedule monitoring
CF Config Sync monitoring RE Relinquish monitoring
IS IRQ storm

Cluster ID: 1
Node Priority Status Preempt Manual Monitor-failures

Redundancy group: 0 , Failover count: 0
node0 200 primary no no None
node1 2 secondary no no None

Redundancy group: 1 , Failover count: 2
node0 200 primary no no None
node1 20 secondary no no None

Redundancy group: 2 , Failover count: 2
node0 200 primary no no None
node1 2 secondary no no None

{primary:node0}"
| fields - _time
``` data emulation above= ```

You can verify that the output is exactly what you prescribed when you combine this with the above.

[richgalloway]

What about the current SPL and the desired output?

[tha_ghost99]

the SPL is a mixture of your suggested but i just made a few modification (which is probably all wrong) lol

| rex field="show chassis cluster status _ no_more" max_match=0 "(?s)Monitor-failures[\n][\n](?<Redundancy_group>.*)[\n][\n]"
| fields Redundancy_group
| rex "(?<group>Redundancy.*)"
| rex "(?<node>node.*)"
| filldown group
| where isnotnull(node)
| eval result=group . "," . group
|table group node result

desired result is still the original 🙂 desired result.

Redundancy group: 0 , Failover count: 0,node0 200 primary no no None
Redundancy group: 0 , Failover count: 0,node1 2 secondary no no None
Redundancy group: 1 , Failover count: 0,node0 200 primary no no None
Redundancy group: 1 , Failover count: 0,node1 2 secondary no no None
Redundancy group: 2 , Failover count: 0,node0 200 primary no no None
Redundancy group: 2 , Failover count: 0,node1 2 secondary no no None

[tha_ghost99]

@richgalloway 

hi rich, im sorry, i tried to use exact code you gave and it didnt work, so i kinda tried some other stuff. as far as the data. it is what i sent. attach is the image.

but when i try to use yours which looks similar/identical to the one here its not. i am thinking its something to do with the formatting of splunk when i paste the output here.

[richgalloway]

I can't do anything with a screenshot other than see you have a lot of information in one multi-value field.

Let's step back and look at how this result was produced.  Perhaps if another approach is used we can get the desired result.

Please share the raw data, the current SPL, and the desired output.

[richgalloway]

How would Splunk know to which Redundancy group any particular node belongs?  There appears to be nothing in a "node" event that connects it to a Redundancy group event.

[tha_ghost99]

hi @richgalloway that is correct.

its basically just node0 and node1 because its below redundancy group0 for example.

so each redudancy group 1,2,3 has their own Node0 and Node1 output. would simply want to group them in 1 line.

hope it makes sense.