Splunk Search

Easier way to search a stanza?

clintla
Contributor

Trying to parse out a set of stanza

Node 1

Device 1 Healthy
Device 2 Healthy
Device 3 Healthy

Node 2

Device 1 Healthy
Device 2 Healthy
Device 3 Healthy

Node 3

Device 1 Healthy
Device 2 FAULT
Device 3 Healthy

If I linebreak on "Node\s+\n+" I just regex the first device status (healthy or not) it only takes the first line when I search so I dont get an acurate device number fault or whatever the status is. No way to apply regex to other parts of the line if they apply?

If I dont linebreak then I dont get the node number.

What are some other ways to look at this? Is there something I can do w/ a transaction to capture the last "Node" prior to something not healthy?

Seems like there should be an easy way to do this.

Tags (1)
0 Karma
1 Solution

Gilberto_Castil
Splunk Employee
Splunk Employee

Just so that we are on the same level of understanding, the assumption here is that the data is broken in such a way that a Node and its Devices reflect a single, multi-line message. This is the line breaker that I used to ensure that assumption in this test.

#props.conf
[answers-1375232025]
SHOULD_LINEMERGE = False
LINE_BREAKER = ([\r\n]+)Node\s\d+

Now the data set contains three events, one for each Node and its related Devices.

alt text

At this point you will want to extract the Node so it can be associated with the message.

| rex "(?<node>Node\s\d+)" 

alt text

Because the Device identity and status is part of a single event, it is not possible to isolate the interesting status by itself. We need to break this up into single lines. Notice how the Node identity is preserved

| multikv noheader=t 

alt text

You are now ready to extract the Device identity and status.

| rex "(?<device>Device\s+\d+)\s+(?<status>\w+)" 

alt text

At this stage you can isolate those devices that are not in healthy state.

| search status="*" NOT status="Healthy" 

alt text

And, finally, prettify the result with a simple table.

| stats list(device) AS device list(status) AS status by node

alt text

All together, the search looks like this:

index=test sourcetype="answers-1375232025" 
| rex "(?<node>Node\s\d+)" 
| multikv noheader=t 
| rex "(?<device>Device\s+\d+)\s+(?<status>\w+)" 
| search status="*" NOT status="Healthy" 
| stats list(device) AS device list(status) AS status by node

I hope this helps.

--
gc

View solution in original post

Gilberto_Castil
Splunk Employee
Splunk Employee

Just so that we are on the same level of understanding, the assumption here is that the data is broken in such a way that a Node and its Devices reflect a single, multi-line message. This is the line breaker that I used to ensure that assumption in this test.

#props.conf
[answers-1375232025]
SHOULD_LINEMERGE = False
LINE_BREAKER = ([\r\n]+)Node\s\d+

Now the data set contains three events, one for each Node and its related Devices.

alt text

At this point you will want to extract the Node so it can be associated with the message.

| rex "(?<node>Node\s\d+)" 

alt text

Because the Device identity and status is part of a single event, it is not possible to isolate the interesting status by itself. We need to break this up into single lines. Notice how the Node identity is preserved

| multikv noheader=t 

alt text

You are now ready to extract the Device identity and status.

| rex "(?<device>Device\s+\d+)\s+(?<status>\w+)" 

alt text

At this stage you can isolate those devices that are not in healthy state.

| search status="*" NOT status="Healthy" 

alt text

And, finally, prettify the result with a simple table.

| stats list(device) AS device list(status) AS status by node

alt text

All together, the search looks like this:

index=test sourcetype="answers-1375232025" 
| rex "(?<node>Node\s\d+)" 
| multikv noheader=t 
| rex "(?<device>Device\s+\d+)\s+(?<status>\w+)" 
| search status="*" NOT status="Healthy" 
| stats list(device) AS device list(status) AS status by node

I hope this helps.

--
gc

clintla
Contributor

worked nicely.. need to learn more about
| multikv noheader=t
Probably the most thorough answer I've seen on Answers!

THANKS!

0 Karma
Get Updates on the Splunk Community!

Announcing Scheduled Export GA for Dashboard Studio

We're excited to announce the general availability of Scheduled Export for Dashboard Studio. Starting in ...

Extending Observability Content to Splunk Cloud

Watch Now!   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to leverage ...

More Control Over Your Monitoring Costs with Archived Metrics GA in US-AWS!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...