topic Re: Extract server name, HealthState, and HoggingTrheadCount from txt file in Splunk Search

Extract server name, HealthState, and HoggingTrheadCount from txt file

wilcoxj — Tue, 29 Sep 2020 19:20:40 GMT

I was thinking that I could do a rex to grab everything up to the newline but I am trying to categorize the below output into three fields.
Server, Health, and ThreadCount. I think a rex would do it but I am still new at writing rex/regex. Please assist or direct me to what I should try.

Date = "03-May-2018 07:04:31 -0400"
From = ""
To = ", "
Subject = "Daily Server Health Report"
CC = ""
mailbox = "INBOX"
size = 51929
____________________ Message Body ____________________
coreServer1
HealthState HEALTH_OK
HoggingThreadCount 0
coreServer2
HealthState HEALTH_OK
HoggingThreadCount 0
coreServer3
HealthState HEALTH_WARN,[ThreadPool has stuck threads]
HoggingThreadCount 1
fsServer1
HealthState HEALTH_OK
HoggingThreadCount 0
fsServer2
HealthState HEALTH_OK
HoggingThreadCount 0
fsServer3
HealthState HEALTH_OK
HoggingThreadCount 0
genServer1
HealthState HEALTH_OK
HoggingThreadCount 0
genServer2
HealthState HEALTH_OK
HoggingThreadCount 0
ociServer1
HealthState HEALTH_OK
HoggingThreadCount 0
ociServer2
HealthState HEALTH_OK
HoggingThreadCount 0
ociServer3
HealthState HEALTH_OK
HoggingThreadCount 0
sourceServer1
HealthState HEALTH_OK
HoggingThreadCount 0
sourceServer2
HealthState HEALTH_OK
HoggingThreadCount 0

Re: Extract server name, HealthState, and HoggingTrheadCount from txt file

xpac — Thu, 03 May 2018 15:17:39 GMT

Do you want to keep those in a single event?
If you do field extraction, you might get a field "servername", but it would contain multiple values, e.g. (fsServer1, ociServer2, etc).
Can you show us how you would like your result to look like?

Re: Extract server name, HealthState, and HoggingTrheadCount from txt file

wilcoxj — Tue, 29 Sep 2020 19:21:01 GMT

I am hoping something like

Server HealthState ThreadCount
coreServer1 Health_OK 0
coreServer2 Health_OK 0

If that is possible.

Re: Extract server name, HealthState, and HoggingTrheadCount from txt file

PowerPacked — Thu, 03 May 2018 17:49:46 GMT

Hi @wilcoxj

Please see this Regex & also i think you need to break this event properly into mutiple events.

Thanks

Re: Extract server name, HealthState, and HoggingTrheadCount from txt file

xpac — Thu, 03 May 2018 17:54:55 GMT

You could break your mail into multiple events, as @PowerPacked suggested. Alternatively, you could do do something like this:

| makeresults
| eval test="Date = '03-May-2018 07:04:31 -0400'
From = ''
To = ', '
Subject = 'Daily Server Health Report'
CC = ''
mailbox = 'INBOX'
size = 51929
Message Body 
coreServer1
HealthState HEALTH_OK
HoggingThreadCount 0
coreServer2
HealthState HEALTH_OK
HoggingThreadCount 0
coreServer3
HealthState HEALTH_WARN,[ThreadPool has stuck threads]
HoggingThreadCount 1
fsServer1
HealthState HEALTH_OK
HoggingThreadCount 0
fsServer2
HealthState HEALTH_OK
HoggingThreadCount 0
fsServer3
HealthState HEALTH_OK
HoggingThreadCount 0
genServer1
HealthState HEALTH_OK
HoggingThreadCount 0
genServer2
HealthState HEALTH_OK
HoggingThreadCount 0
ociServer1
HealthState HEALTH_OK
HoggingThreadCount 0
ociServer2
HealthState HEALTH_OK
HoggingThreadCount 0
ociServer3
HealthState HEALTH_OK
HoggingThreadCount 0
sourceServer1
HealthState HEALTH_OK
HoggingThreadCount 0
sourceServer2
HealthState HEALTH_OK
HoggingThreadCount 0"
| rex max_match=0 field=test "(?<allevents>\S+\nHealthState\s+\S+\nHoggingThreadCount\s+\d+)"
| fields _time allevents
| mvexpand allevents
| rex field=allevents "^(?<Server>.*?)\s+HealthState\s+(?<HealthState>.*?)\s+HoggingThreadCount\s+(?<ThreadCount>.*?)$"
| table _time Server HealthState ThreadCount

Hope that helps - if it does I'd be happy if you would upvote/accept this answer, so others could profit from it. 🙂

Re: Extract server name, HealthState, and HoggingTrheadCount from txt file

wilcoxj — Thu, 03 May 2018 19:38:02 GMT

How would I break up an IMAP message into multiple records? Would I have to create a custom props.conf or transforms.conf?

Re: Extract server name, HealthState, and HoggingTrheadCount from txt file

xpac — Thu, 03 May 2018 19:51:41 GMT

Yes, exactly.
You could put this in a custom props.conf:

[yoursourcetype]
LINE_BREAKER = ((?:Body\s*)|(?:HoggingThreadCount \d+))([\r\n]+)

You can see it work here:
https://regex101.com/r/iygoQ2/3

I've inserted empty lines and ##### where the events would get broken.

You must however be aware of the following:
- The part before your first data (the mail headers) would be a single event on it's own, you would have to either live with that or build another props entry to drop it
- This will be done to all your data from the same sourcetype
- You will loose the data from the header, especially the timestamp

Depending on the requirements, you might be better off with one or the other solution.