I was thinking that I could do a rex to grab everything up to the newline but I am trying to categorize the below output into three fields.
Server, Health, and ThreadCount. I think a rex would do it but I am still new at writing rex/regex. Please assist or direct me to what I should try.
Date = "03-May-2018 07:04:31 -0400"
From = ""
To = ", "
Subject = "Daily Server Health Report"
CC = ""
mailbox = "INBOX"
size = 51929
____________________ Message Body ____________________
coreServer1
HealthState HEALTH_OK
HoggingThreadCount 0
coreServer2
HealthState HEALTH_OK
HoggingThreadCount 0
coreServer3
HealthState HEALTH_WARN,[ThreadPool has stuck threads]
HoggingThreadCount 1
fsServer1
HealthState HEALTH_OK
HoggingThreadCount 0
fsServer2
HealthState HEALTH_OK
HoggingThreadCount 0
fsServer3
HealthState HEALTH_OK
HoggingThreadCount 0
genServer1
HealthState HEALTH_OK
HoggingThreadCount 0
genServer2
HealthState HEALTH_OK
HoggingThreadCount 0
ociServer1
HealthState HEALTH_OK
HoggingThreadCount 0
ociServer2
HealthState HEALTH_OK
HoggingThreadCount 0
ociServer3
HealthState HEALTH_OK
HoggingThreadCount 0
sourceServer1
HealthState HEALTH_OK
HoggingThreadCount 0
sourceServer2
HealthState HEALTH_OK
HoggingThreadCount 0
Hi @wilcoxj
Please see this Regex & also i think you need to break this event properly into mutiple events.
Thanks
You could break your mail into multiple events, as @PowerPacked suggested. Alternatively, you could do do something like this:
| makeresults
| eval test="Date = '03-May-2018 07:04:31 -0400'
From = ''
To = ', '
Subject = 'Daily Server Health Report'
CC = ''
mailbox = 'INBOX'
size = 51929
Message Body
coreServer1
HealthState HEALTH_OK
HoggingThreadCount 0
coreServer2
HealthState HEALTH_OK
HoggingThreadCount 0
coreServer3
HealthState HEALTH_WARN,[ThreadPool has stuck threads]
HoggingThreadCount 1
fsServer1
HealthState HEALTH_OK
HoggingThreadCount 0
fsServer2
HealthState HEALTH_OK
HoggingThreadCount 0
fsServer3
HealthState HEALTH_OK
HoggingThreadCount 0
genServer1
HealthState HEALTH_OK
HoggingThreadCount 0
genServer2
HealthState HEALTH_OK
HoggingThreadCount 0
ociServer1
HealthState HEALTH_OK
HoggingThreadCount 0
ociServer2
HealthState HEALTH_OK
HoggingThreadCount 0
ociServer3
HealthState HEALTH_OK
HoggingThreadCount 0
sourceServer1
HealthState HEALTH_OK
HoggingThreadCount 0
sourceServer2
HealthState HEALTH_OK
HoggingThreadCount 0"
| rex max_match=0 field=test "(?<allevents>\S+\nHealthState\s+\S+\nHoggingThreadCount\s+\d+)"
| fields _time allevents
| mvexpand allevents
| rex field=allevents "^(?<Server>.*?)\s+HealthState\s+(?<HealthState>.*?)\s+HoggingThreadCount\s+(?<ThreadCount>.*?)$"
| table _time Server HealthState ThreadCount
Hope that helps - if it does I'd be happy if you would upvote/accept this answer, so others could profit from it. 🙂
How would I break up an IMAP message into multiple records? Would I have to create a custom props.conf or transforms.conf?
Yes, exactly.
You could put this in a custom props.conf
:
[yoursourcetype]
LINE_BREAKER = ((?:Body\s*)|(?:HoggingThreadCount \d+))([\r\n]+)
You can see it work here:
https://regex101.com/r/iygoQ2/3
I've inserted empty lines and ##### where the events would get broken.
You must however be aware of the following:
- The part before your first data (the mail headers) would be a single event on it's own, you would have to either live with that or build another props entry to drop it
- This will be done to all your data from the same sourcetype
- You will loose the data from the header, especially the timestamp
Depending on the requirements, you might be better off with one or the other solution.
I am hoping something like
Server HealthState ThreadCount
coreServer1 Health_OK 0
coreServer2 Health_OK 0
If that is possible.
Do you want to keep those in a single event?
If you do field extraction, you might get a field "servername", but it would contain multiple values, e.g. (fsServer1, ociServer2, etc).
Can you show us how you would like your result to look like?