Windows Logs and splitting up Message field

brentmatlock — Mon, 04 Feb 2019 11:39:03 GMT

I have a custom module that pulls Event logs via WMI and then converts the output to JSON. I have everything parsing correctly via the custom Sourcetype. I am looking to see how exactly you divide up the "Message" field. Currently the message field is just one giant blob, I have spent the last couple of hours trying to google it and even checked all the threads suggested by this site before posting.

Maybe I am missing something small.

Inputs.conf:

[default]
host = splunk_server

[batch://C:\scripts\events\*.json]
move_policy = sinkhole
index = script_dev
sourcetype = script_event_json
initCrcLength = 2000

props.conf:

[scripts_event_json]
BREAK_ONLY_BEFORE = ^{
LINE_BREAKER = ^{
NO_BINARY_CHECK = true
SHOULD_LINEMERGE = false
TIME_FORMAT = %Y%m%d%h%M%S.%6N
TIME_PREFIX = TimeWritten
category = structured
pulldown_type = true

I have tried splitting message with | eval Message = Split(Message,".") this allows me to create the Short message but I cant for the life of me figure out how to parse the additional fields out of message.

Sample Log:

{   "Category": {},     "CategoryString": "Logon",  "ComputerName": "DESKTOP-8F1C40T",  "Container": null,  "Data": null,   "EventCode": {},    "EventIdentifier": 4624,    "EventType": {},    "Logfile": "Security",  "Message": "An account was successfully logged on.\r\n\r\nSubject:\r\n\tSecurity ID:\t\tS-1-5-18\r\n\tAccount Name:\t\tDESKTOP-8F1C40T$\r\n\tAccount Domain:\t\tWORKGROUP\r\n\tLogon ID:\t\t0x3E7\r\n\r\nLogon Information:\r\n\tLogon Type:\t\t5\r\n\tRestricted Admin Mode:\t-\r\n\tVirtual Account:\t\tNo\r\n\tElevated Token:\t\tYes\r\n\r\nImpersonation Level:\t\tImpersonation\r\n\r\nNew Logon:\r\n\tSecurity ID:\t\tS-1-5-18\r\n\tAccount Name:\t\tSYSTEM\r\n\tAccount Domain:\t\tNT AUTHORITY\r\n\tLogon ID:\t\t0x3E7\r\n\tLinked Logon ID:\t\t0x0\r\n\tNetwork Account Name:\t-\r\n\tNetwork Account Domain:\t-\r\n\tLogon GUID:\t\t{00000000-0000-0000-0000-000000000000}\r\n\r\nProcess Information:\r\n\tProcess ID:\t\t0x328\r\n\tProcess Name:\t\tC:\\Windows\\System32\\services.exe\r\n\r\nNetwork Information:\r\n\tWorkstation Name:\t-\r\n\tSource Network Address:\t-\r\n\tSource Port:\t\t-\r\n\r\nDetailed Authentication Information:\r\n\tLogon Process:\t\tAdvapi  \r\n\tAuthentication Package:\tNegotiate\r\n\tTransited Services:\t-\r\n\tPackage Name (NTLM only):\t-\r\n\tKey Length:\t\t0\r\n\r\nThis event is generated when a logon session is created. It is generated on the computer that was accessed.\r\n\r\nThe subject fields indicate the account on the local system which requested the logon. This is most commonly a service such as the Server service, or a local process such as Winlogon.exe or Services.exe.\r\n\r\nThe logon type field indicates the kind of logon that occurred. The most common types are 2 (interactive) and 3 (network).\r\n\r\nThe New Logon fields indicate the account for whom the new logon was created, i.e. the account that was logged on.\r\n\r\nThe network fields indicate where a remote logon request originated. Workstation name is not always available and may be left blank in some cases.\r\n\r\nThe impersonation level field indicates the extent to which a process in the logon session can impersonate.\r\n\r\nThe authentication information fields provide detailed information about this specific logon request.\r\n\t- Logon GUID is a unique identifier that can be used to correlate this event with a KDC event.\r\n\t- Transited services indicate which intermediate services have participated in this logon request.\r\n\t- Package name indicates which sub-protocol was used among the NTLM protocols.\r\n\t- Key length indicates the length of the generated session key. This will be 0 if no session key was requested.",   "PSComputerName": "192.168.0.1",    "PSShowComputerName": true,     "RecordNumber": 34183,  "RunspaceId": { "Guid": "6ffd6d75-40ae-44ce-a242-b23f2f01057d"},    "Site": null,   "SourceName": "Microsoft-Windows-Security-Auditing",    "TimeGenerated": "20190204085533.708994-000",   "TimeWritten": "20190204085533.708994-000",     "Type": "Audit Success",    "User": null}

Any help would be greatly appreciated. If you need any additional information please feel free to ask.

Re: Windows Logs and splitting up Message field

brentmatlock — Mon, 04 Feb 2019 13:35:04 GMT

I found a solution of sorts. I can get it to work at search time with:
rex field=Message "\sAccount Name:\s(?<AccountName>.*)"
My NEW question is, how would you go about doing this at Index time?

topic Windows Logs and splitting up Message field in Getting Data In

Windows Logs and splitting up Message field

Re: Windows Logs and splitting up Message field