Getting Data In

Windows Logs and splitting up Message field

brentmatlock
Engager

I have a custom module that pulls Event logs via WMI and then converts the output to JSON. I have everything parsing correctly via the custom Sourcetype. I am looking to see how exactly you divide up the "Message" field. Currently the message field is just one giant blob, I have spent the last couple of hours trying to google it and even checked all the threads suggested by this site before posting.

Maybe I am missing something small.

Inputs.conf:

[default]
host = splunk_server

[batch://C:\scripts\events\*.json]
move_policy = sinkhole
index = script_dev
sourcetype = script_event_json
initCrcLength = 2000

props.conf:

[scripts_event_json]
BREAK_ONLY_BEFORE = ^{
LINE_BREAKER = ^{
NO_BINARY_CHECK = true
SHOULD_LINEMERGE = false
TIME_FORMAT = %Y%m%d%h%M%S.%6N
TIME_PREFIX = TimeWritten
category = structured
pulldown_type = true

I have tried splitting message with | eval Message = Split(Message,".") this allows me to create the Short message but I cant for the life of me figure out how to parse the additional fields out of message.

Sample Log:

{   "Category": {},     "CategoryString": "Logon",  "ComputerName": "DESKTOP-8F1C40T",  "Container": null,  "Data": null,   "EventCode": {},    "EventIdentifier": 4624,    "EventType": {},    "Logfile": "Security",  "Message": "An account was successfully logged on.\r\n\r\nSubject:\r\n\tSecurity ID:\t\tS-1-5-18\r\n\tAccount Name:\t\tDESKTOP-8F1C40T$\r\n\tAccount Domain:\t\tWORKGROUP\r\n\tLogon ID:\t\t0x3E7\r\n\r\nLogon Information:\r\n\tLogon Type:\t\t5\r\n\tRestricted Admin Mode:\t-\r\n\tVirtual Account:\t\tNo\r\n\tElevated Token:\t\tYes\r\n\r\nImpersonation Level:\t\tImpersonation\r\n\r\nNew Logon:\r\n\tSecurity ID:\t\tS-1-5-18\r\n\tAccount Name:\t\tSYSTEM\r\n\tAccount Domain:\t\tNT AUTHORITY\r\n\tLogon ID:\t\t0x3E7\r\n\tLinked Logon ID:\t\t0x0\r\n\tNetwork Account Name:\t-\r\n\tNetwork Account Domain:\t-\r\n\tLogon GUID:\t\t{00000000-0000-0000-0000-000000000000}\r\n\r\nProcess Information:\r\n\tProcess ID:\t\t0x328\r\n\tProcess Name:\t\tC:\\Windows\\System32\\services.exe\r\n\r\nNetwork Information:\r\n\tWorkstation Name:\t-\r\n\tSource Network Address:\t-\r\n\tSource Port:\t\t-\r\n\r\nDetailed Authentication Information:\r\n\tLogon Process:\t\tAdvapi  \r\n\tAuthentication Package:\tNegotiate\r\n\tTransited Services:\t-\r\n\tPackage Name (NTLM only):\t-\r\n\tKey Length:\t\t0\r\n\r\nThis event is generated when a logon session is created. It is generated on the computer that was accessed.\r\n\r\nThe subject fields indicate the account on the local system which requested the logon. This is most commonly a service such as the Server service, or a local process such as Winlogon.exe or Services.exe.\r\n\r\nThe logon type field indicates the kind of logon that occurred. The most common types are 2 (interactive) and 3 (network).\r\n\r\nThe New Logon fields indicate the account for whom the new logon was created, i.e. the account that was logged on.\r\n\r\nThe network fields indicate where a remote logon request originated. Workstation name is not always available and may be left blank in some cases.\r\n\r\nThe impersonation level field indicates the extent to which a process in the logon session can impersonate.\r\n\r\nThe authentication information fields provide detailed information about this specific logon request.\r\n\t- Logon GUID is a unique identifier that can be used to correlate this event with a KDC event.\r\n\t- Transited services indicate which intermediate services have participated in this logon request.\r\n\t- Package name indicates which sub-protocol was used among the NTLM protocols.\r\n\t- Key length indicates the length of the generated session key. This will be 0 if no session key was requested.",   "PSComputerName": "192.168.0.1",    "PSShowComputerName": true,     "RecordNumber": 34183,  "RunspaceId": { "Guid": "6ffd6d75-40ae-44ce-a242-b23f2f01057d"},    "Site": null,   "SourceName": "Microsoft-Windows-Security-Auditing",    "TimeGenerated": "20190204085533.708994-000",   "TimeWritten": "20190204085533.708994-000",     "Type": "Audit Success",    "User": null}

Any help would be greatly appreciated. If you need any additional information please feel free to ask.

0 Karma

brentmatlock
Engager

I found a solution of sorts. I can get it to work at search time with:
rex field=Message "\sAccount Name:\s(?<AccountName>.*)"
My NEW question is, how would you go about doing this at Index time?

0 Karma
Get Updates on the Splunk Community!

Earn a $35 Gift Card for Answering our Splunk Admins & App Developer Survey

Survey for Splunk Admins and App Developers is open now! | Earn a $35 gift card!      Hello there,  Splunk ...

Continuing Innovation & New Integrations Unlock Full Stack Observability For Your ...

You’ve probably heard the latest about AppDynamics joining the Splunk Observability portfolio, deepening our ...

Monitoring Amazon Elastic Kubernetes Service (EKS)

As we’ve seen, integrating Kubernetes environments with Splunk Observability Cloud is a quick and easy way to ...