How to extract values from multiple events and create a new event with those values?

[skoelpin]

I want to extract a key-value pair from multiple events and create a single event with those extractions.

We have events coming in with a unique EventCode. I only want the Event code, everything else can be "thrown out". I want to create a single event with multiple event codes

Event 1:

10/12/2016 03:30:23 PM
LogName=Microsoft-Windows-WLAN-AutoConfig/Operational
SourceName=Microsoft-Windows-WLAN-AutoConfig
EventCode=12000
EventType=4
Type=Information
ComputerName=xxxxxxxxxxxxxxxxxxxxxx
User=NOT_TRANSLATED
Sid=S-x-x-xx
SidType=0
TaskCategory=OneXAuthentication
OpCode=Start
RecordNumber=xxxxx
Keywords=None
Message=Wireless xxx.xx authentication started.

Network Adapter: xxxxxxxxxxxxxxxxxxxxxxxxxx
Interface GUID: {xxxxxxxxxxxxxxxxxxxxxxxx}
Local MAC Address: xxxxxxxxxxxxxxxxxx
Network SSID: xxxxxxx
BSS Type: Infrastructure
Eap Information: Type 25, Vendor ID 0, Vendor Type 0, Author ID 0

Event 2:

10/12/2016 03:30:24 PM
LogName=Microsoft-Windows-WLAN-AutoConfig/Operational
SourceName=Microsoft-Windows-WLAN-AutoConfig
EventCode=19000
EventType=4
Type=Information
ComputerName=xxxxxxxxxxxxxxxxxxxxxx
User=NOT_TRANSLATED
Sid=S-x-x-xx
SidType=0
TaskCategory=OneXAuthentication
OpCode=Start
RecordNumber=xxxxx
Keywords=None
Message=Wireless xxx.xx authentication started.

Network Adapter: xxxxxxxxxxxxxxxxxxxxxxxxxx
Interface GUID: {xxxxxxxxxxxxxxxxxxxxxxxx}
Local MAC Address: xxxxxxxxxxxxxxxxxx
Network SSID: xxxxxxx
BSS Type: Infrastructure
Eap Information: Type 25, Vendor ID 0, Vendor Type 0, Author ID 0

After extracting the EventCode, I want to discard everything else and have a single event look like this

10/12/2016 03:30:23 PM -- EventCode=12000
10/12/2016 03:30:24 PM -- EventCode=19000 

I was thinking about extracting the event code and populating it in a summary index so I can create a new event from the extracted values. Is there a better way of going about this? Any recommendations would be great!

[sundareshr]

Accelerated Datamodel maybe?

[skoelpin]

@sundareshr, whats your thoughts on creating a line breaking rule in props.conf so it will see these events in series and break the line after all the events are merged into one?

I'm tasked with creating a report which will show how frequently a series of 4 events happen in series within 2 seconds

[sundareshr]

If you really mean discard as-in not even index, how about SEDCMD

https://regex101.com/r/3AvJwR/1

[skoelpin]

Hmm this may be a good approach.. We want to keep all the data in one index, but have a separate index where the "garbage" is thrown out. I'm thinking about creating a summary index, pushing this data into the SI, using SEDCMD to discard everything I don't need, then use streamstats to correlate the events and send an alert if they happen sequentially. Whats your thoughts on this? Would you take a different approach?

[gokadroid]

yourBaseSearch
| rex field=_raw ".*EventCode=(?< event_code>[\d]+)"
| eval desired_time=strftime(_time, "%m/%d/%Y %H:%M:%S %p")
| eval desired_event = desired_time." -- EventCode=".event_code
| complete your search

Remove the space in the tag < event_code>

[skoelpin]

Thanks for helping, but not exactly what I was looking for. I would like to extract the event_code field and value with it's timestamp of several events and create a new event with those extracted fields.

An alternative approach I'm thinking about would be to create a special line breaking rule in the props.conf so if it see's a pattern in the event_code over several events, it will treat all those events as one while indexing the data. What's your thoughts on this?