How to send events in JSON format to NullQueue?

kiran331 — Tue, 31 Oct 2017 20:15:35 GMT

Hello,

We are puling JSON data from cloud, can I trim out the events with EventId=5156 and 5158 from the events with sourcetype "mscs:storage:table". Below is the sample event and _raw event?

{ [-]
Channel: Security

DeploymentId: fgdfgfdgfdgfgngzser3

Description: The Windows Filtering Platform has permitted a connection.

Application Information:
Process ID: 964
Application Name: \device\harddisk\program files\splunkuniversalforwarder\bin\splunkd.exe

Network Information:
Direction: Outbound
Source Address: 1.11.12.13
Source Port: 57564
Destination Address: 21.22.23.24
Destination Port: 9997
Protocol: 6

Filter Information:
Filter Run-Time ID: 119665
Layer Name: Connect
Layer Run-Time ID: 48

EventId: 5156

EventTickCount: 4545656687812

EventTickCount@odata.type: Edm.Int64

Level: 0

Opcode: 0

PartitionKey: 565656548896

Pid: 4

PreciseTimeStamp: 2017-10-31T19:50:52.5322979Z

PreciseTimeStamp@odata.type: Edm.DateTime

ProviderGuid: {asa-dfdfdf-4994-sads-fdfdf}

ProviderName: Microsoft-Windows-Security-Auditing

RawXml: 5156101281000x80200000000000004344544Securitytest.tt.com964\device\harddisk\program files\splunkuniversalforwarder\bin\splunkd.exe%%145931.11.12.135756421.22.23.2499976119665%%1461148S-1-0-0S-1-0-0

Role: IaaS

RoleInstance: _test.tt.com

RowIndex: 000000010755656

RowKey: dfttresttvsdfsfsf000000019
TIMESTAMP: 2017-10-31T19:50:00Z

TIMESTAMP@odata.type: Edm.DateTime

Task: 12810

Tid: 14808

Timestamp: 2017-10-31T19:51:26.4589637Z

odata.etag: W/"datetime'2017-10-31T19%3A51%3A26.4589637Z'"
}

_raw event:

{"Timestamp": "2017-10-31T19:51:26.4589637Z", "ProviderName": "Microsoft-Windows-Security-Auditing", "RawXml": "5156101281000x8020000000000000fdfdfe323Securitytest.tt.com964\device\harddis3\program files\splunkuniversalforwarder\bin\splunkd.exe%%145931.11.12.135756421.22.23.2499976119665%%1461148S-1-0-0S-1-0-0", "RowIndex": "0000000107374703779", "TIMESTAMP": "2017-10-31T19:50:00Z", "EventTickCount": "dfdf", "PartitionKey": "0636988789789835", "Tid": 14808, "Role": "IaaS", "EventTickCount@odata.type": "Edm.Int64", "Channel": "Security", "Task": 12810, "PreciseTimeStamp@odata.type": "Edm.DateTime", "PreciseTimeStamp": "2017-10-31T19:50:52.5322979Z", "Level": 0, "ProviderGuid": "{erer-5478-4994-errer-3E3B0328C30D}", "RoleInstance": "_test.tt.com", "TIMESTAMP@odata.type": "Edm.DateTime", "EventId": 5156, "Description": "The Windows Filtering Platform has permitted a connection.\n\nApplication Information:\n\tProcess ID:\t\t964\n\tApplication Name:\t\device\harddisk\program files\splunkuniversalforwarder\bin\splunkd.exe\n\nNetwork Information:\n\tDirection:\t\tOutbound\n\tSource Address:\t\t1.11.12.13\n\tSource Port:\t\t57564\n\tDestination Address:\t21.22.23.24\n\tDestination Port:\t\t9997\n\tProtocol:\t\t6\n\nFilter Information:\n\tFilter Run-Time ID:\t119665\n\tLayer Name:\t\tConnect\n\tLayer Run-Time ID:\t48", "Pid": 4, "DeploymentId": "c9f4631c-fdfdfff-6a27dbd29a02", "odata.etag": "W/\"datetime'2017-10-31T19%3A51%3A26.4589637Z'\"", "RowKey": "c9f4631c-bf16-dferersfssdf

Re: How to send events in JSON format to NullQueue?

skoelpin — Wed, 01 Nov 2017 15:39:18 GMT

Try this

props.conf

[mscs:storage:table]
 TRANSFORMS-DiscardWinEvents = eliminate-eventids

transforms.conf

 [eliminate-eventids]
 REGEX=EventId=(5156|5158)
 DEST_KEY=queue
 FORMAT=nullQueue

Re: How to send events in JSON format to NullQueue?

kiran331 — Thu, 02 Nov 2017 14:15:20 GMT

I tried this one on the Heavy forwarder, its not working.

Re: How to send events in JSON format to NullQueue?

skoelpin — Thu, 02 Nov 2017 14:18:17 GMT

Did you restart splunkd after making changes?

Re: How to send events in JSON format to NullQueue?

kiran331 — Thu, 02 Nov 2017 14:34:47 GMT

Yes, I restarted it.

Re: How to send events in JSON format to NullQueue?

macvili — Tue, 05 Dec 2017 07:54:27 GMT

Your regex won't match. The _raw data contais this: ...,"EventId": 5156,... therefore your regex in the transforms.conf stanza should go like this:

REGEX = \"EventId\":\s*(?:5156|5158)

topic Re: How to send events in JSON format to NullQueue? in Splunk Search

How to send events in JSON format to NullQueue?

Re: How to send events in JSON format to NullQueue?

Re: How to send events in JSON format to NullQueue?

Re: How to send events in JSON format to NullQueue?

Re: How to send events in JSON format to NullQueue?

Re: How to send events in JSON format to NullQueue?