Getting Data In

Can you help props.conf to break the event and mask the data?

vrmandadi
Builder

I have the below sample event

{"timestamp": 1553559218742, "message": "(0133108c-4f5c-11e9-82ca-1b5bad0211a1) Method request path: {serverId=s-2f9b4670b10148058, username=mike}", "ingestionTime": 1553559219615}, {"timestamp": 1553559218742, "message": "(0133108c-4f5c-11e9-82ca-1b5bad0211a1) Method request query string: {}", "ingestionTime": 1553559219615},{"timestamp": 1553559218743, "message": "(0133108c-4f5c-11e9-82ca-1b5bad0211a1) Endpoint request body after transformations: {\n  \"username\": \"mike\",\n  \"password\": \"mike\",\n  \"serverId\": \"s-2f9b4670b10148058\"\n}\n", "ingestionTime": 1553559219615}, {"timestamp": 1553559218743, "message": "(0133108c-4f5c-11e9-82ca-1b5bad0211a1) Sending request to https://lambda.us-east-1.amazonaws.com/2015-03-31/functions/arn:aws:lambda:us-east-1:114086737623:fu..., "ingestionTime": 1553559219615}
  1. I am trying to break the event after the ingestion time, like below

{"timestamp": 1553559218742, "message": "(0133108c-4f5c-11e9-82ca-1b5bad0211a1) Method request path: {serverId=s-2f9b4670b10148058, username=mike}", "ingestionTime": 1553559219615}
2. In some of the events, the user name has a password in it, and I want to mask that one .Below is the sample event:

{"timestamp": 1553559218743, "message": "(0133108c-4f5c-11e9-82ca-1b5bad0211a1) Endpoint request body after transformations: {\n \"username\": \"mike\",\n **\"password\": \"mike\"**,\n \"serverId\": \"s-2f9b4670b10148058\"\n}\n", "ingestionTime": 1553559219615}

Thanks In advance !

0 Karma
1 Solution

somesoni2
Revered Legend

Give this a try

[YourSourcetype]
SHOULD_LINEMERGE=false
LINE_BREAKER=(\{\"timestamp\")
TIME_PREFIX=^\:\s+
TIME_FORMAT = %s%N
# line breaker actually remove timestamp, so putting it back
SEDCMD-header=s/^/{"timemstamp"/
#masking password
SEDCMD-passwordmask=s/(\"password[^\"]+\"[^\"]+)(\"[^\"]+\")/\1"##MASKED##"/

View solution in original post

somesoni2
Revered Legend

Give this a try

[YourSourcetype]
SHOULD_LINEMERGE=false
LINE_BREAKER=(\{\"timestamp\")
TIME_PREFIX=^\:\s+
TIME_FORMAT = %s%N
# line breaker actually remove timestamp, so putting it back
SEDCMD-header=s/^/{"timemstamp"/
#masking password
SEDCMD-passwordmask=s/(\"password[^\"]+\"[^\"]+)(\"[^\"]+\")/\1"##MASKED##"/

View solution in original post

Register for .conf21 Now! Go Vegas or Go Virtual!

How will you .conf21? You decide! Go in-person in Las Vegas, 10/18-10/21, or go online with .conf21 Virtual, 10/19-10/20.