Hello All ,
I have a json data format , which I am trying to import into splunk .I want to extract the timestamp from the last field value a multivalue field .For instance there is a field called appid which is a multivalue field with values 1573503539877 , 1573503539875,1573503539878,1573503539873 .I want to make the last value as the timestamp .
The last timestamp for the multivalue field appid has the following format with closed flower brackets and a square bracket but the others have just a flower bracket
MULTIVALUE FIELD "APPID" -first event
apps: [ [-]
{ [-]
addedById: 5d013c468
appId: 5d0d1fc13d418bdf5
dateAdded: /Date(1573503009489)/
MULTIVALUE FIELD APPID-last value which needs to be extracted
addedById: 398
appId:ccaaadb
dateAdded: /Date(1584128055615)/
}
]
This worked.
[ alt ]
CHARSET=UTF-8
INDEXED_EXTRACTIONS=JSON
KV_MODE=JSON
LINE_BREAKER=([\r\n]+)
MAX_TIMESTAMP_LOOKAHEAD=60
NO_BINARY_CHECK=true
SHOULD_LINEMERGE=true
TRUNCATE=10000
category=Structured
description=JavaScript Object Notation format. For more information, visit http://json.org/
disabled=false
pulldown_type=true
TIME_PREFIX=(Date()\d+)\\/\"}]
This worked.
[ alt ]
CHARSET=UTF-8
INDEXED_EXTRACTIONS=JSON
KV_MODE=JSON
LINE_BREAKER=([\r\n]+)
MAX_TIMESTAMP_LOOKAHEAD=60
NO_BINARY_CHECK=true
SHOULD_LINEMERGE=true
TRUNCATE=10000
category=Structured
description=JavaScript Object Notation format. For more information, visit http://json.org/
disabled=false
pulldown_type=true
TIME_PREFIX=(Date()\d+)\\/\"}]
[alt ]
CHARSET=UTF-8
INDEXED_EXTRACTIONS=none
KV_MODE=JSON
LINE_BREAKER=([\r\n]+){
MAX_TIMESTAMP_LOOKAHEAD=1000
NO_BINARY_CHECK=true
SHOULD_LINEMERGE=false
TRUNCATE=0
category=Structured
description=JSON
disabled=false
pulldown_type=true
TIME_PREFIX=.*Date\(
TIME_FORMAT=%s%3N
INDEXED_EXTRACTIONS OR KV_MODE should be set only one.
If LINE_BREAKER is good, TIME_PREFIX 's REGEX match greedy.
so, latest Date
match _time
I am making a guess at how the raw json looks... but since the raw JSON will be all on one line with no carriage returns. try something like so in your props.conf:
TIME_PREFIX = (dateAdded:\s\/Date\()\d+\)\/\}\]
TIME_FORMAT = %s%3N
MAX_TIMESTAMP_LOOKAHEAD = 25
The regex is looking for a dateAdded: /Date(1234567891234) followed by the end strings of your event : }]
https://regex101.com/r/918sTd/1
Hope this helps...
.//D
Thank You @darrenfuller for your reply . I tried the props you told me but that did not work .
[alt ]
CHARSET=UTF-8
INDEXED_EXTRACTIONS=JSON
KV_MODE=JSON
LINE_BREAKER=([\r\n]+)
MAX_TIMESTAMP_LOOKAHEAD=25
NO_BINARY_CHECK=true
SHOULD_LINEMERGE=true
TRUNCATE=10000
category=Structured
description=JavaScript Object Notation format. For more information, visit http://json.org/
disabled=false
pulldown_type=true
TIME_PREFIX=(dateAdded:\s\/Date()\d+)\/}]
TIME_FORMAT=%s%3N
I am pasting the raw format of data how it looks like.the one in bold before collection id is what I am looking at
"dateAdded":"\/Date(1576263356219)\/"},{"addedById":"5d013cd01758d3c468","appId":"5d013d418c2cf","dateAdded":"\/Date(1576263482497)\/"},{"addedById":"5d013cd013c468","appId":"5d35d43d17588644c6c25","dateAdded":"\/Date(1576263489027)\/"},{"addedById":"5d013cd084d3c468","appId":"5e5dc7827acaa","dateAdded":"\/Date(1583177463548)\/"},{"addedById":"5d013cd01d3c468","appId":"5e5d5c7827af0c","dateAdded":"\/Date(1583177467959)\/"}],"collectionId"