- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
How can I strip the headers for JSON to parse correctly? Currently not working!
I am trying to remove the header of my JSON data but my current setup will not work, it continues to parse with the header (and therefore doesn't actually parse!).
The header / beginning of the JSON looks like this:
2016-12-29T14:52:53-05:00 17.1.9.10 {"creationTime": 1483993708000
I tried this:
[sourcename]
SHOULD_LINEMERGE=true
NO_BINARY_CHECK=true
CHARSET=UTF-8
INDEXED_EXTRACTIONS=json
category=Structured
disabled=false
pulldown_type=true
TZ=America/New_York
SEDCMD-StripHeader = s/^[^{]+//
KV_MODE = json
EXTRACT-_time={"creationTime":
TIME_PREFIX=\{\"creationTime\"\:
TZ=America/New_York
MAX_TIMESTAMP_LOOKAHEAD=13
Any idea why this is not working?
Many thanks in advance!
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I had similar issue and was solved using answers
https://answers.splunk.com/answers/117121/extract-json-data-within-the-logs.html
I would split the logic into two parts. (1) To extract whole JSON out (2) To extract key value pairs within JSON
In props.conf
### props.conf
[myjson]
REPORT-json = report-json,report-json-kv
In transforms.conf
[report-json]
# This will get the json payload from the logs.
REGEX = (?P<json1>{\"creationTime\".+)
[report-json-kv]
SOURCE_KEY = json1
KV_MODE = json
NO_BINARY_CHECK = 1
TRUNCATE = 0
TIME_PREFIX=\{\"creationTime\"\:
TZ=America/New_York
SHOULD_LINEMERGE = true
BREAK_ONLY_BEFORE = someregularexpression
MAX_TIMESTAMP_LOOKAHEAD=13
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I really don't understand how this works/breaks down. Can you describe the approach, please?
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Unfortunately this didn't work for me, if I am adding data in manually to test, would I be able to see this in the UI or only at search time?
Additionally, for the BREAK_ONLY_BEFORE, do I want everything leading up the the {\"creationTime or everything after?
Thanks!
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
The above settings are REPORT. So search time. You can change to "TRANSFORMS-json" to make it index time.
break_only_before is something you need to try out and test as I don't have the full json. normally it is }\n
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Ok I will try to make this work; it may be unorthodox but do you have Skype or teamviewer? This is really driving me in circles!