Hi ,
I have this json data which I am unable to parse through any of the props.conf mechanisms.
{"meta": {"limit": 20, "next": "/api/v1/indicators/?username=xxxx&api_key=xxxxxxxxxxxxx&limit=20&offset=20", "offset": 0, "previous": null, "total_count": 570289}, "objects": [{"_id": "570f4f34c011fb78b52434d7", "actions": [], "activity": [], "attack_type": "Unknown", "bucket_list": ["Basic", "BR", "botnet", "low_confidence"], "campaign": [], "confidence": {"analyst": "Basic_Feed", "rating": "low"}, "created": "2016-04-14 04:05:08.677000", "impact": {"analyst": "Basic_Feed", "rating": "unknown"}, "locations": [], "modified": "2016-04-14 04:05:08.680000", "objects": [], "relationships": [], "releasability": [], "schema_version": 3, "screenshots": [], "sectors": [], "source": [{"instances": [{"analyst": "Basic_Feed", "date": "2016-04-14 04:05:08.679000", "method": "basic", "reference": "REF: http://botscout.com/last_caught_cache.htm||Report Date:2016-04-13T19:26:46Z||Confidence: 65"}], "name": "osint"}], "status": "New", "threat_type": "Unknown", "tickets": [], "type": "IPv4 Address", "value": "177.33.224.193"}, {"_id": "570f4f62c011fb78b52435a1", "actions": [], "activity": [], "attack_type": "Unknown", "bucket_list": ["Basic", "US", "botnet", "low_confidence"], "campaign": [], "confidence": {"analyst": "Basic_Feed", "rating": "low"}, "created": "2016-04-14 04:05:54.227000", "impact": {"analyst": "_Basic_Feed", "rating": "unknown"}, "locations": [], "modified": "2016-04-14 04:05:54.229000", "objects": [], "relationships": [], "releasability": [], "schema_version": 3, "screenshots": [], "sectors": [], "source": [{"instances": [{"analyst": "_Basic_Feed", "date": "2016-04-14 04:05:54.229000", "method": "_basic", "reference": "REF: http://botscout.com/last_caught_cache.htm||Report Date:2016-04-13T21:26:52Z||Confidence: 65"}], "name": "osint"}], "status": "New", "threat_type": "Unknown", "tickets": [], "type": "IPv4 Address", "value": "104.238.191.144"},
The log is json like format, although events appear to be in one single line and I'm unable to break them using line breakers.
This is how my props.conf looks like after several different tries:
[sourcetype = _json]
BREAK_ONLY_BEFORE_DATE = false
BREAK_ONLY_BEFORE = (\{|\[\s+{)
MUST_BREAK_AFTER = (\}|\}\s+\])
SEDCMD-remove_header = s/(\{\s+.+?\[)//g
SEDCMD-remove_trailing_commas = s/\},/}/g
SEDCMD-remove_footer = s/\]\s+\}//g
TIME_PREFIX = \"modified\":\s+\"
Please help.
... View more