Hi,
this query should give you an idea, on how this can be done:
| makeresults | eval _raw="2019-10-10T17:51:40+00:00 action=\"updateDate->saveDatesFromDataMining\", 0={\"urlupdateid\":1371955,\"datetype\":\"Review date\",\"datevalue\":\"10/03/2019\"},1={\"urlupdateid\":1371955,\"datetype\":\"somethingElse\",\"datevalue\":\"10/03/2020\"},2={\"urlupdateid\":1371955,\"datetype\":\"False datetype\",\"datevalue\":\"10/03/2019\"}`, approved=true host=stuff.localsource PCDAccuracy.txtsourcetype=Text"
| rex field=_raw "action=\"(?<action>[^\"]+)\"" | rex field=_raw "(?<values>\d+\=\{.*)`"
| eval values=replace(values,"},","}|")
| makemv delim="|" values | mvexpand values
| rex field=values "urlupdateid\":(?<urlupdateid>[^,]+),.*:\"(?<datetype>[^\"]+)\".*:\"(?<datevalue>.*)\""
| fields - _raw,values
Let me explain:
1. The messy part on top is your event data. In order to reproduce it in search, I assign it to a field called _raw. In your events, this field is already present and filled with the event data.
2. The line with the two rex commands, one for action and one for values extracts two fields from your events. Cut & Paste from the start including only this line to take a look at the content. You should extract those fields in your sourcetype. Doing it with a rex at search time is somewhat ok, the sourcetype is better.
3. Now the fun part starts: Values contains a list of results. Since the list is dynamic, it can hold an arbitrary number of results, one per datetype. A static mapping is useless. Therefore we trick the machine by making the individual datetype values clearly indentifyable. That's the replace for: We replace the , that separates two datatype records by | in order to distinguish it from the other commas around.
4. Now we make the values field a multi-value field based on the | that we introduced and we split the event into multiple events. The result is, that for every datetype in the original single event, we now have one event.
5. Almost done: We extract the relevant fields from the single datetype values field
6. With the last fields command we remove the unwanted noise from the result
This was the heavy lifting. From here on, you can do your further statistics and analysis:
- Choose the timeframe for the query, so that it has 3 results ( 3 days, if you want 3 results for every datetype and get them daily)
- Do a transaction on the urlupdateid field, this will group all events based on the urlupdateid into one. You may use the startswith and endswith parameter to include the action field
- Analyse the datetype and datevalue fields to calculate the Accuracy
Hope it helps
Oliver
... View more