yes, this: spath input=msg query{} is probably just some cheating to see some data, but it does not actually work. It is just some overcomplicated syntax to see the whole array, but not individual items; if the array would have more items, we won't get first item. If I need to get first item in array to work further, this just does not work ... View more

Thanks for your help, I really appreciate the time you put it. I asked in good faith I can learn this. streaming languages — like JQ I mentioned — are harder, yes, functional languages are harder, yes, but all is doable. Yes, shown json is tricker, but I just happen to know, that there will be just one query there, and if not, I can call one more reduce in JQ and I'm good, still oneliner. Regarding the SPL — the spl solution is just crazy. I just cannot see what each individual part does and why. And the documentation does it make it harder instead of easier. But I probably miss some basic premise of splunk. Ok, simple sample, simpler task.       | makeresults | eval _raw="{ \"json\": { \"class\": \"net.ttddyy.dsproxy.support.SLF4JLogUtils\", }, }"       Use regex to remove word ttddyy and return .json.class. Should be exceptionally trivial. If I ran that just thing above I see the json. OK, now projection:       … | table _raw.json.class       no.       … | table json.class       no.       ... | table _raw       yes. OK, so maybe it's not parsed or whatever. spath to the rescue.       … | spath _raw.json.path | table json.path //please consider this as all potential combinations of cartesian products of all subpaths from _raw.json.path and same with table.       no! Ok(!!) maybe it's input parameter:       ... | spath input=_raw.json path=json.class | table json.class //all cartesian products again.       no.  ok. .. So maybe we need output for something I don't know what:       | spath input=_raw path=json.class output=aaaaa | table aaaaa //all cartesian products of subpaths again.       30 minutes passed... It's just-a-simple-projection.  No luck and I didn't even get to regex, which will be true struggle (all regex flavors are really easy, that's not the crux of the problem). chatgpt thinks this is the solution:       | makeresults | eval _raw="{\"json\": {\"class\": \"net.ttddyy.dsproxy.support.SLF4JLogUtils\"}}" | spath input=_raw path=json.class | table json.class       but it isn't. Does not print anything(it's so confusing, that trained chatgpt cannot do projection OR it does not work on our heavily paid sw). Can you please explain, how simple projection works in splunk and what steps the engine really do internally since it's really mystery? In jq to compare:       jq -n '{ "json": { "class": "net.ttddyy.dsproxy.support.SLF4JLogUtils", } } | .json | .class | sub("ttddyy";"goddamm-easy")'       to explain: 1) declare json 2) take .json subtree 3) take .class subtree 4) do replacement. 40s, straightforward. UPDATE: if I export data returned for table, and then download the data, I got different results in file than on screen, so I guess that what-is-shown on screen is different to what-the-data-of-query-is, which is probably source of confusion, since I'm working with something I can see, while splunk probably work with some datastructure I'm not aware of. { "preview": false, "result": { "json.msg": //:| ... ... View more

I have no idea why it printed this. And it's surprising, that adding  | spath json.msg output=msg | spath input=msg query{}   in this context yields different results than if I place it after my query in our system. And that the query for some reason also extract query[] part of json(in your context, not in mine). Why? Who asked for that? But even then I still cannot access 'parsed' field named 'batch'... I think the query is some generic function doing some guess-what-is-important extractions. I've got an idea, which will force us to avoid clever functions doing random data extractions. Can you please show me, how could I transform this little json sample into string: select * from whatever.whatever w where w.whatever in (1,2,3) Equivalent for this in jq would look like: jq -r '.json.msg|fromjson|.query[0] as $q| .params[]|reduce .[] as $param ($q; sub("\\?";$param))' full bash command including input being: echo '{"time":"2024-09-19T08:03:02.234663252Z","json":{"ts":"2024-09-19T15:03:02.234462341+07:00","logger":"<anonymized>","level":"WARN","class":"net.ttddyy.dsproxy.support.SLF4JLogUtils","method":"writeLog","file":"<anonymized>","line":26,"thread":"pool-1-thread-1","arguments":{},"msg":"{\"name\":\"\", \"connection\":22234743, \"time\":20000, \"success\":false, \"type\":\"Prepared\", \"batch\":false, \"querySize\":1, \"batchSize\":0, \"query\":[\"select * from whatever.whatever w where w.whatever in (?,?,?) \"], \"params\":[[\"1\",\"2\",\"3\"]]}","scope":"APP"},"kubernetes":{"pod_name":"<anonymized>","namespace_name":"<anonymized>","labels":{"whatever":"whatever"},"container_image":"<anonymized>"}}' | jq -r '.json.msg|fromjson|.query[0] as $q| .params[]|reduce .[] as $param ($q; sub("\\?";$param))';   ... View more

{ "time": "2024-09-19T08:03:02.234663252Z", "json": { "ts": "2024-09-19T15:03:02.234462341+07:00", "logger": "<anonymized>", "level": "WARN", "class": "net.ttddyy.dsproxy.support.SLF4JLogUtils", "method": "writeLog", "file": "<anonymized>", "line": 26, "thread": "pool-1-thread-1", "arguments": {}, "msg": "{\"name\":\"\", \"connection\":22234743, \"time\":20000, \"success\":false, \"type\":\"Prepared\", \"batch\":false, \"querySize\":1, \"batchSize\":0, \"query\":[\"select * from whatever.whatever w where w.whatever in (?,?,?) \"], \"params\":[[\"1\",\"2\",\"3\"]]}", "scope": "APP" }, "kubernetes": { "pod_name": "<anonymized>", "namespace_name": "<anonymized>", "labels": { "whatever": "whatever" }, "container_image": "<anonymized>" } }     to begin with, I'd like to do equivallent`jq '.json.msg|fromjson|.query[0]'`. After that, eventually, do the actual parameter substitutions, deduplication, counting, min/max time, but that's way beyond of scope of this question. ... View more