i had that feeing to, but it is still not working, more over , i tried the simplest query that does nor inclde any specail chars and it is still throwing the same error. this also fails:  index=* | eval bytes=10+20 | table id.orig_h, id.resp_h, bytes    

The Github app for Splunk supports both Github and Splunk Cloud, it should be possible to set up the first part of your log path using the documentation for it.   https://splunkbase.splunk.com/app/5596    

I would recommend making the following checks: 1. The props.conf file is on the indexer machines 2. The props.conf file is readable by the splunk user 3. The TZ value in the props.conf file reflects the timezone of the logs 4. In your Splunk User Preferences in the webUI, your timezone is set to your current timezone

I don't see any events when filtering index=_internal and source=<path_to_splunkd.log> (with my path obviously) but I do see errors when looking in the splunkd.log file in my SOAR machine - lots of "connection to host <indexer>:9997 failed", which is weird because 9997 is open on the splunk indexer, the machines are in the same segment and the "test connectivity" worked.

Did you set up your SOAR to forward logs? Go to Administration->Administration Settings->Forwarder Settings->New Group Then add your indexers, e.g.: indexer1:9997 Check the boxes for which logs you would like to see. Add an optional TCP token if it applies for your environment. Then if you save this configuration, your SOAR should start sending logs to Splunk Enterprise.   Ref: https://docs.splunk.com/Documentation/SOARApp/1.0.57/Install/ConnectremotesearchSOAR6.2 https://docs.splunk.com/Documentation/SOARonprem/latest/Admin/Forwarders

Hi Giuseppe, Thank you for your reply.  But just wanted to understand bit more on this, All default collections.conf and transforms.conf will be synced with Secondary SH from Primary SH. Once we bring up the services on Secondary SH in future, It should populate all the data as is in primary as we got default KVstore in secondary as well ? Is my understanding correct ?  Regards VK  

For this problem, using the lookup in subsearch is more direct and potentially more efficient. |mstats sum(faliure.count) as Failed where index=metric-logs by service application_codes | search type = error1 [inputlookup app.csv]  

This is an interesting challenge because of the leading underscore (_) in the root node key.  spath can't seem to recognize the path as is. (Could be a subtle bug.)  One potential way to work around this is to use text replacement to get rid of the underscore. But I prefer a more syntactic method.  In Splunk 9, you can do fronjson with an arbitrary prefix so spath can do its job. (You can also use fromjson repeatedly.  But with deep paths like this problem, that's undesirable.)   | fromjson jsondata prefix=my | spath input=my_embedded path=metadata.data.results{} | mvexpand metadata.data.results{} | spath input=metadata.data.results{} | foreach notifications.* [eval ErrorCode = if(isnull(ErrorCode), "<<MATCHSTR>>", ErrorCode), ErrorMessage = mvappend(ErrorMessage, '<<FIELD>>')] | stats count by ErrorCode ErrorMessage   Your sample data results in ErrorCode ErrorMessage count 212 Document not detected 1 212 The image could not be found 1 212 The image quality was poor 1 This is an emulation you can play with and compare with real data   | makeresults | eval jsondata = "{ \"_embedded\": { \"metadata\": { \"environment\": { \"id\": \"6b3dc\" }, \"data\": { \"results\": [ { \"documentId\": \"f18a20f1\", \"notifications\": { \"212\": \"The image quality was poor\" } }, { \"documentId\": \"f0fdf5e8c\", \"notifications\": { \"680\": \"The image could not be found\" } }, { \"documentId\": \"95619532\", \"notifications\": { \"809\": \"Document not detected\" } } ] } } } }"   Note: Once you get past that leading underscore in JSON path, you can then use the text manipulation method proposed by @ITWhisperer (with a small path correction), like this   | fromjson jsondata prefix=my | spath input=my_embedded path=metadata.data.results{} | mvexpand metadata.data.results{} ``` not metadata.data.results{}.notifications ``` | rex field=metadata.data.results{} "\"(?<ErrorCode>\d+)\":\s*\"(?<ErrorMessage>[^\"]*)\"" | stats count by ErrorCode ErrorMessage   But text manipulation on structured data is not as robust because a developer/software can always change format in the future without altering semantics.