Hello Splunkers: This question is for the splunkers who are running their instances with splunk user. Three logs have stopped logging since April 17, 2018. They are splunkd.log, metrics.log and splunk-access.log. Also, now I am getting access denied error for splunkd.log while restarting splunk and also KV store failed due to access denied error for /mnt/data/splunk/kvstore/mongo/_tmp directory. Our splunk instance runs with splunk user. On 17th, I, by mistake started with root user. After realizing it, I stopped and restarted splunk with splunk user. Just after that, the problem started. All the three logs files and_tmp directory mentioned above are currently running with root. All others are running with splunk user all other files have splunk ownership. I am not sure whether the above mentioned log files and _tmp direct should be with splunk ownership. I have struggling with issue for the past few days. Any help on this will be highly appreciated. Can you please check your instances and let me know the ownership of the splunkd.log, metrics.log, splunk-access.log and /mnt/data/splunk/kvstore/mongo/_tmp directory? Thank you in advance. Deb ... View more

New in handling json files in splunk with little knowledge in rex. Need help on parsing/formatting json logs in search time. Recently collected auth0 json logs. Every event starts with different character and ends with different character. Looks like Splunk break based on the max limit of the characters per event. I am trying to read the data in Search with proper formatting in table form with all fields. props.conf stanza: [auth0] INDEXED_EXTRACTIONS = json KV_MODE = none logs looks like: ol: ...: {}\n...., ...."}, hostname":"...","user_id":"...", user_name":"...", "strategy":"...", "strategy_type":"enterprise","_id":"...", "log_id":"...", "isMobile":...},{"date":"2018-03-21T23:28:59.296Z","type":"...","connection_id":"","client_id":"...","client_name":"...", "ip":"...","user_agent":"...", "details":{"return_to":"...","allowed_logout_url":[..."]},"hostname":"...","user_id":"...","user_name":"...","_id":"...","log_id":"...","isMobile":...}, {"date":"2018-03-21T23:28:59.286Z","type":"...","connection":"...","connection_id":"...","client_id":"...","client_name":"...","ip":"...","user_agent":"...", "details":{"prompts":[{"name":"....","initiatedAt":..., ....several line from json file Sometimes the event starts with [{"date", sometimes with something else and ends before closing } . I went through the logs and understood that, it should be something like below in each event. { "date":"2018-03-09T23:41:05.437Z", "type":"..", "connection":"... "connection_id":"...", "client_id":"...", "client_name":"...", "ip":"...", "user_agent":"...", "details":{ "timings":{ "connection":..., "all_rules":..., "internal":..., "total":... }, "stats":{ "loginsCount":... }, "consoleOut":"..." }, "hostname":"...", "user_id":"...", "user_name":"...", "strategy":"...", "strategy_type":"...", "_id":"...", "log_id":"...", "isMobile":... }, ends up comma and then another one starts. Is there any way, I can extract log in search. Please advise. ... View more