added sample data ... View more

@LukeMurphey  Hi Luke, Will yo be able to guide me on this issues. ... View more

@PavelP : Some time we use to have Splunk install and forwarder on same box so we might have changed management port to 9089 for forwarder to avoid conflict. Forwarder is original Universal forwarder. I tried changing port back to 8089 but still not working 😞 ... View more

I tried your command on Forwarder but its not working. eventhough splunkd is up its saying its down. Am I missing something here ? bash$ bin/splunk status splunkd is running (PID: 18089). splunk helpers are running (PIDs: 18090). mx25089vm autoengine /data/apps/splunkforwarder_MIC_v44_ORCH1/ bash$ bin/splunk _internal call /services/admin/inputstatus/TailingProcessor:FileStatus QUERYING: 'https://127.0.0.1:9089/services/admin/inputstatus/TailingProcessor:FileStatus' This command [GET /services/admin/inputstatus/TailingProcessor:FileStatus] needs splunkd to be up, and splunkd is down. ... View more

@skalliger : thanks for reply. Props.conf First data is injested to GC_11_RAW and only relevant events trasffered to GC11 sourcetype [GC11] disabled = false SHOULD_LINEMERGE = false TIME_PREFIX = ^\[ LINE_BREAKER = ([\r\n]+) MAX_TIMESTAMP_LOOKAHEAD = 100 EXTRACT-PAUSE_FULL = (\[(?P\d*-\d*-\d\dT\d\d:\d\d:\d\d.\d*\+\d*)\])?(\[(?P\d+.\d+)s])?.*GC\((?P\d*)\) (?PPause Full)(.*) (?P[0-9]*)M->(?P[0-9]*)M\((?P[0-9]*)M\) (?P[0-9]*.[0-9]*)ms [GC11_RAW] LINE_BREAKER = ([\r\n]+) SHOULD_LINEMERGE = false NO_BINARY_CHECK = true category = Custom TRANSFORMS-sourcetye_routing = JAVA_GC11_sourcetye_routing transforms.conf [JAVA_GC11_sourcetye_routing] DEST_KEY = MetaData:Sourcetype REGEX = GC\(\d*\)|Using G1 FORMAT = sourcetype::GC11 There is no heavy forwarder in between. even no unusal activity on monitoing console. data volume is same what we ususally gets. _time is correct because its been extracted from logs. Just indexing is lagging. ... View more

thanks but it doesn't seem to be working. Even though I have given full permission to this app and my app which has dashboard to export, it always display page not found (404 error). ... View more

For the same reason this app is desinged for. Default pdf export misses many GUI functionality. for example: Table colors are removed, not all charts been exported ( Timeline chart etc...) Smart pdf export exact snapshot of dashboard. ... View more

Actually it worked with result.host_token but its not solving my problem. Second search is still running with all time. (Same results as using subsearch to override time) ... View more

Thanks @skoelpin I have tried this approch but still I am not be able to get desired outcome. 1. Outcome of first search is not been passed to second search properly. This is how I am using my queries. **first search output in table** | table host_token service_earliest_time earliest latest time_token.earliest time_token.latest **Second search** | savedsearch "TEST_KPI_MTE_ALERT_FUNCTION" host_token="$host_token$" service_earliest_time="$service_earliest_time$" earliest="$earliest$" latest="$latest$" time_token.earliest="$time_token.earliest$" time_token.latest="$time_token.latest$" Inspite arguments are not passed to second search, I can see second query is running but results are stored in "main" index in json format. which will be diffcult to parse. I was hopping to get results either via email or dumping in some file. extracting from logs will be difficult. I am not sure if second search will use earliest and latest time from output of first search. This is results what I am getting in logs (tokens not having values from first search) {"messages": [{"type": "INFO", "text": "[subsearch]: Successfully read lookup file '/hp737srv2/apps/splunk/etc/apps/Murex/lookups/KPI_MASTER_LIST.csv'."}, {"type": "WARN", "text": "Unable to run query '| savedsearch KPI_MTE_NO_OF_CRITICAL_INFRA host_token= SERVICE_EARLIEST_TIME= time_token.earliest= time_token.latest= | appendcols [ | makeresults | eval Order=7 | fillnull count ] | table ALERT count Order '."}], "earliest": "1970-01-01T01:00:00.000+01:00", "description": "Splunk Adaptive Response Search", "latest": "2020-01-29T13:27:05.%L", "search": "| savedsearch \"TEST_KPI_MTE_ALERT_FUNCTION\" host_token=\"\" service_earliest_time=\"\" earliest=\"\" latest=\"\" time_token.earliest=\"\" time_token.latest=\"\"", "results": [{"TYPE": "MTE_GENERIC", "Category": "INFRA", "ALERT": "KPI_MTE_NO_OF_CRITICAL_INFRA", "Alert_Description": "Total No of Critical Alerts (INFRA)", "Dashboard": "MTE_ALERTS_CRITICAL_INFRA", "Threshold": "0"}, {"TYPE": "MTE_GENERIC", "Category": "INFRA", "ALERT": "KPI_EnvStatus_INFRA", "Alert_Description": "EnvStatus INFRA", "Dashboard": "MTE_envStatus_INFRA", "Threshold": "0"}, {"TYPE": "MTE_GENERIC", "Category": "MTE", "ALERT": "KPI_MTE_RUNTIME_BA", "Alert_Description": "Business Activities Failed", "Dashboard": "MTE_RUNTIME_BA", "Threshold": "0"}, {"TYPE": "MTE_GENERIC", "Category": "MTE", "ALERT": "KPI_MTE_RUNTIME_BA_EOD", "Alert_Description": "BA_EOD Failed Scripts", "Dashboard": "MTE_RUNTIME_BA_EOD", "Threshold": "0"}, {"TYPE": "MTE_GENERIC", "Category": "MTE", "ALERT": "KPI_MTE_RUNTIME_TOOLING", "Alert_Description": "MTE RUNTIME TOOLING (Not Executed)", "Dashboard": "MTE_RUNTIME_TOOLING", "Threshold": "0"}, {"TYPE": "MTE_GENERIC", "Category": "MX", "ALERT": "KPI_MTE_NO_OF_CRITICAL_MX", "Alert_Description": "Total No of Critical Alerts (MX)", "Dashboard": "MTE_ALERTS_CRITICAL_MX", "Threshold": "0"}, {"TYPE": "MTE_GENERIC", "Category": "MTE", "ALERT": "KPI_MTE_NO_OF_CORES", "Alert_Description": "Total No of Cores", "Dashboard": "MTE_ALERTS_CORES", "Threshold": "0"}, {"TYPE": "MTE_GENERIC", "Category": "MTE", "ALERT": "KPI_MTE_ALERT_RECIPIENT_COUNT", "Alert_Description": "MTE Alerts > 20 Per User", "Dashboard": "MTE_ALERT", "Threshold": "25"}, {"TYPE": "MTE_GENERIC", "Category": "MTE", "ALERT": "KPI_EnvStatus_MXTECH", "Alert_Description": "EnvStatus MXTECH", "Dashboard": "MTE_envStatus_MXTECH", "Threshold": "0"}, {"TYPE": "MTE_GENERIC", "Category": "MTE", "ALERT": "KPI_EnvStatus_LOAD", "Alert_Description": "EnvStatus LOAD (Value<=3)", "Dashboard": "MTE_envStatus_LOAD", "Threshold": "3"}, {"TYPE": "MTE_GENERIC", "Category": "MTE", "ALERT": "KPI_EnvStatus_BAU", "Alert_Description": "EnvStatus BAU (Value<=3)", "Dashboard": "MTE_envStatus_BAU", "Threshold": "3"}, {"TYPE": "MTE_GENERIC", "Category": "MTE", "ALERT": "KPI_EnvStatus_BP", "Alert_Description": "EnvStatus BP", "Dashboard": "MTE_envStatus_BP", "Threshold": "0"}]} ... View more

Thanks, I tried this approach but it doen't work. eventhought earliest/ latest time is overriden by subsearch but still savedsearch use all time. ... View more

thanks but as I mentioned before, NPID, NICKNAME, Machine_Name are not same. there could be hundreds of different value. And 1st solution will fill wrong values without checking the condition of matching NPID and Machine_Name ... View more

just using filldown won't work for me. NICKNAME, NPID, Machine_Name are not always having same value. NPID and Machine name should match before updating the empty NICKNAME. Attached another example in question. ... View more

@BainM : thanks for response. I have reverted server.conf. I have attached screenshot of relevant errors I am getting in Splunkd.log There errors related to python and getting this error for multiple apps. ... View more