All Topics

Find Answers
Ask questions. Get answers. Find technical product solutions from passionate members of the Splunk community.

All Topics

I have a lookup file which contains a list of jobnames, description and their SLAs. Example:  jobName Description SLA job1 Example1 08.00 job2 Example2 10.00 job5 Example3 05.00... See more...
I have a lookup file which contains a list of jobnames, description and their SLAs. Example:  jobName Description SLA job1 Example1 08.00 job2 Example2 10.00 job5 Example3 05.00   My index data (index=autosys) contains events for hundreds of jobs, their status and updated time.  An event example would be: job1, FAILED, 07.00 I'm trying to write a query to output the below. However, there are cases where there won't be any events for a specific job. In that case, I need to display, "NOT RUNNING" jobName Description SLA Status Updatetime job1 Example1 08.00 FAILED 07.00 job2 Example2 10.00 SUCCESS 09.00 job5 Example3 05.00 NOT RUNNING NULL   I'm using the following query, but it is not displaying the row that does not have any event/data in the index       | inputlookup append=t lookup_job.csv | table jobName, SLA, Description | join jobName [search index=autosys | inputlookup lookup_job.csv | fields jobName ]] | table jobName, Description, SLA, Status, Updatedtime          
Hi all, I'm leveraging the splunk/splunk:8.0.4.1 image for some development. At this time, I've pulled the splunk/splunk image, and configured it according to my needs. I now want to commit that con... See more...
Hi all, I'm leveraging the splunk/splunk:8.0.4.1 image for some development. At this time, I've pulled the splunk/splunk image, and configured it according to my needs. I now want to commit that container to save it as a new image, so that I have a baseline container to fall back on and start. However, my newly saved image fails to start when I perform a `docker run`. I've checked the original container for the `cacert.pem` file in question and it does exist. Since the new image I'm attempting to start is essentially a clone of the running container, I'm surprised that it fails to start. Is this a use case that can be achieved with the splunk/splunk image? Thank you. ``` TASK [splunk_common : Start Splunk via CLI] ************************************ fatal: [localhost]: FAILED! => { "changed": false, "cmd": [ "/opt/splunk/bin/splunk", "start", "--accept-license", "--answer-yes", "--no-prompt" ], "delta": "0:00:01.506092", "end": "2020-07-15 19:59:26.957025", "rc": 1, "start": "2020-07-15 19:59:25.450933" } STDOUT: Splunk> Take the sh out of IT. Checking prerequisites... Checking http port [8000]: open Checking mgmt port [8089]: open Checking appserver port [127.0.0.1:8065]: open Checking kvstore port [8191]: open Checking configuration... Done. Creating: /opt/splunk/var/lib/splunk Creating: /opt/splunk/var/run/splunk Creating: /opt/splunk/var/run/splunk/appserver/i18n Creating: /opt/splunk/var/run/splunk/appserver/modules/static/css Creating: /opt/splunk/var/run/splunk/upload Creating: /opt/splunk/var/run/splunk/search_telemetry Creating: /opt/splunk/var/spool/splunk Creating: /opt/splunk/var/spool/dirmoncache Creating: /opt/splunk/var/lib/splunk/authDb Creating: /opt/splunk/var/lib/splunk/hashDb STDERR: The CA file specified (/opt/splunk/etc/auth/cacert.pem) does not exist. Cannot continue. SSL certificate generation failed. MSG: non-zero return code PLAY RECAP ********************************************************************* localhost : ok=43 changed=6 unreachable=0 failed=1 skipped=44 rescued=0 ignored=0 ```
Hi, From a set of log events I need to get the daily Top 1000 calls by each appId and clientId. How can I do this in a single query without using the append command? log events: timestamp c... See more...
Hi, From a set of log events I need to get the daily Top 1000 calls by each appId and clientId. How can I do this in a single query without using the append command? log events: timestamp call=123 client=a appId=1 timestamp call=123 client=a appId=1 timestamp call=123  client=a appId=1 timestamp call=12 client=a appId=1 timestamp call=123 client=b appId=1 timestamp call=123 client=b appId=2 .... Nth event So the result set based on these logs is : day | call | client | appId | count 07/14/2020 | 123 | a | 1 | 3 07/14/2020 | 12 | a | 1 | 2 07/14/2020 | 123 | b | 1 | 1 07/14/2020 | 123 | b | 2 | 1  The idea is to get the Top 1000 calls by each tuple of client : appId. Thanks for your help.
Hi! I'm working on a bar graph visualization for a report. I need some help in achieving the attached bar graph type of visualization. I have a count for all the apps and I need to overlay user on ... See more...
Hi! I'm working on a bar graph visualization for a report. I need some help in achieving the attached bar graph type of visualization. I have a count for all the apps and I need to overlay user on top of the app bar to show which user holds the maximum amount of data. Appreciate your help! TIA.  
I'm currently using PersistentServerConnectionApplication to implement custom API endpoints inside Splunk. Any changes I make in code delays about 1 or 2 minutes to start working.  Is this the corre... See more...
I'm currently using PersistentServerConnectionApplication to implement custom API endpoints inside Splunk. Any changes I make in code delays about 1 or 2 minutes to start working.  Is this the correct behavior or am I missing something? If it is the correct behavior, is there any form to test the endpoint immediately after save the code? Also, I would like to know if BaseRestHandler approach still works on Splunk newer versions (8.x), because I coudn't make this work (python can´t find splunk.rest.BaseRestHandler class). Best regards.
I am sending data to my splunk instance like https://docs.splunk.com/Documentation/Splunk/8.0.4/Data/HECExamples says to: curl -k "https://mysplunkserver.example.com:8088/services/collector" \ -H "... See more...
I am sending data to my splunk instance like https://docs.splunk.com/Documentation/Splunk/8.0.4/Data/HECExamples says to: curl -k "https://mysplunkserver.example.com:8088/services/collector" \ -H "Authorization: Splunk CF179AE4-3C99-45F5-A7CC-3284AA91CF67" \ -d '{"sourcetype": "_json","event": {'....over 10,000 character json event' I get a success return {"text":"Success","code":0}   When query for the event and i do a character count on it, I see it always truncates my events to 10,000 characters. On my server, looking at ./etc/system/default/limits.conf It has inside: [http_input] # The max number of tokens reported by logging input metrics. max_number_of_tokens = 10000 # The interval (in seconds) of logging input metrics report. metrics_report_interval = 60 # The max request content length (800MB, to match HTTP server). max_content_length = 838860800 # The max number of ACK channels. max_number_of_ack_channel = 1000000 # The max number of acked requests pending query. max_number_of_acked_requests_pending_query = 10000000 # The max number of acked requests pending query per ACK channel. max_number_of_acked_requests_pending_query_per_ack_channel = 1000000 Seems like everything is ok there.   And in ./etc/system/local/limits.conf it has: [search] allow_batch_mode = 1 allow_inexact_metasearch = 0 always_include_indexedfield_lispy = 0 default_allow_queue = 1 disabled = 0 enable_conditional_expansion = 1 enable_cumulative_quota = 0 enable_datamodel_meval = 1 enable_history = 1 enable_memory_tracker = 0 force_saved_search_dispatch_as_user = 0 load_remote_bundles = 0 record_search_telemetry = 1 remote_timeline = 1 search_retry = 0 timeline_events_preview = 0 track_indextime_range = 1 track_matching_sourcetypes = 1 truncate_report = 0 unified_search = 0 use_bloomfilter = 1 use_metadata_elimination = 1 write_multifile_results_out = 1   Why is Splunk truncating my events to 10,000 characters that I am sending? If my JSON is less than 10,000 characters, I am able to see all the data and it JSON formatted when I do a splunk query.
We have the following code that ran for one and a half hours last week and consumed lots of cpu. How can we optimize this query?     index=<index name> Error* | eval ERROR_TYPE = if(searchmatch("... See more...
We have the following code that ran for one and a half hours last week and consumed lots of cpu. How can we optimize this query?     index=<index name> Error* | eval ERROR_TYPE = if(searchmatch("SQLException"), "SQL Exceptions", (if(searchmatch("JwtTokenUtil"), "JWTToken Error", (if(searchmatch("JwtAuthenticationTokenFilter"), "Authentication error", (if(searchmatch("java.lang.OutOfMemoryError: Java heap space"), "Out of memory error", (if(searchmatch("sendEmail"), "Send Mail error", (if(searchmatch("message was not delivered"), "IMAP error", (if(searchmatch("sever not reponding"), "Server error", (if(searchmatch("Timed Out"),"Timed Out Error", "OTHER"))))))))))))))) | timechart count by ERROR_TYPE    
I want to set a token value based on different dropdown values. eg: <input type="dropdown" token="endpoint" searchWhenChanged="true">     <label>endpoint</label>     <choice value=“/abc”>/abc</cho... See more...
I want to set a token value based on different dropdown values. eg: <input type="dropdown" token="endpoint" searchWhenChanged="true">     <label>endpoint</label>     <choice value=“/abc”>/abc</choice>     <choice value=“/defb”>/def</choice>     <choice value=“/xyz”>/xyz</choice>     <default>/abc</default>     <initialValue>/abc</initialValue> </input>   <input type="dropdown" token="environment" searchWhenChanged="true">     <label>environment</label>     <choice value="prod">prod</choice>     <choice value="ppe">pre-prod</choice>     <choice value="dev">dev</choice>     <default>prod</default>     <initialValue>prod</initialValue> </input>   In this example, 1. I want to set a token(named as targetName) value to "employee" if the endpoint value is /def and environment is prod.  2. TargetName should be set to "address" for any other endpoint with environment value as prod.    
I never received an email from Splunk after I signed up for the 7 day free trial of the Splunk ES sandbox. Although my "Instances" page shows that I can access my instance, when I attempt to login wi... See more...
I never received an email from Splunk after I signed up for the 7 day free trial of the Splunk ES sandbox. Although my "Instances" page shows that I can access my instance, when I attempt to login with my username and password, I get a login failed message. The password I'm using is the same one I use to login to the Splunk Portal.  Can someone please assist?
I am fairly new to Splunk and only have the basics under my belt at best. I'm having trouble proving out the following use case:  Taking a malicious URL (threatURL) from email threat (index=emailThr... See more...
I am fairly new to Splunk and only have the basics under my belt at best. I'm having trouble proving out the following use case:  Taking a malicious URL (threatURL) from email threat (index=emailThreat) events and then querying the firewall (index=firewall) for any user activity including the referenced malicious URL. So far I have the following search: index=firewall log_subtype=url [search index=emailThreat | dedup threatUrl | table threatUrl] | table time, url, user Separately, the the two searches work and return the expected value. However, when put together, the search returns no results. Not sure what I'm missing.
I need to write a common regex to match all the below patterns  My regular expression written so far is  (?P<timestamp>\w+\s+\d+\s+\d+:\d+:\d+)\s+(?P<src>\w+)\s+(?P<daemon>\w+):\s+(?P<message>(.*?)... See more...
I need to write a common regex to match all the below patterns  My regular expression written so far is  (?P<timestamp>\w+\s+\d+\s+\d+:\d+:\d+)\s+(?P<src>\w+)\s+(?P<daemon>\w+):\s+(?P<message>(.*?)$)|(?J)(?P<timestamp>\w+\s+\d+\s+\d+:\d+:\d+)\s+(?P<src>\w+)\s+(?P<daemon>\w+)\[(?P<process_id>\d+)\]:\s+(?P<entry_type>\w+):\s+(?P<service_id>\w+)\s+\w+=((?P<status>0)|(?<pid>\d+))(\s+from=(?P<origin_ip>(.*?)$)) This matches 1st and 2nd pattern in regex101.com but when I put it in splunk it doesn't work matching unintended fields. Please help how to go with this Jul 15 14:01:32 jiufc1fe330 xinetd[82352]: START: nrpe pid=151239 from=::ffff:14.956.44.41 Jul 15 12:30:36 dyue29200 systemd: Removed slice User Slice of root. Jul 15 12:30:21 dtg280419 xinetd[16211]: EXIT: nrpe status=0 pid=8924 duration=0(sec)
Hi,  how to compare search1 results with search2 and list out how many matched and not matched. EX: search1: index=test sourcetype=sample | rex "type=(?<Job>.*) " |dedup Job |table Job search2: ... See more...
Hi,  how to compare search1 results with search2 and list out how many matched and not matched. EX: search1: index=test sourcetype=sample | rex "type=(?<Job>.*) " |dedup Job |table Job search2:  index=** sourcetype=** |rename JOBS AS Job |dedup Job |table Job sample data from search1: Jobs xxx yyy zzz aaa sample data from search2: Jobs aaa bbb ccc ddd xxx ttt Expected sample output: search1 is returning 100 jobs and search2 is returning 200 jobs, we need to list out the jobs those are not matching search1 with search2 for example: out of 100 jobs if 40 matched with search2 remaining 60 not matched jobs list in search1  Output: Jobs bbb ccc ddd ttt Tried |set diff command but not worked, Please help. Thanks in advance.        
In my logfile there are two separate lines in there, I need to look at first one and get some of the result but then merge with the above event and create a report like in the following example - **... See more...
In my logfile there are two separate lines in there, I need to look at first one and get some of the result but then merge with the above event and create a report like in the following example - ****** NEW WEBSCRIPT REQUEST ****** [1st Line] --- Uploading file to TestDB from \\server1\TestFolder\out\1stTestforme.pdf --- --- --- Upload Completed /Repo/service/ABC/upload succeeded Request: {empNumber=12345, name=1st Test For Me, uploadedBy=User1, DeptCode=6789, source=App1, mimeType=application/pdf}, Response: {MIMETYPE=application/pdf, SIZE=0.0 KB} [Last Line] I need to get a report that Provides me a report of all records that that 0KB (Event 2) but File Name should be from the Event 1 So, result Set can be - empNumber source DeptCode UoloadedBy FileName 12345      App1    6789      User1   1stTestforme.pdf Will appreciate if anyone can help in this. Thanks!
When installing the "universal forwarder" I put a "path to monitor". But I would like to add another path. How do I add more "path to monitor"?
Hi guys, New to Splunk so pardon the simplicity of my question! Im trying to bounce my csv list off another one. I've a list of servers in my spreadsheet, uploaded it to a lookup table files. By my... See more...
Hi guys, New to Splunk so pardon the simplicity of my question! Im trying to bounce my csv list off another one. I've a list of servers in my spreadsheet, uploaded it to a lookup table files. By my problem is (I think) that the other file has the server names in a different format. Eg. my server name is lets say "gb1234" and the other file it's saved as "UK gb1234" or "EU UK gb1234"   I came up with a basic search command:  | inputlookup MY_LIST.csv | eval host=upper(host) | lookup OTHER_LIST asset_key as host | fields host, asset_version, asset_bu   But I'm guessing that because of the format in the search i need to use regex.... But where and how??    
Hi everyone, Has anyone faced the following problem when importing a health rule in JSON format? I'm using the following curl command: curl -sL -w "%{http_code}" -X POST --user "${USER}:${PASS}" ... See more...
Hi everyone, Has anyone faced the following problem when importing a health rule in JSON format? I'm using the following curl command: curl -sL -w "%{http_code}" -X POST --user "${USER}:${PASS}" \ -H "Content-Type: application/json" -F file=@${FILE}.json \ https://$HOST/controller/alerting/rest/v1/applications/$APP_ID/health-rules which in every JSON file I tried, returns me this error: { "statusCode" : 400, "message" : "Unexpected character (- (code 45)) in numeric value: expected digit (0-9) to follow minus sign, for valid numeric value " } ... even if I'm using the same JSON as in the official documentation: https://docs.appdynamics.com/display/PRO45/Health+Rule+API The JSON files I've tried do not contain any hyphen characters, so apparently the request payload is not related to the error cause. Anyone been there? Thanks and regards.
Hi Splunkers, I have a Heavy Forwarder using this IA-G Suite Add-on app. The app worked properly last year, but now is having some issues to retrieve the data from the API. In _internal index ... See more...
Hi Splunkers, I have a Heavy Forwarder using this IA-G Suite Add-on app. The app worked properly last year, but now is having some issues to retrieve the data from the API. In _internal index show the following error:       2020-07-15 10:31:39,492 log_level=ERROR pid=11773 tid=MainThread file="ModularInput.py" function="print_error" line_number="659" version="GSuiteForSplunk.v1.3.1.b261" host=IA-GSuiteForSplunk sourcetype=IA-GSuiteForSplunk:error source=gapps:EXAMPLE.COM {"errors": [{"msg": "<HttpError 500 when requesting https://www.googleapis.com/discovery/v1/apis/drive/v3/rest returned \"Backend Error\">", "input_name": "ga://activity", "filename": "GoogleAppsForSplunkModularInput.py", "exception_type": "HttpError", "line": 511, "exception_arguments": "<HttpError 500 when requesting https://www.googleapis.com/discovery/v1/apis/drive/v3/rest returned \"Backend Error\">"}], "log_level": "ERROR", "timestamp": "Wed, 15 Jul 2020 14:31:39 +0000"}         Its shows that the problem could be in the G Suite side, but we can't find any clue what could it be   Thanks You so much!
HI,  I have splunk enterprise free version installed on AWS instance. When i access the splunk with IP <ip>:8000 iam able  to access the splunk. I need to access the splunk with domain name so i set... See more...
HI,  I have splunk enterprise free version installed on AWS instance. When i access the splunk with IP <ip>:8000 iam able  to access the splunk. I need to access the splunk with domain name so i set in server.conf [root@ipXXXXX local]# cat server.conf [general] serverName = splunkconnect.online.io in input.conf [root@ipxxxxxxx local]# cat inputs.conf [default] host = splunkconnect.online.io in web.conf [root@ip-xxXXXx local]# cat web.conf [settings] enableSplunkWebSSL = 1 httpport = 8000   when i hit the url https://splunkconnect.online.io iam getting this site cant be reached. where else do i need to configure. Can anyone please let me know.
Hi All, Thanks for taking the time to review this message. I attempting to create a Splunk notable that will allow me to detect if our organisation 'breakglass' accounts have been successfully auth... See more...
Hi All, Thanks for taking the time to review this message. I attempting to create a Splunk notable that will allow me to detect if our organisation 'breakglass' accounts have been successfully authenticated. The existing notable uses the Authentication Data Model and this has been disable in the past for some reason. I am attempting to revise this query to search not on one break glass account but for a few. The exist query looks something like this, as shown below. | datamodel Authentication Successful_Authentication search | search Authentication.signature_id=4624 Authentication.user=Administrator NOT (host=AWD* OR ComputerName=EC2* OR ComputerName=WIN*) | rename Authentication.user as user | table src_ip, user, host | eval urgency="critical" I have change the Authentication.user=Administrator specifically to the userid of the one of the breakglass account, how do I include in the search for the other accounts, by using a OR operator?  Appreciate any guidance anyone can offer and refining this SPL query. Many thanks in advance.
My settings in web.conf enableSplunkWebSSL = True privKeyPath = /opt/splunk/etc/auth/myxxx/private.key serverCert = /opt/splunk/etc/auth/myxxx/web.pem (conatins server cer, Intermediate ca and roo... See more...
My settings in web.conf enableSplunkWebSSL = True privKeyPath = /opt/splunk/etc/auth/myxxx/private.key serverCert = /opt/splunk/etc/auth/myxxx/web.pem (conatins server cer, Intermediate ca and root ca) requireClientCert = false   Getting error in browser > NET::ERR_CERT_COMMON_NAME_INVALID splunkd.log SSLCommon - Received fatal SSL3 alert. ssl_state='SSLv3 read client key exchange A', alert_description='certificate unknown'. HttpListener - Socket error from x.x.x.x:xxxx while idling: error:14094416:SSL routines:ssl3_read_bytes:sslv3 alert certificate unknown   All configuration seems correct. anyone resolved similar issue.