All Topics

Find Answers
Ask questions. Get answers. Find technical product solutions from passionate members of the Splunk community.

All Topics

Hi All, I am trying to create a regular expression to extract a value from a given log. Below is the log: 2021-10-05 07:25:42.986, DATUM2="3095", STATUS="2", REQUEST_TYPE="103", PRIORITY="300", OWN... See more...
Hi All, I am trying to create a regular expression to extract a value from a given log. Below is the log: 2021-10-05 07:25:42.986, DATUM2="3095", STATUS="2", REQUEST_TYPE="103", PRIORITY="300", OWNER="490070", COUNT(1)="2" Here I want to extract value of "COUNT(1)" and created the regular expression (?ms)COUNT\(1\)\=\"(?P<COUNT(1)>\d+)\" But with this expression I am not able to get the field name as "COUNT(1)" which is my requirement. Please help modify my expression to get the desired output.   Thank you very much.
Hye. The situation :  an instance of Splunk standalone (test platform), and an UF. The data : JSON Stream with multi level. The problem : the volume of data being important, we would like to reduc... See more...
Hye. The situation :  an instance of Splunk standalone (test platform), and an UF. The data : JSON Stream with multi level. The problem : the volume of data being important, we would like to reduce the _raw at only one field. But all JSON fields are saved as _meta. We have succeeded to update source, sourcetype and host from the JSON datas. But impossible to omit _meta ... (they always appear in the Search Head) IN :  { "input":{      "type":"log"}, "log":{      "file":"c:\log.josn"}, "@metadata":{      "beat":"filebeat",      "version":"7.10.2"}, "message":"bla bla bla", "fields":{      "type":"bdc",      "host":"VLCR03",      "type2":"back"} } OUT :  _raw  : "bla bla bla" <= OK meta "input.***" <= to suppress meta "log.***" <= to suppress meta "@metadata.beat" <= to keep meta "@metadata.version"<= to suppress meta "message"<= to suppress meta "fields.***" <= to suppress props.conf on the UF SHOULD_LINEMERGE = false NO_BINARY_CHECK = true CHARSET = AUTO KV_MODE = none AUTO_KV_JSON = false INDEXED_EXTRACTIONS = JSON TRANSFORMS-x = set_host set_source set_sourcetype TRANSFORMS-y = extract_message TRANSFORMS-z = remove_metadata transforms.conf on the UF [extract_message] SOURCE_KEY = field:message REGEX = (.*) FORMAT = $1 DEST_KEY = _raw [set_host] SOURCE_KEY = field:fields.host REGEX = (.*) FORMAT = host::$1 DEST_KEY = MetaData:Host [set_source] SOURCE_KEY = field:log.file REGEX = (.*) FORMAT = source::$1 DEST_KEY = MetaData:Source [set_sourcetype] SOURCE_KEY = fields:fields.type,fields.type2 REGEX = (.*)\s(.*) FORMAT = sourcetype::$1:$2 DEST_KEY = MetaData:Sourcetype [remove_message] SOURCE_KEY = _meta:message REGEX = (.*) DEST_KEY = queue FORMAT = nullQueue
Hello All, I am testing the upgrade from ES 6.2.0 to 6.6.2.  When I do the upgrade it fails with OSError type 28 no space left of device.  But there is almost 30GB of disk space free.     202... See more...
Hello All, I am testing the upgrade from ES 6.2.0 to 6.6.2.  When I do the upgrade it fails with OSError type 28 no space left of device.  But there is almost 30GB of disk space free.     2021-10-04 19:18:28,028 INFO [615bb5deed7f2dc4595650] _cplogging:216 - [04/Oct/2021:19:18:28] HTTP Request Headers: Remote-Addr: 127.0.0.1 TE: chunked HOST: splunk-sh1.wv.mentorg.com:8000 ACCEPT-ENCODING: gzip, br CACHE-CONTROL: max-age=0 SEC-CH-UA: "Google Chrome";v="93", " Not;A Brand";v="99", "Chromium";v="93" SEC-CH-UA-MOBILE: ?0 SEC-CH-UA-PLATFORM: "Windows" UPGRADE-INSECURE-REQUESTS: 1 ORIGIN: null USER-AGENT: Mozilla/5.0 (Windows NT 6.3; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/93.0.4577.82 Safari/537.36 ACCEPT: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9 SEC-FETCH-SITE: same-origin SEC-FETCH-MODE: navigate SEC-FETCH-USER: ?1 SEC-FETCH-DEST: document ACCEPT-LANGUAGE: en-US,en;q=0.9 COOKIE: splunkweb_csrf_token_8000=[REDACTED]5649; session_id_8000=[REDACTED]5b74; token_key=[REDACTED]5649; experience_id=[REDACTED]b0c2; splunkd_8000=[REDACTED]tgchx REMOTE-USER: admin X-SPLUNKD: SKdIpkhtf8PlfUDwvOLunA== 11626949294704615649 ijbs1HY^4Ms541EE5sF6eqHg^iyD5t6QKZRByWhdMDXkj546^eB1lT6y59b9LewgHbLcz0Xa5SKotHijcl__zWhYqh8MZISrCqYVxuLkY7jijwyyXijSUQ9VAJRlcQA3o7tgchx 0 Content-Type: multipart/form-data; boundary=----WebKitFormBoundaryO0HdVIPxgJr5HUZN Content-Length: 675766277 2021-10-04 19:18:28,029 INFO [615bb5deed7f2dc4595650] error:333 - POST /en-US/manager/appinstall/_upload 127.0.0.1 8065 2021-10-04 19:18:28,029 INFO [615bb5deed7f2dc4595650] error:334 - 500 Internal Server Error The server encountered an unexpected condition which prevented it from fulfilling the request. 2021-10-04 19:18:28,029 ERROR [615bb5deed7f2dc4595650] error:335 - Traceback (most recent call last): File "/opt/splunk/lib/python3.7/site-packages/cherrypy/_cprequest.py", line 628, in respond self._do_respond(path_info) File "/opt/splunk/lib/python3.7/site-packages/cherrypy/_cprequest.py", line 680, in _do_respond self.body.process() File "/opt/splunk/lib/python3.7/site-packages/cherrypy/_cpreqbody.py", line 982, in process super(RequestBody, self).process() File "/opt/splunk/lib/python3.7/site-packages/cherrypy/_cpreqbody.py", line 559, in process proc(self) File "/opt/splunk/lib/python3.7/site-packages/cherrypy/_cpreqbody.py", line 225, in process_multipart_form_data process_multipart(entity) File "/opt/splunk/lib/python3.7/site-packages/cherrypy/_cpreqbody.py", line 217, in process_multipart part.process() File "/opt/splunk/lib/python3.7/site-packages/cherrypy/_cpreqbody.py", line 557, in process self.default_proc() File "/opt/splunk/lib/python3.7/site-packages/cherrypy/_cpreqbody.py", line 717, in default_proc self.file = self.read_into_file() File "/opt/splunk/lib/python3.7/site-packages/cherrypy/_cpreqbody.py", line 732, in read_into_file self.read_lines_to_boundary(fp_out=fp_out) File "/opt/splunk/lib/python3.7/site-packages/cherrypy/_cpreqbody.py", line 702, in read_lines_to_boundary fp_out.write(line) OSError: [Errno 28] No space left on device           As you can see there should be plenty of room for a 670MB upload     splunk@splunk-sh1:~/var/log/splunk> df -kh /opt/splunk Filesystem Size Used Avail Use% Mounted on /dev/mapper/system-splunk 74G 44G 27G 63% /opt splunk@splunk-sh1:~/var/log/splunk>     Web.conf     splunk@splunk-sh1:~/var/log/splunk> more ~/etc/system/local/web.conf [settings] login_content = <h1> <CENTER>Splunk Dev Search Head</CENTER> </h1> max_upload_size = 1024 enableSplunkWebSSL = 1 privKeyPath = /opt/splunk/etc/auth/splunkweb/com.key caCertPath = /opt/splunk/etc/auth/splunkweb/expJun2022.crt splunkdConnectionTimeout = 1400 tools.sessions.timeout = 180 sslVersions = ssl3,tls cipherSuite = TLSv1+HIGH:TLSv1.2+HIGH:@STRENGTH splunk@splunk-sh1:~/var/log/splunk>     So I am confused why it would say that there is no space left of the device. Thanks ed
Hi, Updated: I am trying to break events which is in nested json. Each events start with  { "links":  I have almost got it working. Just small part left is that now after each event there is o... See more...
Hi, Updated: I am trying to break events which is in nested json. Each events start with  { "links":  I have almost got it working. Just small part left is that now after each event there is one " ,  "  and due to this event is not recognized as json event. Any idea how to remove it. Screenshot.   Props.conf 95% working props. CHARSET = UTF-8 DATETIME_CONFIG = KV_MODE = json LINE_BREAKER = ([\r\n,]*(?:{[^[{]+\[)?){"links" NO_BINARY_CHECK = true SEDCMD-removefooter = s/(\]\,).*//g SEDCMD-removeheader = s/\{\"data\": \[//g SHOULD_LINEMERGE = false TIME_FORMAT = %Y-%m-%dT%H:%M:%S.%3NZ TIME_PREFIX = "endTime": " TRUNCATE = category = Custom description = JavaScript Object Notation format. For more information, visit http://json.org/ disabled = false pulldown_type = true Sample data for 3 events ( each event starts with {"links": )  FYI: there is another 4th string  {"links" which is extra value which I will remove using regex. basically consider data only in [ ]. all other will be removed using regex as its unnecessary. {"data": [{"links": {"self": {"href": "/admin/jobs/81913"}, "file-lists": {"href": "https://test"}, "try-logs": {"href": "https://test"}}, "type": "job", "id": "81913", "attributes": {"jobId": 81913, "parentJobId": 0, "activeProcessId": 19776, "startTime": "2021-10-05T08:14:29.000Z", "endTime": "2021-10-05T08:14:53.000Z", "kilobytesDataTransferred": 0}}, {"links": {"self": {"href": "/admin/jobs/81912"}, "file-lists": {"href": "https://test"}, "try-logs": {"href": "https://test"}}, "type": "job", "id": "81912", "attributes": {"jobId": 81912, "parentJobId": 0,"startTime": "2021-10-05T08:14:04.000Z", "endTime": "2021-10-05T08:14:29.000Z", "jobQueueResource": "", "kilobytesDataTransferred": 0}}, {"links": {"self": {"href": "/admin/jobs/81911"}, "file-lists": {"href": "https://test"}, "try-logs": {"href": "https://test"}}, "type": "job", "id": "81911", "attributes": {"jobId": 81911, "parentJobId": 0, "startTime": "2021-10-05T05:44:01.000Z", "endTime": "2021-10-05T05:44:51.000Z", "kilobytesDataTransferred": 0}}], "meta": {"pagination": {"next": 10, "pages": 42, "last": 410, "offset": 0, "limit": 10, "count": 415, "page": 0, "first": 0}}, "links": {"next": {"href": "https://test"}, "self": {"href": "https://test"}, "last": {"href": "https://test"}, "first": {"href": "https://test"}}} Thanks
Hi there, we are trying to configure MS Graph API for Office 365 to process emails from mailboxes. Created an Azure Enterprise Application and gave required api access to the application. Administrat... See more...
Hi there, we are trying to configure MS Graph API for Office 365 to process emails from mailboxes. Created an Azure Enterprise Application and gave required api access to the application. Administrator has done the consent in the Azure portal. However when we try to connect to the app, it’s still asking to do the ‘test connection’ and asked admin consent. Is this a bug? And is there a way to use the phantom app without this consent being done via app (instead to be done in Azure portal)? thanks 
Hi, Im setting up an alert for data flow the alert build is when the application is not running it will send us an alert and i use trigger condition in the alert.  here is the search query  | eval ... See more...
Hi, Im setting up an alert for data flow the alert build is when the application is not running it will send us an alert and i use trigger condition in the alert.  here is the search query  | eval value1=if(like(sample, "value1"), 1,0), value2=if(like(sample, "value2"), 1,0), value3=if(like(sample, "value3"), 1,0) | stats sum(value1) as VALUE1, sum(value2) as VALUE2, sum(value3) as VALUE3 | table VALUE1, VALUE2, VALUE3   and for the alert condition i use this command  search VALUE1 = 0  "0" because in the sum it indicates that the 0 means data is not flowing in splunk meaning the application is down  Thanks in advance
hi I need to do a count on the field "titi" which exist in 2 different sourcetype following 2 conditions : the field "cit" is related to the sourcetype "citrix" and the field "domain" is related to... See more...
hi I need to do a count on the field "titi" which exist in 2 different sourcetype following 2 conditions : the field "cit" is related to the sourcetype "citrix" and the field "domain" is related to the sourcetype "web" And "host" exist in both sourcetype so I am doing something like this but i have no results index=tutu sourcetype=citrix OR sourcetype=web | search (cit<="3") AND domain=west | stats dc(titi) by host Is it enough to add a "by host" clause for matching the events or do I have to use a join command? thanks  
Hi, Has anyone worked with control-m logs in splunk. I want to understand what are the important attributes  we need to consider for building dashboard for control-M logs. Thanks      
I have a small all-in-one testing instance of Splunk Enterprise 8.1.3 (noone bothered to update for now ;-)) I wanted to do some testing on the question I posted yesterday - about permissions for da... See more...
I have a small all-in-one testing instance of Splunk Enterprise 8.1.3 (noone bothered to update for now ;-)) I wanted to do some testing on the question I posted yesterday - about permissions for datamodels and so on. Anyway, I created two indexes - dm_test1 and dm_test2. I created two users - test1 and test2. Created a separate role for each user. Each role has only one capability - search. And only one allowed index - dm_test1 for test1 role and dm_test2 for test2 role. There are no inherited capabilities, because I don't inherit from any other roles and the test1 and test2 are the only roles assigned to test1 and test2 users. So in theory, user test1 should only be able to do searches against test1 index and test2 - against test2. But it doesn't work. Both users can do searches from any index I have. Even from _internal ones. How to debug it? Again - there is no inheritance (at least no explicit one as far as I know about) These are the roles: As you can see - native capabilities - 1 each, no inherited capabilities. The "view indexes" checker shows only one native index per role and no inherited indexes. The users have only one role each So what's going on???
I want to predict Stock price and use Date as Fields to use for predicting, I am getting bad model. Is it I need to change the date to numeric? If so, how do i do it?
I read How do I use a small font size in questions and answers? and Allow the <small> tag, but I don't want to write <sup> </sup> for each paragraph. For example, I want everything below the line to... See more...
I read How do I use a small font size in questions and answers? and Allow the <small> tag, but I don't want to write <sup> </sup> for each paragraph. For example, I want everything below the line to be in a small font, but it fails. First, A fails to be quoted in yellow. Second, neither A nor B are smaller. and also I see a lot of the designing font style on the Fonts Monster website Now I want to use this designing font style in this tag
How to split a text like ""Liam John" "Grayson Roy" "Olivia"" into names filed using eval and text. names: Liam John Grayson Roy Olivia
I have some data from logs in Splunk where I need to determine what other requests were running concurrently at the time of any single event. Using the following query, I was able to have it return ... See more...
I have some data from logs in Splunk where I need to determine what other requests were running concurrently at the time of any single event. Using the following query, I was able to have it return a column for the number of requests that ran at the same time within my start time and duration. index="sfdc" source="sfdc_event_log://EventLog_SFDC_Production_eventlog_hourly" EVENT_TYPE IN (API, RestAPI) RUN_TIME>20000 | eval endTime=_time | eval permitTimeInSecs=(RUN_TIME-20000)/1000 | eval permitAcquiredTime=endTime-permitTimeInSecs | eval dbTotalTime=DB_TOTAL_TIME/1000000 | concurrency start=permitAcquiredTime duration=permitTimeInSecs | table _time API_TYPE EVENT_TYPE ENTITY_NAME apimethod concurrency permitAcquiredTime permitTimeInSecs RUN_TIME CPU_TIME dbtotalTime REQUEST_ID USER_ID | fieldformat dbTotalTime=round(dbTotalTime,0) | rename permitAcquiredTime as "Start Time", permitTimeInSecs as "Concurrency Duration", concurrency as "Concurrent Running Events", API_TYPE as "API Type", EVENT_TYPE as "Event Type", ENTITY_NAME as "Entity Name", apimethod as "API Method", RUN_TIME as "Run Time", CPU_TIME as "CPU Time", dbtotalTime as "DB Total Time", REQUEST_ID as "Request ID", USER_ID as "User ID" | sort "Concurrent Running Events" desc     I am now trying to investigate a single event in these results. For example, the top event says that at the time it ran, there were 108 concurrent requests running in the 20 second window of time. How can I identify those 108 events using this data? I imagine it would be querying the events that had a specific time frame range, but I am not sure if I need to check something like _time + - 10 seconds to see what was running within the 20 second window? I just need to understand the data behind this 108 events a little more for this top example. My end goal here is to be able to add a drill-down to the dashboard so that when I click on the 108, I can see those events that were running concurrently.
Hi  Can any one help me finding out Iphone device information from user agent  Example - iphone 12 , SE , 6 etc  User agent - Mozilla/5.0%20(iPhone;%20CPU%20iPhone%20OS%2014_7%20like%20Mac%20OS%20... See more...
Hi  Can any one help me finding out Iphone device information from user agent  Example - iphone 12 , SE , 6 etc  User agent - Mozilla/5.0%20(iPhone;%20CPU%20iPhone%20OS%2014_7%20like%20Mac%20OS%20X)%20AppleWebKit/605.1.15%20(KHTML,%20like%20Gecko)%20GSA/179.0.398606875%20Mobile/15E148%20Safari/604.1
Hi Guys,      I have a splunk command which returns a filename as the output. But i found that there is an extra space before and after the filename. query i am running here is, filetest | rex "se... See more...
Hi Guys,      I have a splunk command which returns a filename as the output. But i found that there is an extra space before and after the filename. query i am running here is, filetest | rex "sent to:  https://someurl/(?<file>.*)" | table file   Output displayed as: ............................... .............................. ............................. .............................. file.txt ............................. ........................... ............................   how can i get rid of the extra space before and after the file name?   Thanks.  
Hi,    I have two different queries running on same dashboard but a different panel.  Below is the query one which results the "reqid" as the output (ex:123456) Query 1: sourcetype=test*-cloudwatch... See more...
Hi,    I have two different queries running on same dashboard but a different panel.  Below is the query one which results the "reqid" as the output (ex:123456) Query 1: sourcetype=test*-cloudwatch-logs file.txt | rex "RequestId: (?&lt;reqid&gt;[\S+]*)\s" | table reqid | dedup reqid Output return as ex : 123456   Then, i would like feed the output of query1 as input of query2.   Query 2:  $query1_output$ | rex "uploaded to: s3://sample-us-east-1-s3/transmit-os/(?&lt;filename&gt;.*)" | table filename     Can someone suggest me is this right way of passing? How can i update the source code (XML) for the changes?   Thanks.
Hi There,   Log event:   [ 2021-02-04 23:14:28.925 SingleApp log:158] 200 GET /apache/proxy/user/1123123/qsdddqwedqewdqwdqwd/api (@1.1.1.1.1) 0.98ms k8s_label = app=jupyterhub k8s_label = compon... See more...
Hi There,   Log event:   [ 2021-02-04 23:14:28.925 SingleApp log:158] 200 GET /apache/proxy/user/1123123/qsdddqwedqewdqwdqwd/api (@1.1.1.1.1) 0.98ms k8s_label = app=jupyterhub k8s_label = component=server k8s_label = entity_type=jupyter-lab k8s_label = heritage=jupyterhub k8s_label = identity_name=namespace k8s_label = notebook_id=1123324234 k8s_label = notebook_name=learningpython.iypnb k8s_label = project_id=123 k8s_label = sdr.appname=notebooks k8s_label = user=mattparker k8s_label = user_id=1123234 k8s_label = zone=us-east-2b   I am trying to achieve the k8s_labels in table format in splunk but however, when i us the below query i am getting an error.  any help would be appreciated   Query: index=<index_name> sourcetype=<sourcetype_Name> 1123123 | table k8s_label="user_id=1123234",k8s_label="mattparker"   Error: Error in 'table' command: Invalid argument: 'k8s_label=user_id=1123234'   Thank You  
Hello, I am trying to build a new app using the Add-On Builder and am having problems getting OpenSSL functioning with my input. Has anyone successfully accomplished this and has the commands they c... See more...
Hello, I am trying to build a new app using the Add-On Builder and am having problems getting OpenSSL functioning with my input. Has anyone successfully accomplished this and has the commands they can share? This is the command I am using to import the module to "myapp/bin/lib" /Applications/Splunk/bin/splunk cmd pip3 install pyopenssl --target /Applications/Splunk/etc/apps/myapp/bin/lib --trusted-host pypi.org --trusted-host files.pythonhosted.org  I then use this in myapp's code: from lib.OpenSSL import SSL  This is the error I get when testing: File "/Applications/Splunk/etc/apps/myapp/bin/lib/OpenSSL/__init__.py", line 8, in <module> from OpenSSL import crypto, SSL ModuleNotFoundError: No module named 'OpenSSL'
Hello, Is there a way to create a health rule and/or policy to alert when an Agent Diagnostics Event occurs on the database agent?  We've had a few occasions where a database being monitored ran in... See more...
Hello, Is there a way to create a health rule and/or policy to alert when an Agent Diagnostics Event occurs on the database agent?  We've had a few occasions where a database being monitored ran into issues, most recently the database ran out of space for writing an audit trail, resulting in the agent returning the following error under event type "Agent Diagnostic Event": ORA-02002: error while writing to audit trail ORA-09925: Unable to create audit trail file Linux-x86_64 Error: 28: No space left on device Additional information: 9925 We do not have the application itself being monitored due to it's java version not supporting tls 1.2, just the oracle database is monitored. But if we can alert on this error it still helps to see issues with the database. Image attached of the event we are seeing under the Events section of the Database tab. Appreciate any advice or guidance on this. Thanks, Chad
I updated to 8.2.2.1 and suddenly all of our unit test output is polluted with hundreds of Authorization Failed messages,  each coming from various calls to splunk.rest.simpleRequest. The Authorizat... See more...
I updated to 8.2.2.1 and suddenly all of our unit test output is polluted with hundreds of Authorization Failed messages,  each coming from various calls to splunk.rest.simpleRequest. The Authorization failures themselves are perfectly normal - many of our tests actually assert that ownership and permissions are set the right way, and testing that involves trying to do things with the wrong user and asserting that the thing fails.   What's problematic is how formerly nice clean unit test output to the console or to stdout is now polluted with all this stuff about these normal failures. for example,  picture dozens or hundreds of these: Authorization Failed: b'{"success":false,"messages":[{"text":"It looks like your Splunk Enterprise\\nuser account does not have the correct capabilities to be able to post licenses.\\nReach out to your local Splunk admin(s) for help, and/or contact Sideview support\\nfor more detail."}]} Curious if anyone has run into this or knows where the messages might be coming from.