On the bug fix for this issue, Splunk Support have come back with the following ... Observation & Findings: Thanks for flagging this issue with us and we taken this to the development team. We informed you that our development team is having high level discussions on the xpath command whether to deprecate it or enhance it. Once the xpath enhancement or deprecation is done, it will be updated in the official documentation. As this task will undergo through some pre-checks, post-checks and some approvals which might take some time. So workarounds are the only option, for now. Here's a more generic regex to extract different sorts of XML declarations (note, removes CDATA entries too) | ... ``` example: https://regex101.com/r/BqHeX4/3 ``` | eval xml=replace(_raw, "(?s)(\<[\?\!]([^\\>]+\>).+?)*(?=\<[^(?=\/)])(?=[a-zA-Z])*", "") | rex mode=sed field=_raw "s/(?s)(\<[\?\!]([^\\>]+\>).+?)*(?=\<[^(?=\/)])(?=[a-zA-Z])*//g" ``` sed example for a props.conf SEDCMD to remove XML declarations before indexing ``` | xpath ...   Finally, there is another bug (Splunk said they are aware) with the xpath command when it is used more than once.  Any existing multi-value fields become non multi-value fields (like a nomv command has been applied) so any mv manipulations should be done before subsequent xpath commands.  ... View more

It's the calling shell that does the file expansion first so disabling globbing inside the function (which runs in a subshell) will not work.   Here's an example that hopefully demonstrates this more clearly ... $ mkdir empty $ mv test.func empty/.test.func $ cd empty $ ls # no files $ ls -a # globbing ignores hidden files . .. .test.func $ . .test.func $ test * 2 3 # no files so no globbling and * works opt=x**x#x file=* stansa=2 search=3 $ touch newfile $ ls newfile $ test * 2 3 opt=x**x#x file=newfile stansa=2 search=3 $ test \* 2 3 opt=x**x#x file=* stansa=2 search=3 $ set -f $ test * 2 3 opt=x**x#x file=* stansa=2 search=3 $  Agree, that using -a switch may be a cleaner way to represent all files though.  ... View more

Hi @jotne  As you may be aware, the * wildcard breaks due to filename expansion (or globbing) from the calling shell, i.e. what is passed to the btools function is a list of filenames in the directory where the function is called, not the * wildcard. This can be turned off in the shell with the set -f call (set +f to re-enable), or the more useful convention is to escape the wildcard with a backslash or wrap it in single quotes.  Standard *nix commands that use the * wildcard on the command line (e.g. find) use this convention so I think this is a more conventional *nix method than using a ¤. My US keyboard does not provide easy access to this character.  [splunk ~]$ touch test_dummy [splunk ~]$ btools indexes test* coldpath.maxDataSizeMB # shell expands to test_dummy so does not work unless the * is escaped [splunk ~]$ btools indexes test\* coldpath.maxDataSizeMB [test_cust] coldPath.maxDataSizeMB = 0 /opt/splunk/etc/system/default/indexes.conf [splunk ~]$ [splunk@lrlskt02 ~]$ btools indexes * coldpath.maxDataSizeMB [splunk@lrlskt02 ~]$ [splunk@lrlskt02 ~]$ btools indexes '*' coldpath.maxDataSizeMB [_audit] coldPath.maxDataSizeMB = 0 /opt/splunk/etc/system/default/indexes.conf [_internal] coldPath.maxDataSizeMB = 0 /opt/splunk/etc/system/default/indexes.conf [_introspection] coldPath.maxDataSizeMB = 0 /opt/splunk/etc/system/default/indexes.conf [_metrics] coldPath.maxDataSizeMB = 0 /opt/splunk/etc/system/default/indexes.conf [_metrics_rollup] coldPath.maxDataSizeMB = 0 /opt/splunk/etc/system/default/indexes.conf [_telemetry] coldPath.maxDataSizeMB = 0 /opt/splunk/etc/system/default/indexes.conf [_thefishbucket] coldPath.maxDataSizeMB = 0 /opt/splunk/etc/system/default/indexes.conf [default] coldPath.maxDataSizeMB = 0 /opt/splunk/etc/system/default/indexes.conf [history] coldPath.maxDataSizeMB = 0 /opt/splunk/etc/system/default/indexes.conf [main] coldPath.maxDataSizeMB = 0 /opt/splunk/etc/system/default/indexes.conf [summary] coldPath.maxDataSizeMB = 0 /opt/splunk/etc/system/default/indexes.conf [test_cust] coldPath.maxDataSizeMB = 0 /opt/splunk/etc/system/default/indexes.conf [splunk@lrlskt02 ~]$ ... View more

Hi @nithys  Something like this should work ...   index=dummy | append [ | makeresults count=22 | eval json=split("{\"name\":\"\",\"hostname\":\"1\",\"pid\":8,\"level\":\"\",\"claims\":{\"ver\":1,\"jti\":\"h7\",\"iss\":\"https\",\"aud\":\"https://p\",\"iat\":1,\"exp\":17,\"cid\":\"name.id\",\"uid\":\"00\",\"scp\":[\"update:\",\"offline_access\",\"read:\",\"readall:\",\"create:\",\"openid\",\"delete:\",\"execute:\",\"read:\"],\"auth_time\":17,\"sub\":\"[email protected]\",\"groups\":[\"App.PreProd.name\"]},\"msg\":\" JWT Claims -API\",\"time\":\"2025\",\"v\":0} | {\"name\":\"\",\"hostname\":\"1\",\"pid\":8,\"level\":\"\",\"claims\":{\"ver\":1,\"jti\":\"h7\",\"iss\":\"https\",\"aud\":\"https://p\",\"iat\":1,\"exp\":17,\"cid\":\"address.id\",\"uid\":\"00\",\"scp\":[\"update:\",\"offline_access\",\"read:\",\"readall:\",\"create:\",\"openid\",\"delete:\",\"execute:\",\"read:\"],\"auth_time\":17,\"sub\":\"[email protected]\",\"groups\":[\"App.PreProd.address,app.preprod.zipcode\"]},\"msg\":\" JWT Claims -API\",\"time\":\"2025\",\"v\":0}", " | ") ] | mvexpand json | eval _raw=json | spath | streamstats count | eval "claims.sub"=if(count%2=0, count."_".'claims.sub', 'claims.sub') ``` ^^^ create dummy events ^^^ ``` | stats dc(claims.sub) as "Unique Users" dc(claims.cid) as "Unique Clients" BY claims.cid claims.groups{} | rename claims.cid AS app claims.groups{} AS groups | table app "Unique Users" "Unique Clients" groups   Hope that helps  ... View more

Created an answer with workaround for the xpath and prolog header line issue here:  https://community.splunk.com/t5/Splunk-Search/The-xpath-command-does-not-work-with-XML-prolog-header-lines-e-g/td-p/711425 ... View more

Just wanted to put add a xpath command solution that also works, simply as a future reference for users that can go with the spath command solution. | makeresults | eval _raw="<?xml version=\"1.0\" encoding=\"utf-8\"?> <soapenv:Envelope xmlns:soapenv=\"http://schemas.xmlsoap.org/soap/envelope/\"> <soapenv:Body> <ns3:LogResponse xmlns:ns2=\"http://randomurl.com/sample1\" xmlns:ns3=\"http://randomurl.com/sample2\"> <LogResponse > <ResponseCode>OK</ResponseCode> <State>Simple</State> <Transactions> <TransactionName>CHANGED</TransactionName> </Transactions> <Transactions> <TransactionData>CHANGE_SIMPLE</TransactionData> </Transactions> <ServerTime>1649691711637</ServerTime> <SimpleResponseCode>OK</SimpleResponseCode> <nResponseCode> <nResponseCode>OK</nResponseCode> </nResponseCode> <USELESS>VALUES</USELESS> <MORE_USELESS>false</MORE_USELESS> </LogResponse> </ns3:LogResponse> </soapenv:Body> </soapenv:Envelope>" | eval xml=replace(_raw, "^<\?xml.+\?>[\r\n]*", "") ``` xpath does not like ?xml encoding version and text declaration, so remove``` | xpath field=xml outfield=ResponseCode "//*[local-name()='ResponseCode']" ``` use *[local-name()='<value>' to ignore namespace declarations, i.e. xmlns='smomething' ] ``` | xpath field=xml outfield=SimpleResponseCode "//*[local-name()='SimpleResponseCode']" | xpath field=xml outfield=nResponseCode "//*[local-name()='nResponseCode']/nResponseCode"   ... View more

Thought I'd add to this post, in regards to using a curl command to push a lookup file to a Splunk instance, as other Splunk users may find it useful.  It's not a replacement for @mthcht excellent python scripts but it is often easy to use curl commands when testing and validating things.   Here's a worked example that creates a simple lookup file (tested against Cloud stack and lookup editor v4.0.4) ...   curl -sk --request POST https://localhost:8089/services/data/lookup_edit/lookup_contents \ -H "Authorization: Splunk $MYTOKEN" \ -H "Content-Type: application/x-www-form-urlencoded" \ -d timeout=10 \ -d namespace=search \ -d lookup_file=lookupfilename.csv \ -d contents=[[\"field1\",\"field2\"],[\"value1\",\"value2\"]] \ -d owner=nobody # n.b. owner is only needed when creating new lookups - a 'user' name creates the new lookup file with private permissions, whereas 'nobody' results in it being shared globally   Note, the 'contents' format must be a 2D JSON array. To make this easier, 'contents' can also be added via a file, like this ...   $ cat <<EOF > myLocalLookup.json contents=[["field1","field2"],["value1","value2"]] EOF $ curl -sk --request POST https://localhost:8089/services/data/lookup_edit/lookup_contents \ -H "Authorization: Splunk $MYTOKEN" \ -H "Content-Type: application/x-www-form-urlencoded" \ -d timeout=10 \ -d namespace=search \ -d lookup_file=lookupfilename.csv \ -d @myLocalLookup.json \ -d owner=nobody   Now, to really make this useful, existing CSV file's need to be formatted as JSON. There are multiple ways this could be done, but here is a simple python oneliner (*nix tested) that reads in a CSV file on stdin and outputs it as JSON.   (python -c $'import sys;import csv;import json;\nwith sys.stdin as f: csv_array=list(csv.reader(f)); print("contents="+json.dumps(csv_array))' > mylocalLookup.json) < myLocalLookup.csv   Hopefully. others may find this useful too. ... View more

Hi @johnansett  We had something similar after moving our on-prem Splunk to Splunk Cloud.  Our on prem indexed data needed to be searchable as it aged out. As a Splunk customer you should be able to request a zero-ingest license from Splunk Support. This is better than a free or trail license as it gives all the benefit of a standard license (except no data ingestion) and does not need renewing every 6 weeks like a trail license does.  If you have a Splunk account manager then this is something you could discuss with them.  Hope that helps.  ... View more

Anyone who comes across this issue please up vote the following idea for a configuration option to disable INDEXED_EXTRACTIONS via an app's local props.conf. https://ideas.splunk.com/ideas/EID-I-2400 ... View more

Anyone who comes across this issue please upvote the following idea for a configuration option to disable INDEXED_EXTRACTIONS via an app's local props.conf.   https://ideas.splunk.com/ideas/EID-I-2400 ... View more

Hi @Bisho-Fouad  Here's an example search to solve your question...   host=<your host> ``` and whatever else you need to filter your data ```` | eval bytes = length(_raw) ``` generally 1 character = 1 byte ``` | stats sum(bytes) AS bytes BY source ``` this gives the size of each log, assuming the source is the name of the log file ``` | eval kilobytes = bytes/1024) | evenstats sum(kilobytes) AS total_kb   Hope that helps   ... View more

Hi @mik3y  Thanks for the update and the workaround solution. In the end we moved away from this solution anyway as the Salesforce streaming API did not provide ability to track the events that had already been ingested, potentially resulting in missed data during Splunk maintenance.  ... View more

No worries @vikas1  Just some admin, it is good practise to mark a correct answer as the solution as other users may find it useful when searching Splunk answers for the same issue.   Karma points are also greatly appreciated. ... View more

Hi @abi2023  This is a good example of using the chart command.  Here's a run anywhere example (based of dummy events derived from your example SPL) | makeresults | eval _raw="state, risk_rating open, Critical open, Moderate open, Severe open, Sustainable close, Critical close, Moderate close, Sustainable" | multikv forceheader=1 | table state risk_rating ``` ^^^ above is just creating dummy example events ^^^ -> the SPL below creates the table output ``` | chart count OVER state BY risk_rating | sort - state | addtotals | addcoltotals labelfield=state label="Total"  Hope this helps ... View more

Hi @atebysandwich  I'm not 100% sure I understand what you are trying to do but does this run anywhere example help... | makeresults | eval _raw="DNS,Identified_Host host1.domain.com,host1.domain.com host1-admin.domain.com,host1.domain.com host1-mgt.domain.com,host1.domain.com host2.domain.com,host2.domain.com host2-admin.com,host2.domain.com host2-mgt.admin.com,host2.domain.com host3.domain.com,host3.domain.com host3-admin.com,host3.domain.com" | multikv forceheader=1 | table DNS Identified_Host ```^^^ dummy events ^^^``` | where DNS!=Identified_Host | stats values(DNS) BY Identified_Host   ... View more

Hi @azulueta  Try the following query index=tenable sourcetype=tenable:io:assets | stats count values(hostnames) BY agent_uuid | where count > 1 Hope that helps ... View more

Likely message is not an extracted field then.  Try this to extract from the _raw event | rex field=_raw "transactionTraceIdentification-(?<transactionTraceIdentification>[^\"]+)   ... View more

Hi @Roy_9  Assuming the lookup file is called test.csv, does this command work? | inputlookup test.csv If so, it would indicate a problem with the lookup definition.  Maybe try deleting and recreating it. Hope that helps ... View more

Hi @Sekhar  try this.. | rex field=message "transactionTraceIdentification-(?<transactionTraceIdentification>[^\"]+)"   hope that helps ... View more