All Posts

Find Answers
Ask questions. Get answers. Find technical product solutions from passionate members of the Splunk community.

All Posts

Hi @Mrig342, you have to run a simplesearch like the following: ... | rex field=_raw "(?ms)]\|(?P<host>\w+\-\w+)\|" | rex field=_raw "(?ms)]\|(?P<host>\w+)\|" | rex field=_raw "\]\,(?P<host>[^\,]+)... See more...
Hi @Mrig342, you have to run a simplesearch like the following: ... | rex field=_raw "(?ms)]\|(?P<host>\w+\-\w+)\|" | rex field=_raw "(?ms)]\|(?P<host>\w+)\|" | rex field=_raw "\]\,(?P<host>[^\,]+)\," | rex field=_raw "\]\|(?P<host>[^\|]+)\|" | regex _raw!="^\d+(\.\d+){0,2}\w" | regex _raw!="/apps/tibco/datastore" | lookup Master_List.csv "host" | search "Tech Stack"="Apple" | search Region="Asia" | eval host=lower(host) | stats count BY host | append [ | inputlookup Master_List.csv WHERE Region="Asia" AND "Tech Stack"="Apple" | eval eval host=lower(host), count=0 | fields host ] | stats sum(count) AS total BY host | where total=0 | rename host as "Not_Reporting_Servers" | table "Not_Reporting_Servers" Ciao. Giuseppe
Hi All, I have got two queries to populate the host, region, tech stack & environment details. One query is a lookup table that has the list of total number of host.    | inputlookup Master_List.c... See more...
Hi All, I have got two queries to populate the host, region, tech stack & environment details. One query is a lookup table that has the list of total number of host.    | inputlookup Master_List.csv | search Region="Asia" | search "Tech Stack"="Apple" | rename host as Total_Servers | table Total_Servers   which gives below table: Total_Servers Apple1 Apple2 Apple3 Apple4 Apple5 Apple6 The second query gives us the list of hosts that are currently populating in splunk.   ... | rex field=_raw "(?ms)]\|(?P<host>\w+\-\w+)\|" | rex field=_raw "(?ms)]\|(?P<host>\w+)\|" | rex field=_raw "\]\,(?P<host>[^\,]+)\," | rex field=_raw "\]\|(?P<host>[^\|]+)\|" | regex _raw!="^\d+(\.\d+){0,2}\w" | regex _raw!="/apps/tibco/datastore" | lookup Master_List.csv "host" | search "Tech Stack"="Apple" | search Region="Asia" | rename host as "Reporting_Servers" | table "Reporting_Servers"   which gives below table: Reporting_Servers Apple1 Apple4 Apple5 Now I want to create a query to compare these two table and populate the servers that are missing from the total servers. So that output of the above tables after comparison should be like below: Non_Reporting_Servers Apple2 Apple3 Apple6   Please help me to create a query to achieve the expected output table. Your kind inputs are highly appreciated.   Thank You..!!!
1. It's not really a Splunk question - it's about your Python scripting. 2. What do you mean by "the output is not sequential so I'm not able to ingest"? You have timestamps on the events so Splunk ... See more...
1. It's not really a Splunk question - it's about your Python scripting. 2. What do you mean by "the output is not sequential so I'm not able to ingest"? You have timestamps on the events so Splunk should be able to line up the events properly. 3. You can't beat maths. You do the checks sequentially, have some percentage of "dead" hosts and have relatively long default TCP SYN timeout - it's bound to take some time. You could try adjusting your timeouts to make it fail faster.
Any way you do it, it _will_ be inefficient (that's the "beauty" of matching ipv6 addresses). In this case it probably would be best to use additional "external" mechanics if possible - maybe try to ... See more...
Any way you do it, it _will_ be inefficient (that's the "beauty" of matching ipv6 addresses). In this case it probably would be best to use additional "external" mechanics if possible - maybe try to expand the addresses on ingest to index the full form and have it easier matchable on search later. Or at least add an indexed field with a flag to easily identify the fields having ipv6 field version.
E_NOT_ENOUGH_INFO We don't know your setup. We don't know how you're ingesting data, we don't know where you have your SEDCMD defined. We don't know what your effective config is. Did you verify yo... See more...
E_NOT_ENOUGH_INFO We don't know your setup. We don't know how you're ingesting data, we don't know where you have your SEDCMD defined. We don't know what your effective config is. Did you verify your sed syntax? Did you do the btool? Did you put the props.conf in the proper place?
To all the python masters out there :: Python Execution Time Optimization using multi-threading. I have a python script which takes a list of 1000 IP from a file and does port monitoring 3389 and 22... See more...
To all the python masters out there :: Python Execution Time Optimization using multi-threading. I have a python script which takes a list of 1000 IP from a file and does port monitoring 3389 and 22 respectively using the os module of python . It is taking as of now 40 minutes to run. The requirement is to run the same scripted input within 10 minutes.   I have tried multi threading but the output is not sequential so I am not able to ingest...
You may be able to use a TRANSFORM or INGEST_EVAL to edit the indexed field as well, although INDEXED_EXTRACTIONS from a forwarder would need to be routed back through pasrsingQueue. Since SEDCMD is ... See more...
You may be able to use a TRANSFORM or INGEST_EVAL to edit the indexed field as well, although INDEXED_EXTRACTIONS from a forwarder would need to be routed back through pasrsingQueue. Since SEDCMD is already working, the re-route probably isn't necessary. Note the use of := to edit the existing field, if present: [remove-foo] INGEST_EVAL = foo:=null() # or [mask-and-replace-foo] INGEST_EVAL = foo:=""
Hi @jslamcle, If the input includes indexed extractions, either in a monitor stanza or in a modular input script, then SEDCMD wouldn't mask the extracted field values; it would only mask the _raw va... See more...
Hi @jslamcle, If the input includes indexed extractions, either in a monitor stanza or in a modular input script, then SEDCMD wouldn't mask the extracted field values; it would only mask the _raw value. E.g., given /tmp/foo.json: {"foo":"bar"} {"foo":"baz"} and: # inputs.conf [monitor:///tmp/foo.json] sourcetype = foo_json INDEXED_EXTRACTIONS = json # props.conf [foo_json] SEDCMD-foo = s/"foo":"[^"]+"/"foo":""/ _raw will be indexed as: {"foo":""} {"foo":""} but the events will have indexed values of foo=bar and foo=baz, respectively:
Hi @Taruchit, You can group counts by week of year, which works well for line and column charts: | timechart span=1d count | eval week_of_year=strftime(_time, "%V") | timechart span=1d count as c... See more...
Hi @Taruchit, You can group counts by week of year, which works well for line and column charts: | timechart span=1d count | eval week_of_year=strftime(_time, "%V") | timechart span=1d count as count by week_of_year and then use trellis to split the charts by week_of_year: However, scatter charts want integral x and y-values. You can use strftime again to convert _time values into reasonable integer values, e.g. the day of the week: | timechart span=1d count | eval week_of_year=strftime(_time, "%V") | eval day_of_week=strftime(_time, "%w") | xyseries day_of_week week_of_year count and then use trellis to split the scatter chart by week_of_year: If you prefer, you can use some other split-by value: | timechart span=1d count | eval day_of_week=strftime(_time, "%w") | eval split_by="Week of ".strftime(_time-(86400*day_of_week), "%d-%b") | xyseries day_of_week split_by count The markers can be made slightly more visually appealing using a Simple XML dashboard and the charting.chart.markerSize option: <option name="charting.chart.markerSize">1</option>
Splunk (and most data query languages) treat columns as solemn.  But for display purposes, you can fool the system by converting columns to rows and take out those you don't want.  Of course we are t... See more...
Splunk (and most data query languages) treat columns as solemn.  But for display purposes, you can fool the system by converting columns to rows and take out those you don't want.  Of course we are talking about transpose.   | transpose 0 | search "row 1" > 0 | transpose 0 header_field=column | fields - column   To demonstrate, run this search   index=_internal sourcetype!=splunkd_ui_access json OR python OR foobar | stats count(eval(searchmatch("json"))) as json count(eval(searchmatch("python"))) as python count(eval(searchmatch("foobar"))) as foobar ``` data emulation above ```   It gives 0 for foobar.   json python foobar 405 1135 0 But this search   index=_internal sourcetype!=splunkd_ui_access earliest=-5h json OR python OR foobar | stats count(eval(searchmatch("json"))) as json count(eval(searchmatch("python"))) as python count(eval(searchmatch("foobar"))) as foobar ``` data simulation above ``` | transpose 0 | search "row 1" > 0 | transpose 0 header_field=column | fields - column   eliminates foobar from table json python 442 1232 (The numbers changed because this is a live splunkd.)
Ah, the same situation as you expressed in Re: How to perform lookup on inconsistent IPv6 for....  The solution is also the same: Use host CIDR expressions instead of host IP address in search.  This... See more...
Ah, the same situation as you expressed in Re: How to perform lookup on inconsistent IPv6 for....  The solution is also the same: Use host CIDR expressions instead of host IP address in search.  This time, it is right in search command, no lookup required. (Absolutely no regex.  Always suppress your urge to manipulate structured data as string.)  See CIDR matching. So, instead of Instead of Use If select IPv4 dropdown box > select 1.1.1.1 ip_token="1.1.1.1" If select IPv4 dropdown box > select 1.1.1.1 ip_token="1.1.1.1/32" if select IPv6 dropdown box > select  2001:db8:3333:4444:5555:6666::2101 ip_token="2001:db8:3333:4444:5555:6666::2101"  if select IPv6 dropdown box > select  2001:db8:3333:4444:5555:6666::2101 ip_token="2001:db8:3333:4444:5555:6666::2101/128" You will be using the same efficient search for both no matter whether the address representation is compressed or not. Let me guess your next question (because I did answered your follow-up IPv6 questions:-): the tokens are populated by a search, so you need to know which host bitmask to apply to which value.  Well, that answer was a hack on ipmask function: https://community.splunk.com/t5/Splunk-Search/How-to-perform-lookup-on-inconsistent-IPv6-format-in-CSV-file/m-p/657104/highlight/true#M226967
As @VatsalJagani said, you need to talk to someone who understands IPv4 address space to help you formulating what you call "IP address ranges" into mathematical formula(e) that a computer can easily... See more...
As @VatsalJagani said, you need to talk to someone who understands IPv4 address space to help you formulating what you call "IP address ranges" into mathematical formula(e) that a computer can easily understand.  Maybe a network engineer in your organization. (You don't want to give your internal IP ranges to a public forum like this.)  These days, CIDR is the most common way of expressing IPv4 address ranges, and Splunk supports it right in the search command.  See CIDR matching.  Once you know your CIDR, you can do something like   index = <source> ip != <CIDR1> ip != <CIDR2>, ... | stats count by ip   To give you an example using the same data as given in the document, you want to exclude 10.10.10.1 through 10.10.10.255, and your data contains these _time ip 2023-09-23 16:08:12 10.10.10.12 2023-09-23 16:08:12 50.10.10.17 2023-09-23 16:08:12 10.10.10.23 You run   index = <source> ip != 10.10.10.0/24 | stats count by ip   The result is ip count 50.10.10.17 1 You can try this with the following emulation   | makeresults | eval ip = split("10.10.10.12 50.10.10.17 10.10.10.23", " ") | mvexpand ip ``` the above emulates raw data ``` | search ip!=10.10.10.0/24 | stats count by ip   If you can't find suitable people to ask for help, there are also CIDR calculators online.  Just be careful with your proprietary data.
Hello, I am using a regex because the ipv6 on the index is not in compressed format. The search with regex is slower than regular search, that is the reason why I want to bypass the regex for ipv4. ... See more...
Hello, I am using a regex because the ipv6 on the index is not in compressed format. The search with regex is slower than regular search, that is the reason why I want to bypass the regex for ipv4. Please suggest. Thanks
Also remember that by filtering early you make search more efficient.
Also remember that if you do manual extraction with the rex command and only then search on its results it will be much much slower than by simply searching the index because instead of finding the v... See more...
Also remember that if you do manual extraction with the rex command and only then search on its results it will be much much slower than by simply searching the index because instead of finding the value in the index splunk has to pass every event through the regex extraction and only then find matching events.
Your question is a bit vague so I'm not sure what you want so please be a little more descriptive. But from what you wrote I assume that you do some comditional aggregation and want to "go back" to r... See more...
Your question is a bit vague so I'm not sure what you want so please be a little more descriptive. But from what you wrote I assume that you do some comditional aggregation and want to "go back" to raw events fulfilling your conditions. You can't do that this way. Splunk "loses" all information not being explicitly passed from the command. So when you're doing the stats command only results of the stats command are available for further processing - the original events are no longer known in your pipeline. So you have to approach it differently. Probably adding some artificial "classifier" field or two but can't really say without knowing what exactly you want to achieve.
@akshada_s - If you are trying to run Splunk search from outside the script then jobs endpoint is usually the answer. Find more info here - https://docs.splunk.com/Documentation/Splunk/9.1.1/RESTTUT... See more...
@akshada_s - If you are trying to run Splunk search from outside the script then jobs endpoint is usually the answer. Find more info here - https://docs.splunk.com/Documentation/Splunk/9.1.1/RESTTUT/RESTsearches   I hope this helps!!!
i have a query where i am looking for multiple values with OR and then counting the occurrence with the stats the query is something like this  index=**** ("value1") OR ("Value3") OR ... | stats ... See more...
i have a query where i am looking for multiple values with OR and then counting the occurrence with the stats the query is something like this  index=**** ("value1") OR ("Value3") OR ... | stats count(eval(searchmatch("vlaue1"))) as value1, count(eval(searchmatch("vlaue2"))) as value2 now I just want to collect only those values which are found which mean there count is greater than 0. How can I achieve this where only stats of the values are displayed which are found in the events   also search values are mostly ips, URLs , domains, etc Note: I'm making this query for dashboard
@chrisyounger is the developer for the App, he should be able to help here.  
Hello, The first search does not work because ipv6 from the dropdown is in a compressed format from a different data source, while the ipv6 in the index is in not in a compressed format, so it has t... See more...
Hello, The first search does not work because ipv6 from the dropdown is in a compressed format from a different data source, while the ipv6 in the index is in not in a compressed format, so it has to go through a regex or function to convert it to a compressed format in the second search. Thank you for your help