All Posts

Find Answers
Ask questions. Get answers. Find technical product solutions from passionate members of the Splunk community.

All Posts

1. This is a very old thread. Starting a new one would give you more visibility. 2. Well, not every type of input supports this parameter so I'm not sure if specifying it here is syntactically corre... See more...
1. This is a very old thread. Starting a new one would give you more visibility. 2. Well, not every type of input supports this parameter so I'm not sure if specifying it here is syntactically correct. Try and see (with btool check)
Adding to that answer - your search term if you just search for "1.2.3.4" might not encompass a whole major-breaker-delimited search term but be somewhere in the middle  of a "word" delimited by mino... See more...
Adding to that answer - your search term if you just search for "1.2.3.4" might not encompass a whole major-breaker-delimited search term but be somewhere in the middle  of a "word" delimited by minor breakers - like "version.1.2.3.4". So Splunk searches for 1, 2, 3 and 4 separately and checks if the events matching all of those partial terms match the literal search term. If you explicitly tell it to find TERM(1.2.3.4), it will find only those events for which the term 1.2.3.4.
For Splunk to Splunk connectivity Splunk uses aptly named s2s (splunk-to-splunk) protocol. It can be either used "raw" or embedded into HTTP.
Hi @payl_chdhry, Splunk uses the proprietary Splunk-To-Splunk (S2S) protocol over TCP between tcpout outputs and splunktcp inputs, optionally encapsulated in TLS. Splunk also natively supports HTTP... See more...
Hi @payl_chdhry, Splunk uses the proprietary Splunk-To-Splunk (S2S) protocol over TCP between tcpout outputs and splunktcp inputs, optionally encapsulated in TLS. Splunk also natively supports HTTP output and input via HTTP Event Collector, syslog output, raw TCP output,   and raw TCP and UDP input.
In some of my earlier posts, if you see them, I incorrectly stated unarchive_cmd would gives us checkpoint tracking etc. It does not. When the file is modified, it is re-read from the beginning. This... See more...
In some of my earlier posts, if you see them, I incorrectly stated unarchive_cmd would gives us checkpoint tracking etc. It does not. When the file is modified, it is re-read from the beginning. This is still useful for dropping complete files into a monitored directory. Also: Don't forget to either keep your SEDCMD setting intact or add code to the Python script to mask the password. We'll assume all of this is being done for research and offline analysis.
My first instinct is to use the mvmap eval function to iterate over the alert_data.csv.content array and join the results into a new JSON array: | makeresults | eval _raw="{\"alert_data\": {\"domain... See more...
My first instinct is to use the mvmap eval function to iterate over the alert_data.csv.content array and join the results into a new JSON array: | makeresults | eval _raw="{\"alert_data\": {\"domain\": \"abc.com\", \"csv\": {\"id\": 12345, \"name\": \"credentials.csv\", \"mimetype\": \"text/csv\", \"is_safe\": true, \"content\": [{\"username\": \"test1@abc.com\", \"password\":\"1qaz@WSX#EDC\"}, {\"username\": \"test2@abc.com\", \"password\":\"NotComplex\"}]}}}" | eval _raw=json_set(json(_raw), "alert_data.csv.content", json("[".mvjoin(mvmap(json_array_to_mv(json_extract(json(_raw), "alert_data.csv.content{}")), json_set(_raw, "is_password_meet_complexity", if(len(json_extract(_raw, "password")) >= 8 AND (if(match(json_extract(_raw, "password"), "[[:digit:]]"), 1, 0) + if(match(json_extract(_raw, "password"), "[[:upper:]]"), 1, 0) + if(match(json_extract(_raw, "password"), "[[:lower:]]"), 1, 0) + if(match(json_extract(_raw, "password"), "[[:punct:]]"), 1, 0)) >= 3, "Yes", "No"))), ",")."]")) However, mvmap is not supported by INGEST_EVAL: "The following search-time eval functions are not currently supported at index-time with INGEST_EVAL: mvfilter, mvmap, searchmatch, now, and commands." See https://docs.splunk.com/Documentation/Splunk/latest/Data/IngestEval. To work around the missing functionality, we must analyze the input stream using an external process. We have several options available, but my preference lately for file (monitor) inputs is the props.conf unarchive_cmd setting. unarchive_cmd streams data to an external command over stdin and sends the command's stdout stream to the Splunk ingest pipeline. If we assume your file input is newline delimited JSON, unarchive_cmd allows us to read each object from stdin, process each content array item individually, and write the resulting object to stdout. Given alert_data.ndjson: {"alert_data": {"domain": "abc.com", "csv": {"id": 12345, "name": "credentials1.csv", "mimetype": "text/csv", "is_safe": true, "content": [{"username": "test1@abc.com", "password":"1qaz@WSX#EDC"}, {"username": "test2@abc.com", "password":"NotComplex"}]}}} {"alert_data": {"domain": "abc.com", "csv": {"id": 67890, "name": "credentials2.csv", "mimetype": "text/csv", "is_safe": true, "content": [{"username": "test3@abc.com", "password":"passw0rd"}, {"username": "test4@abc.com", "password":"j#4kS.0e"}]}}} let's introduce an alert_data source type and construct inputs.conf and props.conf: # inputs.conf [monitor:///tmp/alert_data.ndjson] sourcetype = alert_data # props.conf [source::...alert_data.ndjson] unarchive_cmd = python $SPLUNK_HOME/bin/scripts/preprocess_alert_data.py unarchive_cmd_start_mode = direct sourcetype = preprocess_alert_data NO_BINARY_CHECK = true [preprocess_alert_data] invalid_cause = archive is_valid = False LEARN_MODEL = false [alert_data] DATETIME_CONFIG = CURRENT SHOULD_LINEMERGE = false LINE_BREAKER = ([\r\n]+) EVENT_BREAKER_ENABLE = true EVENT_BREAKER = ([\r\n]+) Now let's write $SPLUNK_HOME/bin/scripts/preprocess_alert_data.py to read, process, and write JSON objects: import json import re import sys for line in sys.stdin: line = line.strip() if not line: continue else: try: json_object = json.loads(line.rstrip()) for item in json_object["alert_data"]["csv"]["content"]: meets_length_requirement = len(item["password"]) >= 8 digit_score = 1 if re.search(r"\d", item["password"]) else 0 upper_score = 1 if re.search(r"[A-Z]", item["password"]) else 0 lower_score = 1 if re.search(r"[a-z]", item["password"]) else 0 punct_score = 1 if re.search(r"[^a-zA-Z0-9\s]", item["password"]) else 0 meets_complexity_requirement = True if (digit_score + upper_score + lower_score + punct_score) >= 3 else False if meets_length_requirement and meets_complexity_requirement: item["is_password_meet_complexity"] = "Yes" else: item["is_password_meet_complexity"] = "No" print(json.dumps(json_object)) except Exception as err: print(err, file=sys.stderr) print(line) On a full instance of Splunk Enterprise, i.e. a heavy forwarder, Splunk will use its local copy of Python. On a universal forwarder, we'll need to install Python 3.x and make sure the executable is in the path. At scale, this solution is better implemented as a modular input, but that's a separate topic for a larger discussion.
What transfer protocol splunk uses like FTP, sFTP.. That is what transfer method is used to transfer data via TCP when sending data from UF to Splunk core cluster (enterprise)
Hi @munang, Depending on your segmentation configuration, 127.0.0.1 will be indexed as: 0 1 127 127.0.0.1 or 0 1 127 127.0 127.0.0 127.0.0.1 You can verify this (relatively) easily with ... See more...
Hi @munang, Depending on your segmentation configuration, 127.0.0.1 will be indexed as: 0 1 127 127.0.0.1 or 0 1 127 127.0 127.0.0 127.0.0.1 You can verify this (relatively) easily with with an empty index and the walklex command. See https://docs.splunk.com/Documentation/Splunk/latest/SearchReference/Walklex. With segmenters.conf INTERMEDIATE_MAJORS = false (the default): | makeresults | eval _raw="127.0.0.1" | collect index=walklex_test $ /opt/splunk/bin/splunk _internal call /data/indexes/walklex_test/roll-hot-buckets | walklex type=term index=walklex_test | table term term 0 1 127 127.0.0.1 Using a source type and segmentation with INTERMEDIATE_MAJORS = true: term 0 1 127 127.0 127.0.0 127.0.0.1 Both 127.0.0.1 and "127.0.0.1" will use the following base lispy at the indexing tier: [ AND 0 1 127 ] You can judge the efficiency of your search using the method you observed in your second question. In the search job inspector, you'll see: "This search has completed and has returned X results by scanning Y events in Z seconds." If X != Y, your search is scanning more events than needed, and introducing TERM() or otherwise modifying your search may improve efficiency. TERM(127.0.0.1)  will use the following base lispy at the indexing tier, also as you observed: [ AND 127.0.0.1 ] With INTERMEDIATE_MAJORS = false, TERM(127.0) will return no results. With INTERMEDIATE_MAJORS = true, TERM(127.0) will return events with 127.0.0.1, 127.0.a$b. 127.0-foo, etc. If searches with and without TERM() return X results by scanning Y events and X == Y, then the same number of events contain the segmented terms as contain the complete term. In this case, there is no direct efficiency to be gained by using TERM(). If your observations contradict this, i.e. you have events that match 127.1.1.24 but do not match TERM(127.1.1.24), then the answer may have something to do inconsistent time ranges across searches, misconfigured search peers, or misconfigured indexer clustering.
What have you tried so far?
up !!
I need to create below dashboard .This will be the main dashboard and from here I can navigate to any of the other mentioned dashboards. AAA, BBB, CCC are the separate dashboards and all these should... See more...
I need to create below dashboard .This will be the main dashboard and from here I can navigate to any of the other mentioned dashboards. AAA, BBB, CCC are the separate dashboards and all these should be accessed form this main dashboard.       
Thanks!!  That do work for the first object! But I do have multiple objects in the array and the number of objects is not fixed, how can I refactor the solution to accommodate them?  Sorry I have do... See more...
Thanks!!  That do work for the first object! But I do have multiple objects in the array and the number of objects is not fixed, how can I refactor the solution to accommodate them?  Sorry I have done the research but totally have no idea how to do that.
Hello, I'm Splunk Newbie. This is a post that I found while looking for improvement of Splunk's search performance, but I'm asking you a question because it's a little confusing.   I referred to t... See more...
Hello, I'm Splunk Newbie. This is a post that I found while looking for improvement of Splunk's search performance, but I'm asking you a question because it's a little confusing.   I referred to the two posts below. https://splunk.illinois.edu/splunk-at-illinois/using-splunk/searching-splunk/how-to-optimize-your-searches/ https://idelta.co.uk/3-easy-ways-to-speed-up-your-splunk-searches-and-why-they-help/     Question 1) - index=firewall_data 127.0.0.1 Or - index=firewall_data "127.0.0.1" If I search that, because of the internal segmentation process 127 127 1 127 0 1 Is it right to search by dividing it into three approach? Because of this, If I use index=firewall_data TERM (127.1.1.24), is it correct that the breaker is not used and it shows better performance? Question 2) index=firewall_data "127.0.0.1" has more resources if the assumptions in question 1 are correct The index= firewall_data TERM (127.1.1.24) should perform better, but when tested, it actually did the same. It says that the data I searched for and the resource (time) are all the same, why?  
<form version="1.1" theme="light"> <label>Multi-select filtered</label> <fieldset submitButton="false"> <input type="multiselect" token="alloptions" searchWhenChanged="true"> <label>Sel... See more...
<form version="1.1" theme="light"> <label>Multi-select filtered</label> <fieldset submitButton="false"> <input type="multiselect" token="alloptions" searchWhenChanged="true"> <label>Select site</label> <choice value="All">All</choice> <search> <query> | makeresults format=csv data="Country USA Romania Turkey" | table Country </query> </search> <fieldForLabel>Country</fieldForLabel> <fieldForValue>Country</fieldForValue> <valuePrefix>"</valuePrefix> <valueSuffix>"</valueSuffix> <delimiter>,</delimiter> <change> <eval token="form.alloptions">case(mvcount('form.alloptions')=0,"All",mvcount('form.alloptions')&gt;1 AND mvfind('form.alloptions',"All")&gt;0,"All",mvcount('form.alloptions')&gt;1 AND mvfind('form.alloptions',"All")=0,mvfilter('form.alloptions'!="All"),1==1,'form.alloptions')</eval> <eval token="countrychoice">if($form.alloptions$=="All","","| where Country IN (".$alloptions$.")")</eval> </change> </input> <input type="multiselect" token="campus" searchWhenChanged="true"> <label>Select Campus</label> <search> <query> | makeresults format=csv data="Country,Campus USA,USA1 USA,USA2 Romania,Romania1 Romania,Romania2 Romania,Romania3 Turkey,Turkey1 Turkey,Turkey2 Turkey,Turkey3 Turkey,Turkey4" $countrychoice$ | table Campus </query> </search> <fieldForLabel>Campus</fieldForLabel> <fieldForValue>Campus</fieldForValue> <valuePrefix>"</valuePrefix> <valueSuffix>"</valueSuffix> <delimiter>,</delimiter> </input> </fieldset> </form>
I thought about that but didn't succeed to edit the dynamic options for the Campus value. I tried  | search $site.token$=$campus.token$* When $site.token$ is for Site value and $campus.token$* ... See more...
I thought about that but didn't succeed to edit the dynamic options for the Campus value. I tried  | search $site.token$=$campus.token$* When $site.token$ is for Site value and $campus.token$* is for Campus value.
For the campus dropdown, use a search which filters the campuses based on the token value from the countries dropdown
Hey all, I'm building new dashboard that contains 2 multiselect values: Site: USA, Romania, Turkey.... (only countries) Campus: USA1,USA2,Romania1,Romania2.... (contains the country's name and num... See more...
Hey all, I'm building new dashboard that contains 2 multiselect values: Site: USA, Romania, Turkey.... (only countries) Campus: USA1,USA2,Romania1,Romania2.... (contains the country's name and number). I want that when I select country/countires in Site multiselect value I will see only options to select the relevant campuses in Campus multiselect value. How can I create inherited rule that the Campus will inherit from Site value? Thanks.
anyone?
Hi Bhavya, What add-ons did you need on Splunk enterprise to receive logs from rsyslog client? Was rsyslog on an external system?  Thanks. Joanna.
Can I configure something like this?  [default] persistentQueueSize=100MB   so that it will applied for all the inputs?