I like this option, as it allows calculation of the effective registered domain using a wildcard lookup based on the PublicSuffixList. | makeresults format=json data="[{\"id\": 1, \"domain\": [\"test.com\",\"sample.com\",\"example.co.uk\", \"nxdomain\"]}, {\"id\": 2, \"domain\": [\"www.example.com\"]}]" | spath path="domain{}" output=domain | eval domain=mvappend(domain, "") ``` append empty string to force to mv field ``` | foreach mode=multivalue domain [ eval domain_segment_len=mvappend( domain_segment_len, json_extract( lookup( "wildcard_public_suffix_list", json_object("tld_wildcard", '<<ITEM>>'), json_array("length") ), "length" ) ), domain_split = mvappend(domain_split, split('<<ITEM>>', ".")) ``` Use mvcount, mvappend, case, and mvjoin to create the effective TLD ``` ] Although not a single command, using a macro like this with builtin lookups is more scalable for large inputs than using an external lookup that generates the effective TLD or registered domain. ... View more

for anyone that would like to see this work better, please consider voting for my idea here to support long query urls: https://ideas.splunk.com/ideas/EID-I-2569 to me, this is not uncommon at all.  it's a daily problem that I have to work around.   (I'm aware of the current solutions and already use them.) ... View more

Finding this much later and love it. I made some slight edits to include calculated fields (the mvfilter NOT match is for the sub-model names that start with capital letters and the is_/is_not_ stuff for each sub-model): | datamodel | rex field=_raw "\"modelName\"\s*\:\s*\"(?<modelName>[^\"]+)\"" | spath output=fieldList objects{}.calculations{}.outputFields{}.displayName | spath output=fieldList2 objects{}.fields{}.displayName | eval fieldList = mvappend(fieldList,fieldList2) | where modelName!="Splunk_CIM_Validation" | table modelName fieldList | eval fieldList = mvdedup(mvfilter(NOT match(fieldList,"is_.*|^[A-Z]")))  The check index/sourcetype is a handy addition. I also highly recommend Outpost's Data Model Mechanic for troubleshooting DMs. ... View more

Hi there, Kara here, Splunk Community Manager. Thanks for your reply, but if you have an additional question, I recommend posting a new question for more visibility.  Cheers! ... View more

I know I'm late to the game here but here is another option for determining the difference in time between two events. {base search} | streamstats window=2 min(_time) as prevTime | eval diffTime = _time-prevTime | {the rest of your search here} ... View more

Yes, it appears that most (if not all) non-ASCII character are major breakers. The lispy I see for a simple search for  тестирование is: [ AND index::main а в е и н о р с т ] This is a bigger issue if the data is ingested in ASCII JSON format. {"data": "\u0442\u0435\u0441\u0442\u0438\u0440\u043e\u0432\u0430\u043d\u0438\u0435"}  If the data above is ingested, "data="тестирование"  or "тестирование" will not find the data.  An initial search like "u04*" must be included.  A similar issue occurs when the raw JSON includes a newline, as a string like {"data": "line_one\nline_two"} cannot be found with a search for "line_two". ... View more

Hey, I was actually able to fix it by adding the following lines to the `Splunk_TA_my_addon/README/input.conf.spec`. Then I stopped getting that warning from slim. ``` [my_input://<name>] # ... my other values start_by_shell = <bool> python.version = <string> interval = <integer> ```   ... View more

Just wanted to post a quick thanks for these macros. They came in handy to replace a custom command |distance that was included with the Okta app which did not pass the Cloud app vetting process. Thanks for posting! ... View more

The sed command is probably too aggressive, as it removes escaped backslashes. The regular expression to extract the message didn't include all the characters. By using an alternative of matching every escaped quote or other characters, it will keep any single JSON string.  REGEX = \"message\":\s*\"((?:\\\"|[^\"]*)\" SEDCMD-unescapetab = s/\\t/    /g SEDCMD-unescapenewline = s/\\n/\n/g SEDCMD-removeescape = s/\\(.)/\1/g   ... View more

This isn't an issue that is limited to liberal use of the wildcard.  I don't believe it is intended behavior. When a search is made in Smart or Verbose mode, the base search returns schema for events that are not included in the result. The number of fields shown to the user in the "All Fields" can be a small subset of the fields exposed in the table command or a data export.  In the search example that @rbechtold provided, the "alert_name" placeholder may have matched 144 of ~2000 events returned from the indexers.  The schema from those other ~2000 events is not shown in the UI.  Because of this, I would consider inclusion of null fields for events that are displayed to be unexpected.   index _raw source Event 1 _internal "alert name", k1=val1 scheduler.log Event 2 _internal "name alert", k1=val1, k2=val2, k3=val3 scheduler.log   If I searched that data with ` index=_internal "alert name" source=schedular.log `,  k2 and k3 would not be shown in the web result GUI.  Including them in the table command or a data export is a surprise, especially if Event 2 included hundreds of fields.  I guess something like this could work, and would also work in Fast mode.    index=_internal "alert_name" source="/opt/splunk/var/log/splunk/scheduler.log" | fields _time, _raw | extract     ... View more

I understand the frustration, but the issue may be partially due to the email provider blocking the messages. I also hope that Splunk will help identify the root cause and help fix this issue. Have you tried resetting the password for your instance and using the username above? ... View more

  I agree that mode function is most likely what @HeinzWaescher  was looking for.  Because it is a stats function, you can also use it with eventstats to keep the events.  | eventstats mode(country_code) by user_id For the example initially provided, the top command does not appear any better than the stats mode() command provided by @p3hndrx. But if you the top N values (N>=2), there is no function like values() that returns them sorted by frequency of occurrence.  ... View more

Hi.. not sure if this is the right post to ask this question.. but, after search, I see this is the one closer..  soo, I had around 5000 karma points on answers.splunk.com but after conversion to Community, my karma points came to 270 something. I assumed that there are some conversions from answers to community, but i missed that something wrong with the conversion.  may i know if someone else also felt that... can someone please check my old answers karma points and new community karma points please, thanks.  ... View more

@kiran331 Did you find the correct regex to blacklist Account name ending with $ at index time ... View more

I would recommend using an eventstats command to exclude  index="idx" earliest=-360s latest=-0s source="server.log" "<Request" | xmlkv | eventstats earliest_time(clientId) as earliest_clientId by clientId | where relative_time(now(), "-60s") > earliest_clientId |`enter code here`   ... View more

I just posted an "idea" for the return command to be able to do what you would like it to do. https://ideas.splunk.com/ideas/EID-I-532  The following is your command, rewritten with a rex command that should create a simple boolean expression from one subsearch.   (index=winsec sourcetype="WinSecurityEvent") OR index=DNS [ search sourcetype="dns" "specific urls" | dedup src | return 100 src_ip=src OR query | eval search = if(len(search)<3, "NOTHING FOUND") | rex mode=sed field=search "s/ (OR|AND|NOT)=\"\" / \1 /g" ] ) | dedup accountname | stats values(accountname) AS accounts | table query, src_ip, accounts     The string in the eval command should be changed to something that will never be found.  Depending on the use, perhaps "index!=*" might be more efficient.  Otherwise, if nothing is found then the search will return an empty string that matches all events. This may be desired, in which case the eval statement should be removed.   ... View more

Check my add-on up the thread. I can push the code to some public git so that more people can work on it and make it better. https://splunkbase.splunk.com/app/5116/ ... View more

That is there, under the "DESCRIPTION FORMATTING" section. The earlier "FORMATTING" section states to use \" to represent a quote. This is not directly aligned with the Description Formatting and does not work in a description. The "\" is also shown as "\\", which is not documented. Based on the descriptions shown above that were written by Splunk, it appears that this behavior was not understood by Splunk authors, or the parsing behavior changed and the descriptions were not updated. The only concern I have with @lstewart_splunk's accurate answer, is that the current behavior breaks the description syntax for stock Splunk commands, as shown above for regex.  The Splunk authors for the regex syntax apparently expected that (\")?string(\")? would be shown as (")?string(")?, but instead, it is strangely highlighted. ... View more

To modify @martin_mueller's answer to find where the underscores ("_") are, the "rex" command option, "offset_field", will gather the locations of your match. The "offset_field" option has been available since at least Splunk 6.3.0, but I can't go back farther in the documentation to check when it was introduced. If you only want the first match index, or a limited number of indexed locations, the "max_match" parameters can be changed. ... | rex field=message offset_field=message_offsets max_match=0 "(?P<char_match>_)" | rex field=message_offsets max_match=0 "char_match=(?P<offset_range>[0-9]+-[0-9]+)" | chart values(offset_range) by message  The first "rex" command creates a field named "message_offsets" will contain data like the results of these eval statements, if the character(s) are found. | eval example_message_offsets = "char_match=7" | eval example_message_offsets = "char_match=12-12&char_match=19-19" The second "rex" extracts the index from those values into "offset_range". For one character, the values are the same and separated with a "-".  If there is no match, both the char_match and the message_offsets fields are null(), so the chart will not include field values that don't contain the string of interest. Unless fillnull, coalesce, or another eval statement is used to populate null values. ... View more

I don't know which version it was introduced in, but mvrange and mvzip can be used. ... | eval keys = split(rawKeys, ",") | eval index_and_key = mvzip(mvrange(0, mvcount(keys)), keys, ",")   ... View more

csv.QUOTE_NONNUMERIC also works, with potentially less overhead that csv.QUOTE_ALL ... View more

I don't know where the crossover for performance is, and I expect it changes depending on the deployment design. But staying in the tens is definitely safe.  Although "filter as soon as possible" is the general recommendation, the search inspector and introspection can help you choose the best command (inputlookup, lookup) for your data. I believe that the server sends back a response that includes the entire expanded search string, which includes expanded inputlookup subsearches. In one case where a table was significantly increased in size, a search with a few results generated over 75MB of traffic from the server. ... View more

EVAL-_time = if(match(_raw,"SB"), _time+86400, _time) Fixing the formatting issues caused by the transition to Khoros, copied from the @tiagofbmm post ... View more

For me, the only backslash sequences that worked for sed replacement were the newline (\n) and back references (\1, \2, etc.). I think I'm going to have to put the replacement into an external lookup/command to include the ability to use other characters, like \r, \x0D, \t, or \x09. ... View more