About yuanliu

yuanliu

My problem is I cannot get them tabled together in the same results view. If I try and add P1Sender to This statement confirms @PickleRick and @ITWhisperer 's diagnosis: that field P1Sender only exists in one group of events, let's call them "event type 1", whereas fields Item.RecipientsCount, Item.Recipients{}.Name, Item.Subject are only present in a totally different group of events, let's call them "event type 2". (Most likely they are from two different sourcetypes in index office365.) The two groups have no overlap. The implied message they are trying to tell you when they ask you what your data look like is this: Unless you can demonstrate that the two groups have something in common, the result you desire, namely "quick view into emails a user has recently received," is an impossibility. To make this obvious, let me give you two groups of mock events P1Sender joe@example1.com jab@example2.com joey@example3.com and Item.RecipientsCount Item.Recipients{}.Name Item.Subject 2 bob@myco.com bobby@myco.com You owe me $1,000,000. Where is my money? 1 rob@myco.com I paid $1,000,000. Where is my shipment? Can you tell me who receive E-mail from whom? Volunteers in this forum all understand privacy concerns. We ask what your data look like because we want to help you identify the link between these two groups. You have two paths forward: Find out which datum/data are in common among the two, and craft a logic to link the two to form a "quick view into emails a user has recently received," or Share format of raw events from the two groups which you believe can help forming a "quick view into emails a user has recently received." You can use mock events but the format must be the same as your raw events and any redaction must retain enough distinct features to establish possible logical connections between two groups. As you have not realized that the two groups of events have no overlap, let me help you with the second path. Run these two searches separately. index=office365 P1Sender=* ``` event type 1 ``` and index=office365 Item.Recipients{}.Name=*, Item.Subject=* ``` event type 2 ``` Inspect events from each type, find events that can form a "quick view into emails a user has recently received." Then, anonymize those events to share here. This is some homework you should have done before posting question into a data analytics forum. These are four golden rules; nay, call them the four commandments: Illustrate data input (in raw text, anonymize as needed), whether they are raw events or output from a search (SPL that volunteers here do not have to look at). Illustrate the desired output from illustrated data. Explain the logic between illustrated data and desired output without SPL. If you also illustrate attempted SPL, illustrate actual output and compare with desired output, explain why they look different to you if that is not painfully obvious.

yuanliu

@PickleRick nailed the core problem in your case: unclear business logic. In addition to impossibly large timestamp uncertainty (> 30 second?), "the order of my events can be unpredictable" and "doesnt matter which one comes first, starting or ending" contradict the basic premise of a transaction. In other words, before you can find a SPL solution, much additional work is needed, e.g., Get timestamp among different data sources to be accurate within 1ms. This is not a very strict requirement today. Manually review events of interest ("to FAIL", "from FAIL", something of interest in between), and ask tech/business owners how they decide which "pairs" should be flagged. Arrive at a rule/set of rules that they (owners) can manually perform without exception. After this, you can see if their rules fall within definition of transaction. IF there indeed is a transaction, you can start with PickleRick's suggestion to test transaction command, then see if some more efficient commands can apply the same rules as @livehybrid suggested. If not, you will need to workshop with owners to come up with an algorithm before working on SPL.

yuanliu · ‎04-17-2026

Like @livehybrid says, there are different ways to work around such timeouts. One suggestion about using summary index followed by saved searches: Run the collect before hand as one saved search, not as part of the pipe to run multiple regional searches. Your regional searches should run against this summary index that you named initial-results. Whether this solves your timeout (or resource) limits will depend on what is hitting such limits in your initial setup. In your statement, you used "scheduled reports" as plural. But I cannot relate your mock code to multiple reports. Could you explain that initial setup, why do you think that the use of dbxlookup is contributing to the problem, and what are the symptoms observed in the initial setup? What is the expected outcome, whether it is one report broken down by region, or multiple reports (plural), one for each region? Your mock code also references a lookup named host-regions.csv. Does this relate to that dbxlookup in your first statement?

yuanliu · ‎04-06-2026

Thank you for providing the context and desired results. Just note that ContactId, DNIS, and ANI, should not require rex in a "normal" Splunk instance. I am not certain about ApiLogs; however, I cannot get your rex to work with the mock data you provided. So, the following assume that all the rex work as you desire. Now, you have this array of JSON objects that have the same keys. You want each pair of keys to have a different name; but the length of the array is indeterminate. Here is one of several possibilities provided your instance is 8.1 or above where JSON functions are available. index=XXXX source=XXXX sourcetype=XXXX | rex field=_raw "ContactId=\"(?<ContactId>[^\"]+)\"" | rex field=_raw "DNIS=\"(?<DNIS>[^\"]+)\"" | rex field=_raw "ANI=\"(?<ANI>[^\"]+)\"" | rex field=_raw "ApiLogs=\"(?<ApiLogs>\[.*?\])\"" | eval ApiLogs=replace(ApiLogs, "\\\"", "\"") | spath input=ApiLogs path={} output=api_array | eval api_logs_kv = json_object() | eval idx = mvrange(0, mvcount(api_array)) | foreach idx mode=multivalue [eval element = mvindex(api_array, <<ITEM>>), api_logs_kv = json_set(api_logs_kv, "request_" . '<<ITEM>>', spath(element, "request"), "statusCode_" . '<<ITEM>>', spath(element, "statusCode"))] | fields ContactId ApiLogs api_logs_kv | spath input=api_logs_kv | fields - _* api_logs_kv | table ContactId ApiLogs request_* statusCode_* Your mock data should give ContactId ApiLogs request_0 request_1 request_3 statusCode0 statusCode_1 statusCode_2 <CONTACT_ID> [ {"request":"APINAME1","statusCode":"200"}, {"request":"APINAME2","statusCode":"200"}, {"request":"APINAME3","statusCode":"200"} ] APINAME1 APINAME2 APINAME3 200 200 200 Here, I name the fields with 0-base. You can always make it 1-base. Hope this helps.

yuanliu · ‎04-06-2026

Can you rephrase your question by following these four golden rules, nay, four commandments of asking an answerable question? Illustrate data input (in raw text, anonymize as needed), whether they are raw events or output from a search (SPL that volunteers here do not have to look at). Illustrate the desired output from illustrated data. Explain the logic between illustrated data and desired output without SPL. If you also illustrate attempted SPL, illustrate actual output and compare with desired output, explain why they look different to you if that is not painfully obvious.

yuanliu · ‎04-06-2026

Good thing you attempt to illustrate data. But as several have already pointed out, you need to be precise when discussing data analytics - qualitatively precise. Not only did you use underscore in one place and dashes in another place as @bowesmana notices, but in this illustration, you give prefix cs in most fields, but sc in the last field. Is that supposed to be cs-status as opposed to sc-status? As you change field names, you are asking volunteers here to read you mind about what each field name means in your original post. I think @richgalloway already gives you the solution if you just apply it with the correct field names. If I have to guess, cs-uri-stem is what you call cs_uri_stem in OP, and cs-status is what you call status in OP. Hence, <your query for relevant events> | stats count as requests, sum(eval('cs-status'=200)) as success by cs-uri-stem | eval pct=success*100/requests | table cs-uri-stem requests pct

yuanliu · ‎04-06-2026

Could you explain why a value stored in a bit counter would be a sore subject? I do not know how many bits are in userAccountControl, but if you can tell which bit represents use of smartcard-based auth, you can always calculate whether that bit is modified. For example, | eval smartCardBit = tonumber(userAccountControl, 16) % pow(2, smartCardBitPos - 1) Here, smartCardBitPos is 1-base from LSB. If your instance is 9.2 or above, there are also a set of Bitwise functions.

yuanliu · ‎03-18-2026

I'm not extracting, just a standard search in spl, the full TEXT field has already been extracted as I said. TEXT=" +MYMSGIDI: An FFDC Incident has been created: \"java.io.IOException: EDC5133I No space left on device. 397\" at ffdc_26.03.15_04.30.06.0.log" This contradicts with your observations that | search TEXT=*MYMSG* works fine but: | search TEXT=*EDC5133I* doesn't work. As PickeRick indicates, Splunk cannot automatically extract the full field when escaped quotation marks is in between even though the full field is in fact inside two unescaped quotation marks. You can test this condition by the following without additional manipulation: | table TEXT You will see "+MYMSGIDI: An FFDC Incident has been created: \" (no quotation mark) which does NOT include the string EDC5133I that you are looking for, but does include MYMSG. Next, try this | rename TEXT as TEXT_init | rex mode=sed "s/\\\\\"/'/g" | fields - TEXT _kv | extract | table TEXT* You'll see TEXT TEXT_init +MYMSGIDI: An FFDC Incident has been created: 'java.io.IOException: EDC5133I No space left on device. 397' at ffdc_26.03.15_04.30.06.0.log +MYMSGIDI: An FFDC Incident has been created: \ Now, | search TEXT_init=*EDC5133I* will continue to fail but | search TEXT=*EDC5133I* will succeed.

yuanliu · ‎03-17-2026

As the other commenters hinted at, you need to clarify what exactly do you mean by this I'm trying to create an alert based on a field as shown below, I want to search for the EDC5133I text. In richgalloway's reading, you simply want to pick out events with a specific string to generate that alert. In many cases, searching _raw is probably sufficient. In PickleRick's reading, you want to search for the string in this field named TEXT. Like PickleRick said, there is no generic way to do this. If you need to search in that quoted string, one possibility is to replace escaped quotation marks with a different marker. For example | rex mode=sed "s/\\\\\"/'/g" | fields - TEXT _kv | extract Here is result from emulation TEXT _raw +MYMSGIDI: An FFDC Incident has been created: 'java.io.IOException: EDC5133I No space left on device. 397' at ffdc_26.03.15_04.30.06.0.log blah ... TEXT=" +MYMSGIDI: An FFDC Incident has been created: \"java.io.IOException: EDC5133I No space left on device. 397\" at ffdc_26.03.15_04.30.06.0.log" If the single quotes bothers you, you can also replace them back. Hope this helps.

yuanliu · ‎03-14-2026

Thank you for explaining the dataset and logic. So, my speculation was not far off. The difference is that I didn't realize that names var1, var2, etc., are your choice, not field names carried in data. The solution is the same. Just use the field names in data. For the three types, you can do index=notable search_name=* [| inputlookup approved_detections.csv] | fillnull value=MISSING action file_name host signature url user | stats count by search_name action file_name host signature url user | stats count Now, there is a chance that the actual field names are very diverse and difficult to maintain in code. In that case, you may want to maintain that monthly_alert_matrics.csv instead. If that is the case, you can easily maintain the above code by using this search | inputlookup monthly_alert_metrics.csv | foreach val* [eval everyfield = mvappend(everyfield, <<FIELD>>)] | stats values(everyfield) as everyfield | eval everyfield = mvjoin(everyfield, ",") Then, copy the value of everyfield into the code.

yuanliu · ‎03-12-2026

Let me know if I get this right: the unique combination in each row of monthly_alert_metrics.csv (i.e., not just alert_name) represents an alert type; you want to know how many of such unique combinations are there in a month; however, some rows may not have all val1, val2, val3, val4 populated. Is this correct? First, if you really, really need to use dedup, know that it has an keepempty option (Optional arguments). If the above is what you want, this will do: index=notable search_name=* [| inputlookup approved_detections.csv] ``` no need for a second search command ``` | dedup keepempty=true search_name val1 val2 val3 | stats count But dedup often is slower than stats. You can simply fillnull any missing field. index=notable search_name=* [| inputlookup approved_detections.csv] | fillnull value=MISSING search_name val1 val2 val3 | stats count by search_name val1 val2 val3 | stats count If my interpretation is correct, I cannot find any use of monthly_alert_metrix.csv. Your description stats "whose search_name field is not found within the approved_detections.csv", but your code does the opposite. If you truly mean "NOT found within" (as opposed to "found within"), index=notable search_name=* NOT [| inputlookup approved_detections.csv] | fillnull value=MISSING search_name val1 val2 val3 | stats count by search_name val1 val2 val3 | stats count Bottom line: map is usually not the answer. Hope this helps.

yuanliu · ‎02-19-2026

@yuanliu Hello! because foreach is a "template" command that can handle wildcards (like foreach *), it forces the search engine to switch to Full Materialization. This is so meta😆 In my Slack thread on this subject, madscient used "'all bets are off' mode" to describe such a scenario. Meanwhile, foreach itself doesn't always degrade performance. In fact my macro contains another foreach command that does regex and some multivalue calculations. No degradation. That's why I speculate that name space manipulation may have triggered foreach to go wild in that thread. I also specifically avoided wildcard with foreach in hope that the compiler will treat the tokens as constant and set rename at compile time. Obviously that was misguided. Can you try run your search again, but put | fields a b c d (the fields you are renaming) immediately before your macro. This tells Splunk to throw away the extra data before the foreach The reason why I want to preempt potential name collision by adding such rename's is for the macro to stand alone without worrying about side effects. In my real macro, a b c d are names that wouldn't usually exist in practical dataset. (In other words, the intention is similar to declaring a set of variables as local in some other languages.) Meanwhile, I am also aware of the effect of narrowing field set - again, in relationship to name space manipulation. A while ago, I posted another Slack thread about tojson command vs json_set() call. Here was my observation: I did some tests with the same dataset comparable to that used in my original observation. Baseline | stats count tojson | tojson output_field=json field1 field2 field3 | stats count eval | json = json_object("field1", field1, "field2", field2, "field3", field3) | stats count So a | stats count command is used to terminate all three for job inspector. All three would give me event count of 102,365. Baseline runs for 0.20s, tojson runs for 38.6s, eval runs for 0.26s. ... I suspect that the number of fields carried in raw events may have some effect. So, I ran | fields field1 field2 field3 | tojson output_field=json field1 field2 field3 | stats count . In the half-million dataset, run time has reduced to 1.6s. Still meaningfully longer than eval but the slowdown is much reduced. Of course, determining which fields might be needed downstream is tedious work and things can change in the future. So, as a practical measure, I will write the long json_object where I can. (Which fields I will need in a JSON object is much easier to determine; future maintenance is more straightforward.) Knowing that number of fields has effect, I also tested Smart Mode against Fast Mode. No difference. Is tojson examining every field in a raw event even when a list is provided? (I do understand that syntax allows tojson to be used without list.) Could this be a potential bug? Basically, my observations in both foreach + rename combo and tojson suggest that whenever a command syntax potentially allows unlimited name tokens, even when actual name tokens are explicitly enumerated, the command goes into that "all bets are off" mode and starts to operate on imaginary list of tokens.

yuanliu · ‎02-18-2026

I'm really confused about performance related to use of foreach + rename. I have a macro that renames potential name collisions. My original search takes about 5 seconds on some 100K events; after applying the macro, the search runs > 50 seconds! Then, I removed foreach + rename from the macro, any commands in between add practically no additional run time, not even 1 second. To test, I set up several macros. The macro that resembles the one I have is ``` This macro does nothing except renaming a bunch of fields back and forth using foreach ``` foreach a b c d [rename <<FIELD>> as _macro_<<FIELD>>] | noop | foreach a b c d [rename _macro_<<FIELD>> as <<FIELD>>] Running this over 400K _internal events with simple stats takes >4s. Running the same stats with a macro that has no foreach + rename, or a macro with rename fully spelled out takes <0.4s. How to explain the dramatic increase of run time in my 100K events when foreach + rename is used on a handful of names? Then, why the degradation is far less in my 400K events when the equivalent amount of foreach + rename is applied? (I have also tested the effect of foreach + rename without macro. It is just as bad. Macro is just a place where foreach + rename can have more practical application.)

yuanliu · ‎02-13-2026

Meanwhile, Dashboard Studio has built-in drag-and-adjust. It seems to have some built-in constraint so free-hand adjustment is not completely free. Then, there is the options.columnformat JSON node where you can manually enter desired width for each column.

yuanliu · ‎02-12-2026

What is the harm? User's dashboards will only be visible by the user.

yuanliu · ‎02-12-2026

Based on the example given, I think the OP wants to adapt dropdowns to actual deployed application and environment combination dynamically. So, I'd take a different route. The key is the composition of index names. I think what you want is like For environment token | tstats count by index | rex field=index "[^_]+_(?<environment>.+)" | stats count by environment | fields - count for application token | tstats count by index | rex field=index "(?<application>[^_]+)_" | stats count by application | fields - count Here is a complete mockup dashboard: <form version="1.1" theme="light"> <label>multiple dependent dropdown Dashboard</label> <description>https://community.splunk.com/t5/Dashboards-Visualizations/Create-multiple-dependent-dropdown-Dashboard/m-p/758134#M59378</description> <fieldset submitButton="false"> <input type="dropdown" token="environment_tok"> <label>Environment</label> <fieldForLabel>environment</fieldForLabel> <fieldForValue>environment</fieldForValue> <search> <query>| makeresults | eval environment = split("stage,test,prod", ","), application = mvappend("s1", "s2", "s3") | mvexpand environment | mvexpand application | eval index = application . "_" . environment | stats count by index ``` the above emulates | tstats count by index ``` | rex field=index "\S+_(?<environment>.+)" | stats count by environment | fields - count</query> <earliest>-24h@h</earliest> <latest>now</latest> </search> </input> <input type="dropdown" token="application_tok"> <label>Application</label> <fieldForLabel>application</fieldForLabel> <fieldForValue>application</fieldForValue> <search> <query>| makeresults | eval environment = split("stage,test,prod", ","), application = mvappend("s1", "s2", "s3") | mvexpand environment | mvexpand application | eval index = application . "_" . environment | stats count by index ``` the above emulates | tstats count by index ``` | rex field=index "(?<application>[^_]+)_" | stats count by application | fields - count </query> <earliest>-24h@h</earliest> <latest>now</latest> </search> </input> </fieldset> <row> <panel> <title>This is what you put in your search</title> <html> <pre> index="$application_tok$_$environment_tok$" blah, blah </pre> </html> </panel> <panel> <title>This is what your search renders into</title> <html> <pre> index="$application_tok$_$environment_tok$" blah, blah </pre> </html> </panel> </row> <row> <panel> <title>Use this search for environment</title> <html> <pre> | tstats count by index | rex field=index "[^_]+_(?<environment>.+)" | stats count by environment | fields - count </pre> </html> </panel> <panel> <title>Use this search for application</title> <html> <pre> | tstats count by index | rex field=index "(?<application>[^_]+)_" | stats count by application | fields - count </pre> </html> </panel> </row> </form> This way, no matter what combination of deployment is, the dropdowns always have the correct elements.

yuanliu · ‎02-11-2026

While others have provided technical paths, the big question, in fact an input you should provide for others to answer your question, is what is your general approach to this problem? Forget Splunk. Do you have/are you trying to construct a white list? (Not to be used in WAP but for Splunk to use.) Are you trying to construct a black list? (Again, for Splunk to use.) Some other approach? If you cannot answer this question without Splunk, no amount of Splunk discussion is going to solve the problem.

yuanliu · ‎02-10-2026

As a fun exercise, yes, you can still reuse this users.csv to hunt for duplicate. Just use lookup command, not inputlookup. (Generally speaking, inputlookup in the middle is not often useful.) And directly count groups instead of trying to hunt for a different group. | inputlookup users.csv WHERE group = "GU" | dedup fname lname init | eval name = lname.", ".fname." ".init | lookup users.csv output group | eval duplicate = if(mvcount(group) > 1, "Yes", "No") | fields name, duplicate You can achieve the final desired output using this technique as well. | inputlookup users.csv WHERE group = "GU" | dedup fname lname init | eval name = lname.", ".fname." ".init | lookup users.csv output group | eval training=if(mvcount(group)>1, "GU/PU", "GU") | table lname fname init training dep phone email Performance-wise, I suspect stats is better.

yuanliu · ‎02-09-2026

This is quite fun learning. I think @richgalloway has got the most comprehensive. Maybe a little refinement of search as the interest is probably in CSV files, not KMZ or files that are not lookup files at all. index=_audit isdir=0 size path=*/lookups/*.csv action IN (update created modified add) | stats latest(eval(size/1024/1024)) as size_mb latest(_time) as _time by path Meanwhile, as OP mentioned, search interval can be a challenge if creation-update times spread very widely. I looked into the OP's method. Whereas inputlookup cannot work directly inside foreach and subsearch, it can be modified to use with map. The big problem with a search other than in _audit index, however, is that inputlookup can only run within a given app context. If the lookup returned from the rest command is not visible in this app context, inputlookup will error out. In addition, unlike _audit index, | rest/servicesNS/-/-/data/lookup-table-files cannot retrieve private lookup files. (This, of course, depends on actual use case. Maybe you don't want to count private files in.) Nevertheless, here is an example usable in search app: | rest/servicesNS/-/-/data/lookup-table-files | rename "eai:acl.app" as app, "eai:acl.owner" as owner, "eai:acl.sharing" as sharing, "eai:data" as path | table title owner sharing app | search (app = search OR sharing = global) title = *.csv | map maxsearches=100000 search="| inputlookup $title$ | foreach * [| eval b_<<FIELD>>=len('<<FIELD>>') + 1 ] | addtotals b_* fieldname=b | stats sum(b) as b | eval mb=b/1024/1024 | fields mb" | stats sum(mb)

yuanliu · ‎02-07-2026

This was already suggested above in 2018. But an obvious alternative is not: add a line | fillnull sort_field value=0 Additionally, it is better to use boolean function true() (which is equivalent to boolean constant in some other languages), instead of a boolean evaluation such as 1==1.

yuanliu · ‎02-01-2026

I first notice that Dashboard Studio's input configuration UI sometimes treats a time string as "number"; as a result, that input would not even be populated. I had to use tostring or another trick to force the UI to accept it as string. After trying for two days, I haven't been able to reproduce the "number" scenario with trivial data. With trivial data, DS's configuration UI always treats a time string as "time" (as opposed to "string") unless otherwise forced. In addition to strange effects in input selector (see example below), DS also passes token of type "time" in a specific format that may be different from the original string. This may also be undesired. Here is a sample dashboard to illustrate this. Source data: timestring _value 2026-01-23 11:44:25 2026-01-23 11:44:25 2026-01-23 11:44:25 2026-01-23 11:44:25 2026-01-27 11:44:25 2026-01-27 11:44:25 2026-01-27 11:44:25 2026-01-27 11:44:25 2026-01-27 11:44:25 2026-01-27 11:44:25 2026-01-29 11:44:25 2026-01-29 11:44:25 2026-01-30 11:44:25 2026-01-30 11:44:25 2026-01-30 11:44:25 2026-01-30 11:44:25 2026-01-31 11:44:25 2026-01-31 11:44:25 2026-01-31 11:44:25 2026-01-31 11:44:25 2026-01-31 11:44:25 2026-01-31 11:44:25 As shown, timestring is of format "%F %T", which a white space between date and time. The idea is to use the date part as label and pass the timestring as token value. For this, I tried several ways to obtain single-value timestring such as values() stats function, max() stats function, groupby timestring, and mvexpand timestring. I also tried different ways to obtain label from timestring including using rex command and using replace function. To make values very clear in each case, I use top panels to show output from respective searches for the token; these outputs include an evaluation of type of timestring and label to verify that Splunk search shows all of them as String. The bottom panels simply displays the token value as passed by Dashboard Studio. As shown, all these token values are transformed into "%FT%T", i.e., with letter "T" between date and time. { "title": "Dashboard Studio thinks time string is other than string", "description": "", "inputs": { "input_1kIxDHKk": { "context": { "formattedConfig": { "number": { "prefix": "" } }, "formattedStatics": ">statics | formatByType(formattedConfig)", "label": ">primary | seriesByName(\"label\") | renameSeries(\"label\") | formatByType(formattedConfig)", "statics": [], "value": ">primary | seriesByName(\"timestring\") | renameSeries(\"value\") | formatByType(formattedConfig)" }, "dataSources": { "primary": "ds_Uh3WYGcG" }, "options": { "items": ">frame(label, value) | prepend(formattedStatics) | objects()", "selectFirstSearchResult": true, "token": "groupby_rex_tok" }, "title": "groupby with rex label", "type": "input.dropdown" }, "input_2qavhtpJ": { "context": { "formattedConfig": { "number": { "prefix": "" } }, "formattedStatics": ">statics | formatByType(formattedConfig)", "label": ">primary | seriesByName(\"timestring\") | renameSeries(\"label\") | formatByType(formattedConfig)", "statics": [], "value": ">primary | seriesByName(\"timestring\") | renameSeries(\"value\") | formatByType(formattedConfig)" }, "dataSources": { "primary": "ds_GAZj9qSN" }, "options": { "items": ">frame(label, value) | prepend(formattedStatics) | objects()", "selectFirstSearchResult": true, "token": "max_timestring_tok" }, "title": "max timestring", "type": "input.dropdown" }, "input_R3Q1RvBK": { "context": { "formattedConfig": { "number": { "prefix": "" } }, "formattedStatics": ">statics | formatByType(formattedConfig)", "label": ">primary | seriesByName(\"label\") | renameSeries(\"label\") | formatByType(formattedConfig)", "statics": [], "value": ">primary | seriesByName(\"timestring\") | renameSeries(\"value\") | formatByType(formattedConfig)" }, "dataSources": { "primary": "ds_nZMqnAeL" }, "options": { "items": ">frame(label, value) | prepend(formattedStatics) | objects()", "selectFirstSearchResult": true, "token": "mvexpand_rex_tok" }, "title": "mvexpand with rex label", "type": "input.dropdown" }, "input_qjIV5Ix1": { "context": { "formattedConfig": { "number": { "prefix": "" } }, "formattedStatics": ">statics | formatByType(formattedConfig)", "label": ">primary | seriesByName(\"label\") | renameSeries(\"label\") | formatByType(formattedConfig)", "statics": [], "value": ">primary | seriesByName(\"timestring\") | renameSeries(\"value\") | formatByType(formattedConfig)" }, "dataSources": { "primary": "ds_beX8QYoY" }, "options": { "items": ">frame(label, value) | prepend(formattedStatics) | objects()", "selectFirstSearchResult": true, "token": "label_replace_tok" }, "title": "label by replace", "type": "input.dropdown" } }, "defaults": { "dataSources": { "ds.search": { "options": { "queryParameters": { "earliest": "$global_time.earliest$", "latest": "$global_time.latest$" } } } }, "visualizations": { "global": { "showProgressBar": true } } }, "visualizations": { "viz_93ySZ5Cc": { "dataSources": { "primary": "ds_Uh3WYGcG" }, "description": "| stats count as _count by timestring | rex field=timestring \"(?<label>\\S+)\" | sort - label | eval v_type = typeof(timestring), l_type = typeof(label) | fields label l_type timestring v_type", "options": { "columnFormat": { "l_type": { "width": 84 }, "label": { "width": 86 }, "timestring": { "width": 136 } }, "showInternalFields": false }, "title": "groupby with rex label", "type": "splunk.table" }, "viz_HaKyhXKv": { "dataSources": { "primary": "ds_sfp5IIKQ" }, "type": "splunk.table" }, "viz_KJJE0cri": { "dataSources": { "primary": "ds_GAZj9qSN" }, "description": "| stats max(timestring) as timestring by _value | eval v_type = typeof(timestring) | sort - timestring", "options": { "columnFormat": { "timestring": { "width": 127 } }, "showInternalFields": false }, "title": "max timestring", "type": "splunk.table" }, "viz_SMBNQ4XA": { "dataSources": { "primary": "ds_nZMqnAeL" }, "description": "| stats values(timestring) as timestring | mvexpand timestring | rex field=timestring \"(?<label>\\S+)\" | eval v_type = typeof(timestring), l_type = typeof(label) | sort - label | fields label l_type timestring v_type", "options": { "columnFormat": { "label": { "width": 98 }, "timestring": { "width": 158 } }, "showInternalFields": false }, "title": "mvexpand with rex label", "type": "splunk.table" }, "viz_UsOo59WW": { "dataSources": { "primary": "ds_beX8QYoY" }, "description": "| stats values(timestring) as timestring by _value | rex field=timestring \"(?<label>\\S+)\" | eval v_type = typeof(timestring), l_type = typeof(label) | sort - label | fields label l_type timestring v_type", "options": { "columnFormat": { "l_type": { "width": 93 }, "label": { "width": 108 }, "timestring": { "width": 147 } }, "showInternalFields": false }, "title": "label by replace", "type": "splunk.table" }, "viz_amq7ntem": { "containerOptions": {}, "dataSources": { "primary": "ds_lWSFXLao" }, "options": { "columnFormat": { "max_timestring_tok": { "width": 190 } } }, "showLastUpdated": false, "showProgressBar": true, "type": "splunk.table" }, "viz_bMEWY9PX": { "dataSources": { "primary": "ds_z6WkSe6U" }, "options": { "columnFormat": { "label_replace_tok": { "width": 90 } } }, "type": "splunk.table" }, "viz_dvN9WkEw": { "dataSources": { "primary": "ds_Nwgz464R" }, "options": { "showInternalFields": false }, "type": "splunk.table" } }, "dataSources": { "ds_34OFi4Mu": { "name": "multivalue timestring", "options": { "query": "| makeresults format=csv data=\"timestring\n2026-01-23 11:44:25\n2026-01-23 11:44:25\n2026-01-27 11:44:25\n2026-01-27 11:44:25\n2026-01-27 11:44:25\n2026-01-29 11:44:25\n2026-01-30 11:44:25\n2026-01-30 11:44:25\n2026-01-31 11:44:25\n2026-01-31 11:44:25\n2026-01-31 11:44:25\"\n| eval _value = timestring\n", "queryParameters": { "earliest": "-24h@h", "latest": "now" } }, "type": "ds.search" }, "ds_GAZj9qSN": { "name": "max timestring", "options": { "extend": "ds_34OFi4Mu", "query": "| stats max(timestring) as timestring by _value\n| eval v_type = typeof(timestring)\n| sort - timestring" }, "type": "ds.chain" }, "ds_Nwgz464R": { "name": "timestring as groupby passed", "options": { "query": "| makeresults format=csv data=\"groupby_rex_tok\n$groupby_rex_tok$\"\n| eval type = typeof(groupby_rex_tok)", "queryParameters": { "earliest": "-24h@h", "latest": "now" } }, "type": "ds.search" }, "ds_Uh3WYGcG": { "name": "timestring as groupby", "options": { "extend": "ds_34OFi4Mu", "query": "| stats count as _count by timestring\n| rex field=timestring \"(?<label>\\S+)\"\n| sort - label\n| eval v_type = typeof(timestring), l_type = typeof(label)\n| fields label l_type timestring v_type" }, "type": "ds.chain" }, "ds_beX8QYoY": { "name": "label from replace", "options": { "extend": "ds_34OFi4Mu", "query": "| stats values(timestring) as timestring by _value\n| eval label=replace(timestring, \"(\\S+).+\", \"\\1\")\n| eval v_type = typeof(timestring), l_type = typeof(label)\n| sort - label\n| fields label l_type timestring v_type" }, "type": "ds.chain" }, "ds_lWSFXLao": { "name": "time string passed", "options": { "query": "| makeresults format=csv data=\"max_timestring_tok\n$max_timestring_tok$\"\n| eval type = typeof(max_timestring_tok)", "queryParameters": { "earliest": "-24h@h", "latest": "now" } }, "type": "ds.search" }, "ds_nZMqnAeL": { "name": "mvexpand with rex label", "options": { "extend": "ds_34OFi4Mu", "query": "| stats values(timestring) as timestring\n| mvexpand timestring\n| rex field=timestring \"(?<label>\\S+)\"\n| eval v_type = typeof(timestring), l_type = typeof(label)\n| sort - label\n| fields label l_type timestring v_type" }, "type": "ds.chain" }, "ds_sfp5IIKQ": { "name": "mvexpand timestring passed", "options": { "query": "| makeresults format=csv data=\"mvexpand_rex_tok\n$mvexpand_rex_tok$\"\n| eval type = typeof(mvexpand_rex_tok)", "queryParameters": { "earliest": "-24h@h", "latest": "now" } }, "type": "ds.search" }, "ds_z6WkSe6U": { "name": "time string passed by label", "options": { "query": "| makeresults format=csv data=\"label_replace_tok\n$label_replace_tok$\"\n| eval type = typeof(label_replace_tok)", "queryParameters": { "earliest": "-24h@h", "latest": "now" } }, "type": "ds.search" } }, "layout": { "globalInputs": [ "input_2qavhtpJ", "input_qjIV5Ix1", "input_1kIxDHKk", "input_R3Q1RvBK" ], "layoutDefinitions": { "layout_1": { "options": { "height": 960, "width": 1440 }, "structure": [ { "item": "viz_KJJE0cri", "position": { "h": 339, "w": 273, "x": 0, "y": 0 }, "type": "block" }, { "item": "viz_amq7ntem", "position": { "h": 136, "w": 273, "x": 0, "y": 339 }, "type": "block" }, { "item": "viz_UsOo59WW", "position": { "h": 339, "w": 376, "x": 273, "y": 0 }, "type": "block" }, { "item": "viz_bMEWY9PX", "position": { "h": 136, "w": 376, "x": 273, "y": 339 }, "type": "block" }, { "item": "viz_93ySZ5Cc", "position": { "h": 339, "w": 403, "x": 649, "y": 0 }, "type": "block" }, { "item": "viz_dvN9WkEw", "position": { "h": 136, "w": 403, "x": 649, "y": 339 }, "type": "block" }, { "item": "viz_SMBNQ4XA", "position": { "h": 339, "w": 388, "x": 1052, "y": 0 }, "type": "block" }, { "item": "viz_HaKyhXKv", "position": { "h": 136, "w": 388, "x": 1052, "y": 339 }, "type": "block" } ], "type": "grid" } }, "options": {}, "tabs": { "items": [ { "label": "New tab", "layoutId": "layout_1" } ] } } } As the screenshot shows, even with labels correctly extracted, when DS considers the field as type time, the input dropdown uses extracted date followed by letter "T" and 0 hour 0 minute 0 second. Are these all intentional?

yuanliu · ‎01-31-2026

I also need to use rex and eval to normalize the fields in both my searches, since they look quite different at index time. I don't think i can use rex and eval in a search like that? Yes, you can. @PickleRick gives you the formula. If you need concrete help in a data analytics forum, you need to illustrate input data. How are the two indices different? Which exact part of the events in the two gives you the user name? Which rex command do you use with each index? Without knowing the rex you actually use, here is a generic formula to combine what @gcusello and @PickleRick have said: (index=x sourcetype=y) OR (index=a sourcetype=b) | rex "(<?user_from_index_x_sourcetype_y>matches username in index x, sourcetype y)" | rex "(<?user_from_index_a_sourcetype_b>username in index x, sourcetype y will be extracted)" | eval username = coalesce(user_from_index_x_sourcetype_y, user_from_index_a_sourcetype_b) | bin _time span=5m | stats values(username) AS username count BY _time | where mvcount(username) > 1 Normally, I would consider the job done here. username will always contain two users, one from each index. But if you want to record which user comes from which index, how many counts from each user contribute total count, such information can be retained, too.

yuanliu · ‎01-31-2026

Maybe we should go back to basics: The data. Instead of trying to filter out, let's see if those username's are even in. How about testing index IN ("pisupport", "pisupport-np") sourcetype=PIMessage ("procbook" AND "Successful Login") | rex field=_raw "Identity\sList:\s(?<identity>[^.]+)" | rex field=_raw "Username\s:\s(?<username>[^.]+)" | where username=="NAM\\OT00564" | table username I have a suspicion that these user names you are trying to exclude do not match the expressions, therefore the above will not return anything. In other words, the task is not to find or fine tune SPL, but to determine the exact users you are really trying to exclude. If you have difficulty with string for matching, please post exact username you get from raw data. One more pointer about data and match string. Looking at the samples you give, "NAM\\OT00564" and "NAM\\CHawki5", they look too much like Windows domain users, but with double backslashes. Have you tried single backslash? Like this index IN ("pisupport", "pisupport-np") sourcetype=PIMessage ("procbook" AND "Successful Login") | rex field=_raw "Identity\sList:\s(?<identity>[^.]+)" | rex field=_raw "Username\s:\s(?<username>[^.]+)" | where NOT username IN ("NAM\OT00564", "NAM\CHawki5") | table username

yuanliu · ‎01-22-2026

This is very useful information! Meanwhile, Splunk equates "saved search" with "Report". If you save a search with token, the saved search can no longer be used as Report because the common sense of a Report is such that you open it and see output. In such use, the "saved search" is very much just another way of saving a macro.

yuanliu · ‎01-21-2026

is there any SPL answer to the original question? The answer, it turns out, is yes. Instead of listing intermediate field names for exclusion, name all intermediate fields to be "hidden", i.e., start with underscore _. The above search can be written as | makeresults count=10 | eval value = "value" . random() % 3 | stats count as _count by value | eventstats sum(_count) as _total | eval ratio = _count / _total Splunk visualization will conveniently ignore hidden fields. This writing has more elaborate use cases of this technique: Up Your Textual Viz with Splunk.

Posts	2726
Solutions	418
Karma Given	228
Karma Received	926
Member Since	‎11-12-2013

Online Status	Offline
Date Last Visited	2 weeks ago

How does foreach + rename impact performance?

Does Dashboard Studio force time-like string to no...

Different behavior regarding JSON null

splunkd causes OutOfMemory when searching data wit...

Extraction does not take place in raw events

Why does Dashboard Studio grey out "Open in Search...

Why does Dashboard Classic not return result even ...

How to fix BTree unexpected offset 0 while on orde...

How to prevent Splunk on MacOS from "freezing"

How to fix wandering trellis order?

Re: Trouble tabling email details

Re: Transaction with starting and ending event

Re: Run savedsearch from main search using lookup ...

Re: pars out dynamic number of request and respons...

Re: pars out dynamic number of request and respons...

Re: How to get percent of http 200 codes per file

Re: Displaying Changes in Smartcard-based authenti...

Re: search past escaped characters for an alert

Re: search past escaped characters for an alert

Re: Dedup different types of events by different f...

Re: Dedup different types of events by different f...

Re: How does foreach + rename impact performance?

How does foreach + rename impact performance?

Re: I want to adjust of field of table.

Re: Prevent users from creating saved searches, da...

Re: Create multiple dependent dropdown Dashboard

Re: How do I filter a large number of ip_client ad...

Re: Search lookup for user in specific group and s...

Re: How do I get a list of all my lookups with the...

Re: eval case statement

Does Dashboard Studio force time-like string to no...

Re: Compare fields from two different searches in ...

Re: How to search for values not in a listing

Re: Can we pass a parameter to a report?

Re: How to drop extra fields while maintaining gro...

Join the Conversation