Splunk uses PCRE and also the search parser will handle unescaping, whereas Ingest Processor uses RE2 - although that appears to be changing. So you probably need to use 2 \\ characters, not 3 as the Splunk parser will take away one. But validate ... View more

Have you tried the search I showed - does it give you something approximating what you are after?     ... View more

I find that match() is generally more useful for most problems than searchmatch - as @livehybrid says, searchmatch is effectively giving you the ability to do matching done through the search command syntax, so there is no case sensitivity, wildcards can be used as needed AND and OR can be used, whereas match is a much more specific regex based matching and you can match any regex against any field.   ... View more

The general way to "join" is to use stats by X and in your case you are searching two data sets, so you could do something like this index=provisioning_index sourcetype=PCF:log source_type=APP/PROC/WEB message_type=OUT cf_org_name=org1 cf_app_name=APP1 (LOG_LEVEL="ERROR" service=service1 errorCd="DOC-MGMT*" OR (NOT letterId=null operation=generateInstantDocument) | rex field=_raw "errorDetails=(?<errorDetails>.*?)\s*:" | fields _time errorCd errorDetails stateCode letterId documentId | stats values(*) as * by documentId and depending on what output you are trying to achieve, you can add in BY fields to split by. Not sure which data is coming from which data source as your fields statements are the same for both cases. Not sure why you originally had the errorDetails field in the fields statement BEFORE the rex statement if you are extracting that fields in the rex. ... View more

Here are a couple of links to other posts here https://community.splunk.com/t5/Getting-Data-In/JSON-field-extraction-beyond-5000-chars/m-p/549963#:~:text=The%20auto%2Dfield%2Dextraction%20stops,result%20list%20after%20a%20search. https://community.splunk.com/t5/Getting-Data-In/Missing-events-JSON-payload-and-indexed-extractions/m-p/489113 If you start changing limits.conf - which is not simple with Cloud, it will affect general settings, so is not always the best way to go. If you have a field that is not extracted and it's a simple field - i.e. a single value inside a JSON object with no multivalue component then a simple calculated field can work, i.e. | eval type=spath(_raw, "data.tree.fruit.type") In the conf just use the spath... part for the eval definition ... View more

searchmatch is a somewhat odd command in that it is looking at the :"EVENT" i.e. it must have a _raw. If you run this | makeresults | eval _raw="ONE TWO THREE" | eval result=if(searchmatch("THREE TWO"), 1, 0) You will see result=1, but if you run | makeresults | eval _raw="ONE TWO THREE" | eval result=if(searchmatch("THREEX TWO"), 1, 0) You will see result=0 Also if you run  | makeresults | eval fieldstring="ONE TWO THREE" | eval result=if(searchmatch("XX YY"), 1, 0) You will also see result=1 - odd - but that's the way it seems to handle a null _raw field. I am not sure why it finds a result when _raw is not present. Note the example given in the documentation, which further confuses https://docs.splunk.com/Documentation/Splunk/9.4.1/SearchReference/ConditionalFunctions#searchmatch.28.26lt.3Bsearch_str.26gt.3B.29 | makeresults 1 | eval _raw = "x=hi y=bye" | eval x="hi" | eval y="bye" | eval test=if(searchmatch("x=hi y=*"), "yes", "no") | table _raw test x y If you set _raw to be "x=low..." then the match will fail, so in this case, it's comparing the match against the specific field x where it has a value different to the _raw content. Anyway, your example sets a specific single string to be a fixed value, so if you do this | makeresults | eval fieldstring="ONE TWO THREE" | eval result=if(searchmatch("fieldstring=\"ONE TWO THREE\""), 1, 0) You will get a correct match, but if you change the match text, it will give you result=0. Hope I've not managed to confuse you too much!     ... View more

You made a point of emphasizing the different test_id (sane test_name BUT a different test_ID)  Is it possible to have a row with the SAME test_id at some point, i.e. could you insert a row at row 3 with test_name=test 1 and test_id="0.98"? If so, the simple streamstats solution suggested by @livehybrid won't work. Is it possible to have the same test_id and if so, what should be the behaviour? ... View more

It's likely that your auto extracted JSON fields are not extracting the entire object, i.e. if you search type=* and it does not find some values, then those values do not exist in that field in the auto extracted field. The fact that they DO give results after the spath, indicates this. What is the size of your JSON object. By default I believe it will only auto extract the first 5000 (5k?) bytes of a JSON object, so if you show "raw" in your display rather than the syntax highlighted view of the JSON, you can see where your fruit type field exists in the raw. If this is the case, then you can add some calculated fields using spath eval statement to extract the fields, so they are always present before the search is run. BTW, I'm not totally sure of the best practice way to manage this 5k limit, but the above will work. ... View more

Look at using INGEST_EVAL, where you can remove data from the JSON simply using eval statements, e.g. the following eval statement   _raw=json_delete(_raw, "avg_ingress_latency_fe", "conn_est_time_fe", "client_insights")   https://docs.splunk.com/Documentation/Splunk/9.4.0/Data/IngestEval   ... View more

Maybe you can share your current search that does this grouping. What's your logic for deciding that server 2 and server 3 need to be in a different server group to server 4, is it simply on the presence of a connection between those 2 services? @danspav  ... View more

Please show what your mailto link looks like in your XML  ... View more

The mvfunctions generally take an MV field as an input and then perform an operation on each of the values of the MV, so the solution | eval filtered=mvfilter(mvfield!="N/A") is saying - for each value of the MV field called mvfield match each one against the string "N/A" and if it does NOT match (!="N/A") then return that value to the new field filtered, appending each non-matching value to that new field. That new field will then contain all the values of the original mvfield that did not match the string. The eval is then finally putting back the "N/A" string to the filtered field so that if ALL values of the original field contained N/A then the new field will have a single N/A value. If you wanted ALL the N/A instances to be present, then replace the mvfilter line with | eval filtered=coalesce(mvfilter(mvfield!="N/A"), mvfield) which if you have N/A 3 times in your original, you will have N/A 3 times in your final result.   ... View more

Excellent - mvmap is very powerful when handling MV fields. One small point I noticed on your use of single quotes to wrap field names. Using single quotes is essential where field names start with numbers or have spaces or other non-ascii characters, although some characters are ok, e.g. underscore, otherwise the search just won't work. Here, you have one use substr(session_time, ...) where you are not using single quotes, and it works because it's not needed for either of the two fields names here, so it's good to be aware of when it's necessary on the right hand side of an eval. When you know the field name, you can make the call, but a command that is very useful is foreach and when using templated fields <<FIELD>>, it's generally good practice to always wrap that usage.   ... View more

I am not sure how you would get two 'records' for the same jobname as all the aggregations you are doing are by JOBNAME. But do you mean you have two (or more) values of END_TIME for the same single JOBNAME in the output. That would be because you are doing  | stats values(Date_of_reception) as Date_of_reception values(*_TIME) as *_TIME by JOBNAME i.e. values(*_TIME)... is giving you all the values of the START and END TIME values. If you just want the latest END_TIME then you just need to use max and min as needed, i.e.  | stats values(Date_of_reception) as Date_of_reception max(END_TIME) as END_TIME min(START_TIME) as START_TIME by JOBNAME would give you the earliest start and the latest end. But if you have getting more than one event then any use of values will give you all values, but note also that values() will sort the values in the multivalue field, so bear that in mind. ... View more

I'm not clear on what the data in type 1 is used for. You have object class in type 2 source as well as type 1 - I assume CartmanUser is meant to be cartmanUser What are the IAL to enforce values 1 and 2 supposed to correlate to, to give you the 1 and 2 in your tables?   ... View more

So, you would use it like this index = main source=xyz (TERM(A1) OR TERM(A2) ) ("- ENDED" OR "- STARTED" ) | rex field=TEXT "((A1-) |(A2-) )(?<Func>[^\-]+)" | eval Function=trim(Func), DAT = strftime(relative_time(_time, "+0h"), "%d/%m/%Y") | rename DAT as Date_of_reception | eval {Function}_TIME=_time | stats values(Date_of_reception) as Date_of_reception values(*_TIME) as *_TIME by JOBNAME ``` This adds in all the entries in the lookup at the end of the current results ``` | inputlookup append=t File.csv ``` This then joins all the lookup fields to your result data based on JOBNAME ``` | stats values(*) as * by JOBNAME ``` Now order the fields as needed and sort ``` | table JOBNAME Description Date_of_reception STARTED_TIME ENDED_TIME | sort -STARTED_TIME i.e. append all the lookup data to the end and collapse it on JOBNAME Note that your _time handling is a little strange - not sure what you're trying to do, but what's wrong with just | eval DATE_of_reception=strftime(_time, "%d/%m/%Y") Note also that if you have more than 1 START_TIME or END_TIME, the sort will not work correctly on the multivalue field. ... View more

Glad it worked, these types of makeresults search are insignificant, they only ever sit on the search head as they are never searching data from the indexers. I often use background searches and tokens to create data that can then be used in <html> panels. They don't consume much. ... View more

Firstly join is almost never a solution to a Splunk problem.  Secondly, you do not have Column1 as an output of your tstats search, so how can it match up Col1+Col2 with the start/end times. Generally if you want to enrich the start/end times with info from a lookup, you would run the tstats, then lookup the common fields (Column1)  from the lookup and output the other fields (Column2) If you want to end up with all the rows from the lookup in the output, even where there is no data for some of the rows, you would then do a final couple of commands, i.e. | inputlookup File.csv append=t | stats values(*) as * by Column1 which would then give you all the rows from the lookup and start/end times from data for Column1 found in the tstats search. ... View more

Technically you could use both base searches, but it's a bit fiddly and isn't really going to save you anything as the searches have to run anyway. You would get the job ids of each base search and then in your panel search you would use loadjob to load each of the jobs.  However, you're still going to have to load the second job in some kind of subsearch (join?) so not sure where you're trying to go with this. If you are simply trying to speed up a join search, you can't achieve this with two base searches, as you are simply not changing anything and it will take the time it takes. The solution for a poor performing search using join is to remove the use of join and rewrite the search in another way. Looking at your existing searches I'm not sure why you are trying to combine these in the first place, because you have appcode in your first search and you simply want appcode to get the list of details from the lookup. You are doing a lookup in the primary search but doing nothing with the retrieved data. Why don't you just do the lookup in your primary search after the chart, i.e. index=serverdata | rex "host_name=\"(?&lt;server_host_name&gt;[^\"]*)" |chart dc(host_name) over appcode by host_environment | eval TOTAL_servers=DEV+PAT+PROD | table appcode DEV PAT PROD TOTAL_servers | lookup servers_businessgroup_appcode.csv appcode output Business_Group as New_Business_Group   ... View more

If you want to do this at search time, create a composite field and expand that, e.g. | eval composite_field=mvzip('users{}.userhost', 'users{}.username', "###") | fields - users{}* | mvexpand composite_field | rex field=composite_field "(?<userhost>.*)###(?<username>.*)" | fields - composite_field it will only zip correctly if there are exactly equal elements in each of the MV fields.   ... View more

If you want the Total to represent the Total of the individual MINUTE in the period, then it will always be different to the max of all the minutes in the period for each LPAR. e.g.  Time,TRX,TRX2,STM,Total 00:01,4,5,6,15 00.02,1,1,16,18 When this is summed over the hour it will be 00.00,4,5,16,18 and of course 18 is less than the sum of 4+5+16 - so you can't have it both ways - in your original post, you wanted the total to reflect the sum of LPAR for the biggest MINUTE, so it can't also represent the sum of the biggest DAY unless you do another addtotals to a new field which is the total for the day. ... View more

OK, so it's getting created in the stats command. So don't use that technique. Do something like | stats dc(hostname) as server_count | eval n=if($env|s$="*", "ALL", $env|s$) | eval server_{n}_count=server_count | fields - server_count but do you really need the $env$ in the field name? Is this dashboard studio - you can probably assign an additional token $env_name$ based on the selected NAME of the environment rather than the token value. I'm not familiar enough with DS to say how to do this, but you can then use $env$ as the search constraint and $env_name$ as the server name. ... View more