headers, but I'm still unsure of how to parse each individual value ("meteoTemp", or "meteolunarPercent" for example) into separate objects so they can represented by separate and I am confused.  Have you viewed my sample output?  meteoTemp and meteolunarPercent are extracted by spath, and tabulated in my example.  You can plot them however you want.  For example, source="mqtt://MeteoMQTT" | rex "msg=(?<msg>.+)" | spath input=msg | timechart avg(meteoTemp) as avgMeteoTemp max(meteolunaPercent) as maxMeteolunaPercent If you do not get those fields, you need to play with my emulation and carefully compare with your raw data and post data that is representative of the actual data structure. ... View more

Yes.  The data is organized in KV pairs.  What is different is that it uses two different connectors, "=" and ":".  It also does not quote the value.  So, I am not sure if automatic extraction is feasible.  But at search time, you can simply do   | kv pairdelim=" " kvdelim="=:"   Your sample data will give the following fields: field name field value ComputerName sacreblue Domaine_du_compte AUTORITE NT EventCode 4672 EventType 0 ID de sécurité AUTORITE NT\Système ID_d_ouverture_de_session 0x3e7 Keywords Succès de l'audit LogName Security Message Privilèges spéciaux attribués à la nouvelle ouverture de session. Nom_du_compte Système OpCode Informations Privilèges SeAssignPrimaryTokenPrivilege RecordNumber 2746 SourceName Microsoft Windows security auditing. TaskCategory Ouverture de session spéciale Type Information Here is an emulation that you can play with and compare with real data         | makeresults | eval _raw = "04/29/2014 02:50:23 PM LogName=Security SourceName=Microsoft Windows security auditing. EventCode=4672 EventType=0 Type=Information ComputerName=sacreblue TaskCategory=Ouverture de session spéciale OpCode=Informations RecordNumber=2746 Keywords=Succès de l'audit Message=Privilèges spéciaux attribués à la nouvelle ouverture de session. Sujet : ID de sécurité : AUTORITE NT\Système Nom du compte : Système Domaine du compte : AUTORITE NT ID d'ouverture de session : 0x3e7 Privilèges : SeAssignPrimaryTokenPrivilege SeTcbPrivilege SeSecurityPrivilege SeTakeOwnershipPrivilege SeLoadDriverPrivilege SeBackupPrivilege SeRestorePrivilege SeDebugPrivilege SeAuditPrivilege SeSystemEnvironmentPrivilege SeImpersonatePrivilege" ``` data emulation above ```         I still have a question about your conversion to XML.  Do you mean that you use an external tool to convert that raw text into XML before ingesting into Splunk?  If you have this option, why not convert the raw text into JSON for which Splunk has better support? ... View more

Just extract the content of "msg" into a new field, then apply spath   | rex "msg=(?<msg>.+)" | spath input=msg   Here is the output from your sample data meteoHumidity meteoRainlasthour meteoTemp meteoWindDirection meteoWindSpeed meteolunaPercent msg 64 0 17.9 SW 6.04 67.3 {"meteoTemp":17.9,"meteoHumidity":64,"meteoRainlasthour":0,"meteoWindSpeed":6.04,"meteoWindDirection":"SW","meteolunarPercent":67.3} This is an emulation for you to play with and compare with real data.   | makeresults | eval _raw = "Fri Jul 26 15:24:46 BST 2024 name=mqtt_msg_received event_id= topic=meteobridge msg={\"meteoTemp\":17.9,\"meteoHumidity\":64,\"meteoRainlasthour\":0,\"meteoWindSpeed\":6.04,\"meteoWindDirection\":\"SW\",\"meteolunarPercent\":67.3}" ``` data emulation above ```     ... View more

This is because you have a multisegment data path and eval doesn't like it.  Use a single quote to tell eval log.level is a field name not some random string. index="prod_k8s_onprm_dig-k8-prod1" "k8s.namespace.name"="apl-secure-dig-svc-prod1" "k8s.container.name"="abc-def-cust-prof" NOT k8s.container.name=istio-proxy NOT log.level IN(DEBUG,INFO) (error OR exception)(earliest="07/25/2024:11:30:00" latest="07/25/2024:12:30:00") | addinfo | bin _time span=30m@m | stats count(eval('log.level'="ERROR")) as error_count by _time | eventstats stdev(error_count)   ... View more

First, using subsearch should not be your first choice.  Second, Splunk is not procedural; forced recursion on command will result in some unmaintainable code. You need to provide additional information about your data in addition to that your second dataset doesn't have eventId readily extracted.  I assume that the first "search" and second have different source types.  I also assume that search period is roughly identical in all three.  But I don't understand what is the dataset for the third "search".  Is it yet another indexed source?  Is it some sort of lookup table? To ask answerable questions in this forum, follow the following golden rules that I call the Four Commandments: Illustrate data input (in raw text, anonymize as needed), whether they are raw events or output from a search that volunteers here do not have to look at. Illustrate the desired output from illustrated data. Explain the logic between illustrated data and desired output without SPL. If you also illustrate attempted SPL, illustrate actual output and compare with desired output, explain why they look different to you if that is not painfully obvious. ... View more

That's better.  So, you are looking at adjacent, and equal time intervals.  In this case, time bucket is perhaps the simplest.   Let me first give you a hard-coded example. index="prod_k8s_onprem_dii--prod1" "k8s.namespace.name"="abc-secure-dig-servi-prod1" "k8s.container.name"="abc-cdf-cust-profile" (earliest="07/25/2024:11:30:00" latest="07/25/2024:12:30:00") | addinfo | bin _time span=30m@m | stats count(eval(status="Error")) as error_count by _time | eventstats stdev(error_count) Is this something you are looking for? index="prod_k8s_onprem_dii--prod1" "k8s.namespace.name"="abc-secure-dig-servi-prod1" "k8s.container.name"="abc-cdf-cust-profile" (earliest=first_earliest latest=first_latest) OR (earliest=second_earliest latest=second_latest) | eval period=if(_time>=first_earliest AND _time<first_latest,"First","Second") | stats count(eval(status="Error")) as error_count count as event_count by period   ... View more

Assuming your search gives you some useful fields to make the determination based on some logic, the next step is to follow my Four Commandments of posing answerable questions in this forum: Illustrate data input (in raw text, anonymize as needed), whether they are raw events or output from a search that volunteers here do not have to look at. Illustrate the desired output from illustrated data. Explain the logic between illustrated data and desired output without SPL. If you also illustrate attempted SPL, illustrate actual output and compare with desired output, explain why they look different to you if that is not painfully obvious. ... View more

You will need a common value in the two types of events to correlate events.  For example, if each pair has a unique transaction ID, you can do | stats values(Resp_time) as Resp_time values(Req_time) as Req_time by transaction_id | eval Resp_time - Req_time Alternatively, if you have some other ways to determine a pairing, e.g., the two always happen within a deterministic interval,  e.g., request comes in at 5 minute into the hour, a unique response is sent within the hour and NO other request would come in during the same hour, you can use that as criterion.  There may be other conditions where you would use transaction.  Unless you give us the exact condition, mathematically there is no solution. ... View more

Instead of stats, use eventstats. index="oap" | eventstats perc25(tt) as P25, perc50(tt) as P50, perc75(tt) as P75 by oper | foreach P25 P50 P75 [eval <<FIELD>>count = if(tt><<FIELD>>, 1, 0)] | stats values(P*count) as P*count by oper P25 P50 P75 ... View more

You need to tell volunteers what kind of "two time frames" are you concerned about.  Two adjacent, equal time intervals? Two equal intervals days apart?  Or some random intervals? ... View more

This is confusing.  Could you explain "convert them?" Do you mean the raw events are not in XML?  In that case, could you share raw events?  Also, French should not stop Splunk as long as it is encoded in UTF-8 or another compatible scheme. ... View more

Your requirement isn't really clear.  Not to point to the obvious difference between last (set in first panel) and $latest$ (used in second panel), but are you sure you can even add an additional field in the first panel and still maintain your original timechart? (Hint: It will ruin it all; at least it will distort the chart.) Another important question is: What is that $latest$ expected  supposed to be?  It seems that you want it to be the interactive token because you set it according to _time which varies by row.  I already mentioned that setting a new field after timechart will ruin your chart.  But in addition, Dashboard Studio has its own regiment to manage tokens.  You cannot set a variable in one search and call that variable with $$ and expect it to be a passable token.  This is the document about setting interactive token with search result: Setting tokens from search results or search job metadata. Then, to add 1 week to the click value, run that result in another search. (Just like you would do in Simple XML.)  Lastly, use result from that search to drive the second panel.  Here is an example: { "visualizations": { "viz_7yE1ZwsT": { "type": "splunk.line", "dataSources": { "primary": "ds_DmIKSSCN" }, "title": "First panel", "eventHandlers": [ { "type": "drilldown.setToken", "options": { "tokens": [ { "token": "latest_tok", "key": "row._time.value" } ] } } ], "options": { "legendDisplay": "top" } }, "viz_OIqDnl0b": { "type": "splunk.line", "options": { "legendDisplay": "bottom" }, "dataSources": { "primary": "ds_79fdaiuf" }, "showProgressBar": false, "showLastUpdated": false } }, "dataSources": { "ds_DmIKSSCN": { "type": "ds.search", "options": { "query": "| tstats count where index=_internal by _time span=1d sourcetype\n| timechart span=1d sum(count) by sourcetype\n| eval _last = relative_time(_time, \"+1w\")" }, "name": "first panel" }, "ds_79fdaiuf": { "type": "ds.search", "options": { "query": "index=_introspection latest=$make token:result.week_after$\n| timechart span=1d count by sourcetype" }, "name": "dependent panel" }, "ds_EHm1QhZI": { "type": "ds.search", "options": { "query": "| makeresults\n| eval week_after = relative_time($latest_tok$, \"+1w\")", "enableSmartSources": true }, "name": "make token" } }, "defaults": { "dataSources": { "ds.search": { "options": { "queryParameters": { "latest": "$global_time.latest$", "earliest": "$global_time.earliest$" } } } } }, "inputs": { "input_global_trp": { "type": "input.timerange", "options": { "token": "global_time", "defaultValue": "-3w@w,now" }, "title": "Global Time Range" } }, "layout": { "type": "grid", "options": { "width": 1440, "height": 960 }, "structure": [ { "item": "viz_7yE1ZwsT", "type": "block", "position": { "x": 0, "y": 0, "w": 1440, "h": 400 } }, { "item": "viz_OIqDnl0b", "type": "block", "position": { "x": 0, "y": 400, "w": 1440, "h": 400 } } ], "globalInputs": [ "input_global_trp" ] }, "description": "https://community.splunk.com/t5/Splunk-Search/Dashboard-Studio-earliest-latest-tokens/m-p/691740", "title": "Pass time token" } In this dashboard, when you click a point on July 13 in the first panel, the second panel will end on July 20.  Is this something you are looking at? ... View more

This is structured data in XML.  Splunk's extraction tool is either regex or delimiter based and will not be robust.  I recommend that you forget about extraction, just run spath or xmlkv at search time. ... View more

The problem is actually deeper because appendcols works only if the lookup and index search has the same number of rows (and sort order).  In this use case, that's opposite to the premise.  I will have to look deeper - but there should be something - it could be even more cumbersome. ... View more

If the lookup is of any size and if the number of events from the index search is large, yes, this will take vary long because it is effectively performing | search url = "foo.com" OR url = "bar.com" OR url = "barz" OR ... Moving the subsearch to index search can save some time because you won't be streaming as many events: index=myindex sourcetype=mysource [|inputlookup LCL_url.csv | fields url] | stats count by url | fields - count | sort url However, using a lookup file as subsearch may not be the best option to begin with. ... View more

Like this?   | rex "message: \((?<in_parentheseses>[^\)]+)"   You can test with   | makeresults format=csv data="_raw message: (c4328dd3-d16e-4df8-a8e6-b2ebcab9d8bc)" ``` data emulation above ``` | rex "message: \((?<in_parentheses>[^\)]+)"   _raw in_parentheses message: (c4328dd3-d16e-4df8-a8e6-b2ebcab9d8bc) c4328dd3-d16e-4df8-a8e6-b2ebcab9d8bc ... View more

Something like this?   | eventstats range(count) as varies by HOST | where varies > 0   Here is an emulation you can play with and compare with real data. (I know that # is not a real field.  It doesn't affect calculation here.)   | makeresults format=csv data="#,HOST,BGP_NEIGHBOR,BGP_STATUS,count 1,Router A,neighbor 10.1.1.1,Down,1 2,Router A,neighbor 10.1.1.1,Up,1 3,Router B,neighbor 10.2.2.2,Down,1 4,Router B,neighbor 10.2.2.2,Up,1 5,Router C,neighbor 10.3.3.3,Down,2 6,Router C,neighbor 10.3.3.3,Up,1 7,Router D,neighbor 10.4.4.4,Down,2 8,Router D,neighbor 10.4.4.4,Up,2" ``` the above emulates ..... | rex field=_raw "(?<BGP_NEIGHBOR>neighbor\s\b\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})" | rex field=_raw "(?<BGP_STATUS>(Up|Down))" | stats count by HOST, BGP_NEIGHBOR, BGP_STATUS ```   Combining this with the above search gives # BGP_NEIGHBOR BGP_STATUS HOST count varies 5 neighbor 10.3.3.3 Down Router C 2 1 6 neighbor 10.3.3.3 Up Router C 1 1 ... View more

As we always say in this forum, illustration of raw input (in text format) is critical for the question to be answerable.  Thank you for finally getting to data.  My previous answer was based on KendallW's emulation.  This latest illustration is not only different from that emulation, but also different from your initial screenshot.  One fundamental difference is that this data includes multiple days potentially in the future.  It seems that the input is from a prediction of sorts. This said, I also realized that JSON keys themselves can be utilized to simply solution if you are using Splunk 8.1 or later.  Again, regex is NOT the correct tool for structured data. Here is the code you can try:   | eval today = strftime(now(), "%F"), tomorrow = strftime(relative_time(now(), "+1d"), "%F") | eval today = json_extract(_raw, "result.watt_hours_day." . today) | eval tomorrow = json_extract(_raw, "result.watt_hours_day." . tomorrow)   Here is an emulation for you to play with and compare with real data.  Because your illustrated data is way in the past, I randomly pick 2019-06-26 as search time and establishes a "fake_now" field instead of using now() function. (As a result, "tomorrow" corresponds to 2019-06-27.)   | makeresults | eval _raw="{ \"result\": { \"watts\": { \"2019-06-22 05:15:00\": 17, \"2019-06-22 05:30:00\": 22, \"2019-06-22 05:45:00\": 27, \"2019-06-29 20:15:00\": 14, \"2019-06-29 20:30:00\": 11, \"2019-06-29 20:45:00\": 7 }, \"watt_hours\": { \"2019-06-22 05:15:00\": 0, \"2019-06-22 05:30:00\": 6, \"2019-06-22 05:45:00\": 12, \"2019-06-29 20:15:00\": 2545, \"2019-06-29 20:30:00\": 2548, \"2019-06-29 20:45:00\": 2550 }, \"watt_hours_day\": { \"2019-06-22\": 2626, \"2019-06-23\": 2918, \"2019-06-24\": 2526, \"2019-06-25\": 2866, \"2019-06-26\": 2892, \"2019-06-27\": 1900, \"2019-06-28\": 2199, \"2019-06-29\": 2550 } }, \"message\": { \"type\": \"success\", \"code\": 0, \"text\": \"\" } }" | spath | eval fake_now = strptime("2019-06-26 18:15:06", "%F %T") | eval today = strftime(fake_now, "%F"), tomorrow = strftime(relative_time(fake_now, "+1d"), "%F") | eval today = json_extract(_raw, "result.watt_hours_day." . today) | eval tomorrow = json_extract(_raw, "result.watt_hours_day." . tomorrow) | fields result.watt_hours_day.2019-06-26 result.watt_hours_day.2019-06-27 today tomorrow   Output is today tomorrow result.watt_hours_day.2019-06-26 result.watt_hours_day.2019-06-27 _raw 2892 1900 2892 1900 { "result": { "watts": { "2019-06-22 05:15:00": 17, "2019-06-22 05:30:00": 22, "2019-06-22 05:45:00": 27, "2019-06-29 20:15:00": 14, "2019-06-29 20:30:00": 11, "2019-06-29 20:45:00": 7 }, "watt_hours": { "2019-06-22 05:15:00": 0, "2019-06-22 05:30:00": 6, "2019-06-22 05:45:00": 12, "2019-06-29 20:15:00": 2545, "2019-06-29 20:30:00": 2548, "2019-06-29 20:45:00": 2550 }, "watt_hours_day": { "2019-06-22": 2626, "2019-06-23": 2918, "2019-06-24": 2526, "2019-06-25": 2866, "2019-06-26": 2892, "2019-06-27": 1900, "2019-06-28": 2199, "2019-06-29": 2550 } }, "message": { "type": "success", "code": 0, "text": "" } } ... View more

Let's get back to basics: When your events are broken, using search technique to cope is the last thing to consider. Can you post sample raw file, the exact event contents Splunk receives, and your properties.conf stanza corresponding to this sourcetype?  Without data, volunteers have nothing to go on. ... View more

If you forwarder is not forwarding the complete file, there might be a problem with linebreaker.  This has nothing to do with how to search.  Getting Data In is a better forum. ... View more

Assuming you have lookup file asn_user.csv and you set up a lookup called asn_user.csv (my preference is to not use .csv in lookup name, but many others do not make the distinction), you can do index="okta" actor.alternateId=*@* authenticationContext.externalSessionId!="unknown" | eval "ASN"='securityContext.asNumber' | eval "Session ID"='authenticationContext.externalSessionId' | eval "User"='actor.alternateId' | eval "Risk"='debugContext.debugData.risk' | stats dc("user_agent") as "Agent Count" values(user_agent) AS "User Agent" dc(ASN) as "ASN Count" values(ASN) as ASN dc(Risk) as "Risk Count" values(Risk) as Risk by User "Session ID" | table "Session ID", ASN, "ASN Count", "User Agent", "Agent Count", User, Risk | lookup asn_user.csv User output ASN as ASNfound | where "ASN Count" > 1 AND "Agent Count" > 1 AND NOT mvfind(ASNfound, ASN)   ... View more

An easier, and perhaps more semantic method is to use JSON functions introduced in 8.1 to restructure data. (As we have seen before, you have developers who overload JSON's key name to convey data, which is never a good thing.  If you have any influence on them, maybe ask them to change structure before it reaches data consumer.) With the data you illustrated, Splunk would have given you fields like key1.field1, key2.field2.  Iterate over them using foreach.   | foreach *.* [eval temp = json_object(), temp = json_set(temp, "Name A", "<<MATCHSEG1>>", "Name B", "<<MATCHSEG2>>", "Value", '<<FIELD>>'), reformat = mvappend(reformat, temp)] | mvexpand reformat | spath input=reformat | fields - _* key* temp    Your example results in Name A Name B Value reformat key1 field1 x {"Name A":"key1","Name B":"field1","Value":"x"} key2 field2 xx {"Name A":"key2","Name B":"field2","Value":"xx"} key3 field3 xxx {"Name A":"key3","Name B":"field3","Value":"xxx"} Here is an emulation you can play with and compare with real data   | makeresults | eval _raw="{ \"key1\": { \"field1\": \"x\" }, \"key2\": { \"field2\": \"xx\" }, \"key3\": { \"field3\": \"xxx\" } }" | spath ``` data emulation above ```   ... View more