This is a JSON object (except you should add quotes to those bare XXXX).  Do not use regex on structured data.  See the other thread you started.  Your search is inefficient because you use wildcard at the beginning of a term.  And there's a solution to that. ... View more

Your search is slower compared with what?  You don't need to run spath according to my analysis.  Because Splunk has already extracted it, running spath simply wastes CPU and memory.   But running a search with leading wildcard always slows things down considerably. (The way you try to use regex doesn't make things better.)  Why do you need wildcards, anyway?  Your search can be conducted in bare terms without considering the field.  Try index="sample_idx" $serialnumber$ log_level=info Unit state update from cook client target Here's an emulation for you to play with and compare with real data | makeresults | eval _raw = "{\"bootcount\":8,\"device_id\":\"XXXX\",\"environment\":\"prod_walker\",\"event_source\":\"appliance\",\"event_type\":\"GENERIC\",\"local_time\":\"2025-02-20T00:34:58.406-06:00\", \"location\":{\"city\":\"XXXX\",\"country\":\"XXXX\",\"latitude\":\"XXXX\",\"longitude\":\"XXXX\",\"state\":\"XXXX\"},\"log_level\":\"info\", \"message\":\"martini::hardware_controller: Unit state update from cook client target: Elements(temp: -, [D, D, D, D, D, F: 0]), hw_state: Elements(temp: -, [D, D, D, D, D, F: 115])\\u0000\", \"model_number\":\"XXXX\",\"sequence\":372246,\"serial\":\"XXXX\",\"software_version\":\"2.3.0.276\",\"ticks\":0,\"timestamp\":1740033298,\"timestamp_ms\":1740033298406}" | eval _time = json_extract(_raw, "timestamp") | spath ``` the abovee emulates index="sample_idx" $serialnumber$ log_level=info ``` | search Unit state update from cook client target   ... View more

Are you sure those bare XXXX are not quoted, like this?   {"bootcount":8,"device_id":"XXXX","environment":"prod_walker","event_source":"appliance","event_type":"GENERIC","local_time":"2025-02-20T00:34:58.406-06:00", "location":{"city":"XXXX","country":"XXXX","latitude":"XXXX","longitude":"XXXX","state":"XXXX"},"log_level":"info", "message":"martini::hardware_controller: Unit state update from cook client target: Elements(temp: -, [D, D, D, D, D, F: 0]), hw_state: Elements(temp: -, [D, D, D, D, D, F: 115])\u0000", "model_number":"XXXX","sequence":372246,"serial":"XXXX","software_version":"2.3.0.276","ticks":0,"timestamp":1740033298,"timestamp_ms":1740033298406}   If so, a "normal" Splunk instance should have given you message as a field with value "martini::hardware_controller: Unit state update from cook client target: Elements(temp: -, [D, D, D, D, D, F: 0]), hw_state: Elements(temp: -, [D, D, D, D, D, F: 115])". If, for whatever reason your instance doesn't, spath command suffices.  Try this example:   | makeresults | eval _raw = "{\"bootcount\":8,\"device_id\":\"XXXX\",\"environment\":\"prod_walker\",\"event_source\":\"appliance\",\"event_type\":\"GENERIC\",\"local_time\":\"2025-02-20T00:34:58.406-06:00\", \"location\":{\"city\":\"XXXX\",\"country\":\"XXXX\",\"latitude\":\"XXXX\",\"longitude\":\"XXXX\",\"state\":\"XXXX\"},\"log_level\":\"info\", \"message\":\"martini::hardware_controller: Unit state update from cook client target: Elements(temp: -, [D, D, D, D, D, F: 0]), hw_state: Elements(temp: -, [D, D, D, D, D, F: 115])\\u0000\", \"model_number\":\"XXXX\",\"sequence\":372246,\"serial\":\"XXXX\",\"software_version\":\"2.3.0.276\",\"ticks\":0,\"timestamp\":1740033298,\"timestamp_ms\":1740033298406}" | eval _time = json_extract(_raw, "timestamp") ``` data emulation above ``` | spath | table message   Hint: output is message martini::hardware_controller: Unit state update from cook client target: Elements(temp: -, [D, D, D, D, D, F: 0]), hw_state: Elements(temp: -, [D, D, D, D, D, F: 115]) Alternatively, use json_extract function if your Splunk is 8.1 or later.  Try this example:   | makeresults | eval _raw = "{\"bootcount\":8,\"device_id\":\"XXXX\",\"environment\":\"prod_walker\",\"event_source\":\"appliance\",\"event_type\":\"GENERIC\",\"local_time\":\"2025-02-20T00:34:58.406-06:00\", \"location\":{\"city\":\"XXXX\",\"country\":\"XXXX\",\"latitude\":\"XXXX\",\"longitude\":\"XXXX\",\"state\":\"XXXX\"},\"log_level\":\"info\", \"message\":\"martini::hardware_controller: Unit state update from cook client target: Elements(temp: -, [D, D, D, D, D, F: 0]), hw_state: Elements(temp: -, [D, D, D, D, D, F: 115])\\u0000\", \"model_number\":\"XXXX\",\"sequence\":372246,\"serial\":\"XXXX\",\"software_version\":\"2.3.0.276\",\"ticks\":0,\"timestamp\":1740033298,\"timestamp_ms\":1740033298406}" | eval _time = json_extract(_raw, "timestamp") ``` data emulation above ``` | eval message = json_extract(_raw, "message")   If your instance is older, you can also use spath function.  Try this example   | makeresults | eval _raw = "{\"bootcount\":8,\"device_id\":\"XXXX\",\"environment\":\"prod_walker\",\"event_source\":\"appliance\",\"event_type\":\"GENERIC\",\"local_time\":\"2025-02-20T00:34:58.406-06:00\", \"location\":{\"city\":\"XXXX\",\"country\":\"XXXX\",\"latitude\":\"XXXX\",\"longitude\":\"XXXX\",\"state\":\"XXXX\"},\"log_level\":\"info\", \"message\":\"martini::hardware_controller: Unit state update from cook client target: Elements(temp: -, [D, D, D, D, D, F: 0]), hw_state: Elements(temp: -, [D, D, D, D, D, F: 115])\\u0000\", \"model_number\":\"XXXX\",\"sequence\":372246,\"serial\":\"XXXX\",\"software_version\":\"2.3.0.276\",\"ticks\":0,\"timestamp\":1740033298,\"timestamp_ms\":1740033298406}" | eval _time = spath(_raw, "timestamp") ``` data emulation above ``` | eval message = spath(_raw, "message")     ... View more

Name, Description, and Location are all multi value fields that directly corresponds to each other.  Here is the sample for one of the hosts:     Name Description Location name1 description1 location1 name2 description2 location2 name3 description3 location3 name4 description4 location4     Can you explain how is this sample for ONE of the hosts?  Does the above represent one field with five lines, the first line being "Name Description Location"?  Or do you mean to say a sample for one of the hosts looks like Name Description Location Name1 Name2 Name3 Name4 description1 description2 description3 description4 location1 location2 location3 location4 Or something totally different? Also, your SPL snippet doesn't show the mvexpand command that causes the memory error.  How are you using mvexpand? Additionally, what is the expected output from the sample, after you clarify how the sample actually look like? ... View more

Second what @ITWhisperer says.  If the raw event is not completely in JSON, the event must have included a JSON message.  In that case, Splunk would not have extracted JSON fields.  But it is strongly recommended that you treat structured data as structured data and do not use regex to extract from them.   The way to do this is to extract the JSON part into its own field so you can make structured extraction.  Please post sample of complete event. ... View more

is there something wrong in the logic or alternate way to do it  Yes, the logic is wrong with the given dataset.  But before I explain, please remember to post sample data in text for others to check even when screenshot helps illustrate the problem you are trying to diagnose.  So, here is your sample data: GroupA GroupB 353649273 353648649 353649184 353648566 353649091 353616829 353649033 353638941 353648797   353648680   353648745   353648730   353638941   From this dataset, it is easy to see that there is no match in any event. (One event is represented by one row.)  In addition to this, if you are going to compare GroupA and GroupB in their original names, there is no need to use foreach.  The logic expressed in your SPL can easily be implemented with   eval match=if(GroupA=GroupB,GroupA ,null())   Two SPL pointers: 1) use eval function null() is more expressive AND does not spend CPU cycles to look for a nonexistent field name such as NULL; more importantly, 2) foreach operates on a each event (row) individually.  If there is no match within the same event, match will always receive null value. On the second point, @livehybrid makes a speculation of your real intent, which seems to be to seek not a match in individual events between string/numerical fields GroupA and GroupB, but to seek matches in the sets of all values of GroupA and all values of GroupB.  Is this the correct interpretation?  If so, your first logical mistake is to misinterpret the problem to be comparison within individual events. A second mistake you make is in problem statement. i have data from two columns and using a third column to display the matches Given that there is no same-event match, your intention of "using a third column to display the matches" becomes impossible for volunteers here to interpret.  @livehybrid made an effort to interpret your intention as "if any value in the set of all values of GroupA matches any values in the set of all values of GroupB, display the matching values in GroupA together with ALL values of GroupB. (As opposed to any specific values of GroupB.)"  The output from that code is GroupA GroupB match 353638941 353616829 353638941 353648566 353648649 1 Is this what you expect?  What if there are two distinct values in GroupA matching two values GroupB, should the column GroupA display the two matching values and the column GroupB still displaying the same five values? It all comes down to the four golden rules in asking questions in this forum that I call Four Commandments: Illustrate data input (in raw text, anonymize as needed), whether they are raw events or output from a search (SPL that volunteers here do not have to look at). Illustrate the desired output from illustrated data. Explain the logic between illustrated data and desired output without SPL. If you also illustrate attempted SPL, illustrate actual output and compare with desired output, explain why they look different to you if that is not painfully obvious. ... View more

You need to clarify your constraints.  The most obvious solution is to send a field "environment" along with log events.  There are a million ways to do this. Then, if the deployment team is sympathetic to your course, they can name hosts according to environment in some way.  There is at least a dozen ways to do this. (One obvious way is to dedicate a special domain to environment.)  So, that's at least 1,000,012. You can also do an automatic lookup on hostname.  That's at least 1,000,013 ways. ... View more

Get rid of that dedup host.  You will see some events with error_msg, some without.  I cannot decipher what that dedup is supposed to accomplish, or what real problem you are trying to solve.  So, I cannot suggest an alternative.  But if you have that dedup and if for each host the last event is NOT a failure or disconnect, you will get no error_msg.  Maybe you mean this? index=kafka-np sourcetype="KCON" connName="CCNGBU_*" ERROR=ERROR OR ERROR=WARN | eval error_msg = case(match(_raw, "Disconnected"), "disconected", match(_raw, "restart failed"), "restart failed", match(_raw, "Failed to start connector"), "failed to start connector") | search error_msg = * | dedup host | table host connName error_msg   ... View more

As many here will tell you, map is probably the wrong answer to the problem you are trying to solve.  That aside, you need to clarify what "not working", "problem ... with 'map' command" mean, how does "things" grey out, etc. (In which window, for example, do "things" grey out?) It is atypical to use a makeresults subsearch to produce search terms inside a map command.  But I think I get that return search should be interpreted verbatim by the compiler.   My wild guess is that the search inside the map command does not return any result.  Is this the case?  It seems that the map command can be simplified to | map maxsearches=100 search="search $time_token$ $index_token$ $rule$ | eval rule_found=\"$rule$\", rule_id=\"$id$\"" If this is semantically correct, try turning substitute select values of time_token, index_token and rule from earlier search, and perform the mapped search manually for diagnosis.  Hope this helps. (Still, there might be more elegant and less error-prone method than using map command to solve the problem you are attacking.) ... View more

This needs to search any group we want to include by specific name and then table out a list of the users that are members of each group sorted by the group name First, you need to illustrate how a user is represented in Splunk data.  Is memberOf already extracted into one string?   Second, you need to illustrate what is the form of that "specific name" by which you wish to use in Splunk search.  As your example LDAP search indicates, LDAP group is more than just CN, but a group of attributes strung together as a unique identifier.  Are you going to search only by CN? ... View more

You need to be more specific about your requirements.  Based on the sample you provided, what is the input and expected output of a query for sender?  What is the input and expected output of a query for recipients?  Are you combining given values of sender, recipients, sender's IP address in one query and expect some specific output?  Or are you expecting to give an input of a sender (email), and find out all recipients the sender has sent and the IP addresses this sender has used?  How does "X and S have the same values for given single message in the logs and will change from message to message" affect the outcome?  Is this information even relevant to your quest? (It didn't help that your sample data contains one X value and one S value.) There are a million different ways to interpret "query to pull sender (env_from value), recipient(s) (env_rcpt values) and IP address;" this, combined with dozens of ways to implement each interpretation, it is impossible for volunteers to help you. If you mean to say that a unique X, S combination marks one unique E-mail transaction, and you want to base your search on X and S values, all you need is from, ip, and rcpt.  Something like this:   | stats values(from) as sender values(ip) as ip values(rcpt) as recipients by s x   Your sample data should give s x sender ip recipients 44pnhtdtkf 44pnhtdtkf-1 sender@company.com 10.10.10.10 recipient.one@company.com recipient.two@DifferentCompany.net Is this what you are looking for? Here is an emulation of your sample.  Play with it and compare with real data.   | makeresults | eval data = split("Feb 11 10:04:12 host.company.com 2025-02-11T15:04:12.436109+00:00 host filter_instance1[1394]: rprt s=44pnhtdtkf m=1 x=44pnhtdtkf-1 mod=mail cmd=env_from value=sender@company.com size= smtputf8= qid=44pnhtdtkf-1 tls= routes= notroutes=tls_fallback host=host123.company.com ip=10.10.10.10 Feb 11 10:04:12 host.company.com 2025-02-11T15:04:12.438453+00:00 host filter_instance1[1394]: rprt s=44pnhtdtkf m=1 x=44pnhtdtkf-1 mod=mail cmd=env_rcpt r=1 value=recipient.two@DifferentCompany.net orcpt=recipient.two@DifferentCompany.NET verified= routes= notroutes=RightFax,default_inbound,journal Feb 11 10:04:12 host.company.com 2025-02-11T15:04:12.440714+00:00 host filter_instance1[1394]: rprt s=44pnhtdtkf m=1 x=44pnhtdtkf-1 mod=mail cmd=env_rcpt r=2 value=recipient.one@company.com orcpt=recipient.one@company.com verified= routes=default_inbound notroutes=RightFax,journal Feb 11 10:04:12 host.company.com 2025-02-11T15:04:12.446326+00:00 host filter_instance1[1394]: rprt s=44pnhtdtkf m=1 x=44pnhtdtkf-1 mod=session cmd=data from=sender@company.com suborg= Feb 11 10:04:12 host.company.com 2025-02-11T15:04:12.446383+00:00 host filter_instance1[1394]: rprt s=44pnhtdtkf m=1 x=44pnhtdtkf-1 mod=session cmd=data rcpt=recipient.two@DifferentCompany.net suborg= Feb 11 10:04:12 host.company.com 2025-02-11T15:04:12.446405+00:00 host filter_instance1[1394]: rprt s=44pnhtdtkf m=1 x=44pnhtdtkf-1 mod=session cmd=data rcpt=recipient.one@company.com suborg= Feb 11 10:04:12 host.company.com 2025-02-11T15:04:12.446639+00:00 host filter_instance1[1394]: info s=44pnhtdtkf m=1 x=44pnhtdtkf-1 mod=session cmd=data rcpt_routes= rcpt_notroutes=RightFax,journal data_routes= data_notroutes= Feb 11 10:04:12 host.company.com 2025-02-11T15:04:12.450566+00:00 host filter_instance1[1394]: info s=44pnhtdtkf m=1 x=44pnhtdtkf-1 mod=session cmd=headers hfrom=sender@company.com routes= notroutes= Feb 11 10:04:12 host.company.com 2025-02-11T15:04:12.455141+00:00 host filter_instance1[1394]: info s=44pnhtdtkf m=1 x=44pnhtdtkf-1 mod=mimelint cmd=getlint lint= Feb 11 10:04:12 host.company.com 2025-02-11T15:04:12.455182+00:00 host filter_instance1[1394]: info s=44pnhtdtkf m=1 x=44pnhtdtkf-1 mod=mimelint cmd=getlint mime=1 score=0 threshold=100 duration=0.000 Feb 11 10:04:12 host.company.com 2025-02-11T15:04:12.455201+00:00 host filter_instance1[1394]: info s=44pnhtdtkf m=1 x=44pnhtdtkf-1 mod=mimelint cmd=getlint warn=0", " ") | mvexpand data | rename data as _raw | extract ``` data emulation above ```   ... View more

First, thank you for illustrate sample events and clearly state desired output and the logic.  Before I foray into action, I'm deeply curious: Who is asking for this transformation in Splunk?  Your boss?  You be your own boss?  Homework?  If it's your boss, ask for a raise because semantic transformation is best done with real language transformers such as DeepSeek.😉  If it's homework, tell them they are insane. This said, I have done a lot of limited-vocabulary, limited-grammar transformations to satisfy myself.  The key to the solution is to study elements (both vocabulary and concepts) and linguistic constraints.  Most limited-vocabulary, limited-grammar problems can be solved with lookups.  In my code below, I use JSON structure for this purpose but lookups are easier to maintain, and result in more readable code. (Using inline JSON has the advantage of reducing the amount of lookups, as you will see.)   | fillnull Status value=Success ``` deal with lack of Status in Logout; this can be refined if blanket success is unwarranted ``` | eval status_adverb = json_object("Success", "succeeded to ", "Failure", "failed to ") | eval action_verb = json_object("Login", "login from " . IPAddress . " (" . Location . ")", "Logout", "logout", "ProfileUpdate", "update " . lower(ElementUpdated), "ItemPurchase", "buy " . ItemName . " for " . Amount) | eval EventDescription = mvappend("User " . json_extract(status_adverb, Status) . json_extract(action_verb, ActionType), if(isnull(FailureReason), null(), "(" . FailureReason . ")")) | table _time SessionId ActionType EventDescription   Output from your sample data is _time SessionId ActionType EventDescription 2025-02-10 01:09:00 123abc Logout User succeeded to logout 2025-02-10 01:08:00 123abc ItemPurchase User failed to buy Item2 for 200.00 (Not enough funds) 2025-02-10 01:07:00 123abc ItemPurchase User succeeded to buy Item1 for 500.00 2025-02-10 01:06:00 123abc ProfileUpdate User failed to update password (Password too short) 2025-02-10 01:05:00 123abc ProfileUpdate User succeeded to update email 2025-02-10 01:04:00 123abc Login User succeeded to login from 10.99.99.99 (California) Here, instead of jumping between indefinite and adverb forms, I adhere to indefinite for both success and failure. Note: If the sample events are as you have shown, you shouldn't need to extract any more field.  Splunk should have extracted everything I referred to in the code.  Here is an emulation of the samples.  Play with it and compare with real data. (Also note that you misplaced purchase failure to the success event.  Below emulation corrects that.)   | makeresults | fields - _time | eval data = mvappend("2025-02-10 01:09:00, EventId=\"6\", SessionId=\"123abc\", ActionType=\"Logout\"", "2025-02-10 01:08:00, EventId=\"5\", SessionId=\"123abc\", ActionType=\"ItemPurchase\", ItemName=\"Item2\", Amount=\"200.00\", Status=\"Failure\", FailureReason=\"Not enough funds\"", "2025-02-10 01:07:00, EventId=\"4\", SessionId=\"123abc\", ActionType=\"ItemPurchase\", ItemName=\"Item1\", Amount=\"500.00\", Status=\"Success\"", "2025-02-10 01:06:00, EventId=\"3\", SessionId=\"123abc\" ActionType=\"ProfileUpdate\", ElementUpdated=\"Password\", NewValue=\"*******\", OldValue=\"***********\", Status=\"Failure\", FailureReason=\"Password too short\"", "2025-02-10 01:05:00, EventId=\"2\", SessionId=\"123abc\" ActionType=\"ProfileUpdate\", ElementUpdated=\"Email\", NewValue=\"NewEmail@somenewdomain.com\", OldValue=\"OldEmail@someolddomain.com\", Status=\"Success\"", "2025-02-10 01:04:00, EventId=\"1\", SessionId=\"123abc\", ActionType=\"Login\", IPAddress=\"10.99.99.99\", Location=\"California\", Status=\"Success\"") | mvexpand data | rename data as _raw | extract | rex "^(?<_time>[^,]+)" ``` data emulation above ```     ... View more

There are many ways to do this, but using if function is perhaps my last choice.  Try this:   | rex field=index "_(?<app_id>\w+?)_(?<environment>(non_)*prod)"   Here is an emulation for you to play with and compare with real data.   | makeresults format=csv data="index sony_app_XXXXXX_non_prod sony_app_XXXXXX_prod sony_app_123456_non_prod sony_app_xyzabc_prod" ``` the above emulates index = sony_* ```   Output from this emulation is app_id environment index app_XXXXXX non_prod sony_app_XXXXXX_non_prod app_XXXXXX prod sony_app_XXXXXX_prod app_123456 non_prod sony_app_123456_non_prod app_xyzabc prod sony_app_xyzabc_prod Hope this helps. ... View more

First, when posting type 2 which is in JSON, please use raw text.  Splunk's "syntax highlights" view is non-compliant and very difficult to process. (See the crazy rex in my emulation below; you also introduced additional syntax errors when attempting to simplify or anonymize.)  Also in type 2, you should preserve the uuid's value as that's the only key that distinguishes between the two.  For everyone's benefit, I'm posting reconstructed raw events from type 2:   { "@message": { "attributeContract": { "extendedAttributes": [ ], "maskOgnlValues": false, "uniqueUserKeyAttribute": "uuid" }, "attributeMapping": { "attributeContractFulfillment": { "uuid": { "source": { "type": "ADAPTER" }, "value": "9c5b94b1-35ad-49bb-b118-8e8fc24abf80" } }, "attributeSources": [ ], "issuanceCriteria": { "conditionalCriteria": [ ] } }, "configuration": { "fields": [ { "name": "Application ObjectClass", "value": "cartmanUser" }, { "name": "Application Entitlement Attribute", "value": "cartmanRole" }, { "name": "IAL to Enforce", "value": 2 } ], "id": "Cartman", "name": "Cartman" } }, "@timestamp": "2025-01-01T00:00:01.833685" } { "@message": { "attributeContract": { "extendedAttributes": [ ], "maskOgnlValues": false, "uniqueUserKeyAttribute": "uuid" }, "attributeMapping": { "attributeContractFulfillment": { "uuid": { "source": { "type": "ADAPTER" }, "value": "550e8400-e29b-41d4-a716-446655440000" } }, "attributeSources": [ ], "issuanceCriteria": { "conditionalCriteria": [ ] } }, "configuration": { "fields": [ { "name": "Application ObjectClass", "value": "cartmanUser" }, { "name": "Application Entitlement Attribute", "value": "cartmanRole" }, { "name": "IAL to Enforce", "value": 1 } ], "id": "Cartman", "name": "Cartman" } }, "@timestamp": "2025-01-02T00:00:01.833685" }   Like @bowesmana, I fail to see see the relevance of type 1.  Type 2 is all you need to produce the results you want.  I also don't see why you want to print two tables rather than printing one table with two rows (differentiated by UUID).  So, this is what I'm going to show. Actual code is pretty simple.  My main time was sunken in reconstruct valid JSON data from your pasted text.   | fields @message.attributeMapping.attributeContractFulfillment.uuid.value ``` ^^^ this line is just to declutter output ``` | spath path=@message.configuration.fields{} | eval restructured_fields = json_object() | foreach @message.configuration.fields{} mode=multivalue [eval restructured_fields = json_set(restructured_fields, json_extract(<<ITEM>>, "name"), json_extract(<<ITEM>>, "value"))] | spath input=restructured_fields   (This foreach syntax above requires Splunk 9.0.)  Output from the two reconstructed events is as follows: @message.attributeMapping.attributeContractFulfillment.uuid.value Application Entitlement Attribute Application ObjectClass IAL to Enforce 9c5b94b1-35ad-49bb-b118-8e8fc24abf80 cartmanRole cartmanUser 2 550e8400-e29b-41d4-a716-446655440000 cartmanRole cartmanUser 1 Does this satisfy your requirements? It is useful to print out the two intermediate JSON objects used in this search so you can clearly see dataflow: @message.configuration.fields{} restructured_fields { "name": "Application ObjectClass", "value": "cartmanUser" } { "name": "Application Entitlement Attribute", "value": "cartmanRole" } { "name": "IAL to Enforce", "value": 2 } {"Application ObjectClass":"cartmanUser","Application Entitlement Attribute":"cartmanRole","IAL to Enforce":2} { "name": "Application ObjectClass", "value": "cartmanUser" } { "name": "Application Entitlement Attribute", "value": "cartmanRole" } { "name": "IAL to Enforce", "value": 1 } {"Application ObjectClass":"cartmanUser","Application Entitlement Attribute":"cartmanRole","IAL to Enforce":1} @message.configuration.fields{}, of source, is extracted directly from raw data. Here is an emulation for you to play with and compare with real data type 2:   | makeresults | fields - _time | eval sourcetype = "type2", data = mvappend("{ [-] @message: { [-] attributeContract: { [-] extendedAttributes: [ [-] ] maskOgnlValues: false uniqueUserKeyAttribute: uuid } attributeMapping: { [-] attributeContractFulfillment: { [-] uuid: { [-] source: { [-] type: ADAPTER } value: 9c5b94b1-35ad-49bb-b118-8e8fc24abf80 } } attributeSources: [ [-] ] issuanceCriteria: { [-] conditionalCriteria: [ [-] ] } } configuration: { [-] fields: [ [-] { [-] name: Application ObjectClass value: cartmanUser } { [-] name: Application Entitlement Attribute value: cartmanRole } { [-] name: IAL to Enforce value: 2 } ] id: Cartman name: Cartman } } @timestamp: 2025-01-01T00:00:01.833685 }", "{ [-] @message: { [-] attributeContract: { [-] extendedAttributes: [ [-] ] maskOgnlValues: false uniqueUserKeyAttribute: uuid } attributeMapping: { [-] attributeContractFulfillment: { [-] uuid: { [-] source: { [-] type: ADAPTER } value: 550e8400-e29b-41d4-a716-446655440000 } } attributeSources: [ [-] ] issuanceCriteria: { [-] conditionalCriteria: [ [-] ] } } configuration: { [-] fields: [ [-] { [-] name: Application ObjectClass value: cartmanUser } { [-] name: Application Entitlement Attribute value: cartmanRole } { [-] name: IAL to Enforce value: 1 } ] id: Cartman name: Cartman } } @timestamp: 2025-01-02T00:00:01.833685 }") | rex field=data mode=sed "s/\[-]//g s/\n+([\w@])/\n\"\1/g s/([^\"]): (true|false|\d+\n)/\1\": \2/g s/([^\"]):(\W+\n)/\1\":\2/g s/([^\"]): (.+)/\1\": \"\2\"/g s/([\w\"}\]])\n([\"{\[])/\1,\n\2/g" | mvexpand data | rename data AS _raw | spath ``` data type 2 emulation above ``` (Can you see how crazy that rex command is?)   For completeness, this is how you extract data from type 1 in case it is of use to you:   | eval message = replace(message, "'", "") | spath input=message   message field should have been present at search type.  The result from your sample data is UserAccessSubmission.csp UserAccessSubmission.mail UserAccessSubmission.objectClass UserAccessSubmission.trackingId UserAccessSubmission.uuid sourcetype trackingid Butters sean@southpark.net cartmanUser tid:13256464 abc123 type1 tid:13256464 Butters sean@southpark.net cartmanUser tid:13256464 abc123 type1 tid:13256464 Butters sean@southpark.net cartmanUser tid:13256464 abc123 type1 tid:13256464 Butters sean@southpark.net StanUser tid:13256464 abc123 type1 tid:13256464 Butters sean@southpark.net StanUser tid:13256464 abc123 type1 tid:13256464 This is emulation of data type 1 used to extract the above.   | makeresults | fields - _time | eval sourcetype = "type1", data = split("2025-01-01 00:00:00,125 trackingid=\"tid:13256464\"message='{\"UserAccessSubmission\":{\"uuid\":\"abc123\",\"mail\":\"sean@southpark.net\",\"trackingId\":\"tid:13256464\",\"objectClass\":\"cartmanUser\",\"csp\":\"Butters\"}}' 2025-01-01 00:01:00,125 trackingid=\"tid:13256464\"message='{\"UserAccessSubmission\":{\"uuid\":\"abc123\",\"mail\":\"sean@southpark.net\",\"trackingId\":\"tid:13256464\",\"objectClass\":\"cartmanUser\",\"csp\":\"Butters\"}}' 2025-01-02 00:01:00,125 trackingid=\"tid:13256464\"message='{\"UserAccessSubmission\":{\"uuid\":\"abc123\",\"mail\":\"sean@southpark.net\",\"trackingId\":\"tid:13256464\",\"objectClass\":\"cartmanUser\",\"csp\":\"Butters\"}}' 2025-01-02 00:01:00,125 trackingid=\"tid:13256464\"message='{\"UserAccessSubmission\":{\"uuid\":\"abc123\",\"mail\":\"sean@southpark.net\",\"trackingId\":\"tid:13256464\",\"objectClass\":\"StanUser\",\"csp\":\"Butters\"}}' 2025-01-02 00:01:00,125 trackingid=\"tid:13256464\"message='{\"UserAccessSubmission\":{\"uuid\":\"abc123\",\"mail\":\"sean@southpark.net\",\"trackingId\":\"tid:13256464\",\"objectClass\":\"StanUser\",\"csp\":\"Butters\"}}'", " ") | mvexpand data | rename data AS _raw | extract ``` data type 1 emulation above ```     ... View more

As @ITWhisperer points out, neither substring or regex is the correct tool to extract information from structured data such as JSON.  I assume that that so-called "string" is not the entire event because otherwise Splunk would have automatically extracted role at search time.  Suppose you have an event like this (I'm quite convinced that you missed a comma between '00"' and '"name'.)   stuff before ... {"code":"1234","bday":"15-02-06T07:02:01.731+00:00", "name":"Alex", "role":"student","age":"16"} stuff after ...   What you do is to use regex to extract the part that is compliant JSON (not a portion of it), then use spath or fromjson to extract all key-value pairs. The following should usually work:   | rex "^[^{]*(?<json_message>{.+})" | spath input=json_message   That sample data will return age bday code json_message name role 16 15-02-06T07:02:01.731+00:00 1234 {"code":"1234","bday":"15-02-06T07:02:01.731+00:00", "name":"Alex", "role":"student","age":"16"} Alex student Here is an emulation for you to play with and compare with real data   | makeresults | eval _raw = "stuff before ... {\"code\":\"1234\",\"bday\":\"15-02-06T07:02:01.731+00:00\", \"name\":\"Alex\", \"role\":\"student\",\"age\":\"16\"} stuff after ..." ``` data emulation above ```   ... View more

Always, always describe your use case in terms of data.  Without data, it is very difficult for another person to understand what you are trying to achieve.  Let me give you a starter. Suppose your raw data is A B C _time 1 38 299 2025-02-04 23:59:00 2 36 296 2025-02-04 23:56:00 3 34 291 2025-02-04 23:51:00 4 32 284 2025-02-04 23:44:00 5 30 275 2025-02-04 23:35:00 6 28 264 2025-02-04 23:24:00 7 26 251 2025-02-04 23:11:00 8 24 236 2025-02-04 22:56:00 9 22 219 2025-02-04 22:39:00 10 20 200 2025-02-04 22:20:00 11 18 179 2025-02-04 21:59:00 12 16 156 2025-02-04 21:36:00 13 14 131 2025-02-04 21:11:00 14 12 104 2025-02-04 20:44:00 15 10 75 2025-02-04 20:15:00 16 8 44 2025-02-04 19:44:00 17 6 11 2025-02-04 19:11:00 18 4 -24 2025-02-04 18:36:00 19 2 -61 2025-02-04 17:59:00 20 0 -100 2025-02-04 17:20:00 21 -2 -141 2025-02-04 16:39:00 22 -4 -184 2025-02-04 15:56:00 23 -6 -229 2025-02-04 15:11:00 24 -8 -276 2025-02-04 14:24:00 25 -10 -325 2025-02-04 13:35:00 This mock sequence spans roughly three 4-hour intervals.  Now, if you bucket the sequence into 4-hour bins,   | bin _time span=4h@h   You get A B C _time 1 38 299 2025-02-04 20:00 2 36 296 2025-02-04 20:00 3 34 291 2025-02-04 20:00 4 32 284 2025-02-04 20:00 5 30 275 2025-02-04 20:00 6 28 264 2025-02-04 20:00 7 26 251 2025-02-04 20:00 8 24 236 2025-02-04 20:00 9 22 219 2025-02-04 20:00 10 20 200 2025-02-04 20:00 11 18 179 2025-02-04 20:00 12 16 156 2025-02-04 20:00 13 14 131 2025-02-04 20:00 14 12 104 2025-02-04 20:00 15 10 75 2025-02-04 20:00 16 8 44 2025-02-04 16:00 17 6 11 2025-02-04 16:00 18 4 -24 2025-02-04 16:00 19 2 -61 2025-02-04 16:00 20 0 -100 2025-02-04 16:00 21 -2 -141 2025-02-04 16:00 22 -4 -184 2025-02-04 12:00 23 -6 -229 2025-02-04 12:00 24 -8 -276 2025-02-04 12:00 25 -10 -325 2025-02-04 12:00 If you do stats/timechart on this, you get what you get. Now, what do you mean by "adjust the starting point of the spans"?  What will the bucketed sequence look like?  Give a concrete example using this dataset. You can reproduce the above sequence using the following code:   | makeresults count=25 | streamstats count as A | eval _time = strptime("2025-02-05", "%F") - 60 * A * A | eval B = 40 - 2 * A, C = 300 - A * A | bin _time span=4h@h     ... View more

As @PickleRick says, do not use append unless absolutely necessary.  I'd suggest a direct expression to do what you want: index=index1 OR (index=index2 sourcetype="api") | eval EventId = coalesce(EventId, Number__c) | stats count values(index) as indices by EventId | search count < 2 indices = index1   ... View more

You must have heard the phrase, time is of essence.  This is especially true in time series such as Splunk.  Could you start from the beginning and describe your use case?  What is the input, what is the expected output, and what is the logic between input and expected output without SPL? ... View more

To receive help in Splunk search, it is best to give more concrete information, even if you use mock names and values. Assuming the two different sources are sourcetype sourceA and sourceB.  The 3 parameters in sourceA are named "ID", "param2", and "param3".  Further assume that sourceB has the same field name "ID" to match that in sourceA, and that "actual name of the object" is in field named "name".  Assuming that all these fields are already extracted. sourcetype IN (sourceA, sourceB) | stats values(name) as name values(param2) as param2 values(param3) as param3 by ID   ... View more

Like @bowesmana says but do you really need the $env$ in the field name? Wouldn't  | stats dc(hostname) by host_environment make more sense in a dashboard?  ... View more

First, when illustrating structured data, please post compliant raw text.  In your case, a compliant JSON should be   { "application": "app1", "feature": "feature1", "timestamp": "01/29/2025 23:02:00 +0000", "users": [ { "userhost": "client1", "username": "user1" }, { "userhost": "client2", "username": "user2" } ] }   The trick here is to reach into the JSON array to perform mvexpand and ignore Splunk's default flattening of array.   | spath path=users{} | mvexpand users{} | spath input=users{}   Your sample data will give application feature timestamp userhost username users{} app1 feature1 01/29/2025 23:02:00 +0000 client1 user1 { "userhost": "client1", "username": "user1" } app1 feature1 01/29/2025 23:02:00 +0000 client2 user2 { "userhost": "client2", "username": "user2" } Here is an emulation for you to play with and compare with real data   | makeresults | eval _raw = "{ \"application\": \"app1\", \"feature\": \"feature1\", \"timestamp\": \"01/29/2025 23:02:00 +0000\", \"users\": [ { \"userhost\": \"client1\", \"username\": \"user1\" }, { \"userhost\": \"client2\", \"username\": \"user2\" } ] }" | spath ``` data emulation above ```     ... View more

Your question seems to be really about extracting user ID.  In other words, given the illustrated data, you want _raw restoftext trx user 10 1/17/2025 15:03:20 account record user100 does not exist account record user100 does not exist 10 user100 12 1/17/2025 15:03:20 login as admin login as admin 12 admin  Is this correct? The above can be achieve by regex.  But you have to enumerate all those login events and write alternative expressions.  For these two,   | rex "^(?<trx>\d+)\s+\S+\s+\S+\s+(?<restoftext>.+)" | rex field=restoftext "(login as|account record) (?<user>\S+)"   Here is an emulation for you to play with and compare with real data   | makeresults format=csv data="_raw 10 1/17/2025 15:03:20 account record user100 does not exist 12 1/17/2025 15:03:20 login as admin, raising privileges" ``` data emulation above ```   Hope this helps. ... View more

Given that a transaction_id would either not exist if a user never calls service 1, or it doesn't matter to your problem, service1_status is superfluous.  Is this correct?  I also do not see service URL as part of required output.  As such, @gcusello's solution can be further simplified.  More than that, I'm not sure if service is an existing field.  On the other hand, the two services are probably logged into different sources or different sourcetypes or both different.  I will assume that service 1 logs into source1 and service 2 logs into source 2.   source IN (source1, source2) | stats dc(source) AS service_count BY transaction_id | eval status1 = "yes", status2=if(service_count > 1,"yes","no") | table transaction_id status1 status2     ... View more