Can someone tell me what each part of this search does so I can learn more about the search processing language?

[Justin1224]

Could someone please tell me what this does? I'm in the process of learning Splunk and knowing what each part of this search does would really help me. Thank you so much!

| tstats `summariesonly` count from datamodel=Change_Analysis where nodename=All_Changes.Account_Management All_Changes.result="lockout" by All_Changes.src,All_Changes.Account_Management.src_nt_domain,All_Changes.user | sort 100 - count | `drop_dm_object_name("All_Changes")` |  `drop_dm_object_name("Account_Management")`

[somesoni2 · Accepted solution]

Read these

For understanding tstats portion: http://docs.splunk.com/Documentation/Splunk/6.4.3/SearchReference/Tstats (also see the section "An accelerated data model object")

For understanding sort : http://docs.splunk.com/Documentation/Splunk/6.4.3/SearchReference/Sort

Last two portions of the search are macro calling, so refer below for general search macro help.
http://docs.splunk.com/Documentation/Splunk/6.4.3/Knowledge/Usesearchmacros

For details on your macro, go to your Splunk web Settings->Advanced search -> Search macro, to find the definition of those macros.

[somesoni2]

Seems like we exceeded the amount/number of comment content under one post. staring a new thread here.

The splunk provides a web UI URL using which you can access Splunk. You don't need a local client of sort to access it. I would suggest to contact your Splunk admin/support people in your company on how to access Splunk.

I would urge you to read the Splunk documentation to understand what Splunk macros are and how they work. (included in my answer). These are custom Splunk code about which only it's creator/user will know.

For details on what by clause does and how it affects the result, first read the documentation for all Splunk Search command that support by clause like tstats, stats, chart etc (see examples as well). I would not write anything different than what these docs will say

[Justin1224]

I posted a comment earlier but it looks like it didn't go through. Just posting this to test if a comment will go through now.

[Justin1224]

Ok so I didn't clarify what I was asking very well. I meant to ask what the line IN the by clause did. Specifically, what this line does: All_Changes.src,All_Changes.Account_Management.src_nt_domain,All_Changes.user.

Also, I looked at the macros page a few times but I can't find how to actually make macros or what they're made of.

Thank you for all your help.

[Justin1224]

@somesoni2 Could you please help me understand what the line: All_Changes.src,All_Changes.Account_Management.src_nt_domain,All_Changes.user does in my search query. I've looked at the definitions for the terms in that line but I still can't figure out what the line does. Thank you very much.

[somesoni2]

These are the fields available in your data model. It's easier to understand if you could run the query and see the output yourself.

[Justin1224]

I don't have Splunk installed and I can't get in contact with the person who originally made the search query at the moment.

I know that you know the best way for me to understand what they do, which is to run the query, but I can't really do that.

If you could try and describe their purpose in layman's terms, that would really help me. I kind of understand what they do as a whole, they summarize the statistics that my search is finding, but how do they summarize them? And how do they work individually? For example, I found the definition of the function of "All_Changes.src" to be: "The resource where the change was originated", but I don't understand what that is doing in my search. What change is it talking about?

Thank you so much again for trying to help me.

[Justin1224]

Ok sorry one last question and then I will stop bugging you (sorry)

Do you have any guesses as to what the macros do in this particular search? Just the two at the end, the: drop_dm_object_name ones.

If I had at least a good guess for what they do I'm confident that I would have a good understanding of what the search query does, thanks to all of your help.

I just want to thank you for all your help, without your help, I wouldn't understand Splunk nearly as well.

[somesoni2]

Generally, It would be nearly impossible to guess what the macro does as we don't know it's code and how the parameter is being used. But I think I saw similar macro used in some Enterprise security app searches, and their it was renaming the fields. (drop_dm_object_name("All_Changes") will rename All_Changes.src as src , and All_Changes.Account_Management.src_nt_domain as Account_Management.src_nt_domain and All_Changes.user as user. Then drop_dm_object_name("Account_Management") will rename Account_Management.src_nt_domain as src_nt_domain)

[Justin1224]

Thank you so much for your help. I really appreciate it. This will help a lot.

[Justin1224]

Ok sorry, real last question:

What does the All_Changes.result="lockout" part do?

I'm certain that it constrains the search somehow by searching for the string "lockout" somehow, but I don't know how.

I found this definition for All_Changes.result: The vendor-specific result of a change, or clarification of an action status. For instance, status=failure may be accompanied by result=blocked by policy or result=disk full.
But I fully don't understand that.

Thank you for your help

[somesoni2]

All_Changes.result should a field/column in the data model node All_Changes (this is the data source you're querying. And yes, All_Changes.result="lockout" is a constraint/filter applied so that the aggregation (count) is done only on relevant events.

[Justin1224]

Thank you very much for answering. So from what I understand, is this an accurate description of what that search is doing?:

Search statistics only from tsidx files AND count from ONLY datamodels that are named “Change_Analysis” WHERE the object name is EQUAL to All_Changes.Account_Management. “All_Changes” is EQUAL to the string “lockout”. Then sort this filtered search into the top 100 results. Filter out objects named, “All_Changes”. Filter out objects named, “Account_Managenemnt”. Then display these results.

I'm really sorry for so many questions, it's just that I really need to know what this search is doing. Thank you again for your help.

[somesoni2]

The only thing I would add to description of the first section of the query (before sort) is that it's generating the statistics (count) group by fields mentioned in "by clause" (by All_Changes.src,All_Changes.Account_Management.src_nt_domain,All_Changes.user ).

Based on the name of the macro, it seems like it's doing filtering events, but how and based on which column, can only be known by seeing the macro definition.

[Justin1224]

Thank you again for helping me. The thing is, I don't think I have access to the macro definitions. I really only have a copy of the search query I posted in the question. Basically what I'm trying to do is translate that search query into an accurate description of what it does so that I can search the log files of my company's customers using the log file search software that my company uses. It has a different syntax, which is why I need to make the translation. So I was given that search query, which does what we want, but that's all we have. It was made some time ago and the person who made it isn't here.

Also, I don't fully understand what you mean regarding adding to the description. What fields are mentioned by the "by clause"? I found the definitions of "All_Changes.src", "Account_Management.src_nt_domain", and "All_Changes.user" but I couldn't find anything for just, "All_Changes". How does the period in "All_Changes.Account_Management.src_nt_domain" affect the search? Also, I tried putting the definitions together but I still can't figure out what that by clause does. Lastly, how do the commas affect that clause?

Any more help would be immensely appreciated.

[somesoni2]

By adding to the description I meant the description of what the search is doing.
The tstats query syntax that you've is like this

 | tstats OPTIONS count from SOURCE where FILTERS by GROUPING_FIELDS

So, The fields mentioned as part of GROUPING_FIELDS (comma separated list of fileds) section are the one by which your statistics are summarized. "All_Changes" should be a Root Object/search within your datamodel=Change_Analysis (In Splunk Web-> Settings -> Data Model) . And Account_Management seems like child object/search within node "All_Changes".

The by clause simply provides a way of grouping the results to provide the aggregation that you requested. So here, your search (first portion before sort) is provide count of events, group by/for each unique combination of fields "All_Changes.src, All_Changes.Account_Management.src_nt_domain, All_Changes.user " , for events which are satisfying your FILTERS condition.

[Justin1224]

Thank you so much that helped a lot. Last thing (hopefully), so there is no way to know what the macros at the bottom (drop_dm_object_name("All_Changes") | drop_dm_object_name("Account_Management") do? I don't actually have Splunk installed here. Also, if that's the case and we don't know what they do, does that mean we also don't knwo what 'summariesonly' does at the top? Because it is also enclosed by backticks and thus is a macro if I'm not mistaken. Unless my knowledge of macros is incorrect in which case, what would they do? Is drop_dm a command or function? And what would drop_dm_object_name("All_Changes") do then?

[somesoni2]

You're correct, the option summariesonly is a macro created by your Splunk administrator and my guess will be that it sets the option summariesonly of tstats command to true. (check the tstats link for more details on what this option does).

The macro names are custom (user defined), so drop_dm is just the naming convention following by the creator of the macro. Having a local (on your laptop/desktop/vm) splunk will not help you understand the macros that you see here. You would need to go to the Splunk instance where these exists and check the definition there for it's functionality.

I would ask the person who gave you the splunk search to also provide macro definitions.

[Justin1224]

So when you save a macro, where is it saved to? Some local file on my computer associated with Splunk? For example, am I the only one that can use macros that I create? Or is it saved online so that if you create a macro I can use it if I know what it's named and what it does?

I will look further into what summariesonly for tstats command does.

I know I'm repeating myself here, but drop_dm is a fuction/command created by someone - the person that gave it to me - and anyone could use that command now that it is created, as long as they know what it does?

Lastly, drop_dm_object_name is a command/function and the ("All_Changes") is an argument to that command/function that seems to be searching for the string "All_Changes" for the command/function drop_dm_object_name to do whatever it does for every instance of that string, correct?

Thank you again

[somesoni2]

In summariesonly , summaryiesonly is the name of the search macro.
In drop_dm_object_name("All_Changes"), drop_dm_object_name is the name of the search macro (it's a name so it could be anything) and string "All_Changes" is the argument.

The macro created is saved on Splunk server where it's created (in configuration files) and whether is available for others to use depends upon what permissions and scope is being set for that macro. As I listed in my answer, you can see the macros in Splunk Web UI in path Settings->Advanced search -> Search macro.