Splunk Search

Add a comment to a search?

Motivator

I'm working on a really large search right now (on the order of 35 lines long). Is there a good way to insert a comment into a search query to remind a future search editor what is going on?

There doesn't seem to be a | comment command.

perhaps | rex field=bogus "This could be a comment" ?

1 Solution

Engager

Updated Answer for Splunk 6.5.0+

`comment("THIS IS A COMMENT")` 

http://docs.splunk.com/Documentation/Splunk/6.5.0/Search/Addcommentstosearches

Example from the documentation :

source=usgs `comment("source is the us geological service (usgs)")`
| eval Description=case(depth<=70, "Shallow", depth>70 AND depth<=300, "Mid", 
  depth>300, "Deep") 
  `comment("Creates field Description. Case function specifies earthquake 
  depths, returns Description values - Shallow, Mid, Deep.")`
| stats count min(mag) max(mag) `comment("Counts earthquakes, displays min 
  and max magnitudes")` BY Description 

View solution in original post

Engager

Updated Answer for Splunk 6.5.0+

`comment("THIS IS A COMMENT")` 

http://docs.splunk.com/Documentation/Splunk/6.5.0/Search/Addcommentstosearches

Example from the documentation :

source=usgs `comment("source is the us geological service (usgs)")`
| eval Description=case(depth<=70, "Shallow", depth>70 AND depth<=300, "Mid", 
  depth>300, "Deep") 
  `comment("Creates field Description. Case function specifies earthquake 
  depths, returns Description values - Shallow, Mid, Deep.")`
| stats count min(mag) max(mag) `comment("Counts earthquakes, displays min 
  and max magnitudes")` BY Description 

View solution in original post

Splunk Employee
Splunk Employee

A macro with the following definition would be the best option

[comment(1)]
iseval=1
definition=""
args=text
description=Throw away comment text

This allow the comment to be inserted anywhere in the search where it will always be expanded into the empty string (without quotes). We use the eval based definition so that we can specify the empty string explicitly.

Splunk Employee
Splunk Employee

UPDATE
As of 6.5.0, there is a built-in macro that you can use to add comments to your search strings, instead of creating the macro yourself. Here is a link to the documentation: http://docs.splunk.com/Documentation/Splunk/6.5.0/Search/Addcommentstosearches

OLDER COMMENT
This method provides a lot of flexibility and is now documented in the Search Manual:
http://docs.splunk.com/Documentation/Splunk/6.3.0/Search/Addcommentstosearches

Splunk Employee
Splunk Employee

Which one is better use for performance for comment ? Eval comment or rename comment?

0 Karma

Splunk Employee
Splunk Employee

FYI :

in 6.5.0 there is now a built in macro that you can invoke in your searches, as documented here:
http://docs.splunk.com/documentation/splunk/6.5.0/search/addcommentstosearches

0 Karma

Splunk Employee
Splunk Employee

rename is better than eval

@ewoo suggests that macros are the best however

Splunk Employee
Splunk Employee

Here is a link to using the new built-in macro (as of 6.5.0) to add comments:

http://docs.splunk.com/Documentation/Splunk/6.5.0/Search/Addcommentstosearches

0 Karma

Splunk Employee
Splunk Employee

By using a macro, I think it's possible to get the benefits of both the "eval method" (inline comments near the relevant parts of the search string) and the "rename method" (no per-event search-time performance overhead).

In macros.conf:

[comment(1)]
args = text
definition = noop
description = Throw away comment text.

Usage:

| table ...
| `comment("first comment")`
| fields ...
| `comment("second comment")`

Communicator

A note on the usage provided, you can actually use this for more than just a single "line". I frequently will add in |comment(" in my search and then insert the ") at the end. This allows you to debug your SPL line by line so you can work out any kinks. For instance, I search I worked on recently:

index=mail tag=expectedclients
| eval window=now()-
indextime
| where window < 300
| join type=left msgId
[search index=phish
| eval new=mvjoin(url,"; ")
| eval domain=mvdedup(domain)
| eval newdom=mvjoin(domain,"; ")
| rename From AS atkfrom,Subject AS atksubject,message AS atkbody, new AS atkurl, newdom AS atkdomain,srcip AS atksrcip,fullreceived AS atkreceiveheaders
| fields atk
from,atksubject,atkbody,atkurl,atkdomain,atksrcip,atkreceiveheaders]
| comment("
| fields To,client,msgId,Date,attachment,md5_only,sha256_only,atk_from,atk_subject,atk_body,atk_url,atk_domain,atk_src_ip,atk_receive_headers
| eval payload_type=case(attachment!="NULL","file",atk_url!="NULL","url",1==1,"none")
| iplocation atk_src_ip
| rename City AS atk_city,Country AS atk_country, Region AS atk_region
| rex field=atk_url mode=sed "s/http/hxxp/g"")

As you can see, I insert the beginning of the comment where I want my search to end, and finish it with the close of the comment. This way I can move the comment macro down line by line to make sure I understand what's going on at every intermediate step without removing the rest of the search.

0 Karma

Builder

Cool beans!

0 Karma

Splunk Employee
Splunk Employee

How would this compare, performance wise, to a custom search command?

0 Karma

Splunk Employee
Splunk Employee

To my knowledge, in 6.2.x and prior, running a custom search command requires a Python process to be forked. I'd expect use of a macro to be significantly faster, especially for search strings containing multiple inline comments.

New Member

As an argument to the search, add e.g.

NOT xcomment="This is a comment"

where no field named "xcomment" exists. Comments can be added further down the search by inserting a further "search" command.

Not sure of the performance impact, but it should be small, as it just involves testing for the existence in the data of a field named e.g. xcomment.

0 Karma

Path Finder

I'd like to make comments an enhancement request for Splunk so that comments could be placed throughout the search without affecting it from the current pipe through the next pipe, both to disable portions of the search that aren't currently being used and to allow comments to be placed inline in the search. Any of these formats would be sensible:
|comment
|rem
|#
or even
|<!-- comment -->|
for instance:

index=main source=df
|rex field=raw "(?\w\S)\shas\s(?\d{1,2})\%\sfree" maxmatch=10
| eval disk-pctfree = mvzip(disk, pctfree) | mvexpand disk-pctfree |fields host, disk-pctfree | rex field=disk-pctfree "(?\w\S),(?\d{1,2})" |stats min(pctfree) by host, disk | sort by min(pctfree) | rename min(pctfree) as "Minimum % Free"
| search "Minimum % Free"<11

|comment begin exclusions
|search NOT ( host=hostname1 AND disk=D: )
|search NOT ( host=hostname2 AND disk=D: )
|search NOT ( host=hostname3 AND disk=C: )
|comment use this method to set an alternate minimum: search NOT ( host=hostname4 AND disk=E: AND "Minimum % Free">5 )

Path Finder

The html style comment did not parse properly. It shows up as two pipes instead of pipe less than bang dash dash comment dash dash greater than pipe.

0 Karma

New Member

If the comment supposed to be always the same per category : the best thing seems to create a lookup on a field (like error field), that has 2 columns "error", "comment" and apply the lookup at the end of your search to add the comment. Then you just have to maintain the lookup table.

0 Karma

Builder

I complained to my SE about this. He sugested:

| eval commnet="This is a comment"

Builder

The rename looks better...

0 Karma

Splunk Employee
Splunk Employee

This would be wasteful for large result sets, as it would create a 'comment' field for each result.

0 Karma

Motivator

We use a SVN repository to document all our Splunk queries we have in production.

0 Karma