Splunk Search

Metadata fields of custom search command

ays7abt
New Member

Hi guys,

could you give me a documentation of the metadata fields of the custom search command?
Im searching for something like:
- generating: true,
- type: "streaming", "generating" and so on

Thanks guys

update: Got a answer from Splunk support.
The metadata JSON object sent in reply by the external command during the getinfo exchange must have the following structure. Some fields are optional (as indicated below).

{

    "type": <string>,

    "generating": <bool>,

    "required_fields": [ <string>, ... ],

    "maxwait": <number>,

    "streaming_preop": <string>,

    "finished": <bool>,

    "error": <string>,

    "inspector": {

        "messages": [

            [ <string>, <string> ],

            ...

        ]

    }

}



Field descriptions:

    type: <string>

    - Optional. Defaults to "streaming".

    - Must be one of "streaming", "stateful", "events",

      "reporting". See "Types of Commands" below for details

      on the semantic differences between these types of

      commands.

    generating: <bool>

    - Optional. Defaults to false.

    - Whether or not this command is a "generating" command.

    - Generating commands create search results (rather than

      processing the results of a preceding base search). A

      generating command must be the first command in an SPL

      search string.

    - After the getinfo exchange, Splunk will send generating

      commands "execute" commands with an empty result body.

    required_fields: [ <string>, ... ]

    - Optional. Defaults to not set.

    - This is a list of fields that the command requires as

      input. This list is used to drive field extraction

      during the search.

    - If required_fields is set and this command is a

      "streaming" or "stateful" command, Splunk will send only

      the specified fields to the external process in

      subsequent "execute" chunks, but the output of the

      external process will be merged with all remaining

      fields. See "Selected Fields" below for more detail.



    - If an external command needs all fields, or doesn't know

      what fields it needs, use: [ "*" ]

    - Accurately specifying required_fields can result in

      better search performance than using [ "*" ].

    maxwait: <number>

    - Optional. Defaults to 0.

    - The maximum time (in seconds) that Splunk will wait for

      the external process to produce output on standard

      out. If this time is exceeded, Splunk will terminate the

      search.

    - This can be used to detect external search commands that

      hang.

    - A value of 0 means to wait forever.

    streaming_preop: <string>

    - Optional. Defaults to not set.

    - For non-streaming commands (e.g. stateful, events,

      reporting), this is an SPL snippet ("preop") that will

      get prepended to this search command before Splunk

      executes it. This allows external search commands a

      limited ability to rewrite SPL strings.

    - The preop must be composed only of streaming commands

      (e.g. eval, where, etc.).

    - The preop may execute at the Indexers.

    - The preop may be a streaming external search command,

      which allows external search command developers to

      author "map-reduce" style search commands.

    error: <string>

    - Optional. Defaults to not set.

    - If error given, Splunk will display the given string to

      the user as an error in Splunk Web and terminate the

      search.

    - Only supported in Splunk 6.4.0 or later.

    inspector.messages: [ [ <string>, <string> ], ... ]

    - Optional. Defaults to not set.

    - This field allows external search commands to return

      user-visible messages related to a search.

    - The value must be a list of [ <level>, <message> ] pairs

      where <level> is one of "ERROR", "WARN", "INFO", or

      "DEBUG", and <message> is the string to be displayed to

      the user
0 Karma

ays7abt
New Member

Update: the Splunk support helped me out with a snipped of a documentation.

{
            "type": <string>,
            "generating": <bool>,
            "required_fields": [ <string>, ... ],
            "maxwait": <number>,
            "streaming_preop": <string>,
            "finished": <bool>,
            "error": <string>,
            "inspector": {
                "messages": [
                    [ <string>, <string> ],
                    ...
                ]
            }
        }

        Field descriptions:

            type: <string>
            - Optional. Defaults to "streaming".
            - Must be one of "streaming", "stateful", "events",
              "reporting". See "Types of Commands" below for details
              on the semantic differences between these types of
              commands.

            generating: <bool>
            - Optional. Defaults to false.
            - Whether or not this command is a "generating" command.
            - Generating commands create search results (rather than
              processing the results of a preceding base search). A
              generating command must be the first command in an SPL
              search string.
            - After the getinfo exchange, Splunk will send generating
              commands "execute" commands with an empty result body.

            required_fields: [ <string>, ... ]
            - Optional. Defaults to not set.
            - This is a list of fields that the command requires as
              input. This list is used to drive field extraction
              during the search.
            - If required_fields is set and this command is a
              "streaming" or "stateful" command, Splunk will send only
              the specified fields to the external process in
              subsequent "execute" chunks, but the output of the
              external process will be merged with all remaining
              fields. See "Selected Fields" below for more detail.

            - If an external command needs all fields, or doesn't know
              what fields it needs, use: [ "*" ]
            - Accurately specifying required_fields can result in
              better search performance than using [ "*" ].

            maxwait: <number>
            - Optional. Defaults to 0.
            - The maximum time (in seconds) that Splunk will wait for
              the external process to produce output on standard
              out. If this time is exceeded, Splunk will terminate the
              search.
            - This can be used to detect external search commands that
              hang.
            - A value of 0 means to wait forever.

            streaming_preop: <string>
            - Optional. Defaults to not set.
            - For non-streaming commands (e.g. stateful, events,
              reporting), this is an SPL snippet ("preop") that will
              get prepended to this search command before Splunk
              executes it. This allows external search commands a
              limited ability to rewrite SPL strings.
            - The preop must be composed only of streaming commands
              (e.g. eval, where, etc.).
            - The preop may execute at the Indexers.
            - The preop may be a streaming external search command,
              which allows external search command developers to
              author "map-reduce" style search commands.

            error: <string>
            - Optional. Defaults to not set.
            - If error given, Splunk will display the given string to
              the user as an error in Splunk Web and terminate the
              search.
            - Only supported in Splunk 6.4.0 or later.

            inspector.messages: [ [ <string>, <string> ], ... ]
            - Optional. Defaults to not set.
            - This field allows external search commands to return
              user-visible messages related to a search.
            - The value must be a list of [ <level>, <message> ] pairs
              where <level> is one of "ERROR", "WARN", "INFO", or
              "DEBUG", and <message> is the string to be displayed to
              the user.
0 Karma

somesoni2
Revered Legend
0 Karma

ays7abt
New Member

No, this doc is more for V1. I Answered with the Splunk support answer

0 Karma
Get Updates on the Splunk Community!

Splunk Custom Visualizations App End of Life

The Splunk Custom Visualizations apps End of Life for SimpleXML will reach end of support on Dec 21, 2024, ...

Introducing Splunk Enterprise 9.2

WATCH HERE! Watch this Tech Talk to learn about the latest features and enhancements shipped in the new Splunk ...

Adoption of RUM and APM at Splunk

    Unleash the power of Splunk Observability   Watch Now In this can't miss Tech Talk! The Splunk Growth ...