I am trying to make this query work:
index="main" | eval host=asset_id | collect index="scanned_app"
where asset_id
is a field, not a static value.
Two observations regarding the query:
- without | collect ...
, the search shows data as i expect it - with the meta-field host
changed
- with | collect ...
, the resulting index carries host
unchanged from the main
index
Q: how do i change the host
, so that it can be persisted in another index ?
index="main" | eval *magic_here* | collect index="scanned_app"
You can try and wrap you search in the map command that dynamically let's you generate another search.
This generates an event in the summary index with host=hello set from the outer search.
|makeresults count=1
| eval asset_id="hello"
| map search="
search
index=\"main\"
| collect index=scanned_app host=$asset_id$
"
j
This will work. Remember that the map command, by default, is limited to only 10 sub-search iterations. Use the option maxsearches=10000 or something more appropriate for your data set.
Converting the above to your actual search, see below. You probably don't need "_time=$orig_time$," in the eval.
index="main"
| eval orig_time=_time, orig_raw=_raw
| map maxsearches=10000 search="|makeresults=1
|eval _time=$orig_time$, _raw=$orig_raw$
| collect index="scanned_app" sourcetype=X host=$asset_id$"
@jbjerke_splunk and @pwild_splunk thank you for comments. could you perhaps help me understand why the SPL index="main" | eval host=asset_id | collect index="scanned_app"
works without |collect...
and does not work with |collect...
? What is happening during |collect...
?
To understand why this doesn't work as you're expecting you have to understand how the collect command works. When you pipe a search result into collect, it dumps the output of the command into a text file on your splunk server, which is then picked up by a monitor input for indexing in the same way as any other input. Just like when configuring a monitor input, you can specify the host field once for the input, you can't set it on an event by event basis. With collect, when you define fields like index=A, sourcetype=B, host=C you are defining them in the same way you would in an inputs.conf. Those fields are applied to the output for processing by Splunk's data pipeline.
@mushkevych -Since host is a default field , and collect command will look for default fields for source sourcetype host unless you override it in collect command
@Vijeta thank you for reply. Perhaps you can advise how to override default field host
in collect
command?
Since you want host value to be assigned to a variable asset_id , you will have to use map command as mentioned by @pwild_splunk
Try this
index="main" | eval _raw=_raw.",host=".asset_id | collect index="scanned_app"
This may not do what you want. The events in the summary index will contain a host field that is multi-valued, containing the indexed host field as well as the auto-extracted host value. If the purpose of this is to create a dashboard or graph, you may be able to work with the data by removing the first value with something like this.
| eval host=mvindex(host,1)
can you elaborate? is asset_id
a field or a static value?
also, what is it that you are trying to accomplish? i sense lookup
will serve you better here
asset_id
is a field.
my goal is to transform the data set by changing host
value and persist it in another index.
P.S. Updated the question to reflect your comment