Splunk Search

Substring with value pairs, using format but remove variable names

Loves-to-Learn Lots


I have mail server logs where each mail has the MID number as identifier (for that mailserver =host, for that day)
MID 1234567 From:  someone1@domain.do
MID 1234567 To: someone3@gmail.com
MID 1234567 Subject: ...
MID 1234567 .... 
I'm trying to find the To with the subsearch and extract the host and MID values. 
For using MID only it working perfectly, however it is not fail safe (it might happen that more than one mail server might have the exact same MID on the same day)


index="mail" "MID" [search index="mail" "MID" ("someone1@domain.do" OR "someone2@othdomain.nu")|rex "MID (?<MID>\d+)"|dedup MID|fields + MID|rename MID as query]


This works perfectly. Now I wanted to add the host variable for get string pairs to search for. Important that I want the result as string without variable names:

This is what I've tried: 


index="mail" "MID" [search index="mail" "MID" ("someone1@domain.do" OR "someone2@othdomain.nu")|rex "MID (?<MID>\d+)"|dedup host MID|fields host MID|format "(" "(" "AND" ")" "OR" ")"|rename output as query]


EDIT/REMARK: I've tried to combine the "host" and "MID" variables into "output" in some way, but it just did not work. that is the reason for this non-functioning rename at the end.. 

However seems the variable names are there. Could you please help how to remove both variable names or at least for the "MID" ? (Interested in both solution, but any good solution is perefectly fine)
EDIT1: Checking the inner search result, because the whole search just not working due to this problem.

Have tried to parse the whole output line to variable then replace with either "rex mode=sed" or with "replace" in two way, however seems I can't get the formatted output to variable anymore. (this is the inner query only)


index="mail" "MID" [search index="mail" "MID" ("someone1@domain.do" OR "someone2@othdomain.nu")|rex "MID (?<MID>\d+)"|dedup host MID|fields host MID|format "(" "(" "AND" ")" "OR" ")"|rex "^(?<output>.*$)"|eval output=replace(output,"MID=","") |rename output as query



Labels (1)
0 Karma



with one field you get ride of field name as "rename <field> as search". Have you try this with as "rename host as search, MID as search"? Probably it don't like that you rename two field to the same name, but maybe it helps if you just use "rename MID as search"?

r. Ismo

0 Karma

Loves-to-Learn Lots

>Have you try this with as "rename host as search, MID as search"? 

Replaced the end with 
|rename host as query,MID as query 
and I got: 
"Multiple renames to field 'query' detected. Only the last one will appear, and previous 'from' fields will be dropped."

Also the result (checked only the inner search result) seems does not drop anything. (Using "as query" instead of "as search".  "as search" it is not working in this splunk version)

>Probably it don't like that you rename two field to the same name, but maybe it helps if you just use "rename MID as search"?
When I use only 1 variable (see my first example in my original post) then rename MID as query works perfectly. Not sure what exactly you mean try to include only one. Then I lose the other one, and I won't need format. And my problem won't be solved. 

0 Karma
Get Updates on the Splunk Community!

Dashboard Studio Challenge - Learn New Tricks, Showcase Your Skills, and Win Prizes!

Reimagine what you can do with your dashboards. Dashboard Studio is Splunk’s newest dashboard builder to ...

Introducing Edge Processor: Next Gen Data Transformation

We get it - not only can it take a lot of time, money and resources to get data into Splunk, but it also takes ...

Take the 2021 Splunk Career Survey for $50 in Amazon Cash

Help us learn about how Splunk has impacted your career by taking the 2021 Splunk Career Survey. Last year’s ...