Splunk Search

Substring with value pairs, using format but remove variable names

zolo
Loves-to-Learn Lots

Hi,

I have mail server logs where each mail has the MID number as identifier (for that mailserver =host, for that day)
MID 1234567 From:  someone1@domain.do
MID 1234567 To: someone3@gmail.com
MID 1234567 Subject: ...
MID 1234567 .... 
I'm trying to find the To with the subsearch and extract the host and MID values. 
For using MID only it working perfectly, however it is not fail safe (it might happen that more than one mail server might have the exact same MID on the same day)

 

index="mail" "MID" [search index="mail" "MID" ("someone1@domain.do" OR "someone2@othdomain.nu")|rex "MID (?<MID>\d+)"|dedup MID|fields + MID|rename MID as query]

 

This works perfectly. Now I wanted to add the host variable for get string pairs to search for. Important that I want the result as string without variable names:

This is what I've tried: 

 

index="mail" "MID" [search index="mail" "MID" ("someone1@domain.do" OR "someone2@othdomain.nu")|rex "MID (?<MID>\d+)"|dedup host MID|fields host MID|format "(" "(" "AND" ")" "OR" ")"|rename output as query]

 

EDIT/REMARK: I've tried to combine the "host" and "MID" variables into "output" in some way, but it just did not work. that is the reason for this non-functioning rename at the end.. 

However seems the variable names are there. Could you please help how to remove both variable names or at least for the "MID" ? (Interested in both solution, but any good solution is perefectly fine)
EDIT1: Checking the inner search result, because the whole search just not working due to this problem.

EDIT2:
Have tried to parse the whole output line to variable then replace with either "rex mode=sed" or with "replace" in two way, however seems I can't get the formatted output to variable anymore. (this is the inner query only)

 

index="mail" "MID" [search index="mail" "MID" ("someone1@domain.do" OR "someone2@othdomain.nu")|rex "MID (?<MID>\d+)"|dedup host MID|fields host MID|format "(" "(" "AND" ")" "OR" ")"|rex "^(?<output>.*$)"|eval output=replace(output,"MID=","") |rename output as query

 

 

Labels (1)
0 Karma

isoutamo
SplunkTrust
SplunkTrust

Hi

with one field you get ride of field name as "rename <field> as search". Have you try this with as "rename host as search, MID as search"? Probably it don't like that you rename two field to the same name, but maybe it helps if you just use "rename MID as search"?

r. Ismo

0 Karma

zolo
Loves-to-Learn Lots

>Have you try this with as "rename host as search, MID as search"? 

Replaced the end with 
|rename host as query,MID as query 
and I got: 
"Multiple renames to field 'query' detected. Only the last one will appear, and previous 'from' fields will be dropped."

Also the result (checked only the inner search result) seems does not drop anything. (Using "as query" instead of "as search".  "as search" it is not working in this splunk version)

>Probably it don't like that you rename two field to the same name, but maybe it helps if you just use "rename MID as search"?
When I use only 1 variable (see my first example in my original post) then rename MID as query works perfectly. Not sure what exactly you mean try to include only one. Then I lose the other one, and I won't need format. And my problem won't be solved. 

0 Karma
Get Updates on the Splunk Community!

Announcing Scheduled Export GA for Dashboard Studio

We're excited to announce the general availability of Scheduled Export for Dashboard Studio. Starting in ...

Extending Observability Content to Splunk Cloud

Watch Now!   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to leverage ...

More Control Over Your Monitoring Costs with Archived Metrics GA in US-AWS!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...