Splunk Search

Substring with value pairs, using format but remove variable names

zolo
Loves-to-Learn Lots

Hi,

I have mail server logs where each mail has the MID number as identifier (for that mailserver =host, for that day)
MID 1234567 From:  someone1@domain.do
MID 1234567 To: someone3@gmail.com
MID 1234567 Subject: ...
MID 1234567 .... 
I'm trying to find the To with the subsearch and extract the host and MID values. 
For using MID only it working perfectly, however it is not fail safe (it might happen that more than one mail server might have the exact same MID on the same day)

 

index="mail" "MID" [search index="mail" "MID" ("someone1@domain.do" OR "someone2@othdomain.nu")|rex "MID (?<MID>\d+)"|dedup MID|fields + MID|rename MID as query]

 

This works perfectly. Now I wanted to add the host variable for get string pairs to search for. Important that I want the result as string without variable names:

This is what I've tried: 

 

index="mail" "MID" [search index="mail" "MID" ("someone1@domain.do" OR "someone2@othdomain.nu")|rex "MID (?<MID>\d+)"|dedup host MID|fields host MID|format "(" "(" "AND" ")" "OR" ")"|rename output as query]

 

EDIT/REMARK: I've tried to combine the "host" and "MID" variables into "output" in some way, but it just did not work. that is the reason for this non-functioning rename at the end.. 

However seems the variable names are there. Could you please help how to remove both variable names or at least for the "MID" ? (Interested in both solution, but any good solution is perefectly fine)
EDIT1: Checking the inner search result, because the whole search just not working due to this problem.

EDIT2:
Have tried to parse the whole output line to variable then replace with either "rex mode=sed" or with "replace" in two way, however seems I can't get the formatted output to variable anymore. (this is the inner query only)

 

index="mail" "MID" [search index="mail" "MID" ("someone1@domain.do" OR "someone2@othdomain.nu")|rex "MID (?<MID>\d+)"|dedup host MID|fields host MID|format "(" "(" "AND" ")" "OR" ")"|rex "^(?<output>.*$)"|eval output=replace(output,"MID=","") |rename output as query

 

 

Labels (1)
0 Karma

isoutamo
SplunkTrust
SplunkTrust

Hi

with one field you get ride of field name as "rename <field> as search". Have you try this with as "rename host as search, MID as search"? Probably it don't like that you rename two field to the same name, but maybe it helps if you just use "rename MID as search"?

r. Ismo

0 Karma

zolo
Loves-to-Learn Lots

>Have you try this with as "rename host as search, MID as search"? 

Replaced the end with 
|rename host as query,MID as query 
and I got: 
"Multiple renames to field 'query' detected. Only the last one will appear, and previous 'from' fields will be dropped."

Also the result (checked only the inner search result) seems does not drop anything. (Using "as query" instead of "as search".  "as search" it is not working in this splunk version)

>Probably it don't like that you rename two field to the same name, but maybe it helps if you just use "rename MID as search"?
When I use only 1 variable (see my first example in my original post) then rename MID as query works perfectly. Not sure what exactly you mean try to include only one. Then I lose the other one, and I won't need format. And my problem won't be solved. 

0 Karma
Get Updates on the Splunk Community!

.conf24 | Day 0

Hello Splunk Community! My name is Chris, and I'm based in Canberra, Australia's capital, and I travelled for ...

Enhance Security Visibility with Splunk Enterprise Security 7.1 through Threat ...

(view in My Videos)Struggling with alert fatigue, lack of context, and prioritization around security ...

Troubleshooting the OpenTelemetry Collector

  In this tech talk, you’ll learn how to troubleshoot the OpenTelemetry collector - from checking the ...