Splunk Search

Extract aws service name from source field of metadata

Poojitha
Path Finder

Hi All,

I want to extract service name from sourcetype="aws:metadata" and source field.

Example : 434531263412:eu-central-1:elasticache_describe_reserved_cache_nodes_offerings

I am using this query :

 

 

index=* sourcetype=aws:metadata
| eval aws_service=mvindex(split(source,":"),2)
| rex field=aws_service "(?<aws_service>[^_]+)"
| table aws_service  source| dedup aws_service

 

 


Using this I will get result :  elasticache. But in case of "434531263412:us-west-2:nat_gateways" its just extracting nat. But it should be gateways. S

Similarly in 434531263412:eu-central-1:application_load_balancers, its extracting application.

I was thinking if we can check for the keyword and update the value. I want to add this in props.conf file so aws_service field gets created from source.

Please can anyone of you help me how can I achieve this ?

Regards,
PNV

Labels (3)
0 Karma

Poojitha
Path Finder

@richgalloway  : If dont use rex , it gets entire value i.e nat_gateways. I just want nat.

My requirement is it should just extract service name.

Example :
434531263412:us-west-2:lambda_functions it will be lambda_functions. This is straight forward.
But like in  : "434531263412:us-west-2:nat_gateways",  it should be gateways. 
434531263412:us-west-2:application_load_balancers,  it should be load_balancers

This is my requirement. 

0 Karma

richgalloway
SplunkTrust
SplunkTrust

To summarize:

434531263412:us-west-2:lambda_functions -> lambda_functions
434531263412:us-west-2:nat_gateways -> gateways
434531263412:us-west-2:application_load_balancers -> load_balancers

If this is correct then more information is needed.  What is the rule to use to determine how much of the service is to be used?

---
If this reply helps you, Karma would be appreciated.
0 Karma

Poojitha
Path Finder

@richgalloway  : Sorry I did not get what rule you are mentioning. Could you please be more clear on this ?

434531263412:us-west-2:lambda_functions -> lambda_functions
434531263412:us-west-2:nat_gateways -> gateways
434531263412:us-west-2:application_load_balancers -> load_balancers

yes , this is the requirement. In the above , right side values are the values from source field. I want to extract service name from this field value.

0 Karma

richgalloway
SplunkTrust
SplunkTrust

The requirements are inconsistent.  Sometimes everything after the second : is the service name; other times the service name follows the first _.  How is a computer to decide which method to use?

---
If this reply helps you, Karma would be appreciated.
0 Karma

Poojitha
Path Finder

@richgalloway  I agree to your point. I tried using case statement as well . Unfortunately its not working as expected. Do you know any other way to handle this ? That really helps me. I am also re-searching.

0 Karma

richgalloway
SplunkTrust
SplunkTrust

I can't help if I don't understand what the goal is.  Once we have a deterministic way to set the service name I may be able to help.

---
If this reply helps you, Karma would be appreciated.
0 Karma

richgalloway
SplunkTrust
SplunkTrust

The split function is extracting the desired field, but then rex reduces it to the part before the first underscore (_).  Remove the rex command and the query should work as expected.

In props..conf, add a transform that uses INGEST_EVAL

INGEST_EVAL = aws_service=mvindex(split(source,":"),2)

 

---
If this reply helps you, Karma would be appreciated.
0 Karma
Get Updates on the Splunk Community!

Optimize Cloud Monitoring

  TECH TALKS Optimize Cloud Monitoring Tuesday, August 13, 2024  |  11:00AM–12:00PM PST   Register to ...

What's New in Splunk Cloud Platform 9.2.2403?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.2.2403! Analysts can ...

Stay Connected: Your Guide to July and August Tech Talks, Office Hours, and Webinars!

Dive into our sizzling summer lineup for July and August Community Office Hours and Tech Talks. Scroll down to ...