Splunk Search

Aws dns field extraction

martinnepolean
Explorer

We are trying to do field extraction of the aws dns events, currently we are getting the events with below indexname, source and sourcetype

index = aws-cloudtrail source = us-east-1:/aws/route53/ins.company.com:Z7ZW0F4AW5AIB/IAH50-C3 sourcetype = aws:cloudwatch

I have created props and transforms as separate app for field extraction but it is not working

cat props.conf - for some reason * is not showing in this editor Its (source::asterisk/aws/route53/asterisk/asterisk)
[source::/aws/route53//*]
REPORT-fields = AWS_DNS_route53

cat transforms.conf
[AWS_DNS_route53]
DELIMS = " "
FIELDS = "version","query_timestamp","hosted_zoneid","queryname","querytype","response_code","protocol","edge_location","ip_address","subnet"

_RAW

1.0 2020-01-07T13:13:00Z Z7ZW0F4AW5AIB ins.company.com AAAA NOERROR UDP KJS50-C3 2001:1890:1ff:8c7:124:10:98:184 -

0 Karma

to4kawa
Ultra Champion
| makeresults 
| eval _raw="version query_timestamp hosted_zoneid queryname querytype response_code protocol edge_location ip_address subnet
1. 2020-01-07T13:13:00Z Z7ZW0F4AW5AIB ins.company.com AAAA NOERROR UDP KJS50-C3 2001:1890:1ff:8c7:124:10:98:184 -"
| multikv forceheader=1

props.conf:

[source::*/aws/route53/*/*]
EXTRACT-aws_dns_route53 = (?<version>[^ ]+) (?<query_timestamp>[^ ]+) (?<hosted_zoneid>[^ ]+) (?<queryname>[^ ]+) (?<querytype>[^ ]+) (?<response_code>[^ ]+) (?<protocol>[^ ]+)

Hi, @martinnepolean
Why not try it in props.conf because it can be extracted neatly?

0 Karma

martinnepolean
Explorer
[source::*/aws/route53/*/*]
EXTRACT-aws_dns_route53 = (?<version>[^ ]+) (?<query_timestamp>[^ ]+) (?<hosted_zoneid>[^ ]+) (?<queryname>[^ ]+) (?<querytype>[^ ]+) (?<response_code>[^ ]+) (?<protocol>[^ ]+) (?<edge_location>[^ ]+) (?<ip_address>[^ ]+) (?<subnet>[^ ]+)

Tried above props.cong but not working

0 Karma

to4kawa
Ultra Champion
[source::us-east-1:/aws/route53/ins.company.com:Z7ZW0F4AW5AIB/IAH50-C3]

Since the behavior of the asterisk is unknown, why not write it directly once?

0 Karma

martinnepolean
Explorer

Only /aws/route53/ is common in the source and others will change. My props and transforms.conf are working in my test env where I manually feed the raw event in a text file and ingest into splunk. I am facing this issue in prod where we are getting data from sqs based s3 using aws addon.

0 Karma

to4kawa
Ultra Champion

I see.
I don't know much. I'm sorry.

0 Karma

martinnepolean
Explorer

Thanks, it is app permission issue and my props and transforms.conf is working

0 Karma
Get Updates on the Splunk Community!

.conf24 | Day 0

Hello Splunk Community! My name is Chris, and I'm based in Canberra, Australia's capital, and I travelled for ...

Enhance Security Visibility with Splunk Enterprise Security 7.1 through Threat ...

 (view in My Videos)Struggling with alert fatigue, lack of context, and prioritization around security ...

Troubleshooting the OpenTelemetry Collector

  In this tech talk, you’ll learn how to troubleshoot the OpenTelemetry collector - from checking the ...