Aws dns field extraction

[martinnepolean]

We are trying to do field extraction of the aws dns events, currently we are getting the events with below indexname, source and sourcetype

index = aws-cloudtrail source = us-east-1:/aws/route53/ins.company.com:Z7ZW0F4AW5AIB/IAH50-C3 sourcetype = aws:cloudwatch

I have created props and transforms as separate app for field extraction but it is not working

cat props.conf - for some reason * is not showing in this editor Its (source::asterisk/aws/route53/asterisk/asterisk)
[source::/aws/route53//*]
REPORT-fields = AWS_DNS_route53

cat transforms.conf
[AWS_DNS_route53]
DELIMS = " "
FIELDS = "version","query_timestamp","hosted_zoneid","queryname","querytype","response_code","protocol","edge_location","ip_address","subnet"

_RAW

1.0 2020-01-07T13:13:00Z Z7ZW0F4AW5AIB ins.company.com AAAA NOERROR UDP KJS50-C3 2001:1890:1ff:8c7:124:10:98:184 -

[to4kawa]

| makeresults 
| eval _raw="version query_timestamp hosted_zoneid queryname querytype response_code protocol edge_location ip_address subnet
1. 2020-01-07T13:13:00Z Z7ZW0F4AW5AIB ins.company.com AAAA NOERROR UDP KJS50-C3 2001:1890:1ff:8c7:124:10:98:184 -"
| multikv forceheader=1

props.conf:

[source::*/aws/route53/*/*]
EXTRACT-aws_dns_route53 = (?<version>[^ ]+) (?<query_timestamp>[^ ]+) (?<hosted_zoneid>[^ ]+) (?<queryname>[^ ]+) (?<querytype>[^ ]+) (?<response_code>[^ ]+) (?<protocol>[^ ]+)

Hi, @martinnepolean
Why not try it in props.conf because it can be extracted neatly?

[martinnepolean]

[source::*/aws/route53/*/*]
EXTRACT-aws_dns_route53 = (?<version>[^ ]+) (?<query_timestamp>[^ ]+) (?<hosted_zoneid>[^ ]+) (?<queryname>[^ ]+) (?<querytype>[^ ]+) (?<response_code>[^ ]+) (?<protocol>[^ ]+) (?<edge_location>[^ ]+) (?<ip_address>[^ ]+) (?<subnet>[^ ]+)

Tried above props.cong but not working

[to4kawa]

[source::us-east-1:/aws/route53/ins.company.com:Z7ZW0F4AW5AIB/IAH50-C3]

Since the behavior of the asterisk is unknown, why not write it directly once?

[martinnepolean]

Only /aws/route53/ is common in the source and others will change. My props and transforms.conf are working in my test env where I manually feed the raw event in a text file and ingest into splunk. I am facing this issue in prod where we are getting data from sqs based s3 using aws addon.

[to4kawa]

I see.
I don't know much. I'm sorry.

[martinnepolean]

Thanks, it is app permission issue and my props and transforms.conf is working