topic Re: Easiest method to extend pretrained sourcetype access_common to support X-forward-for? in Getting Data In

Easiest method to extend pretrained sourcetype access_common to support X-forward-for?

rune_hellem — Fri, 16 Aug 2019 11:57:09 GMT

The events indexed via Syslog and stripped for the prefixed date/time using SEDCMD is finally indexed by Splunk like this

192.777.77.77  95.66.66.66 - - [16/Aug/2019:11:34:50.962 +0000] "GET /favicon.ico HTTP/1.1" 404 6144

Ok, it is a slight diff from what I see here https://docs.splunk.com/Documentation/Splunk/7.3.1/Data/Listofpretrainedsourcetypes, the example does not have the X-forward-for address

10.1.1.140 - - [16/May/2005:15:01:52 -0700] "GET /themes/ComBeta/images/bullet.png HTTP/1.1" 404 304

Question is how do I extend access_common to support source ip and x-forward-for and also indexing the other fields as well?

Re: Easiest method to extend pretrained sourcetype access_common to support X-forward-for?

somesoni2 — Fri, 16 Aug 2019 12:21:55 GMT

You can create your custom sourcetype by taking definition of pretained sourcetype access_common (you can use btool to find the config for pretained sourcetype e.g. line breaker, timestamp parsing and field extraction ) and use it to define your custom sourcetype with your modifications. Then update your data inputs to use this custom sourcetype explicitly.

Re: Easiest method to extend pretrained sourcetype access_common to support X-forward-for?

rune_hellem — Fri, 16 Aug 2019 12:55:31 GMT

splunk btool props list access_common

showed me

REPORT-access = access-extractions

pointed me to the file

splunk\etc\system\default\transforms.conf

And then I found some customizations a consultant had implemented for us years ago, so now I should try doing the same as him adding x-forward-for to my custom access_common

Re: Easiest method to extend pretrained sourcetype access_common to support X-forward-for?

somesoni2 — Fri, 16 Aug 2019 13:02:18 GMT

we do not update any file in splunk\etc\system\default\, so you'd create a new app (to be deployed in splunk/etc/apps) with a props.conf and transforms.conf. The props.conf will have definition of your custom sourcetype and will point to transforms.conf which will have your custom field extraction for your custom sourcetype.

Re: Easiest method to extend pretrained sourcetype access_common to support X-forward-for?

rune_hellem — Wed, 30 Sep 2020 01:50:12 GMT

This was how I ended up implemeting it - the first comment to my question by @somesoni2 actually pointed me to previous customizations done by a Splunk consultant.

As described in the comment

splunk btool props list access_common

showed me

 REPORT-access = access-extractions

pointed me to the file

 splunk\etc\system\default\transforms.conf

And then I found some customizations a consultant had implemented for us years ago, so now I should try doing the same as him adding x-forward-for to my custom access_common. This customization is done in an app "TA-KLP", and not in the default app of Splunk.

I created a custom sourcetype 'klp_access_common' and using the extract for access_common adjusted it to our own needs

REPORT-thread = extract-klp_access_common