Getting Data In

Easiest method to extend pretrained sourcetype access_common to support X-forward-for?

rune_hellem
Contributor

The events indexed via Syslog and stripped for the prefixed date/time using SEDCMD is finally indexed by Splunk like this

192.777.77.77  95.66.66.66 - - [16/Aug/2019:11:34:50.962 +0000] "GET /favicon.ico HTTP/1.1" 404 6144

Ok, it is a slight diff from what I see here https://docs.splunk.com/Documentation/Splunk/7.3.1/Data/Listofpretrainedsourcetypes, the example does not have the X-forward-for address

10.1.1.140 - - [16/May/2005:15:01:52 -0700] "GET /themes/ComBeta/images/bullet.png HTTP/1.1" 404 304

Question is how do I extend access_common to support source ip and x-forward-for and also indexing the other fields as well?

0 Karma
1 Solution

rune_hellem
Contributor

This was how I ended up implemeting it - the first comment to my question by @somesoni2 actually pointed me to previous customizations done by a Splunk consultant.

As described in the comment

splunk btool props list access_common

showed me

 REPORT-access = access-extractions

pointed me to the file

 splunk\etc\system\default\transforms.conf

And then I found some customizations a consultant had implemented for us years ago, so now I should try doing the same as him adding x-forward-for to my custom access_common. This customization is done in an app "TA-KLP", and not in the default app of Splunk.

I created a custom sourcetype 'klp_access_common' and using the extract for access_common adjusted it to our own needs

REPORT-thread = extract-klp_access_common

View solution in original post

0 Karma

rune_hellem
Contributor

This was how I ended up implemeting it - the first comment to my question by @somesoni2 actually pointed me to previous customizations done by a Splunk consultant.

As described in the comment

splunk btool props list access_common

showed me

 REPORT-access = access-extractions

pointed me to the file

 splunk\etc\system\default\transforms.conf

And then I found some customizations a consultant had implemented for us years ago, so now I should try doing the same as him adding x-forward-for to my custom access_common. This customization is done in an app "TA-KLP", and not in the default app of Splunk.

I created a custom sourcetype 'klp_access_common' and using the extract for access_common adjusted it to our own needs

REPORT-thread = extract-klp_access_common
0 Karma

somesoni2
Revered Legend

You can create your custom sourcetype by taking definition of pretained sourcetype access_common (you can use btool to find the config for pretained sourcetype e.g. line breaker, timestamp parsing and field extraction ) and use it to define your custom sourcetype with your modifications. Then update your data inputs to use this custom sourcetype explicitly.

0 Karma

rune_hellem
Contributor
splunk btool props list access_common

showed me

REPORT-access = access-extractions

pointed me to the file

splunk\etc\system\default\transforms.conf

And then I found some customizations a consultant had implemented for us years ago, so now I should try doing the same as him adding x-forward-for to my custom access_common

0 Karma

somesoni2
Revered Legend

we do not update any file in splunk\etc\system\default\, so you'd create a new app (to be deployed in splunk/etc/apps) with a props.conf and transforms.conf. The props.conf will have definition of your custom sourcetype and will point to transforms.conf which will have your custom field extraction for your custom sourcetype.

0 Karma
Get Updates on the Splunk Community!

Detecting Brute Force Account Takeover Fraud with Splunk

This article is the second in a three-part series exploring advanced fraud detection techniques using Splunk. ...

Buttercup Games: Further Dashboarding Techniques (Part 9)

This series of blogs assumes you have already completed the Splunk Enterprise Search Tutorial as it uses the ...

Buttercup Games: Further Dashboarding Techniques (Part 8)

This series of blogs assumes you have already completed the Splunk Enterprise Search Tutorial as it uses the ...