The events indexed via Syslog and stripped for the prefixed date/time using SEDCMD is finally indexed by Splunk like this
192.777.77.77 95.66.66.66 - - [16/Aug/2019:11:34:50.962 +0000] "GET /favicon.ico HTTP/1.1" 404 6144
Ok, it is a slight diff from what I see here https://docs.splunk.com/Documentation/Splunk/7.3.1/Data/Listofpretrainedsourcetypes, the example does not have the X-forward-for address
10.1.1.140 - - [16/May/2005:15:01:52 -0700] "GET /themes/ComBeta/images/bullet.png HTTP/1.1" 404 304
Question is how do I extend access_common to support source ip and x-forward-for and also indexing the other fields as well?
This was how I ended up implemeting it - the first comment to my question by @somesoni2 actually pointed me to previous customizations done by a Splunk consultant.
As described in the comment
splunk btool props list access_common
showed me
REPORT-access = access-extractions
pointed me to the file
splunk\etc\system\default\transforms.conf
And then I found some customizations a consultant had implemented for us years ago, so now I should try doing the same as him adding x-forward-for to my custom access_common. This customization is done in an app "TA-KLP", and not in the default app of Splunk.
I created a custom sourcetype 'klp_access_common' and using the extract for access_common adjusted it to our own needs
REPORT-thread = extract-klp_access_common
This was how I ended up implemeting it - the first comment to my question by @somesoni2 actually pointed me to previous customizations done by a Splunk consultant.
As described in the comment
splunk btool props list access_common
showed me
REPORT-access = access-extractions
pointed me to the file
splunk\etc\system\default\transforms.conf
And then I found some customizations a consultant had implemented for us years ago, so now I should try doing the same as him adding x-forward-for to my custom access_common. This customization is done in an app "TA-KLP", and not in the default app of Splunk.
I created a custom sourcetype 'klp_access_common' and using the extract for access_common adjusted it to our own needs
REPORT-thread = extract-klp_access_common
You can create your custom sourcetype by taking definition of pretained sourcetype access_common (you can use btool to find the config for pretained sourcetype e.g. line breaker, timestamp parsing and field extraction ) and use it to define your custom sourcetype with your modifications. Then update your data inputs to use this custom sourcetype explicitly.
splunk btool props list access_common
showed me
REPORT-access = access-extractions
pointed me to the file
splunk\etc\system\default\transforms.conf
And then I found some customizations a consultant had implemented for us years ago, so now I should try doing the same as him adding x-forward-for to my custom access_common
we do not update any file in splunk\etc\system\default\, so you'd create a new app (to be deployed in splunk/etc/apps) with a props.conf and transforms.conf. The props.conf will have definition of your custom sourcetype and will point to transforms.conf which will have your custom field extraction for your custom sourcetype.