Hunting a bit, I am finding evidence that I cannot feed one sourcetype into another. I've had some success in stripping out the openshift related delimiters at the front of the file, but I cannot then pass it into a sourcetype it seems. Or, if I do, it doesn't properly process it. I've also tried cloning the sourcetype, but I think I am missing a step there as it doesn't appear to be picking up by the tomcat_local_access sourcetype (if that even exists).
I also saw some documents that indicated that you might be able to run two transforms in a row, but I do not know how to ensure that will pass properly on a single line. Instead, it looks as if it looks for different criteria.
Edit: It's the end of the day and I need to step away for now. Here's my latest attempt, at trying to get the heavy forwarder to generate search-time field extractions. However, no new fields are showing up in my search results, so I can't tell if it is doing what it should or not.
props.conf
#Intake from Shib containers (OCP specific!)
[ocp:container:ilstu-shib-test]
TRANSFORMS-shibboleth-tomcat-access = shibboleth:tomcat-access-sourcetype
[shibboleth:tomcat-access]
REPORT = shibboleth:tomcat-access-fields
transforms.conf
[shibboleth:tomcat-access-sourcetype]
REGEX = tomcat-access;console;.*?;.*?;(.*)
DEST_KEY = MetaData:Sourcetype
FORMAT = sourcetype::shibboleth:tomcat-access
[shibboleth:tomcat-access-fields]
REGEX = tomcat-access;console;.*?;.*?;(?P<ip>(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)|(?:[A-F0-9]{1,4}:){7}[A-F0-9]{1,4})\s-\s(?P<user>(-|\w+))\s\[\d+\/\w+\/\d{4}:\d{2}:\d{2}:\d{2}\s[\+\-]\d{4}\]\s"(?P<method>[A-Z]{3,7})\s(?P<request_uri>[\S]+)\s(?P<protocol>[\w\/\.]+)"\s(?P<status>\d{3})\s(?P<bytes_sent>(?:\d+|-))$
FORMAT = ip::$1 user::$2 method::$4 request_uri::$5 protocol::$6 status::$7 bytes_sent::$8
WRITE_META = true
