All Apps and Add-ons

Multiple Log format in a single sourcetype?

Path Finder

I have a problem where we had an older revision of Bluecoat and then one by one they upgraded the bluecoats and the field mappings to a newer revision.

Problem - mixed fields in the same sourcetype
- Older format is parsed correctly with correct field mappings
- Newer format is not parsed correctly.

All of our bluecoats are now updated to the latest revision now.

I fixed the field mappings to the new BC fields so current data is being parsed and reported on correctly.

Problem - Historical data does not get reported on with the updated fields.


Is it possible to have multiple log formats within a single sourcetype and be able to parse the data out correctly even through field names may be the same but in a different position?


I think splunk needs to add the ability to perform regex restrictions or filtering prior to field extractions. That would help me in my current set up.

I have 3 different logging formats that change based on a specific field within the same source type.

Basically I need

Case 1: " TEXTA "
rex fileds

Case2: " TEXTB "
rex fields

Case3: " TEXTC "
rex fields

default unparsed, no rex fields.

0 Karma

Path Finder

This will probably get ugly, I kind of figure this was the case.. This is what happens when proper change control is not followed!

0 Karma

Splunk Employee
Splunk Employee

maybe. the neatest way would be to either version or to have different sourcetypes (e.g., bluecoat, bluecoat_v2) but that takes foreknowledge that you might not have, and it's too late now.

you can simply put multiple field extractions to a sourcetype, and both will run, but you might have conflicts. you might be able to deal with this with regex extractions, provided you write the regexes so each one could only match one of the formats, and fail on the other(s).

0 Karma
Get Updates on the Splunk Community!

Take the 2021 Splunk Career Survey for $50 in Amazon Cash

Help us learn about how Splunk has impacted your career by taking the 2021 Splunk Career Survey. Last year’s ...

Using Machine Learning for Hunting Security Threats

WATCH NOW Seeing the exponential hike in global cyber threat spectrum, organizations are now striving more for ...

Observability Newsletter Highlights | March 2023

 March 2023 | Check out the latest and greatestSplunk APM's New Tag Filter ExperienceSplunk APM has updated ...