Typo3 logs to Splunk

[tech_g706]

Hi,

I need recommendations on typo3 logs source type.

Be default, I set source type as "typo3" in inputs.conf but logs are not parsed properly.

I did not find any Splunk TA for typo3 that can help in parsing.

Anyone have experience onboarding typo3 logs?  Thank you!

[livehybrid · Accepted solution]

Hi

There is no official Splunk TA for Typo3, so you need to create a custom sourcetype with appropriate field extractions for your Typo3 logs. Start by identifying the log format (e.g., JSON, key-value, plain text) and create custom props.conf and transforms.conf settings to parse the fields.

Its a few years since Ive used Typo3 and the only instance I still have running just has apache2 logs however in the Typo3 docs I found the following sample event - is this similar to yours?

Fri, 19 Jul 2023 09:45:00 +0100 [WARNING] request="5139a50bee3a1" component="TYPO3.Examples.Controller.DefaultController": Something went awry, check your configuration!

If so then the following props/transforms should help get you started:

== props.conf == 
[typo3] 
SHOULD_LINEMERGE = false 
# Custom timestamp extraction (day, month, year, time, tz) 
TIME_PREFIX = ^ 
TIME_FORMAT = %a, %d %b %Y %H:%M:%S %z 
TRUNCATE = 10000 
# Route event to stanza in transforms.conf for field extractions REPORT-typo3_fields = typo3_field_extractions 

== transforms.conf == 
[typo3_field_extractions] 
# Extract log_level, request id, component, message 
REGEX = \[([^\]]+)\]\s+request="([^"]+)"\s+component="([^"]+)":\s*(.*)$ FORMAT = log_level::$1 request_id::$2 component::$3 message::$4

🌟 Did this answer help you? If so, please consider:

• Adding karma to show it was useful
• Marking it as the solution if it resolved your issue
• Commenting if you need any clarification

  Your feedback encourages the volunteers in this community to continue contributing

[tech_g706]

@livehybrid Thanks for the response.

Regex is working fine, and 4 fields are extracted (log_level, request_id, component, message)

Are these four fields the only ones for typo3 logs, and should this work for every typo3 log format?

I did not find an official documentation on typo3 logs format.

The message field contains some nested field value pairs as well.
In addition, message values have multi-line events as well, so I had to adjust props.conf like this:

SHOULD_LINEMERGE = true
BREAK_ONLY_BEFORE = ^\w{3},\s+\d+\s+\w+\s+\d{4}\s+\d{2}:\d{2}:\d{2}\s+\+\d{4}

Thanks

[livehybrid]

Hi @tech_g706 

The official docs are at https://docs.typo3.org/m/typo3/reference-coreapi/main/en-us/ApiOverview/Logging/Index.html if they're any use to you 🙂

Let me know if theres anything else I can help with.

🌟 Did this answer help you? If so, please consider:

• Adding karma to show it was useful
• Marking it as the solution if it resolved your issue
• Commenting if you need any clarification

Your feedback encourages the volunteers in this community to continue contributing