Getting Data In

Linebreaking and event date in text file

mcomfurf
Path Finder

We have a text file to log row counts for CSV files used in an ETL job. The format of the file is like this:

08/30/15 16:08:51
acme_file.csv: 0
coyote_file.csv: 0
anvil_file.csv: 88835
roadrunner_file.csv: 330333

I want to break this into one line per filename from line 2 onward, into fields {filename}.csv: {rowcount} Note that the first line in the .txt file is the datetime for all of the events.

I think the linebreaking is fairly straightforward via props.conf on my indexers, but that first line with the datetime is in a different format; I want to use that and apply it to all subsequent lines. Little help!

0 Karma

woodcock
Esteemed Legend

Based on this clarification:

Yes, the timestamp is in the filename: 08-30-2015_162551__summary.txt

You can do it like this:

$SPLUNK_HOME/etc/apps/MyApp/default/datetime.xml

<define name="datetimefromfile" extract="month, day, year, hour, minute, second">
   <text><![CDATA[source::.*?(\d{2})-(\d{2})-(\d{4})_(\d{2})(\d{2})(\d{2})]]></text>
</define>
<timePatterns>
   <use name="datetimefromfile"/>
</timePatterns>
<datePatterns>
   <use name="datetimefromfile"/>
</datePatterns>
</datetime>

$SPLUNK_HOME/etc/apps/MyApp/default/props.conf:

[YourSourcetypeHere]
DATETIME_CONFIG = /etc/apps/MyApp/default/datetime.xml
0 Karma

woodcock
Esteemed Legend

Can you have the timestamp added to the filename? If so, you can use datetime.xml easily to do this.

0 Karma

mcomfurf
Path Finder

Yes, the timestamp is in the filename: 08-30-2015_162551__summary.txt

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Your Path to AgenticOps: AI Experiences for Every Splunk Practitioner

Your Path to AgenticOps: AI Experiences for Every Splunk Practitioner   Watch on demand us for a demo-driven ...

The Data Connection - October 2026

We all have data. Lots of it. But that’s not the problem – it’s understanding what we should we do with it, ...

Beyond the Acceleration Zone: Maximizing Your Success with Splunk Customer Experience ...

Customer success is built on tailored guidance and accessible resources. As a Splunk user, you are encouraged ...