Getting Data In

Linebreaking and event date in text file

mcomfurf
Path Finder

We have a text file to log row counts for CSV files used in an ETL job. The format of the file is like this:

08/30/15 16:08:51
acme_file.csv: 0
coyote_file.csv: 0
anvil_file.csv: 88835
roadrunner_file.csv: 330333

I want to break this into one line per filename from line 2 onward, into fields {filename}.csv: {rowcount} Note that the first line in the .txt file is the datetime for all of the events.

I think the linebreaking is fairly straightforward via props.conf on my indexers, but that first line with the datetime is in a different format; I want to use that and apply it to all subsequent lines. Little help!

0 Karma

woodcock
Esteemed Legend

Based on this clarification:

Yes, the timestamp is in the filename: 08-30-2015_162551__summary.txt

You can do it like this:

$SPLUNK_HOME/etc/apps/MyApp/default/datetime.xml

<define name="datetimefromfile" extract="month, day, year, hour, minute, second">
   <text><![CDATA[source::.*?(\d{2})-(\d{2})-(\d{4})_(\d{2})(\d{2})(\d{2})]]></text>
</define>
<timePatterns>
   <use name="datetimefromfile"/>
</timePatterns>
<datePatterns>
   <use name="datetimefromfile"/>
</datePatterns>
</datetime>

$SPLUNK_HOME/etc/apps/MyApp/default/props.conf:

[YourSourcetypeHere]
DATETIME_CONFIG = /etc/apps/MyApp/default/datetime.xml
0 Karma

woodcock
Esteemed Legend

Can you have the timestamp added to the filename? If so, you can use datetime.xml easily to do this.

0 Karma

mcomfurf
Path Finder

Yes, the timestamp is in the filename: 08-30-2015_162551__summary.txt

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Automating Threat Operations and Threat Hunting with Recorded Future

    Automating Threat Operations and Threat Hunting with Recorded Future June 29, 2026 | Register   Is your ...

Keep the Learning Going with the New Best of .conf Hub

Hello Splunkers, With .conf26 getting closer, there’s already a lot of excitement building around this year’s ...

Splunk Community Badges!

  Hey everyone! Ready to earn some serious bragging rights in the community? Along with our existing badges ...