- Splunk Answers
- :
- Splunk Administration
- :
- Getting Data In
- :
- How do i stop a file from being segmented?
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark Topic
- Subscribe to Topic
- Mute Topic
- Printer Friendly Page
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
This is the beginning of the file, line numbers for clarity:
1. Log File for: BatchJobOutput_20150801-0139_13516_MonthlyBatchJob_SAMM191.log
2. Started: Sat Aug 1 01:39:22 CDT 2015
3. Using path to access.properties: /opt/WebSphere/AppServer/lib/app
4. --------------------------
5. /usr/java64/jdk1.6.0_43/bin:/bin:/usr/bin:/opt/gnome/bin:/usr/X11/bin:/home/cd7543/scripts
This is the end of the file:
159. Ended: Sat Aug 1 01:40:55 CDT 2015
160.
There are many date references between these two sections and Splunk takes this one file and splits it up which is then displayed to the end user segmented, in reverse order.
How do I get Splunk to index this as one contiguous file?
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Try something like this for your props.conf
[YourSourceType]
BREAK_ONLY_BEFORE=^Log\s*File\s*for:
MAX_TIMESTAMP_LOOKAHEAD=150
NO_BINARY_CHECK=1
SHOULD_LINEMERGE=true
TIME_PREFIX=Started:\s*
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Try something like this for your props.conf
[YourSourceType]
BREAK_ONLY_BEFORE=^Log\s*File\s*for:
MAX_TIMESTAMP_LOOKAHEAD=150
NO_BINARY_CHECK=1
SHOULD_LINEMERGE=true
TIME_PREFIX=Started:\s*
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Of the solutions offered, this one works the best.
But I am exceeding the 10000 byte limit and the file is being truncated.
Suggestions?
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
10000 bytes limits in the preview screen? If you see data truncated in Preview, don't worry actual data will not get truncated.
If you see the actual data getting truncated, add following to props.conf stanza.
TRUNCATE = 0
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Do not set settings such as TRUNCATE to zero, ever.
Pick a value that seems outlandishly large, sure - but not zero. If something breaks at your source, Splunk will attempt to build infinitely large events...
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
yes in the preview screen
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
You want all 160 lines to appear into one big single event with timestamp being taken from line 2?
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Yes that is correct.
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Add this to your props.conf:
TIME_PREFIX = ^Started:\s+
This won't work if the other timestamps start with the same RegEx.
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
That helps a lot. the file is now broken up into two segments. the first line and the second which contains the timestamp.
I can make do with this but if needed how do I get the previous line included?
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Add these, too:
BREAK_ONLY_BEFORE_DATE = false
BREAK_ONLY_BEFORE = ^Log File for:\s+
SHOULD_LINEMERGE = true
.conf24 | Registration Open!
ICYMI - Check out the latest releases of Splunk Edge Processor
Introducing the 2024 SplunkTrust!
