Deployment Architecture

How to extract time from bash_history # timestamp?

peterm30
Path Finder

I'm dealing with bash_history files in the following format. I would like to extract the timestamp and use that as the event timestamp, but I'm having some issues doing so.

#1579207583
whoami
#1579207584
cd /var/log
#1579207590
cat messages
#1579207595
id
#1579207598
exit

I'm using the following thread as reference: https://answers.splunk.com/answers/60015/splunking-bash-history.html

 [bash_history]
 BREAK_ONLY_BEFORE = #(?=\d+)
 MAX_TIMESTAMP_LOOKAHEAD = 11
 SHOULD_LINEMERGE = true
 TIME_FORMAT = %s
 TIME_PREFIX = #

We've changed a number of variables (set TIME_PREFIX = ^#, set MAX_TIMESTAMP_LOOKAHEAD to a higher value, etc.), but nothing seems to be working correctly.

The events do break in the correct place (#), and they do merge, so we get "groups" of events like:

#1579207583
 whoami

However, the timestamp for the event isn't set to that value. All events are set to the date/time that history was written on, so everything for any given session is the same.

That props.conf configuration -appears- correct, and our sourcetype is named bash_history (we've also tried source::/root/.bash_history, without success). I'm not sure where we are going wrong, but any suggestions would be welcome.

0 Karma
1 Solution

peterm30
Path Finder

I figured it out. The "default/props.conf" in Splunk_TA_nix contains several lines that affect the timestamp. I copied these to "local/props.conf" and unset them (didn't provide a value), and now it's working. Final props.conf looks like...

[bash_history]
BREAK_ONLY_BEFORE = #(?=\d+)
MAX_TIMESTAMP_LOOKAHEAD = 10
SHOULD_LINEMERGE = true
TIME_FORMAT = %s
TIME_PREFIX = ^#
EVENT_BREAKER_ENABLE =
DATETIME_CONFIG =

I also added a field extraction for the command itself:

^#\d+\s+(?P<command>.+) 

TL;DR - It was working from the beginning, but other values in default were affecting the final result.

View solution in original post

Get Updates on the Splunk Community!

Introducing the Splunk Community Dashboard Challenge!

Welcome to Splunk Community Dashboard Challenge! This is your chance to showcase your skills in creating ...

Built-in Service Level Objectives Management to Bridge the Gap Between Service & ...

Wednesday, May 29, 2024  |  11AM PST / 2PM ESTRegister now and join us to learn more about how you can ...

Get Your Exclusive Splunk Certified Cybersecurity Defense Engineer Certification at ...

We’re excited to announce a new Splunk certification exam being released at .conf24! If you’re headed to Vegas ...