Deployment Architecture

How to extract time from bash_history # timestamp?

peterm30
Path Finder

I'm dealing with bash_history files in the following format. I would like to extract the timestamp and use that as the event timestamp, but I'm having some issues doing so.

#1579207583
whoami
#1579207584
cd /var/log
#1579207590
cat messages
#1579207595
id
#1579207598
exit

I'm using the following thread as reference: https://answers.splunk.com/answers/60015/splunking-bash-history.html

 [bash_history]
 BREAK_ONLY_BEFORE = #(?=\d+)
 MAX_TIMESTAMP_LOOKAHEAD = 11
 SHOULD_LINEMERGE = true
 TIME_FORMAT = %s
 TIME_PREFIX = #

We've changed a number of variables (set TIME_PREFIX = ^#, set MAX_TIMESTAMP_LOOKAHEAD to a higher value, etc.), but nothing seems to be working correctly.

The events do break in the correct place (#), and they do merge, so we get "groups" of events like:

#1579207583
 whoami

However, the timestamp for the event isn't set to that value. All events are set to the date/time that history was written on, so everything for any given session is the same.

That props.conf configuration -appears- correct, and our sourcetype is named bash_history (we've also tried source::/root/.bash_history, without success). I'm not sure where we are going wrong, but any suggestions would be welcome.

0 Karma
1 Solution

peterm30
Path Finder

I figured it out. The "default/props.conf" in Splunk_TA_nix contains several lines that affect the timestamp. I copied these to "local/props.conf" and unset them (didn't provide a value), and now it's working. Final props.conf looks like...

[bash_history]
BREAK_ONLY_BEFORE = #(?=\d+)
MAX_TIMESTAMP_LOOKAHEAD = 10
SHOULD_LINEMERGE = true
TIME_FORMAT = %s
TIME_PREFIX = ^#
EVENT_BREAKER_ENABLE =
DATETIME_CONFIG =

I also added a field extraction for the command itself:

^#\d+\s+(?P<command>.+) 

TL;DR - It was working from the beginning, but other values in default were affecting the final result.

View solution in original post

Get Updates on the Splunk Community!

Demo Day: Strengthen Your SOC with Splunk Enterprise Security 8.1

Today’s threat landscape is more complex than ever. Security operation centers (SOCs) are overwhelmed with ...

Dashboards: Hiding charts while search is being executed and other uses for tokens

There are a couple of features of SimpleXML / Classic dashboards that can be used to enhance the user ...

Splunk Observability Cloud's AI Assistant in Action Series: Explaining Metrics and ...

This is the fourth post in the Splunk Observability Cloud’s AI Assistant in Action series that digs into how ...