Getting Data In

How to edit my props.conf to extract the timestamp at the end of my sample syslog events?

dpanych
Communicator

I have the following syslog data and I need help extracting the timestamp field at the end of the event:

Sep  6 06:07:20 2016-09-06 06: 07:20,165 192.168.0.0 CPPM_Dashboard_Summary 17000 1 0 session_id=A00001234-01-56bcde1f,req_source=RADIUS,user_name=XXXXXXXXXX,service_name=ABC802.1X,alerts_present=0,nas_ip=192.168.1.1,nas_port=0,conn_status=Unknown,login_status=ACCEPT,error_code=0,mac_address=12a3123456n2,timestamp=2016-09-06 06:05:48-07,write_timestamp=2016-09-06 06:05:50.133529-07

Sep  6 14:24:18 2016-09-06 14: 24:18,179 192.168.109.102 CPPM_System_Stat 11115 1 0 id=11112,swap_size_used=65757,slash_size_used=11166596,swap_memory_avail=6079780,system_memory_avail=3737736,cpu_raw_user=1,cpu_raw_nice=0,cpu_raw_system=13,cpu_raw_idle=87,mgmt_inf_status=up,data_inf_status=down,uptime=8291850,timestamp=2016-09-06 14:24:07.156076-07

I'm using the following props on the indexers and heavy forwarder, but the timestamp is still not parsing out:

TIME_PREFIX = ,timestamp
TIME_FORMAT = %Y-%d-%m %H:%M:%S.%6N
MAX_TIMESTAMP_LOOKAHEAD = 350
0 Karma

gcusello
SplunkTrust
SplunkTrust

Yout time prefix should be
,timestamp=
And your lookahead should be 26 the exact characters number of your timestamp.
Bye.
Giuseppe

0 Karma

gcusello
SplunkTrust
SplunkTrust

if you're satisfied of the answer, please, accept the answer.
Bye.
Giuseppe

0 Karma

sundareshr
Legend

Try this in your props on your indexer and restart the service. This will only affect new data. Since you have a TIME_PREFIX, the look ahead will start from the point it finds a prefix match.

TIME_PREFIX = timestamp=
TIME_FORMAT = %Y-%d-%m %H:%M:%S.%6N
MAX_TIMESTAMP_LOOKAHEAD = 30
0 Karma

dpanych
Communicator

Your solution did not seem to work. Any other suggestions?

0 Karma

sundareshr
Legend

I just tried this, and it worked. Are you trying it datapreview? Is your props on the right server and has right stanza?

SHOULD_LINEMERGE=true
TIME_FORMAT=%Y-%d-%m %H:%M:%S.%6N
TIME_PREFIX=timestamp=
MAX_TIMESTAMP_LOOKAHEAD=30
0 Karma
Get Updates on the Splunk Community!

Splunk Mobile: Your Brand-New Home Screen

Meet Your New Mobile Hub  Hello Splunk Community!  Staying connected to your data—no matter where you are—is ...

Introducing Value Insights (Beta): Understand the Business Impact your organization ...

Real progress on your strategic priorities starts with knowing the business outcomes your teams are delivering ...

Enterprise Security (ES) Essentials 8.3 is Now GA — Smarter Detections, Faster ...

As of today, Enterprise Security (ES) Essentials 8.3 is now generally available, helping SOC teams simplify ...