Getting Data In

Fixed timestamp location in event. How to extract?

himynamesdave
Contributor

I have a timestamp in %Y%m format - not ideal. Here is an event:

  A       201301    08433 

The timestamp here is 201301.

The first digit of the timestamp (e.g 2 above) is always in the 15th position of the events.

How can I get Splunk to recognise this timestamp at index time? I would want to snap the day in the timestamp to the first of each month (e.g 01/01/2013 above).

Thanks!

0 Karma

aholzer
Motivator

If you want to extract this as the _time of your event, what you want is a combination of the following attributes in your props.conf (documentation for props.conf😞

TIME_PREFIX=(.){15}
MAX_TIMESTAMP_LOOKAHEAD=6
TIME_FORMAT=%Y%m
  • The time prefix tells it to skip the first 15 characters.
  • The max timestamp lookahead tells it that the timestamp is found within 6 characters of the end of the time prefix.
  • The time format tells it to expect the timestamp in "yyyymm" format

If you are extracting this as a new field at search time:

<base search> | rex "(.){15}(?P<your_new_time_field>\d+)"

Or save it as a field extraction using the field extractor GUI feature.

Hope this helps

0 Karma

aholzer
Motivator

Note that the &lt; should be replaced with a less than sign (<) and the &gt; should be replaced with a greater than sign (>)

0 Karma

himynamesdave
Contributor

So this was my props.conf - but it fails to extract the timestamp

TIME_FORMAT=%Y%m
TIME_PREFIX=(.){15}
MAX_TIMESTAMP_LOOKAHEAD=6
SHOULD_LINEMERGE=false
LINE_BREAKER= .{775}()


  A       201301    08433           11                1                             34105                                            201  1    06  X1 4                          32  0502 2   074          01  2    3                         13021                      1308  172  Y00000000    N                  YNNYNNN NXXY01    1   212       NN NNNNNNN  2 2  NNNNN NNNNNN 2     2  1       24U    224 2NNNNNN1Y64 094885  1            F 05  201233   33041       1530    042  NNYNNNN         NNNNNNNNNNNN22       2    2  2 2  NYY42.06999999999999999245262649           11111111111111111111 1111111 111111111111 111111  1111111111111111111111111111 1   11              1  111111111    1   111            1 1      1     1  1            11                11       1    1  1 1    
0 Karma

aholzer
Motivator

I suggest you use the file input wizard to help you with your props.conf.

  1. Save a file on your indexer with a few lines of your log
  2. Go to Settings > Data inputs > Files & directories > new
  3. Browse to the location of the file and choose next
  4. Expand the "Advanced" tab on the left, and add your attributes
  5. Test those attributes until they work for you

Try skipping the MAX_TIMESTAMP_LOOKAHEAD=6 and LINE_BREAKER= .{775}() your first pass.

0 Karma

aholzer
Motivator

@himynamesdave,

Did the above comment help you out at all? If not, can you provide a few lines of samples?

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...