Parsing Multi Line Timestamp

[jroedel]

I have to parse the timestamp of JSON logs and I would like to include subsecond precision. My JSON-Events start like this:

{
  "instant" : {
    "epochSecond" : 1727189281,
    "nanoOfSecond" : 202684061
  },
...

Thus I tried as config in props.conf:

TIME_FORMAT=%s,\n    "nanoOfSecond" : %9N
TIME_PREFIX="epochSecond" :\s
MAX_TIMESTAMP_LOOKAHEAD=500

That did unfortunately not work.

What is the right way to parse this time stamp with subsecond precision?

[jroedel · Accepted solution]

Finally after a lot of testing I found a solution via transforms.conf

[timestamp-fix]
INGEST_EVAL= _time=json_extract(_raw,"instant.epochSecond").".".json_extract(_raw,"instant.nanoOfSecond")

Furthermore, it turned out that regex is not allowed in TIME_FORMAT field in props.conf.

[gcusello]

Hi @jroedel ,

good for you, see next time!

Ciao and happy splunking

Giuseppe

P.S.: Karma Points are appreciated 😉

[gcusello]

Hi @jroedel ,

are you sure about the number of spaces?

please try this:

TIME_FORMAT=%s,\n\s*"nanoOfSecond"\s*:\s*%9N
TIME_PREFIX="epochSecond"\s*:\s*
MAX_TIMESTAMP_LOOKAHEAD=500

Ciao.

Giuseppe

[gcusello]

Hi @jroedel ,

please try this:

TIME_FORMAT=%s,\n\s*\"nanoOfSecond\"\s*:\s*%9N
TIME_PREFIX=\"epochSecond\"\s*:\s*
MAX_TIMESTAMP_LOOKAHEAD=500

Ciao.

Giuseppe

[jroedel]

Thanks for your second attempt. I tried, but still no luck.

Might there be the possibility, that the "Add Data" WebUI Wizard does not support this correctly?

[gcusello]

Hi @jroedel ,

if the Add Data feature doesn't permit to use this feature I suppose that it isn't possible event if it's strange.

I tried but I have the same result

Ciao.

Giuseppe

[jroedel]

I tried, but still no luck

Screenshot 2024-10-01 at 11.46.03.png