Knowledge Management

what should an event look like? best practices, etc...

ra01
Path Finder

I've been asked to create my best case/wished-for Splunk event and our tech team will create it for me. I think I'm in a bit over my
head because I don't know what "best" should look like.

These events are ONLY for the analytics team at the office to do work, these have no purpose outside of our team. I'm pulling data from google analytics and our cookie for users.

Here's what I've come up with so far.

[2016-05-19 12:04:25,979] [ACTION-Track]
[id=1521775661u1442616559]
[utma_1_first=91689306]
[utma_2_first=1526771661]
[utma_3_first=1412616559]
[utma_4_first=1452136054]
[utma_5_first=1464900787
[utma_6_first=211]
[ip_first=255.255.255.255]
[device_akamai_first=MOBILE]
[device_extra_first=MOBILE]
[country_first=US]
[state_first=NY]
[city_first=NEWYORK]
[lat_first=40.7500]
[long_first=-73.9967]
[loggedin=0]
[server_session_id=05B19CF1665B8AC5A8913A3F6FA01DE9]
[utma_1_event=99681306]
[utma_2_event=1811900925]
[utma_3_event=1464912789]
[utma_4_event=1464912789]
[utma_5_event=1464902189]
[utma_6_event=1]
[utmb_1_event=99189306]
[utmb_2_event=1]
[utmb_3_event=10]
[utmb_4_event=1464900717]
[utmc_event=99189306]
[utmz_1_event=99619306]
[utmz_2_event=1456521385]
[utmz_3_event=201]
[utmz_4_event=11]
[utmz_utmcsr_event=admin1:1011]
[utmz_utmccn_event=(referral)]
[utmz_utmcmd_event=referral]
[utmz_utmcct_event=/admin/index.jsp]
[ip_event=255.255.255.2]
[device_akamai_event=NDV]
[device_extra_event=MOBILE]
[country_event=US]
[state_event=NY]
[city_event=NEWYORK]
[lat_event=40.7500]
[long_event=-73.9967]
[basket_event=zzDefault~198840000000~011~JHUN~198540000000~002~ORLANDO~198540000000~021]
[step_event=0]
[url_event=/product/cart/qty.html?token=1411464901183199&referrer=http://www.XXXXX.com/gifts/]
[q_size=10]
[dv=NDV]

Here's what I wonder:

  • does it make sense to create the field-ready lines in the event? "utma_1_first=" or would it be better to extract them in the events?
  • is it better to use new lines, or should it be one long line?
0 Karma
1 Solution

woodcock
Esteemed Legend

Q1: Does it make sense to create the field-ready lines in the event ( utma_1_first= ) or would it be better to extract them in the events?

A1: If you have unlimited license and disk space, yes. Doing so bloats your event size considerably. I personally would not do it. It locks you into a naming convention that you may regret (imagine a lawsuit or the selling/rebranding of your company). Because you have control over everything, I would go with CSV format with the field extractions in props.conf/transforms.conf. Then make sure that you NEVER delete or move around field positions and you only ever add new fields to the end.

Q2: is it better to use new lines, or should it be one long line?

A2: Let's remember that these logs are for the benefit of humans; I would keep the newlines if you are going with a bloated approach (unlimited license) but obviously not if you are doing CSV.

View solution in original post

0 Karma

woodcock
Esteemed Legend

Q1: Does it make sense to create the field-ready lines in the event ( utma_1_first= ) or would it be better to extract them in the events?

A1: If you have unlimited license and disk space, yes. Doing so bloats your event size considerably. I personally would not do it. It locks you into a naming convention that you may regret (imagine a lawsuit or the selling/rebranding of your company). Because you have control over everything, I would go with CSV format with the field extractions in props.conf/transforms.conf. Then make sure that you NEVER delete or move around field positions and you only ever add new fields to the end.

Q2: is it better to use new lines, or should it be one long line?

A2: Let's remember that these logs are for the benefit of humans; I would keep the newlines if you are going with a bloated approach (unlimited license) but obviously not if you are doing CSV.

0 Karma

ra01
Path Finder

thanks. I just talked to one of our architects and he doesn't want a new line because it messes up his ability to do grep searches on the logs if he needs to check on them.

0 Karma
Get Updates on the Splunk Community!

Get ready to show some Splunk Certification swagger at .conf24!

Dive into the deep end of data by earning a Splunk Certification at .conf24. We're enticing you again this ...

Built-in Service Level Objectives Management to Bridge the Gap Between Service & ...

Now On-Demand Join us to learn more about how you can leverage Service Level Objectives (SLOs) and the new ...

Database Performance Sidebar Panel Now on APM Database Query Performance & Service ...

We’ve streamlined the troubleshooting experience for database-related service issues by adding a database ...