Getting Data In

Record Delimeters

gbolcer2
New Member

I'm running free version 4.2.3. I have an email archive that I'm pre-processing the data to allow for key=value settings. I need to have one field that is the string content including multiple lines and >> responses and other characters that I need to stuff into a field. I control the data format, so I've put an #EOR (end of record) string into the data.

I've been through the documentation for multi-line events here: http://docs.splunk.com/Documentation/Splunk/4.2.3/Data/Indexmulti-lineevents

There wasn't a copy of C:/program files/splunk/etc/system/local/props.conf, so I made one.

I tried to use TRUNCATE = 0 but it stuck everything in one single record. Also LINE_BREAKER = "#DCS_EOR"

Please advise on if I'm editing the right file, file location, values, have the right regex for the line breaker, etc.

Please see two records fo data below:

m_id="27001",t_id="26044",t_name="Simpsons: BT Raw supply",p_id="1434",p_firstname="Tabish",p_lastname="Hasan",p_fullname="Tabish Hasan",m_idparent="0",m_type="email",fileid="3",from_firstname="Tabish",from_lastname="Hasan",from_fullname="Tabish Hasan",m_createdate="2007-07-26","to"="torrents","attachments"="multipart/alternative:/opt/dcsemap/visualizer/database/repository/f_file/00/00/00/03/contents/mediadefender@codebrilliant.com/Inbox/attachment/mdexch01.mediadefender.com/F55A72EF782B43428EBF605551914B20E8C474/attachment1.1,text/html:/opt/dcsemap/visualizer/database/repository/f_file/00/00/00/03/contents/mediadefender@codebrilliant.com/Inbox/attachment/mdexch01.mediadefender.com/F55A72EF782B43428EBF605551914B20E8C474/attachment1.1.2",m_content="FYI...for those that look at BT raw supply through the munger.

The BT Raw Supply for The Simpsons Movie is inaccurate. It's picking up
our decoys and counting it as supply. There's not really that much
non-MD supply out there for it yet.

Gerald and Arthur are looking into it.

-TH

"
#DCS_EOR

m_id="27002",t_id="26045",t_name="Status Change - [PM !GOR-850395]: Mark as Single (UMG)",p_id="1432",p_firstname="Gilberto",p_lastname="Vargas",p_fullname="Gilberto Vargas",m_idparent="0",m_type="email",fileid="3",from_firstname="Gilberto",from_lastname="Vargas",from_fullname="Gilberto Vargas",m_createdate="2007-09-05","to"="Jay Mairs","attachments"="multipart/alternative:/opt/dcsemap/visualizer/database/repository/f_file/00/00/00/03/contents/mediadefender@codebrilliant.com/Inbox/attachment/support.mediadefender.com/jnwzva.6mzu7b/attachment1.1,text/html:/opt/dcsemap/visualizer/database/repository/f_file/00/00/00/03/contents/mediadefender@codebrilliant.com/Inbox/attachment/support.mediadefender.com/jnwzva.6mzu7b/attachment1.1.2",m_content="Ticket Status: Mark as Single (UMG)

New Status: Closed
Old Status: On Hold

Ticket Details
===================
Ticket ID: GOR-850395
Department: QA
Priority: Low
Status: Closed
Link:http://support.mediadefender.com/support/staff/index.php?_m=tickets&_a=viewticket&ticketid=14634"
#DCS_EOR
Tags (1)
0 Karma

woodcock
Esteemed Legend

You are using a file that should work and there are many ways to do it. Try this way:

[MySourceType]
LINE_BREAKER = (#DCS_EOR)
SHOULD_LINEMERGE = false

The docs are here:
http://docs.splunk.com/Documentation/Splunk/latest/Data/Indexmulti-lineevents

The key statement I suspect you missed is this:
"The regex must contain a capturing group -- a pair of parentheses that defines an identified subcomponent of the match."

0 Karma
Get Updates on the Splunk Community!

.conf24 | Day 0

Hello Splunk Community! My name is Chris, and I'm based in Canberra, Australia's capital, and I travelled for ...

Enhance Security Visibility with Splunk Enterprise Security 7.1 through Threat ...

 (view in My Videos)Struggling with alert fatigue, lack of context, and prioritization around security ...

Troubleshooting the OpenTelemetry Collector

  In this tech talk, you’ll learn how to troubleshoot the OpenTelemetry collector - from checking the ...