Getting Data In

Record Delimeters

gbolcer2
New Member

I'm running free version 4.2.3. I have an email archive that I'm pre-processing the data to allow for key=value settings. I need to have one field that is the string content including multiple lines and >> responses and other characters that I need to stuff into a field. I control the data format, so I've put an #EOR (end of record) string into the data.

I've been through the documentation for multi-line events here: http://docs.splunk.com/Documentation/Splunk/4.2.3/Data/Indexmulti-lineevents

There wasn't a copy of C:/program files/splunk/etc/system/local/props.conf, so I made one.

I tried to use TRUNCATE = 0 but it stuck everything in one single record. Also LINE_BREAKER = "#DCS_EOR"

Please advise on if I'm editing the right file, file location, values, have the right regex for the line breaker, etc.

Please see two records fo data below:

m_id="27001",t_id="26044",t_name="Simpsons: BT Raw supply",p_id="1434",p_firstname="Tabish",p_lastname="Hasan",p_fullname="Tabish Hasan",m_idparent="0",m_type="email",fileid="3",from_firstname="Tabish",from_lastname="Hasan",from_fullname="Tabish Hasan",m_createdate="2007-07-26","to"="torrents","attachments"="multipart/alternative:/opt/dcsemap/visualizer/database/repository/f_file/00/00/00/03/contents/mediadefender@codebrilliant.com/Inbox/attachment/mdexch01.mediadefender.com/F55A72EF782B43428EBF605551914B20E8C474/attachment1.1,text/html:/opt/dcsemap/visualizer/database/repository/f_file/00/00/00/03/contents/mediadefender@codebrilliant.com/Inbox/attachment/mdexch01.mediadefender.com/F55A72EF782B43428EBF605551914B20E8C474/attachment1.1.2",m_content="FYI...for those that look at BT raw supply through the munger.

The BT Raw Supply for The Simpsons Movie is inaccurate. It's picking up
our decoys and counting it as supply. There's not really that much
non-MD supply out there for it yet.

Gerald and Arthur are looking into it.

-TH

"
#DCS_EOR

m_id="27002",t_id="26045",t_name="Status Change - [PM !GOR-850395]: Mark as Single (UMG)",p_id="1432",p_firstname="Gilberto",p_lastname="Vargas",p_fullname="Gilberto Vargas",m_idparent="0",m_type="email",fileid="3",from_firstname="Gilberto",from_lastname="Vargas",from_fullname="Gilberto Vargas",m_createdate="2007-09-05","to"="Jay Mairs","attachments"="multipart/alternative:/opt/dcsemap/visualizer/database/repository/f_file/00/00/00/03/contents/mediadefender@codebrilliant.com/Inbox/attachment/support.mediadefender.com/jnwzva.6mzu7b/attachment1.1,text/html:/opt/dcsemap/visualizer/database/repository/f_file/00/00/00/03/contents/mediadefender@codebrilliant.com/Inbox/attachment/support.mediadefender.com/jnwzva.6mzu7b/attachment1.1.2",m_content="Ticket Status: Mark as Single (UMG)

New Status: Closed
Old Status: On Hold

Ticket Details
===================
Ticket ID: GOR-850395
Department: QA
Priority: Low
Status: Closed
Link:http://support.mediadefender.com/support/staff/index.php?_m=tickets&_a=viewticket&ticketid=14634"
#DCS_EOR
Tags (1)
0 Karma

woodcock
Esteemed Legend

You are using a file that should work and there are many ways to do it. Try this way:

[MySourceType]
LINE_BREAKER = (#DCS_EOR)
SHOULD_LINEMERGE = false

The docs are here:
http://docs.splunk.com/Documentation/Splunk/latest/Data/Indexmulti-lineevents

The key statement I suspect you missed is this:
"The regex must contain a capturing group -- a pair of parentheses that defines an identified subcomponent of the match."

0 Karma
Get Updates on the Splunk Community!

Accelerate Service Onboarding, Decomposition, Troubleshooting - and more with ITSI’s ...

Accelerate Service Onboarding, Decomposition, Troubleshooting - and more! Faster Time to ValueManaging and ...

New Release | Splunk Enterprise 9.3

Admins and Analyst can benefit from:  Seamlessly route data to your local file system to save on storage ...

2024 Splunk Career Impact Survey | Earn a $20 gift card for participating!

Hear ye, hear ye! The time has come again for Splunk's annual Career Impact Survey!  We need your help by ...