https://community.splunk.com/t5/Splunk-Search/How-to-configure-props-conf-to-index-txt-files-as-one-new-event/m-p/188070#M54178 <P>Start with these specs in the relevant props.conf stanza:</P> <PRE><CODE>SHOULD_LINEMERGE = true TRUNCATE = 0 MAX_EVENTS = 500 BREAK_ONLY_BEFORE = ~~CTRL AS~~ DATETIME_CONFIG = current </CODE></PRE> Wed, 21 Jan 2015 13:28:37 GMT richgalloway 2015-01-21T13:28:37Z https://community.splunk.com/t5/Splunk-Search/How-to-configure-props-conf-to-index-txt-files-as-one-new-event/m-p/188065#M54173 <P>Hello everybody!</P> <P>I could use some help with this project that I've been working with...<BR /> I have some .txt files which show timestamp in some lines like this " ---- FRIDAY, 05 DEC 2014 ---- "<BR /> But the point is, when I index it, it's counting every single datetime as new event, and it should consider the whole .txt as ONE EVENT.<BR /> The text I have in particular that defines this txt is unique is this:</P> <PRE><CODE>~~CTRL AS~~: </CODE></PRE> <P>Any idea how could I make a Regex for this to consider every time a " ~~CTRL AS~~: " is a new event, not based on the timestamps actually.<BR /> Thanks in adv!<BR /> Bst rgds!</P> Wed, 21 Jan 2015 12:08:25 GMT https://community.splunk.com/t5/Splunk-Search/How-to-configure-props-conf-to-index-txt-files-as-one-new-event/m-p/188065#M54173 vtsguerrero 2015-01-21T12:08:25Z https://community.splunk.com/t5/Splunk-Search/How-to-configure-props-conf-to-index-txt-files-as-one-new-event/m-p/188066#M54174 <P>Did you set line_breaker in your props? By default Splunk will break on the time.</P> <P><A href="http://docs.splunk.com/Documentation/Splunk/6.2.1/admin/Propsconf">http://docs.splunk.com/Documentation/Splunk/6.2.1/admin/Propsconf</A></P> Wed, 21 Jan 2015 12:56:29 GMT https://community.splunk.com/t5/Splunk-Search/How-to-configure-props-conf-to-index-txt-files-as-one-new-event/m-p/188066#M54174 thomrs 2015-01-21T12:56:29Z https://community.splunk.com/t5/Splunk-Search/How-to-configure-props-conf-to-index-txt-files-as-one-new-event/m-p/188067#M54175 <P>A REGEX is required to set this prop right?<BR /> I know that " ~ " would match the beggining. but not the complete start of the event...</P> Wed, 21 Jan 2015 12:58:45 GMT https://community.splunk.com/t5/Splunk-Search/How-to-configure-props-conf-to-index-txt-files-as-one-new-event/m-p/188067#M54175 vtsguerrero 2015-01-21T12:58:45Z https://community.splunk.com/t5/Splunk-Search/How-to-configure-props-conf-to-index-txt-files-as-one-new-event/m-p/188068#M54176 <P>How large are the .txt files? If they're too large then Splunk won't be able to treat them as a single event.</P> <P>If you can provide some sample data (not a whole file) we can better help you.</P> Wed, 21 Jan 2015 13:05:57 GMT https://community.splunk.com/t5/Splunk-Search/How-to-configure-props-conf-to-index-txt-files-as-one-new-event/m-p/188068#M54176 richgalloway 2015-01-21T13:05:57Z https://community.splunk.com/t5/Splunk-Search/How-to-configure-props-conf-to-index-txt-files-as-one-new-event/m-p/188069#M54177 <P>Each txt has an average of 400 lines and all of'em start with this " ~~CTRL AS~~: " pattern...</P> <P>The data is similar to this ( I don't have all the source too <span class="lia-unicode-emoji" title=":disappointed_face:">😞</span></P> <PRE><CODE>~~CTRL AS~~:FG8WT09UX86UBB929376293762376M92738263TROKOM S28628ITT86327UPK 293862397263755 *&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt; LOGS UTDNAME: HUTHUTHYGS &lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;&lt;* 06.52.22 UTF8556 ---- THURSDAY, 04 DEC 2014 ---- 06.52.22 UTF8556 HASP HHIAO WLM IFOP 06.52.22 UTF8556 0PLLOAOKWMO </CODE></PRE> <P>And all the rest should be considered as one event even though there's a datetime present.</P> Wed, 21 Jan 2015 13:17:09 GMT https://community.splunk.com/t5/Splunk-Search/How-to-configure-props-conf-to-index-txt-files-as-one-new-event/m-p/188069#M54177 vtsguerrero 2015-01-21T13:17:09Z https://community.splunk.com/t5/Splunk-Search/How-to-configure-props-conf-to-index-txt-files-as-one-new-event/m-p/188070#M54178 <P>Start with these specs in the relevant props.conf stanza:</P> <PRE><CODE>SHOULD_LINEMERGE = true TRUNCATE = 0 MAX_EVENTS = 500 BREAK_ONLY_BEFORE = ~~CTRL AS~~ DATETIME_CONFIG = current </CODE></PRE> Wed, 21 Jan 2015 13:28:37 GMT https://community.splunk.com/t5/Splunk-Search/How-to-configure-props-conf-to-index-txt-files-as-one-new-event/m-p/188070#M54178 richgalloway 2015-01-21T13:28:37Z https://community.splunk.com/t5/Splunk-Search/How-to-configure-props-conf-to-index-txt-files-as-one-new-event/m-p/188071#M54179 <P>You probably don't want the complete start of the event. The matching string is not included in the event so you'd want to use the smallest string. '~~CTRL AS~~' should work.</P> Wed, 21 Jan 2015 13:31:00 GMT https://community.splunk.com/t5/Splunk-Search/How-to-configure-props-conf-to-index-txt-files-as-one-new-event/m-p/188071#M54179 richgalloway 2015-01-21T13:31:00Z https://community.splunk.com/t5/Splunk-Search/How-to-configure-props-conf-to-index-txt-files-as-one-new-event/m-p/188072#M54180 <P>This props.conf should be placed inside the app folder right?<BR /> And I should re-index the data in the preview mode to see any changes...</P> Wed, 21 Jan 2015 13:43:27 GMT https://community.splunk.com/t5/Splunk-Search/How-to-configure-props-conf-to-index-txt-files-as-one-new-event/m-p/188072#M54180 vtsguerrero 2015-01-21T13:43:27Z https://community.splunk.com/t5/Splunk-Search/How-to-configure-props-conf-to-index-txt-files-as-one-new-event/m-p/188073#M54181 <P>Yes, app/local/props.conf. You must re-index.</P> <P>I suggest using a test index until you've found the right settings. That makes it easier to clean up and keeps unusable events out of your regular indexes.</P> Wed, 21 Jan 2015 13:46:39 GMT https://community.splunk.com/t5/Splunk-Search/How-to-configure-props-conf-to-index-txt-files-as-one-new-event/m-p/188073#M54181 richgalloway 2015-01-21T13:46:39Z https://community.splunk.com/t5/Splunk-Search/How-to-configure-props-conf-to-index-txt-files-as-one-new-event/m-p/188074#M54182 <P>Okay @richgalloway <BR /> I'm gonna re-index it here, asap, I'll post results, thanks a lot!</P> Wed, 21 Jan 2015 13:53:07 GMT https://community.splunk.com/t5/Splunk-Search/How-to-configure-props-conf-to-index-txt-files-as-one-new-event/m-p/188074#M54182 vtsguerrero 2015-01-21T13:53:07Z https://community.splunk.com/t5/Splunk-Search/How-to-configure-props-conf-to-index-txt-files-as-one-new-event/m-p/188075#M54183 <P>Still tryin' to re-index it, but when I applyin' this new stanza if keeps on loading, loading, and doesn't show any data at all in the preview mode...</P> Wed, 21 Jan 2015 16:16:22 GMT https://community.splunk.com/t5/Splunk-Search/How-to-configure-props-conf-to-index-txt-files-as-one-new-event/m-p/188075#M54183 vtsguerrero 2015-01-21T16:16:22Z https://community.splunk.com/t5/Splunk-Search/How-to-configure-props-conf-to-index-txt-files-as-one-new-event/m-p/188076#M54184 <P>Try cutting the file down as much as possible. Once you have it working with a few lines, add more data.</P> Wed, 21 Jan 2015 16:21:15 GMT https://community.splunk.com/t5/Splunk-Search/How-to-configure-props-conf-to-index-txt-files-as-one-new-event/m-p/188076#M54184 richgalloway 2015-01-21T16:21:15Z https://community.splunk.com/t5/Splunk-Search/How-to-configure-props-conf-to-index-txt-files-as-one-new-event/m-p/188077#M54185 <P>Just did it, and it worked perfectly as one single event!<BR /> Just had to cut the last line <STRONG>DATETIME_CONFIG = current</STRONG> wasn't allowing to load the stanza config, but once removed, it worked... Thanks a lot @richgalloway !</P> Wed, 21 Jan 2015 17:09:25 GMT https://community.splunk.com/t5/Splunk-Search/How-to-configure-props-conf-to-index-txt-files-as-one-new-event/m-p/188077#M54185 vtsguerrero 2015-01-21T17:09:25Z