Getting Data In

How to number each line in a multiline event?

landen99
Motivator

Is there a way to take a multiline event:

a 1 b 2
c 2 d 4
e 5 c 6

and number each line?

1 a 1 b 2
2 c 2 d 4
3 e 5 c 6

like this?

Maybe using streamstats and current=f?

1 Solution

Gilberto_Castil
Splunk Employee
Splunk Employee

You should try the accum command in the search language. This will allow you to aggregate to a variable value in an ascending fashion. For instance, assume that your data has been indexed.

alt text

Because Splunk will index and show the data in reverse order Last In First Out (LIFO), you need to decide how the order it for presentation. Assume we want the data as it was originally written to the log file (not the way it was indexed); use reverse to align it accordingly.

Create a new variable called No and use accum to increase the variable value by one (1).

alt text

All together the search looks something like this:

sourcetype="answers-1429120234" | reverse | eval No=1 | accum No | table No _raw

Good luck and happy Splunking 🙂

--
gc

View solution in original post

stephane_cyrill
Builder

Hi,
If there is a way you can have your event in the statistic tab, and if you are using splunk 6.2, you can go in format option and select LINE NUMBER to yes

0 Karma

Gilberto_Castil
Splunk Employee
Splunk Employee

You should try the accum command in the search language. This will allow you to aggregate to a variable value in an ascending fashion. For instance, assume that your data has been indexed.

alt text

Because Splunk will index and show the data in reverse order Last In First Out (LIFO), you need to decide how the order it for presentation. Assume we want the data as it was originally written to the log file (not the way it was indexed); use reverse to align it accordingly.

Create a new variable called No and use accum to increase the variable value by one (1).

alt text

All together the search looks something like this:

sourcetype="answers-1429120234" | reverse | eval No=1 | accum No | table No _raw

Good luck and happy Splunking 🙂

--
gc

landen99
Motivator

I love your answer. It introduced me to two new Splunk functions (reverse and accum) which provide a very simple solution. Your solution does begin with the assumption that the data has already been broken data with each row as an event before it is indexed. To apply your solution from my test data where the rows are grouped together into each event when indexed, I would simply alter your solution as follows:

| eventstats count AS event_num | rex max_match=0 "(?P<line>[^\n]+)\n+" | mvexpand line | eval line_num=1 | accum line_num | table event_num line_num line

In addition to your results, this simply adds numbering for each event that each line originally belonged to. It differs from my eventstats solution in that the numbering does not restart with each new event. I like my eventstats solution better though.

| eventstats count AS event_num | rex max_match=0 "(?P<line>[^\n]+)\n+" | mvexpand line | eventstats count AS line_num by event_num | eval line_new=line_num." - ".line | stats list(line_new) AS line_new by event_num

chimell
Motivator

Hi landen99
Try the search code below
examine attentively After extracting the fields which are in your event

   .....|rex max_match=0 field=_raw "\s+(?P<field1>\S+)\s+(?P<field2>\d+)\s+(?P<field3>\S+)\s+(?P<field4>\d+)"| eval fields=mvzip(field1,mvzip(field2,mvzip(field3,field4)))|mvexpand fields|eval fil=split(fields,",")|eval fiel1=mvindex(fil,0)|eval fiel2=mvindex(fil,1)|eval fiel3=mvindex(fil,2)
        |eval fiel4=mvindex(fil,3)|streamstats count as number_line|eval fi1=mvzip(number_line,mvzip(fiel1,mvzip(fiel2,mvzip(fiel3,fiel4," ")," ")," ")," ")|stats list(fi1) as test2
0 Karma

landen99
Motivator

This search appears to do the following:
* extract each column as field1-4 (using rex)
* combine each field into line field called fields (using mvzip)
* separates the multivalue field of lines into separate events
* splits the line field by commas which do not exist in the value ...

It would be so much easier to just separate the event by lines into new events and then streamstats count to track event and line order separately

| streamstats count AS event_num | rex max_match=0 "(?P<line>[^\n]+)\n+" | mvexpand line | streamstats count AS line_num by event_num | eval line_new=line_num." - ".line | stats list(line_new) AS line_new by event_num
0 Karma

fdi01
Motivator

try like this :

| streamstats sum(linecount) as rank
or
| streamstats sum(linecount) as rank| table   rank|
0 Karma

landen99
Motivator

The linecount of the first event is 3, so I believe that would yield:

  a 1 b 2
3 c 2 d 4
  e 5 c 6

If the next event were exactly the same and had therefore had 3 lines, we should see:

  a 1 b 2
6 c 2 d 4
  e 5 c 6

I will test later to verify.

0 Karma

alacercogitatus
SplunkTrust
SplunkTrust

Streamstats should work.

<your_search> | streamstats count as EVTCOUNT 

Each event then should have EVTCOUNT, one per event, increasing.

landen99
Motivator

I am talking about numbering the lines of multiline events You solution provides number of events and not the lines in each event, like this:

  a 1 b 2
1 c 2 d 4
  e 5 c 6
0 Karma
Get Updates on the Splunk Community!

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...