How to extract fields at search-time and display values in one row without modifying props.conf?

[chrismok]

As this sourcetype is used for other searches, the props.conf cannot be modified for adding the line merger, how to I do to get the result in one row

Log

Deploy.name=ABC
Deploy.startime=20140101000000
Deploy.environment=QA3
Deploy.joblist=A,B,C,D,E,F
Deploy.jobselected=B,C,E,F
Deploy.job.A.start=true
Deploy.job.E.start=true
Deploy.job.B.start=true
Deploy.job.A.status=Completed
Deploy.job.C.start=true
Deploy.job.B.status=Failed
Deploy.job.A.start=true
Deploy.job.E.start=true
Deploy.job.B.start=true
Deploy.job.A.status=Completed
Deploy.job.C.start=true
Deploy.job.B.status=Failed
Deploy.job.A.start=true
Deploy.job.E.start=true
Deploy.job.B.start=true;
Deploy.job.A.status=Completed
Deploy.job.C.start=true
Deploy.job.B.status=Failed

Expected Result

Name          |   Startime            |         Environment      |          JobList             |      Jobselected
======================================================================================================================
ABC              2014-01-01 00:00                    QA3                   A,B,C,D,E,F                    B,C,E,F

Search query

 | stats count | eval data = 
       "Deploy.name=ABC;
        Deploy.startime=20140101000000;
        Deploy.environment=QA3;
        Deploy.joblist=A,B,C,D,E,F;
        Deploy.jobselected=B,C,E,F;
        Deploy.job.A.start=true;
        Deploy.job.E.start=true;
        Deploy.job.B.start=true;
        Deploy.job.A.status=Completed;
        Deploy.job.C.start=true;           Deploy.job.B.status=Failed;Deploy.job.A.start=true;Deploy.job.E.start=true;Deploy.job.B.start=true;Deploy.job.A.status=Completed;Deploy.job.C.start=true;Deploy.job.B.status=Failed"
   | makemv data delim=";" | mvexpand
   data | streamstats count | eval
   _time = now() + count | rename data as _raw | table _time _raw

[somesoni2 · Accepted solution]

Gve this a try

sourcetype=test3  ("Deploy.name" OR "Deploy.startime" OR "Deploy.environment" OR "Deploy.joblist" OR "Deploy.jobselected")  | rex "Deploy.name=(?<Name>\S+)"
| rex "Deploy.startime=(?<Startime>\S+)"
| rex "Deploy.environment=(?<Environment>\S+)"
| rex "Deploy.joblist=(?<JobList>\S+)"
| rex "Deploy.jobselected=(?<Jobselected>\S+)"
| eval Startime=strftime(strptime(Startime,"%Y%m%d%H%M%S"),"%Y-%m-%d %H:%M")
| table Name Startime Environment JobList Jobselected | eval id=if(isnotnull(Name),1,0) | accum id | stats list(*) as * by id

[somesoni2]

Just add a sort on _time command before accum.

[chrismok]

Nearly finish~~ But the name field fall to row 2 now.

[lguinn2]

Try this then:

sourcetype=whateverItIs ("Deploy.name" OR "Deploy.startime" OR "Deploy.environment" OR "Deploy.joblist" OR "Deploy.jobselected")
| sort _time
| transaction startswith="Deploy.name=" endswith="Deploy.jobselected="
| rex "Deploy.name=(?<Name>\S+)"
| rex "Deploy.startime=(?<Startime>\S+)"
| rex "Deploy.environment=(?<Environment>\S+)"
| rex "Deploy.joblist=(?<JobList>\S+)"
| rex "Deploy.jobselected=(?<Jobselected>\S+)"
| eval Startime=strftime(strptime(Startime,"%Y%m%d%H%M%S"),"%Y-%m-%d %H:%M)
| table Name Startime Environment JobList Jobselected

[chrismok]

Sorry bother you again.

You can see that the splunk will descend the log seq as log don't provide the timestamp.

As a result, there's an error doing "transcation" command.

There may has another method to sort the log in ascending, but the other problem is the "Deploy.selected" may not append to log due to system in progress or system crashed.

To specific the wording in endswith is not prefect for all the cases.

No luck once delete the transaction command

[lguinn2]

This should do

sourcetype=whateverItIs
| rex "Deploy.name=(?<Name>\S+?);.*Deploy.startime=(?<Startime>\S+?);.*Deploy.environment=(?<Environment>\S+?);Deploy.joblist=(?<JobList>\S+?);.*Deploy.jobselected=(?<Jobselected>\S+?);"
| eval Startime=strftime(strptime(Startime,"%Y%m%d%H%M%S"),"%Y-%m-%d %H:%M)
| table Name Startime Environment JobList Jobselected

[chrismok]

Hi Iguinn,

Not work, also the log do not included a sybmol ";"

And please note each line is a single event

[lguinn2]

Well, you are showing a ";" in all your examples!!

[sk314]

what is the output that you get with your current search query?