topic Re: How to use same field with same value to get result from two different sourcetype? in Getting Data In

How to use same field with same value to get result from two different sourcetype?

titanwss — Thu, 10 Oct 2013 06:55:12 GMT

sample data like followed.

sourcetype A (like access log):

clientip uri_path status other fields.... 10.0.0.1 /bar.html 200 ... 10.0.0.2 /admin/ 200 ...

sourcetype B (like security audit log):

clientip uri_path status other fields.... 192.168.0.1 /foo.html 403 ... 192.168.1.1 /admin/ 403 ...

These sourcetypes have three same fields. I just want get the result like

clientip uri_path statusB statusA 192.168.1.1 /admin/ 403 200

To find out if uri_path in sourcetype B is existed(200)

Re: How to use same field with same value to get result from two different sourcetype?

gfuente — Thu, 10 Oct 2013 07:55:35 GMT

Hello

You can get this by several ways, but i think that the best performance would be something like:

sourcetype="A" OR sourcetype="B" | stats values(statusB), values(statusA) by clientip, uri_path

Other way would be using the join command, but i think it is worse from a performance point of view

Regards

Re: How to use same field with same value to get result from two different sourcetype?

titanwss — Fri, 11 Oct 2013 09:43:48 GMT

Thanks for your reply. I've tried what you said, but it return nothing. Finally, I use append and distinct_count command to deal with it.

Re: How to use same field with same value to get result from two different sourcetype?

bwooden — Fri, 11 Oct 2013 11:20:06 GMT

The append command can be useful in some scenarios.

But generally you'll get better performance (and not be restricted by result limits associated with the append command) if you use one all-encompassing base search.

Using the example in your original question, it would look something like this:

sourcetype=A OR sourcetype=B | eval status{sourcetype}=status | stats count(status*) by clientip uri_path

The status{sourcetype} will actually create distinct fields based on your sourcetype name. In the above example, it would create a field called statusA for events with sourcetype of A and a field called statusB for events having a sourcetype of B. The status* (in the stats command) includes both of those. Using these techniques, we can expand the base search with additional sourcetypes and not have to worry about updating our search downstream to take advantage of that expansion.

Re: How to use same field with same value to get result from two different sourcetype?

bwooden — Fri, 11 Oct 2013 11:22:53 GMT

reading your original question again, you may actually want this search (replacing count with values) --> sourcetype=A OR sourcetype=B | eval status{sourcetype}=status | stats values(status*) by clientip uri_path

Re: How to use same field with same value to get result from two different sourcetype?

titanwss — Sat, 12 Oct 2013 01:32:26 GMT

I did make some mistakes, but I don't know what it is now. Actually yours works well.XD

Re: How to use same field with same value to get result from two different sourcetype?

titanwss — Sat, 12 Oct 2013 01:56:50 GMT

The status{sourcetype} seems to be very useful. There must be something wrong in my search. Thanks for telling these techniques.