I want to build a small dashboard that offers a quick view into emails a user has recently received. If I use this, I can get the sender address tabled properly:
(1). index=office365
| table P1Sender
| search P1Sender=*
If I use this, I can get recipient count, recipient names and subject tabled properly:
(2). index=office365
| table Item.RecipientsCount, Item.Recipients{}.Name, Item.Subject
| search Item.Recipients{}.Name=*, Item.Subject=*
My problem is I cannot get them tabled together in the same results view. If I try and add P1Sender to the (2) SPL table, I get an empty column. If I add P1Sender=* to the (2) search statement I get 0 results in every column.
Any advice? Screenshots of SPL and results in replies.
My problem is I cannot get them tabled together in the same results view. If I try and add P1Sender to
This statement confirms @PickleRick and @ITWhisperer 's diagnosis: that field P1Sender only exists in one group of events, let's call them "event type 1", whereas fields Item.RecipientsCount, Item.Recipients{}.Name, Item.Subject are only present in a totally different group of events, let's call them "event type 2". (Most likely they are from two different sourcetypes in index office365.) The two groups have no overlap. The implied message they are trying to tell you when they ask you what your data look like is this: Unless you can demonstrate that the two groups have something in common, the result you desire, namely "quick view into emails a user has recently received," is an impossibility.
To make this obvious, let me give you two groups of mock events
| P1Sender |
| joe@example1.com |
| jab@example2.com |
| joey@example3.com |
and
| Item.RecipientsCount | Item.Recipients{}.Name | Item.Subject |
| 2 | bob@myco.com bobby@myco.com | You owe me $1,000,000. Where is my money? |
| 1 | rob@myco.com | I paid $1,000,000. Where is my shipment? |
Can you tell me who receive E-mail from whom?
Volunteers in this forum all understand privacy concerns. We ask what your data look like because we want to help you identify the link between these two groups. You have two paths forward:
As you have not realized that the two groups of events have no overlap, let me help you with the second path.
Run these two searches separately.
index=office365 P1Sender=* ``` event type 1 ```and
index=office365 Item.Recipients{}.Name=*, Item.Subject=* ``` event type 2 ```Inspect events from each type, find events that can form a "quick view into emails a user has recently received." Then, anonymize those events to share here.
This is some homework you should have done before posting question into a data analytics forum. These are four golden rules; nay, call them the four commandments:
Your events do not have values for P1Sender at the same time as values for the other fields you are searching on. You need to find a field that has corresponding values in both sets of events so you can correlate the data. As @PickleRick says, we have no idea what your data looks like!
The data seems to be visible in the events. I can see the fields in the Selected Fields section as well as within the events themselves. So the data is there, I'm just not sure why I can see it but not table it with the other information I want to table.
The data might "be there" in general but it might not be all in the same events.
Since you're dealing with email data, often email solutions, unless you have some forms of report-level data, report events from different parts of processing queue separately. And those events might include different sets of fields. For example, "intake" events about the SMTP receiving process would include source IP for the connection from the originating system and possibly authentication data if the connection was over submission port from a user's agent, not from another MTA over standard 25/TCP port. Then you might get some portion of info (for example, sender's envelope adress and receiver's envelope address) in another event from "somewhere in the middle" of the procesing queue and finally you'd have an event on local delivery including expanded destination aliases.
So if you wanted to get the full picture about a single email, you'd have to correlate several separate events (typically by some form of internal ID from your mail system) because no single event would contain the whole set of information you're looking for.
I suspect (since you haven't shown us your data that's all I can do) this is your case. The events which contain values for one field you're searching for probably don't contain values for other fields. Yes, you'd still see values in the aggregated report in "interesting fields" but the values would be scattered among the events.
I hope it's clearer now.
Ok. From the top.
1. What does your data look like? We have no idea what is in your events. From our point of view the info might simply not be available within the same events.
2. As a rule of thumb the table command should not be used anywhere else but at the very end of your search to transform your results to... well, a table for presentation purposes. It's not meant to be used mid-search.
Point 2 is a "good practice" remark but probably it's because of p.1 - your data itself.
Adding P1Sender to search statement blocks everything and nothing gets returned.
Recipient count, recipient names and subject all work as well. But trying to add in P1Sender, nothing happens.
P1Sender by itself works. I blocked out most of the information for privacy reasons.