Skip to main content

HI Team, We’ve got a request from customer.

When previewing we are not seeing the correct data for the merge... We see accounts with missing data from the first fetch but if you filer for that account you see all the data.

To show merge results we need to have matched results from both fetch tasks. As in the preview, we are only fetching 2k records. we might not have matched data from the right dataset (in fetched 2k records). That's why the preview not showing data without a filter.

All data is present in the report as well it is just the preview is misleading and added confusion to the customer

Hi Aditya,

Purpose of preview is to quickly provide feedback to the customers on if their configuration is correct or not. For this, it is important for us to return the results in a short time. To achieve this, what we do is we sample 2000 records from every individual fetch and then process those 2k records. While this solves the problem for most of the scenarios, it wouldn't help in all the scenarios. Assume few customers are not present in 2k records from Fetch from Company and assume when you join Company with Usage data we only have usage data of the customers who are not in the first 2k in fetch from Company. In this scenario, no records will show in preview. 

If we increase from 2k to lets say 5k, we are still not 100% solving this but are making every preview run slower. We have analyzed various combinations and have decided 2k records is ideal for preview. There are 2 things that the admin can do here:

  1. Filter for a single company - see if the data prep config is working or not 
  2. Go to explore, click run and wait for complete data prep to be done and then explore the data.

Reply


Terms & ConditionsCookie settings