Skip to main content

Data Design Previews Don't Function as Expected

  • 10 May 2024
  • 4 replies
  • 66 views

The Data Design preview function, more specifically the Merge Preview function, is borderline useless. Maybe 1 out of 20 times will the preview actually generate anything. Is it because the merge has 0 results? Is it because I merged on the wrong fields? Maybe I have the wrong join and it should be left instead of right?

Impossible to know without fully running the data design, and if you’re working through multiple joins within a design you have to join a potentially non-functioning merge into the full data set - whereas if the preview actually functioned as expected I could adjust as needed without having to complete the design. When working with large data sets, it’s extremely time consuming to always have to run a DD for each Merge to validate the output when the Preview feature should function. 

I don’t remember having this issue so often in the past. 

Lies

 

4 replies

dayn.johnson
Userlevel 5
Badge +4

Definitely noticed this! I’ve ended up previewing each data set, rather than the merge.

At least it’s more functional than the participant preview in Dynamic JO… I’ve found it more useful to build out a Data Design template that I can run a report on to preview the full participant list, and then pull that template into a query to take advantage of the in-program sync when the program runs.

"All knowledge is connected to all other knowledge. The fun is in making the connections." – Arthur C. Aufderheide
lori_dinardo
Badge

I filter down to one or two records that I know should end up in my final merge at every data point. Then I can preview the final and make sure those 2 records followed through the process.  Once I get the result I expect then I remove those filtered records and run the whole thing.  I have wasted endless hours searching for missing records when they weren’t really missing, it was just the preview decided not to show them. 

spencer_engel
Userlevel 7
Badge +3

There has to be a better way for this to work. One of the big benefits of horizon rules was supposed to be the ability to preview each step of the dataset. But when the preview hardly ever works, we just have to test run the rule and download a bunch of CSVs anyway just like we used to.

Dheepak G
Badge

Hi @kelly I understand your frustration with the preview function, especially when dealing with complex merges and large data sets. This issue you are encountering relates to how the preview functionality is designed to work.

The Merge Preview function operates by randomly selecting a subset of records to display in the preview. This subset is not always representative of the entire data set or the results of the merge operation. As a result, if the randomly selected records do not meet the filters or merge conditions set in rule, the preview can appear empty or not reflect the actual results you would get from running the actual rule. 
 

One workaround for now as suggested by @lori_dinardo would be to add a filter condition of the records that could satisfy the conditions and then running a preview, i know that this is not an ideal solution but we are looking into ways specifically from UI on how to make this preview function more effectively

Reply


Terms & ConditionsCookie settings