Skip to main content
Syed
Contributor ⭐️⭐️⭐️
July 10, 2022
Solved

How to remove duplicate values from Union in Data Designer or from Rule?any suggestion?

  • July 10, 2022
  • 4 replies
  • 462 views

Combined 2 tables in Data Designer via Union but gives me duplicate rows.How should i remove them?

    Best answer by Wayne

    You could probably get away with just applying a filter where Serial 1 != null AND Serial 2 != null AND Serial 3 != null AND Serial 4 != null post this union. That would remove those blank rows. 

     

    Unless it’s possible for an ID to be blank in both data sets and you want to reflect that. If that is the case then you will need a different solution.

     

    Merging instead of union and then case statement your duplicate fields to combine them. If Table 1.Serial 1 != null then output Table 1.Serial 1 Else Output Table 2.Serial 1

    4 replies

    rakesh
    Gainsight Employee ⭐️
    rakesh
    Lets put your data to work!
    July 10, 2022

    @Syed 

    The best possible path is to avoid duplicates when you do a union itself. Make sure you use the right filters so that a single record doesnt come in both datasets. If there is any usecase that makes this impossible, could you please post that here? 

     

    That being said, one way to remove duplicates is to load the data to an(other) object and use correct identifiers. 

    SyedAuthor
    Contributor ⭐️⭐️⭐️
    July 10, 2022

    Hi @rakesh 

    Below are as an example 2 samples. when i union them.It gives me 4 rows.what filter i should add that

    it should not  show rows which have not values in Serial 1, Serial 2, serial 3 andIf any row has value iether of these fields then it should show.In this example Table 1 should show ID 2 and Table 2 should show ID 1

     

      jpgamboa
      jpgamboa
      Contributor ⭐️
      August 25, 2022

      @Syed - I believe I’m having the same issue as you. I was using Union to combine based on Advanced Outreach Name but it’s splitting up both like so. Is that correct?

       

       

      WayneAnswer
      Expert ⭐️
      August 26, 2022

      You could probably get away with just applying a filter where Serial 1 != null AND Serial 2 != null AND Serial 3 != null AND Serial 4 != null post this union. That would remove those blank rows. 

       

      Unless it’s possible for an ID to be blank in both data sets and you want to reflect that. If that is the case then you will need a different solution.

       

      Merging instead of union and then case statement your duplicate fields to combine them. If Table 1.Serial 1 != null then output Table 1.Serial 1 Else Output Table 2.Serial 1

      Terms & ConditionsAccessibility statement