Skip to main content
kevin_ly
Helper ⭐️
July 8, 2020
Open

AWS S3 Integration : Have Option To Create New Dated Folder Each Day At the Target To Hold Yesterday's Export

Related products:PX Integrations
  • July 8, 2020
  • 2 replies
  • 76 views

Posting on Behalf of a Customer.

 

Currenlty when getting daily PX  export files from the AWS S3 Integration, it would drop everything at the target destination at one level. Eventually you will have many files in the target destination which can go pretty long. 

 

Example (Its one file here, but each days sends over 10 different files):

GainsightPX_User_Feature_RollUps_Day1

GainsightPX_User_Feature_RollUps_Day2

GainsightPX_User_Feature_RollUps_Day3

GainsightPX_User_Feature_RollUps_Day4

 

It would make more sense to have a folder created each day to contain that days data and would help load time of directory, as you would only open the day that is needed to ingest at the target.

 

Example (Instead of “Day 1” we be doing a date “2020-07-08”:

Day 1(Folder)/GainsightPX_User_Feature_RollUps_Day1

Day 2 (Folder)/GainsightPX_User_Feature_RollUps_Day2

Day 3 (Folder)/GainsightPX_User_Feature_RollUps_Day3

Day 4 (Folder)/GainsightPX_User_Feature_RollUps_Day4

 

2 replies

angelo
Helper ⭐️⭐️
July 8, 2020

Traditionally, customers appreciate that all of the export files are placed on the same directory. This ensures that only unprocessed files are picked up, processed and then moved to their archive folder.

 

 

 

kevin_ly
kevin_lyAuthor
Helper ⭐️
July 8, 2020

Thanks Angelo! That makes sense!