Skip to main content
ashok_dugaputi
Helper ⭐️⭐️
June 28, 2017

Import a file from S3 to Bionic Rules directly

  • June 28, 2017
  • 18 replies
  • 159 views
I've been hearing multiple use cases / suggestions around being able to import the data file from S3 into Bionic Rules. 

1) Eliminate the need for setting up S3 jobs and then running the rule separately and potentially avoid 24 hours of delay before data is processed and available for consumption.

2) Loading data to SFDC directly without needing to store in MDA as a staging area

3) Without having to think about missing schedules. With Bionic Rules, reading the data from S3 and processing can be one rule.

4) Avoid storing unwanted data. Dimensionality reduction, filtering the bad quality records, applying the transformations.

5) Merging multiple files into one data set before loading. 

What are the other supporting use cases that you can think of?

    18 replies

    sdrostgainsightcom
    Gainsight Employee ⭐️⭐️⭐️
    June 28, 2017
    Love the general idea!  Turns Bionic rules into a 1-stop shop for data ingest and transformation.

    In particular, #'s 1, 3, and 5 get my vote for the strongest use cases in the list. Basically sets up a way to allow data transformation, if nothing else, to occur immediately following the arrival of new data.

    I'd want to talk through #2, 'load directly to SFDC' in a very detailed fashion - a lot of implications there for consequences to the SFDC-side data that may not have any bearing on Gainsight, which always makes me nervous - basically would want that to be a feature added later.

    One other use case:

    -- Allowing an opportunity to load custom field mappings (static data points like "load date" or "data source") as part of a file ingest as well - formula fields with Today(), for instance, for Load Date, etc.
    Scott Drost, Gainsight -- Principal Customer Success Architect
    sundar
    Expert ⭐️
    June 30, 2017
    Thanks Ashok for bringing this up! Can't agree more, I would love to hear more perspectives.
    roja_kode
    Helper ⭐️
    June 30, 2017
    Thanks Ashok for the idea!
    (1-3) will address missing connect flight scenarios and in turn reduces data issues. I believe, it's one of the awaiting feature for many of the customers.
    roja_kode
    Helper ⭐️
    July 14, 2017
    Another use case is that for standalone customers, for loading Customers list, currently I have to load to MDA object (staging), load to Customer then load to Company via Gainsight Connect. If we have S3 as source, then directly I can load to Customer Info.
    sri_gali_ibm
    Contributor ⭐️
    July 14, 2017
    Thanks Ashok for the ideas!
    Can't wait for #1,4,5 to happen. There would be tremendous productivity benefits with these ideas. I believe some of them may be in the pipeline. But overall, I like the way Gainsight (You) take client feedback seriously.
    Keep up with the good work!
    jitin_mehndiratta
    Helper ⭐️⭐️⭐️
    January 29, 2018
    Thanks everyone for your inputs.

    With Winter release, you will be able to import data directly from S3 file and be able to do transformations on that.

    There is a new type of task(s3) in bionic rules. This task will read an s3 file (user will get the flexibility to define the format of the s3 file similar to export to S3 functionality that we have today) and load show fields based on the columns in the s3 file. Once the file is loaded, the show fields will be available for selection by user.

    Once a file is loaded, show fields cannot be synced with the S3 file again. It always has to be a new task. 
    tanya_strauss
    Contributor ⭐️⭐️⭐️
    February 27, 2018
    @Ashok - would this solution allow us to connect to a client's own S3 bucket? If so, that's a use case one of my clients would love to see. They are concerned about data at rest and outside their custody.
    Director, Customer Success | Druva
    ashok_dugaputi
    Helper ⭐️⭐️
    February 28, 2018
    @Tanya - Current release does not support reading from client's own S3 bucket but this is something being planned in the near term roadmap.
    darkknight
    Expert ⭐️
    February 28, 2018
    I tried this today, selecting the "GainsightManaged" and clicking either Load Column Details or Preview, but I keep getting an error "Error occured while loading the file. The specified key does not exist. (Service: Amazon S3; Status Code: 404; Error Code: NoSuchKey; Request ID: 6C0DCF3B759BA46B)"

    Meanwhile, S3 jobs in the Connectors tab are running successfully without issue.
    Jeff Kirkpatrick
    darkknight
    Expert ⭐️
    February 28, 2018
    Tried it again in a different (demo) org....same error:

    "Error occured while loading the file. The specified key does not exist. (Service: Amazon S3; Status Code: 404; Error Code: NoSuchKey; Request ID: F3F4D19E1C2050D0)"

    I was able to do an S3 Connector import without a problem after this.
    Jeff Kirkpatrick