Skip to main content
ashok_dugaputi
Helper ⭐️⭐️
Jun 28, 2017 3:47 PM UTC

Import a file from S3 to Bionic Rules directly

I've been hearing multiple use cases / suggestions around being able to import the data file from S3 into Bionic Rules. 

1) Eliminate the need for setting up S3 jobs and then running the rule separately and potentially avoid 24 hours of delay before data is processed and available for consumption.

2) Loading data to SFDC directly without needing to store in MDA as a staging area

3) Without having to think about missing schedules. With Bionic Rules, reading the data from S3 and processing can be one rule.

4) Avoid storing unwanted data. Dimensionality reduction, filtering the bad quality records, applying the transformations.

5) Merging multiple files into one data set before loading. 

What are the other supporting use cases that you can think of?

    18 replies

    sdrostgainsightcom
    Gainsight Employee ⭐️⭐️⭐️
    Jun 28, 2017 7:28 PM UTC
    Love the general idea!  Turns Bionic rules into a 1-stop shop for data ingest and transformation.

    In particular, #'s 1, 3, and 5 get my vote for the strongest use cases in the list. Basically sets up a way to allow data transformation, if nothing else, to occur immediately following the arrival of new data.

    I'd want to talk through #2, 'load directly to SFDC' in a very detailed fashion - a lot of implications there for consequences to the SFDC-side data that may not have any bearing on Gainsight, which always makes me nervous - basically would want that to be a feature added later.

    One other use case:

    -- Allowing an opportunity to load custom field mappings (static data points like "load date" or "data source") as part of a file ingest as well - formula fields with Today(), for instance, for Load Date, etc.
    Translate
    Scott Drost, Gainsight -- Principal Customer Success Architect
    sundar
    Expert ⭐️
    Jun 30, 2017 10:02 AM UTC
    Thanks Ashok for bringing this up! Can't agree more, I would love to hear more perspectives.
    Translate
    roja_kode
    Helper ⭐️
    Jun 30, 2017 10:22 AM UTC
    Thanks Ashok for the idea!
    (1-3) will address missing connect flight scenarios and in turn reduces data issues. I believe, it's one of the awaiting feature for many of the customers.
    Translate
    roja_kode
    Helper ⭐️
    Jul 14, 2017 5:56 AM UTC
    Another use case is that for standalone customers, for loading Customers list, currently I have to load to MDA object (staging), load to Customer then load to Company via Gainsight Connect. If we have S3 as source, then directly I can load to Customer Info.
    Translate
    sri_gali_ibm
    Contributor ⭐️
    Jul 14, 2017 12:50 PM UTC
    Thanks Ashok for the ideas!
    Can't wait for #1,4,5 to happen. There would be tremendous productivity benefits with these ideas. I believe some of them may be in the pipeline. But overall, I like the way Gainsight (You) take client feedback seriously.
    Keep up with the good work!
    Translate
    jitin_mehndiratta
    Helper ⭐️⭐️⭐️
    Jan 29, 2018 11:00 AM UTC
    Thanks everyone for your inputs.

    With Winter release, you will be able to import data directly from S3 file and be able to do transformations on that.

    There is a new type of task(s3) in bionic rules. This task will read an s3 file (user will get the flexibility to define the format of the s3 file similar to export to S3 functionality that we have today) and load show fields based on the columns in the s3 file. Once the file is loaded, the show fields will be available for selection by user.

    Once a file is loaded, show fields cannot be synced with the S3 file again. It always has to be a new task. 
    Translate
    tanya_strauss
    Contributor ⭐️⭐️⭐️
    Feb 27, 2018 6:20 PM UTC
    @Ashok - would this solution allow us to connect to a client's own S3 bucket? If so, that's a use case one of my clients would love to see. They are concerned about data at rest and outside their custody.
    Translate
    Director, Customer Success | Druva
    ashok_dugaputi
    Helper ⭐️⭐️
    Feb 28, 2018 9:47 PM UTC
    @Tanya - Current release does not support reading from client's own S3 bucket but this is something being planned in the near term roadmap.
    Translate
    darkknight
    Expert ⭐️
    Feb 28, 2018 10:12 PM UTC
    I tried this today, selecting the "GainsightManaged" and clicking either Load Column Details or Preview, but I keep getting an error "Error occured while loading the file. The specified key does not exist. (Service: Amazon S3; Status Code: 404; Error Code: NoSuchKey; Request ID: 6C0DCF3B759BA46B)"

    Meanwhile, S3 jobs in the Connectors tab are running successfully without issue.
    Translate
    Jeff Kirkpatrick
    darkknight
    Expert ⭐️
    Feb 28, 2018 11:33 PM UTC
    Tried it again in a different (demo) org....same error:

    "Error occured while loading the file. The specified key does not exist. (Service: Amazon S3; Status Code: 404; Error Code: NoSuchKey; Request ID: F3F4D19E1C2050D0)"

    I was able to do an S3 Connector import without a problem after this.
    Translate
    Jeff Kirkpatrick