Skip to main content
Jef Vanlaer
Jef Vanlaer
Helper ⭐️⭐️⭐️
August 25, 2021
Closed (Won't Do)

Export of >100000 records from Horizon Analytics reports

  • August 25, 2021
  • 7 replies
  • 131 views

Data download from Horizon Analytics reports is apparently limited to 100000 records (lines) and according to documentation, export to S3 via Rules Engine should be used when more records need to be exported. This makes it really hard for end-users without Rules Engine access to export large dataset (e.g., usage data) from reports to do further analysis.

The option to export to csv (via an email link when the file was ready) available on the old reports was great for these purposes. I'd be very happy to see this option (or another way to easily download larger datasets) return.

    7 replies

    sai_ram
    sai_ram
    Expert ⭐️⭐️
    August 27, 2021

    @Jef Vanlaer I will get back to you with the limitation. Thanks.

    rahul_prayakarao
    rahul_prayakarao
    Helper ⭐️⭐️
    September 1, 2021

    @Jef Vanlaer Thanks for the input. A clarification on this - we never supported export of reports with more than 100,000 records. There are two reasons for this

    • When you are exporting that much amount of data, it is more a data export than a report export and 
    • Most importantly, processing that much data will negatively impact the user experience of other users who are exporting reports as they will have to wait longer than expected to download their reports.
    Jef Vanlaer
    Jef VanlaerAuthor
    Helper ⭐️⭐️⭐️
    September 1, 2021

    @rahul_prayakarao I was under the impression that that second reason explained why larger exports were provided via email before. This way, they can be scheduled and processed in the back-end, so performance is not negatively impacted for other users.

    rahul_prayakarao
    rahul_prayakarao
    Helper ⭐️⭐️
    September 3, 2021

    @Jef Vanlaer If I understand your request correctly, it will be sufficient if we give this capability while only scheduling the report exports and not while downloading it directly?

      Jef Vanlaer
      Jef VanlaerAuthor
      Helper ⭐️⭐️⭐️
      September 3, 2021

      @rahul_prayakarao Absolutely

        rahul_prayakarao
        rahul_prayakarao
        Helper ⭐️⭐️
        September 3, 2021

        Thank you for the confirmation. Please give me some time to validate this request and get back to you.

        Kaffersan
        Kaffersan
        Contributor ⭐️⭐️
        April 7, 2022

        Data download from Horizon Analytics reports is apparently limited to 100000 records (lines) and according to documentation, export to S3 via Rules Engine should be used when more records need to be exported. This makes it really hard for end-users without Rules Engine access to export large dataset (e.g., usage data) from reports to do further analysis.

        The option to export to csv (via an email link when the file was ready) available on the old reports was great for these purposes. I'd be very happy to see this option (or another way to easily download larger datasets) return.

        We often suffer the same issue with our end users when exporting customer usage reports. Fully understand the performance impact but the proposed alternative with a link would be just fine.

          Terms & ConditionsAccessibility statement