Skip to main content
Contributor ⭐️
June 3, 2024
New Idea

Query Builder Problem - Distribution of Property value buckets are skewed by data, making them unusable

Related products:PX Analytics
  • June 3, 2024
  • 0 replies
  • 13 views

Context

When using the ‘distribution of property’ option in the query builder, the data is automatically allocated into buckets that are defined by splitting the range of the data into five equal parts. 

ie if the range of the data is 0-100, the buckets will be something like 0-20, 21-40, 41-60, 61-80, 81-100.

 

Problem

Outliers in the data will heavily skew the result, making the feature unusable. For example
 

You have 100 logged events
- 90 of them have a value between 1-10
- 5 have a value of 11-20
- 5 have a value of 100

the buckets will be split from 0-20, 21-40, 41-60, 61-80, 81-100.

You'll then have 95 of your events in bucket 1 (0-20) and 5 in bucket 5 (81-100)

 

As a result you have no granularity on the bulk of the data which is all collated in 0-20, meaning you can’t extract any learnings

 

Suggested solutions

  1. Allow the user to set the bucket sizes, including an outer limit. ie 1”-3, 4-6, 7-9, 10+”
    1. It would also be helpful in this case to be able to set the number of buckets
  2. Allow the user to manually not include certain data samples to remove the skew.
    1. The caveat here is that if you have a lot of data this could take some time...so maybe it could be done as a part of the query
    2. ie ‘”Don’t include # > 20”