Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DateFacet suggestions are incredibly slow against large tables #2407

Closed
simonw opened this issue Aug 21, 2024 · 0 comments
Closed

DateFacet suggestions are incredibly slow against large tables #2407

simonw opened this issue Aug 21, 2024 · 0 comments

Comments

@simonw
Copy link
Owner

simonw commented Aug 21, 2024

I thought the limit 100 would help here, but it turns out that for a column on a huge table (e.g. 20m rows) which has no dates in at all that glob has to scan through every value of the column every time, just to figure out that there are NOT 100 matches!

# Does this column contain any dates in the first 100 rows?
suggested_facet_sql = """
select date({column}) from (
{sql}
) where {column} glob "????-??-*" limit 100;
""".format(
column=escape_sqlite(column), sql=self.sql
)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant