Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixed relevancy ordering for Full-Text #1957

Merged
merged 2 commits into from
Aug 22, 2023
Merged

Conversation

Hydrocharged
Copy link
Contributor

@Hydrocharged
Copy link
Contributor Author

This might be an efficient way to implement the table column requirement for MATCH, but that depends on a big assumption that appears to hold true. While looking through the scope logic, it looked like we always start with the fromScope constructed from the target table, and we then append projected columns, extra columns, etc. onto that initial scope. That leads to the first n columns always being the table columns, in their schema ordering. With that, I can always grab the table row with row[:len(tableSchema)]. This makes everything extremely straightforward, and makes this a relatively small change.

@Hydrocharged Hydrocharged force-pushed the daylon/relevancy-ordering branch from a398c2d to 35f5bbb Compare August 21, 2023 13:30
Copy link
Contributor

@max-hoffman max-hoffman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is the right idea. The fromScope cols are the input deps you want. But my first reaction is that this will panic for any queries that use projection pruning. I see that you disable the projection pruning rule for match expressions, so it is a smaller subset of queries will panic -- aggregations and windows. So either 1) we disable aggregation/window mixing with MA, 2) we ship and cut a ticket expecting someone to hit this panic soon, 3) you use the scope extraCols to register MA dependencies and embed those expressions in MatchAgainst for index fixup/execution reference. The projection pruning guard rail is probably unnecessary after (3).

> ALTER TABLE test ADD COLUMN extracol2 INT DEFAULT 7;
> SELECT pk, count(pk), MATCH(doc) AGAINST('aaaa') as relevancy FROM test ORDER BY relevancy DESC;

panic: runtime error: slice bounds out of range [:4] with capacity 3 [recovered]
	panic: runtime error: slice bounds out of range [:4] with capacity 3

goroutine 10 [running]:
testing.tRunner.func1.2({0x1f72700, 0xc00004aa38})
	/usr/local/go/src/testing/testing.go:1545 +0x238
testing.tRunner.func1()
	/usr/local/go/src/testing/testing.go:1548 +0x397
panic({0x1f72700?, 0xc00004aa38?})
	/usr/local/go/src/runtime/panic.go:914 +0x21f
github.com/dolthub/go-mysql-server/sql/expression.(*MatchAgainst).Eval(0xc0004deca8?, 0x3?, {0xc0004f1ad0?, 0x2?, 0xc00065a500?})
	/Users/max-hoffman/go/src/github.com/dolthub/go-mysql-server/sql/expression/matchagainst.go:87 +0x165
github.com/dolthub/go-mysql-server/sql/expression.(*Alias).Eval(0xc00007d7c0?, 0xc00026fb08?, {0xc0004f1ad0?, 0x3?, 0x1e6f8a0?})
...

@Hydrocharged Hydrocharged force-pushed the daylon/relevancy-ordering branch from 35f5bbb to 67af087 Compare August 22, 2023 12:42
@Hydrocharged Hydrocharged force-pushed the daylon/relevancy-ordering branch from 67af087 to 7e88fc4 Compare August 22, 2023 12:43
@Hydrocharged
Copy link
Contributor Author

Hydrocharged commented Aug 22, 2023

A quick stab at #3 didn't lead to much success, so I'll go with #1 for now and make an issue linking to this PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

order by a fulltext match column does not work
2 participants