Various fixes found with sqlite fuzzer #1021

jussisaurio · 2025-02-16T09:41:42Z

Discovering tons of shit with @sivukhin 's sqlite fuzzer

NOTE: this is an exploration PR and I will probably cherry-pick smaller PRs out of this. Mainly sqlite's affinity rules are giving tons of headaches since we're not implementing it properly right now.

PRs extracted so far:

#1023
#1024
#1025
#1031

Adds missing support:

Support all the same literals in WHERE clause position as in SELECT position
Support CAST in WHERE clause position
Support FunctionCall in WHERE clause position
Support Column in WHERE clause position
Support Rowid in WHERE clause position
Support CASE in WHERE clause position
Support LIKE in SELECT position
Support Unary expressions in WHERE clause position
Support rest of the Binary expressions in WHERE clause position
Support TEXT in remainder operations

Fixes:

Fix concat losing decimal precision of floats in text representation
Fix remainder panic on zero right-hand-side
Fix wrong text->numeric coercion in NOT context
Fix incorrect text->numeric coercion in logical OR
Fix equality comparisons for OwnedValue
Fix incorrectly always returning False for text in exec_if()
Fix never evaluating constant conditions at all
Fix cast_text_as_real() not allowing partial parse of exactly 1 char
Fix incorrect text->numeric coercion in CAST
Fix incorrect implementation of cast_text_to_numerical()
Remove incorrect constant folding optimization for NULL

Testing utils:

Enhance sqlite fuzzer to mostly be able to work with the same set of possible expressions in both SELECT and WHERE clause position

core/translate/expr.rs

…_expr()

…ition_expr()

…dition_expr()

…d ops in translate_condition_expr()

…s incorrect

…icate position

…i Saurio This PR is sourced from the fuzzing exploration PR in #1021 **Adds missing support:** Support all the same literals in WHERE clause position as in SELECT position Support CAST in WHERE clause position Support FunctionCall in WHERE clause position Support Column in WHERE clause position Support Rowid in WHERE clause position Support CASE in WHERE clause position Support LIKE in SELECT position Support Unary expressions in WHERE clause position Support rest of the Binary expressions in WHERE clause position Support TEXT in remainder operations **Fix:** Remove incorrect constant folding optimization for NULL **Testing utils:** Enhance sqlite fuzzer to mostly be able to work with the same set of possible expressions in both SELECT and WHERE clause position Closes #1024

…rom Jussi Saurio This PR is extracted from the sqlite fuzzing exploration effort in #1021 --- We were not evaluating constant conditions (e.g '1 IS NULL') when there were no tables referenced in the query, because our WHERE term evaluation was based on "during which loop" to evaluate them. However, when there are no tables, there are no loops, so they were never evaluated. Closes #1023

PThorpe92 · 2025-02-19T00:24:56Z

core/vdbe/insn.rs

@@ -1023,64 +1057,71 @@ pub fn exec_boolean_not(mut reg: &OwnedValue) -> OwnedValue {
        OwnedValue::Null => OwnedValue::Null,
        OwnedValue::Integer(i) => OwnedValue::Integer((*i == 0) as i64),
        OwnedValue::Float(f) => OwnedValue::Integer((*f == 0.0) as i64),
-        OwnedValue::Text(text) => exec_boolean_not(&cast_text_to_numerical(text.as_str())),
+        OwnedValue::Text(text) => exec_boolean_not(&cast_text_to_real(text.as_str())),
        _ => todo!(),
    }
 }

 pub fn exec_concat(lhs: &OwnedValue, rhs: &OwnedValue) -> OwnedValue {


we can shorten this here (I know maybe unrelated change but if it's being changed anyway)

if let OwnedValue::Agg(agg) = lhs { lhs = agg.final_value(); } // rhs..

Yeah true! Anyway I'm not planning to get this merged as is, I was using this branch as a mega-exploration to discover things

pedrocarlo · 2025-02-21T05:12:35Z

Hey @jussisaurio, with the fuzzer I was trying to solve some issues with cast_text_to_numeric as well. I created a PR #1038. Still trying to port correctly the implementation from sqlite. Would appreciate some feedback with my exploration as well.

jussisaurio force-pushed the expr-fixes branch 3 times, most recently from c7949a6 to 22c7286 Compare February 16, 2025 10:04

jussisaurio commented Feb 16, 2025

View reviewed changes

core/translate/expr.rs Outdated Show resolved Hide resolved

jussisaurio force-pushed the expr-fixes branch from 22c7286 to 423a0ad Compare February 16, 2025 10:25

jussisaurio added 15 commits February 16, 2025 18:10

expr.rs: Literal: call translate_expr() from translate_condition_expr()

aeab678

expr.rs: Cast: call translate_expr() from translate_condition_expr()

cf18a7d

expr.rs: FunctionCall: call translate_expr() from translate_condition…

ff1ad49

…_expr()

expr.rs: Column: call translate_expr() from translate_condition_expr()

66c27a1

expr.rs: Rowid: call translate_expr() from translate_condition_expr()

195966b

expr.rs: Case: call translate_expr() from translate_condition_expr()

3380a08

expr.rs: Like: use shared impl in translate_expr() and translate_cond…

4636a3c

…ition_expr()

expr.rs: Unary: use shared impl in translate_expr() and translate_con…

d0c3122

…dition_expr()

expr.rs: Binary: use translate_expr()'s impl for currently unsupporte…

ae65530

…d ops in translate_condition_expr()

optimizer.rs: remove constant folding optimization for NULL since it'…

9b259c2

…s incorrect

tests/fuzz: use mostly the same expression options in select and pred…

8fb2995

…icate position

Add more TCL tests for exprs in select/where positions

a053e94

Fix equality comparisons for OwnedValue

f5ae011

Fix wrong text->numeric coercion in NOT context

d9bbd29

Fix remainder panic on zero right-hand-side

5d2e21e

jussisaurio force-pushed the expr-fixes branch 2 times, most recently from c644708 to 5a70745 Compare February 16, 2025 17:19

jussisaurio changed the title ~~Expr fixes~~ Various fixes found with sqlite fuzzer Feb 16, 2025

jussisaurio added 4 commits February 16, 2025 19:49

Fix concat losing decimal precision of floats in text representation

c08423b

Support TEXT in remainder operations

2e21e13

Fix incorrect text->numeric coercion in logical OR

75bbd1a

Fix incorrectly always returning False for text in exec_if()

adadfab

jussisaurio force-pushed the expr-fixes branch from c44da4d to c97c3ab Compare February 16, 2025 17:50

jussisaurio added 2 commits February 16, 2025 20:03

Fix never evaluating constant conditions at all

e41f5fb

Fix cast_text_as_real() not allowing partial parse of exactly 1 char

a4c4887

jussisaurio added 2 commits February 16, 2025 20:03

Fix incorrect text->numeric coercion in CAST

d408dec

Fix cast_text_to_* panicing on empty input

686ed2f

jussisaurio force-pushed the expr-fixes branch 2 times, most recently from a6a92d6 to cfff690 Compare February 16, 2025 18:18

Fix incorrect implementation of cast_text_to_numerical()

0c6e417

jussisaurio force-pushed the expr-fixes branch from cfff690 to 0c6e417 Compare February 16, 2025 19:46

This was referenced Feb 17, 2025

Fix not evaling constant conditions when no tables in query #1023

Closed

Add some missing supported features in WHERE clause #1024

Closed

PThorpe92 reviewed Feb 19, 2025

View reviewed changes

pedrocarlo mentioned this pull request Feb 22, 2025

Add support for VALUES statement #1049

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Various fixes found with sqlite fuzzer #1021

Various fixes found with sqlite fuzzer #1021

jussisaurio commented Feb 16, 2025 •

edited

Loading

PThorpe92 Feb 19, 2025

jussisaurio Feb 19, 2025

pedrocarlo commented Feb 21, 2025

Various fixes found with sqlite fuzzer #1021

Are you sure you want to change the base?

Various fixes found with sqlite fuzzer #1021

Conversation

jussisaurio commented Feb 16, 2025 • edited Loading

PThorpe92 Feb 19, 2025

Choose a reason for hiding this comment

jussisaurio Feb 19, 2025

Choose a reason for hiding this comment

pedrocarlo commented Feb 21, 2025

jussisaurio commented Feb 16, 2025 •

edited

Loading