Fix wrong precision

[tangjiangling]

Fixes: #807

com.clickhouse.jdbc.ClickHouseDatabaseMetaData#getColumns
returns the wrong precision, which is against the JDBC standard.

[CLAassistant]

All committers have signed the CLA.

[tangjiangling]

A related issue(trinodb/trino#10541)

[tangjiangling]

cc: @ebyhr @hashhar

[github-actions]

Benchmark                                 (client)  (connection)  (statement)  (type)   Mode  Cnt     Score     Error  Units
Basic.insertOneRandomNumber  clickhouse-http-jdbc1         reuse       normal  object  thrpt   20   240.527 ±  33.461  ops/s
Basic.insertOneRandomNumber  clickhouse-http-jdbc1         reuse     prepared  object  thrpt   20   237.337 ±  26.340  ops/s
Basic.insertOneRandomNumber  clickhouse-http-jdbc1           new       normal  object  thrpt   20   235.198 ±  27.903  ops/s
Basic.insertOneRandomNumber  clickhouse-http-jdbc1           new     prepared  object  thrpt   20   234.410 ±  22.012  ops/s
Basic.insertOneRandomNumber   clickhouse-grpc-jdbc         reuse       normal  object  thrpt   20   247.870 ±  27.479  ops/s
Basic.insertOneRandomNumber   clickhouse-grpc-jdbc         reuse     prepared  object  thrpt   20   248.048 ±  26.457  ops/s
Basic.insertOneRandomNumber   clickhouse-grpc-jdbc           new       normal  object  thrpt   20   246.023 ±  29.453  ops/s
Basic.insertOneRandomNumber   clickhouse-grpc-jdbc           new     prepared  object  thrpt   20   241.346 ±  25.222  ops/s
Basic.selectOneRandomNumber  clickhouse-http-jdbc1         reuse       normal  object  thrpt   20  1576.755 ± 131.857  ops/s
Basic.selectOneRandomNumber  clickhouse-http-jdbc1         reuse     prepared  object  thrpt   20  1646.127 ± 108.495  ops/s
Basic.selectOneRandomNumber  clickhouse-http-jdbc1           new       normal  object  thrpt   20  1658.941 ±  70.198  ops/s
Basic.selectOneRandomNumber  clickhouse-http-jdbc1           new     prepared  object  thrpt   20  1687.920 ±  64.342  ops/s
Basic.selectOneRandomNumber   clickhouse-grpc-jdbc         reuse       normal  object  thrpt   20  1040.702 ±  92.676  ops/s
Basic.selectOneRandomNumber   clickhouse-grpc-jdbc         reuse     prepared  object  thrpt   20  1042.248 ±  97.646  ops/s
Basic.selectOneRandomNumber   clickhouse-grpc-jdbc           new       normal  object  thrpt   20  1051.719 ±  93.958  ops/s
Basic.selectOneRandomNumber   clickhouse-grpc-jdbc           new     prepared  object  thrpt   20  1006.116 ±  90.380  ops/s

[tangjiangling]

The second commit describes:

In method com.clickhouse.jdbc.ClickHouseDatabaseMetaData#getColumns,
we should map DECIMAL_DIGITS and CHAR_OCTET_LENGTH to int32(0),
otherwise it will lead to incorrect results.

Why?

The original implementation maps DECIMAL_DIGITS and CHAR_OCTET_LENGTH to int32(null), which causes the corresponding field type to become com.clickhouse.client.data.ClickHouseEmptyValue, which is unable to update the new value, so it will end up with the wrong result.

[github-actions]

Benchmark                                 (client)  (connection)  (statement)  (type)   Mode  Cnt     Score     Error  Units
Basic.insertOneRandomNumber  clickhouse-http-jdbc1         reuse       normal  object  thrpt   20   224.853 ±  29.809  ops/s
Basic.insertOneRandomNumber  clickhouse-http-jdbc1         reuse     prepared  object  thrpt   20   214.132 ±  25.693  ops/s
Basic.insertOneRandomNumber  clickhouse-http-jdbc1           new       normal  object  thrpt   20   217.588 ±  26.057  ops/s
Basic.insertOneRandomNumber  clickhouse-http-jdbc1           new     prepared  object  thrpt   20   214.580 ±  27.600  ops/s
Basic.insertOneRandomNumber   clickhouse-grpc-jdbc         reuse       normal  object  thrpt   20   228.626 ±  25.056  ops/s
Basic.insertOneRandomNumber   clickhouse-grpc-jdbc         reuse     prepared  object  thrpt   20   231.906 ±  28.852  ops/s
Basic.insertOneRandomNumber   clickhouse-grpc-jdbc           new       normal  object  thrpt   20   231.754 ±  27.423  ops/s
Basic.insertOneRandomNumber   clickhouse-grpc-jdbc           new     prepared  object  thrpt   20   226.982 ±  28.056  ops/s
Basic.selectOneRandomNumber  clickhouse-http-jdbc1         reuse       normal  object  thrpt   20  1236.902 ±  87.694  ops/s
Basic.selectOneRandomNumber  clickhouse-http-jdbc1         reuse     prepared  object  thrpt   20  1226.242 ± 132.556  ops/s
Basic.selectOneRandomNumber  clickhouse-http-jdbc1           new       normal  object  thrpt   20  1214.646 ±  77.618  ops/s
Basic.selectOneRandomNumber  clickhouse-http-jdbc1           new     prepared  object  thrpt   20  1233.896 ±  87.188  ops/s
Basic.selectOneRandomNumber   clickhouse-grpc-jdbc         reuse       normal  object  thrpt   20   786.241 ±  78.995  ops/s
Basic.selectOneRandomNumber   clickhouse-grpc-jdbc         reuse     prepared  object  thrpt   20   784.618 ±  83.369  ops/s
Basic.selectOneRandomNumber   clickhouse-grpc-jdbc           new       normal  object  thrpt   20   757.054 ±  83.106  ops/s
Basic.selectOneRandomNumber   clickhouse-grpc-jdbc           new     prepared  object  thrpt   20   842.684 ±  59.096  ops/s

[zhicwu]

Thanks @tangjiangling for the pull request :) I don't mind to create another patch release, but I'll need to run integration test against trino to see if there's more issues.

In the mean time, before merging the PR, would you mind to sign CLA and rebase? I changed merge target from master to develop, and I'll add some more tests.

[tangjiangling]

would you mind to sign CLA and rebase?

signed.

What does rebase mean?

Is it a combination of all the "Merge pull request ClickHouse#..." into a single commit?

[tangjiangling]

would you mind to sign CLA and rebase?

Sorry, I forgot to set the email, I will do that soon.

[tangjiangling]

Sorry, I forgot to set the email, I will do that soon.

done.

[github-actions]

Benchmark                                 (client)  (connection)  (statement)  (type)   Mode  Cnt     Score     Error  Units
Basic.insertOneRandomNumber  clickhouse-http-jdbc1         reuse       normal  object  thrpt   20   239.145 ±  34.485  ops/s
Basic.insertOneRandomNumber  clickhouse-http-jdbc1         reuse     prepared  object  thrpt   20   232.703 ±  26.263  ops/s
Basic.insertOneRandomNumber  clickhouse-http-jdbc1           new       normal  object  thrpt   20   231.571 ±  25.240  ops/s
Basic.insertOneRandomNumber  clickhouse-http-jdbc1           new     prepared  object  thrpt   20   232.121 ±  27.181  ops/s
Basic.insertOneRandomNumber   clickhouse-grpc-jdbc         reuse       normal  object  thrpt   20   242.690 ±  29.855  ops/s
Basic.insertOneRandomNumber   clickhouse-grpc-jdbc         reuse     prepared  object  thrpt   20   240.499 ±  26.050  ops/s
Basic.insertOneRandomNumber   clickhouse-grpc-jdbc           new       normal  object  thrpt   20   242.497 ±  28.161  ops/s
Basic.insertOneRandomNumber   clickhouse-grpc-jdbc           new     prepared  object  thrpt   20   242.667 ±  27.760  ops/s
Basic.selectOneRandomNumber  clickhouse-http-jdbc1         reuse       normal  object  thrpt   20  1579.735 ± 105.663  ops/s
Basic.selectOneRandomNumber  clickhouse-http-jdbc1         reuse     prepared  object  thrpt   20  1611.489 ±  73.549  ops/s
Basic.selectOneRandomNumber  clickhouse-http-jdbc1           new       normal  object  thrpt   20  1594.689 ±  85.338  ops/s
Basic.selectOneRandomNumber  clickhouse-http-jdbc1           new     prepared  object  thrpt   20  1601.815 ±  95.206  ops/s
Basic.selectOneRandomNumber   clickhouse-grpc-jdbc         reuse       normal  object  thrpt   20  1016.533 ±  87.853  ops/s
Basic.selectOneRandomNumber   clickhouse-grpc-jdbc         reuse     prepared  object  thrpt   20   998.676 ±  82.197  ops/s
Basic.selectOneRandomNumber   clickhouse-grpc-jdbc           new       normal  object  thrpt   20   940.991 ± 116.601  ops/s
Basic.selectOneRandomNumber   clickhouse-grpc-jdbc           new     prepared  object  thrpt   20   999.509 ±  87.986  ops/s

[zhicwu]

Is it a combination of all the "Merge pull request ClickHouse#..." into a single commit?

Yes, squash may work too.

incorrect precision

I think we should consider both cases. When the column is numeric, we use column.getPrecision(), otherwise fall back to column.getDataType().getByteLength(). Make sense?

changed toInt32(null) to toInt32(0)

I'll need to look into this tonight and add some more tests accordingly.

Update:
DECIMAL_DIGITS is nullable according to javadoc, and CHAR_OCTET_LENGTH is nullable as well according to other JDBC driver implementation. The problem here is that toInt32(null) resulted in a column with type Nullable(Nothing), which was mapped to ClickHouseEmptyValue. I think it's better to change it to cast(null as Nullable(Int32)).

I'll fix the these issues and release 0.3.2-patch3 after testing against trino master branch.

[tangjiangling]

I think we should consider both cases. When the column is numeric, we use column.getPrecision(), otherwise fall back to column.getDataType().getByteLength(). Make sense?

+1, I will update the current PR later this evening.

[tangjiangling]

+1, I will update the current PR later this evening.

I fixed it on the basis of this #815 .

[github-actions]

Benchmark                                 (client)  (connection)  (statement)  (type)   Mode  Cnt     Score     Error  Units
Basic.insertOneRandomNumber  clickhouse-http-jdbc1         reuse       normal  object  thrpt   20   208.988 ±  28.144  ops/s
Basic.insertOneRandomNumber  clickhouse-http-jdbc1         reuse     prepared  object  thrpt   20   206.678 ±  26.898  ops/s
Basic.insertOneRandomNumber  clickhouse-http-jdbc1           new       normal  object  thrpt   20   203.693 ±  27.873  ops/s
Basic.insertOneRandomNumber  clickhouse-http-jdbc1           new     prepared  object  thrpt   20   207.810 ±  24.750  ops/s
Basic.insertOneRandomNumber   clickhouse-grpc-jdbc         reuse       normal  object  thrpt   20   217.180 ±  18.722  ops/s
Basic.insertOneRandomNumber   clickhouse-grpc-jdbc         reuse     prepared  object  thrpt   20   207.819 ±  23.562  ops/s
Basic.insertOneRandomNumber   clickhouse-grpc-jdbc           new       normal  object  thrpt   20   214.801 ±  28.415  ops/s
Basic.insertOneRandomNumber   clickhouse-grpc-jdbc           new     prepared  object  thrpt   20   216.493 ±  20.909  ops/s
Basic.selectOneRandomNumber  clickhouse-http-jdbc1         reuse       normal  object  thrpt   20  1087.584 ±  88.325  ops/s
Basic.selectOneRandomNumber  clickhouse-http-jdbc1         reuse     prepared  object  thrpt   20  1115.086 ± 103.262  ops/s
Basic.selectOneRandomNumber  clickhouse-http-jdbc1           new       normal  object  thrpt   20  1111.623 ±  99.610  ops/s
Basic.selectOneRandomNumber  clickhouse-http-jdbc1           new     prepared  object  thrpt   20  1065.311 ±  76.128  ops/s
Basic.selectOneRandomNumber   clickhouse-grpc-jdbc         reuse       normal  object  thrpt   20   715.300 ±  67.225  ops/s
Basic.selectOneRandomNumber   clickhouse-grpc-jdbc         reuse     prepared  object  thrpt   20   753.368 ±  50.600  ops/s
Basic.selectOneRandomNumber   clickhouse-grpc-jdbc           new       normal  object  thrpt   20   678.829 ±  81.305  ops/s
Basic.selectOneRandomNumber   clickhouse-grpc-jdbc           new     prepared  object  thrpt   20   683.853 ±  77.931  ops/s

[github-actions]

Benchmark                                 (client)  (connection)  (statement)  (type)   Mode  Cnt     Score     Error  Units
Basic.insertOneRandomNumber  clickhouse-http-jdbc1         reuse       normal  object  thrpt   20   217.486 ±  24.092  ops/s
Basic.insertOneRandomNumber  clickhouse-http-jdbc1         reuse     prepared  object  thrpt   20   214.531 ±  24.273  ops/s
Basic.insertOneRandomNumber  clickhouse-http-jdbc1           new       normal  object  thrpt   20   213.368 ±  23.031  ops/s
Basic.insertOneRandomNumber  clickhouse-http-jdbc1           new     prepared  object  thrpt   20   212.004 ±  23.038  ops/s
Basic.insertOneRandomNumber   clickhouse-grpc-jdbc         reuse       normal  object  thrpt   20   219.993 ±  25.482  ops/s
Basic.insertOneRandomNumber   clickhouse-grpc-jdbc         reuse     prepared  object  thrpt   20   223.826 ±  26.966  ops/s
Basic.insertOneRandomNumber   clickhouse-grpc-jdbc           new       normal  object  thrpt   20   221.079 ±  24.768  ops/s
Basic.insertOneRandomNumber   clickhouse-grpc-jdbc           new     prepared  object  thrpt   20   221.584 ±  24.142  ops/s
Basic.selectOneRandomNumber  clickhouse-http-jdbc1         reuse       normal  object  thrpt   20  1342.372 ±  94.688  ops/s
Basic.selectOneRandomNumber  clickhouse-http-jdbc1         reuse     prepared  object  thrpt   20  1341.758 ± 109.327  ops/s
Basic.selectOneRandomNumber  clickhouse-http-jdbc1           new       normal  object  thrpt   20  1335.903 ± 110.250  ops/s
Basic.selectOneRandomNumber  clickhouse-http-jdbc1           new     prepared  object  thrpt   20  1357.388 ±  95.751  ops/s
Basic.selectOneRandomNumber   clickhouse-grpc-jdbc         reuse       normal  object  thrpt   20   881.557 ±  53.238  ops/s
Basic.selectOneRandomNumber   clickhouse-grpc-jdbc         reuse     prepared  object  thrpt   20   810.222 ±  76.477  ops/s
Basic.selectOneRandomNumber   clickhouse-grpc-jdbc           new       normal  object  thrpt   20   743.756 ±  62.011  ops/s
Basic.selectOneRandomNumber   clickhouse-grpc-jdbc           new     prepared  object  thrpt   20   830.568 ±  70.877  ops/s

[github-actions]

Benchmark                                 (client)  (connection)  (statement)  (type)   Mode  Cnt     Score    Error  Units
Basic.insertOneRandomNumber  clickhouse-http-jdbc1         reuse       normal  object  thrpt   20   201.640 ± 22.888  ops/s
Basic.insertOneRandomNumber  clickhouse-http-jdbc1         reuse     prepared  object  thrpt   20   203.765 ± 24.698  ops/s
Basic.insertOneRandomNumber  clickhouse-http-jdbc1           new       normal  object  thrpt   20   201.649 ± 26.472  ops/s
Basic.insertOneRandomNumber  clickhouse-http-jdbc1           new     prepared  object  thrpt   20   201.527 ± 23.686  ops/s
Basic.insertOneRandomNumber   clickhouse-grpc-jdbc         reuse       normal  object  thrpt   20   210.149 ± 18.274  ops/s
Basic.insertOneRandomNumber   clickhouse-grpc-jdbc         reuse     prepared  object  thrpt   20   212.332 ± 22.137  ops/s
Basic.insertOneRandomNumber   clickhouse-grpc-jdbc           new       normal  object  thrpt   20   211.663 ± 26.317  ops/s
Basic.insertOneRandomNumber   clickhouse-grpc-jdbc           new     prepared  object  thrpt   20   209.550 ± 23.070  ops/s
Basic.selectOneRandomNumber  clickhouse-http-jdbc1         reuse       normal  object  thrpt   20  1016.752 ± 96.928  ops/s
Basic.selectOneRandomNumber  clickhouse-http-jdbc1         reuse     prepared  object  thrpt   20  1063.180 ± 57.560  ops/s
Basic.selectOneRandomNumber  clickhouse-http-jdbc1           new       normal  object  thrpt   20  1031.810 ± 72.690  ops/s
Basic.selectOneRandomNumber  clickhouse-http-jdbc1           new     prepared  object  thrpt   20  1045.367 ± 60.911  ops/s
Basic.selectOneRandomNumber   clickhouse-grpc-jdbc         reuse       normal  object  thrpt   20   678.027 ± 45.675  ops/s
Basic.selectOneRandomNumber   clickhouse-grpc-jdbc         reuse     prepared  object  thrpt   20   667.163 ± 69.436  ops/s
Basic.selectOneRandomNumber   clickhouse-grpc-jdbc           new       normal  object  thrpt   20   614.581 ± 50.957  ops/s
Basic.selectOneRandomNumber   clickhouse-grpc-jdbc           new     prepared  object  thrpt   20   682.945 ± 73.751  ops/s

[github-actions]

Benchmark                                 (client)  (connection)  (statement)  (type)   Mode  Cnt     Score     Error  Units
Basic.insertOneRandomNumber  clickhouse-http-jdbc1         reuse       normal  object  thrpt   20   209.885 ±  22.759  ops/s
Basic.insertOneRandomNumber  clickhouse-http-jdbc1         reuse     prepared  object  thrpt   20   209.797 ±  24.329  ops/s
Basic.insertOneRandomNumber  clickhouse-http-jdbc1           new       normal  object  thrpt   20   207.236 ±  27.008  ops/s
Basic.insertOneRandomNumber  clickhouse-http-jdbc1           new     prepared  object  thrpt   20   211.582 ±  24.292  ops/s
Basic.insertOneRandomNumber   clickhouse-grpc-jdbc         reuse       normal  object  thrpt   20   206.755 ±  29.760  ops/s
Basic.insertOneRandomNumber   clickhouse-grpc-jdbc         reuse     prepared  object  thrpt   20   217.701 ±  22.935  ops/s
Basic.insertOneRandomNumber   clickhouse-grpc-jdbc           new       normal  object  thrpt   20   215.399 ±  26.640  ops/s
Basic.insertOneRandomNumber   clickhouse-grpc-jdbc           new     prepared  object  thrpt   20   216.043 ±  26.944  ops/s
Basic.selectOneRandomNumber  clickhouse-http-jdbc1         reuse       normal  object  thrpt   20  1083.612 ±  83.767  ops/s
Basic.selectOneRandomNumber  clickhouse-http-jdbc1         reuse     prepared  object  thrpt   20  1027.421 ± 102.281  ops/s
Basic.selectOneRandomNumber  clickhouse-http-jdbc1           new       normal  object  thrpt   20  1051.678 ±  84.544  ops/s
Basic.selectOneRandomNumber  clickhouse-http-jdbc1           new     prepared  object  thrpt   20  1070.535 ±  81.369  ops/s
Basic.selectOneRandomNumber   clickhouse-grpc-jdbc         reuse       normal  object  thrpt   20   683.292 ±  61.748  ops/s
Basic.selectOneRandomNumber   clickhouse-grpc-jdbc         reuse     prepared  object  thrpt   20   714.334 ±  61.043  ops/s
Basic.selectOneRandomNumber   clickhouse-grpc-jdbc           new       normal  object  thrpt   20   651.629 ±  55.758  ops/s
Basic.selectOneRandomNumber   clickhouse-grpc-jdbc           new     prepared  object  thrpt   20   720.645 ±  76.726  ops/s

[tangjiangling]

Fixed by #815. Now we can close this.