Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Better support for nested structure #722

Closed
zhicwu opened this issue Sep 15, 2021 · 0 comments · Fixed by #736 or #747
Closed

Better support for nested structure #722

zhicwu opened this issue Sep 15, 2021 · 0 comments · Fixed by #736 or #747
Labels
enhancement module-client Client API and all implementations
Milestone

Comments

@zhicwu
Copy link
Contributor

zhicwu commented Sep 15, 2021

dbeaver/dbeaver#10334 shows the driver has problem dealing with Array of Tuple, which should be a common case to support. If we mix use Array, AggregateFunction, Tuple, Nested, Map, and Geo types etc., we'll run into similar issue too.

It's not simply a deserialization issue but also related to how we parse column types, for example:
/~https://github.com/ClickHouse/clickhouse-jdbc/blob/5e6f74b89354e60af7d0c9437eef0520ee9ffda1/clickhouse-jdbc/src/main/java/ru/yandex/clickhouse/response/ClickHouseColumnInfo.java#L45-L49

This is actually supported in new Java client, but it's limited to RowBinary format and is currently lack of optimization(primitive types and maybe byte-code generation). We need to 1) enhance TabSeparated data processor to support nested structure; and 2) add clickhouse-client dependency in clickhouse-jdbc(better with an option to turn it on/off).

// insert into x values([{ 'a' : (null, 3), 'b' : (1, 2), 'c' : (2, 1)}])
ClickHouseValue value = newProcessor(1, 3, 1, 0x61, 1, 3, 0, 1, 0x62, 0, 1, 2, 0, 1, 0x63, 0, 2, 1, 0)
        .deserialize(ClickHouseColumn.of("a", "Array(Map(String, Tuple(Nullable(UInt8), UInt16)))"), null);
Assert.assertTrue(value instanceof ClickHouseArrayValue);
Object[] array = (Object[]) value.asObject();
Assert.assertEquals(array.length, 1);
Map<String, Object[]> map = (Map<String, Object[]>) array[0];
Assert.assertEquals(map.size(), 3);
array = map.get("a");
Assert.assertEquals(array.length, 2);
Assert.assertEquals(array[0], null);
Assert.assertEquals(array[1], 3);
array = map.get("b");
Assert.assertEquals(array.length, 2);
Assert.assertEquals(array[0], Short.valueOf("1"));
Assert.assertEquals(array[1], 2);
array = map.get("c");
Assert.assertEquals(array.length, 2);
Assert.assertEquals(array[0], Short.valueOf("2"));
Assert.assertEquals(array[1], 1);
@zhicwu zhicwu added this to the 0.3.2 Release milestone Sep 15, 2021
@zhicwu zhicwu added module-http HTTP/HTTPS client module-client Client API and all implementations and removed module-http HTTP/HTTPS client labels Oct 6, 2021
@zhicwu zhicwu linked a pull request Oct 7, 2021 that will close this issue
@zhicwu zhicwu linked a pull request Nov 28, 2021 that will close this issue
18 tasks
@zhicwu zhicwu closed this as completed Dec 10, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement module-client Client API and all implementations
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant