Skip to content

Commit

Permalink
docs: auto-generate connection properties documentation (#1860)
Browse files Browse the repository at this point in the history
* docs: auto-generate connection properties documentation

* docs: add link to README

* chore: only run during test phase
  • Loading branch information
olavloite authored Dec 16, 2024
1 parent 6c08ffa commit c22e654
Show file tree
Hide file tree
Showing 7 changed files with 234 additions and 23 deletions.
6 changes: 5 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -105,6 +105,9 @@ The Cloud Spanner JDBC driver supports the following connection URL properties.
these can also be supplied in a Properties instance that is passed to the
`DriverManager#getConnection(String url, Properties properties)` method.

See [Supported Connection Properties](documentation/connection_properties.md) for a full list of all
supported connection properties.

#### Commonly Used Properties
- credentials (String): URL for the credentials file to use for the connection. If you do not specify any credentials at all, the default credentials of the environment as returned by `GoogleCredentials#getApplicationDefault()` is used. Example: `jdbc:cloudspanner:/projects/my-project/instances/my-instance/databases/my-db;credentials=/path/to/credentials.json`
- autocommit (boolean): Sets the initial autocommit mode for the connection. Default is true.
Expand Down Expand Up @@ -135,7 +138,8 @@ these can also be supplied in a Properties instance that is passed to the
- oauthToken (string): A valid pre-existing OAuth token to use for authentication for this connection. Setting this property will take precedence over any value set for a credentials file.
- lenient (boolean): Enable this to force the JDBC driver to ignore unknown properties in the connection URL. Some applications automatically add additional properties to the URL that are not recognized by the JDBC driver. Normally, the JDBC driver will reject this, unless `lenient` mode is enabled.

For a full list of supported connection properties see /~https://github.com/googleapis/java-spanner/blob/main/google-cloud-spanner/src/main/java/com/google/cloud/spanner/connection/ConnectionProperties.java.
For a full list of supported connection properties, see
[Supported Connection Properties](documentation/connection_properties.md).

### Jar with Dependencies
A single jar with all dependencies can be downloaded from https://repo1.maven.org/maven2/com/google/cloud/google-cloud-spanner-jdbc/latest
Expand Down
56 changes: 56 additions & 0 deletions documentation/connection_properties.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
# Supported Connection Properties

This file contains all supported connection properties for the Spanner JDBC driver. These properties can be specified both in the connection URL and in the Properties map that is used to create a connection.

The 'Context' value indicates whether the property can only be set when a connection is created (STARTUP), or whether the value can also be changed after a connection has been created.

| Name | Description | Default | Enum values | Context |
|------|-------------|---------|-------------|---------|
| auto_batch_dml | Automatically buffer DML statements that are executed on this connection and execute them as one batch when a non-DML statement is executed, or when the current transaction is committed. The update count that is returned for DML statements that are buffered is by default 1. This default can be changed by setting the connection variable auto_batch_dml_update_count to value other than 1. This setting is only in read/write transactions. DML statements in auto-commit mode are executed directly. | false | | USER |
| auto_batch_dml_update_count | DML statements that are executed when auto_batch_dml is set to true, are not directly sent to Spanner, but are buffered in the client until the batch is flushed. This property determines the update count that is returned for these DML statements. The default is 1, as that is the update count that is expected by most ORMs (e.g. Hibernate). | 1 | | USER |
| auto_batch_dml_update_count_verification | The update count that is returned for DML statements that are buffered during an automatic DML batch is by default 1. This value can be changed by setting the connection variable auto_batch_dml_update_count. The update counts that are returned by Spanner when the DML statements are actually executed are verified against the update counts that were returned when they were buffered. If these do not match, a com.google.cloud.spanner.DmlBatchUpdateCountVerificationFailedException will be thrown. You can disable this verification by setting auto_batch_dml_update_count_verification to false. | true | | USER |
| autocommit | Should the connection start in autocommit (true/false) | true | | USER |
| autocommit_dml_mode | Should the connection automatically retry Aborted errors (true/false) | TRANSACTIONAL | | USER |
| autoconfigemulator | Automatically configure the connection to try to connect to the Cloud Spanner emulator (true/false). The instance and database in the connection string will automatically be created if these do not yet exist on the emulator. Add dialect=postgresql to the connection string to make sure that the database that is created uses the PostgreSQL dialect. | false | | STARTUP |
| autopartitionmode | Execute all queries on this connection as partitioned queries. Executing a query that cannot be partitioned will fail. Executing a query in a read/write transaction will also fail. | false | | USER |
| channelprovider | The name of the channel provider class. The name must reference an implementation of ExternalChannelProvider. If this property is not set, the connection will use the default grpc channel provider. | | | STARTUP |
| connection_state_type | The type of connection state to use for this connection. Can only be set at start up. If no value is set, then the database dialect default will be used, which is NON_TRANSACTIONAL for GoogleSQL and TRANSACTIONAL for PostgreSQL. | | | STARTUP |
| credentials | The location of the credentials file to use for this connection. If neither this property or encoded credentials are set, the connection will use the default Google Cloud credentials for the runtime environment. | | | STARTUP |
| credentialsprovider | The class name of the com.google.api.gax.core.CredentialsProvider implementation that should be used to obtain credentials for connections. | | | STARTUP |
| databaserole | Sets the database role to use for this connection. The default is privileges assigned to IAM role | | | STARTUP |
| databoostenabled | Enable data boost for all partitioned queries that are executed by this connection. This setting is only used for partitioned queries and is ignored by all other statements. | false | | USER |
| ddlintransactionmode | Determines how the connection should handle DDL statements in a read/write transaction. | ALLOW_IN_EMPTY_TRANSACTION | | USER |
| delaytransactionstartuntilfirstwrite | Enabling this option will delay the actual start of a read/write transaction until the first write operation is seen in that transaction. All reads that happen before the first write in a transaction will instead be executed as if the connection was in auto-commit mode. Enabling this option will make read/write transactions lose their SERIALIZABLE isolation level. Read operations that are executed after the first write operation in a read/write transaction will be executed using the read/write transaction. Enabling this mode can reduce locking and improve performance for applications that can handle the lower transaction isolation semantics. | false | | USER |
| dialect | Sets the dialect to use for new databases that are created by this connection. | GOOGLE_STANDARD_SQL | | STARTUP |
| directed_read | The directed read options to apply to read-only transactions. | | | USER |
| enableapitracing | Add OpenTelemetry traces for each individual RPC call. Enable this to get a detailed view of each RPC that is being executed by your application, or if you want to debug potential latency problems caused by RPCs that are being retried. | | | STARTUP |
| enableendtoendtracing | Enable end-to-end tracing (true/false) to generate traces for both the time that is spent in the client, as well as time that is spent in the Spanner server. Server side traces can only go to Google Cloud Trace, so to see end to end traces, the application should configure an exporter that exports the traces to Google Cloud Trace. | false | | STARTUP |
| enableextendedtracing | Include the SQL string in the OpenTelemetry traces that are generated by this connection. The SQL string is added as the standard OpenTelemetry attribute 'db.statement'. | | | STARTUP |
| encodedcredentials | Base64-encoded credentials to use for this connection. If neither this property or a credentials location are set, the connection will use the default Google Cloud credentials for the runtime environment. | | | STARTUP |
| endpoint | The endpoint that the JDBC driver should connect to. The default is the default Spanner production endpoint when autoConfigEmulator=false, and the default Spanner emulator endpoint (localhost:9010) when autoConfigEmulator=true. This property takes precedence over any host name at the start of the connection URL. | | | STARTUP |
| keeptransactionalive | Enabling this option will trigger the connection to keep read/write transactions alive by executing a SELECT 1 query once every 10 seconds if no other statements are being executed. This option should be used with caution, as it can keep transactions alive and hold on to locks longer than intended. This option should typically be used for CLI-type application that might wait for user input for a longer period of time. | false | | USER |
| lenient | Silently ignore unknown properties in the connection string/properties (true/false) | false | | STARTUP |
| maxcommitdelay | The max delay that Spanner may apply to commit requests to improve throughput. | | | USER |
| maxpartitionedparallelism | The max partitions hint value to use for partitioned queries. Use 0 if you do not want to specify a hint. | 1 | | USER |
| maxpartitions | The max partitions hint value to use for partitioned queries. Use 0 if you do not want to specify a hint. | 0 | | USER |
| maxsessions | The maximum number of sessions in the backing session pool. The default is 400. | | | STARTUP |
| minsessions | The minimum number of sessions in the backing session pool. The default is 100. | | | STARTUP |
| numchannels | The number of gRPC channels to use to communicate with Cloud Spanner. The default is 4. | | | STARTUP |
| oauthtoken | A valid pre-existing OAuth token to use for authentication for this connection. Setting this property will take precedence over any value set for a credentials file. | | | STARTUP |
| optimizerstatisticspackage | Sets the query optimizer statistics package to use for this connection. | | | USER |
| optimizerversion | Sets the default query optimizer version to use for this connection. | | | USER |
| read_only_staleness | The read-only staleness to use for read-only transactions and single-use queries. | strong | | USER |
| readonly | Should the connection start in read-only mode (true/false) | false | | USER |
| retryabortsinternally | Should the connection automatically retry Aborted errors (true/false) | true | | USER |
| returncommitstats | Request that Spanner returns commit statistics for read/write transactions (true/false) | false | | USER |
| routetoleader | Should read/write transactions and partitioned DML be routed to leader region (true/false) | true | | STARTUP |
| rpcpriority | Sets the priority for all RPC invocations from this connection (HIGH/MEDIUM/LOW). The default is HIGH. | | | USER |
| savepoint_support | Determines the behavior of the connection when savepoints are used. | FAIL_AFTER_ROLLBACK | | USER |
| tracing_prefix | The prefix that will be prepended to all OpenTelemetry traces that are generated by a Connection. | CloudSpanner | | STARTUP |
| trackconnectionleaks | Capture the call stack of the thread that created a connection. This will pre-create a LeakedConnectionException already when a connection is created. This can be disabled, for example if a monitoring system logs the pre-created exception. If disabled, the LeakedConnectionException will only be created when an actual connection leak is detected. The stack trace of the exception will in that case not contain the call stack of when the connection was created. | true | | STARTUP |
| tracksessionleaks | Capture the call stack of the thread that checked out a session of the session pool. This will pre-create a LeakedSessionException already when a session is checked out. This can be disabled, for example if a monitoring system logs the pre-created exception. If disabled, the LeakedSessionException will only be created when an actual session leak is detected. The stack trace of the exception will in that case not contain the call stack of when the session was checked out. | true | | STARTUP |
| useautosavepointsforemulator | Automatically creates savepoints for each statement in a read/write transaction when using the Emulator. This is no longer needed when using Emulator version 1.5.23 or higher. | false | | STARTUP |
| useplaintext | Use a plain text communication channel (i.e. non-TLS) for communicating with the server (true/false). Set this value to true for communication with the Cloud Spanner emulator. | false | | STARTUP |
| useragent | The custom user-agent property name to use when communicating with Cloud Spanner. This property is intended for internal library usage, and should not be set by applications. | | | STARTUP |
| usevirtualgrpctransportthreads | Use a virtual thread instead of a platform thread for the gRPC executor (true/false). This option only has any effect if the application is running on Java 21 or higher. In all other cases, the option is ignored. | false | | STARTUP |
| usevirtualthreads | Use a virtual thread instead of a platform thread for each connection (true/false). This option only has any effect if the application is running on Java 21 or higher. In all other cases, the option is ignored. | false | | STARTUP |
19 changes: 19 additions & 0 deletions pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -334,6 +334,25 @@
<groupId>org.codehaus.mojo</groupId>
<artifactId>flatten-maven-plugin</artifactId>
</plugin>

<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>exec-maven-plugin</artifactId>
<executions>
<execution>
<id>generate_connection_properties_documentation</id>
<phase>test</phase>
<goals>
<goal>java</goal>
</goals>
<configuration>
<mainClass>com.google.cloud.spanner.connection.ConnectionPropertiesFileGenerator</mainClass>
<classpathScope>test</classpathScope>
<skip>false</skip>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
<pluginManagement>
<plugins>
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
/*
* Copyright 2024 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package com.google.cloud.spanner.connection;

import com.google.api.core.InternalApi;
import com.google.common.collect.ImmutableList;
import java.sql.DriverPropertyInfo;
import java.util.Arrays;
import java.util.Comparator;
import java.util.regex.Matcher;
import java.util.regex.Pattern;
import java.util.stream.Collectors;

// TODO: Remove this class when the Connection API has made the list of properties public.
@InternalApi
public class ConnectionPropertiesHelper {
public static ImmutableList<ConnectionProperty<?>> VALID_CONNECTION_PROPERTIES =
ImmutableList.copyOf(
ConnectionProperties.CONNECTION_PROPERTIES.values().stream()
.sorted(Comparator.comparing(ConnectionProperty::getName))
.collect(Collectors.toList()));

public static DriverPropertyInfo toDriverPropertyInfo(
String connectionUri, ConnectionProperty<?> connectionProperty) {
DriverPropertyInfo result =
new DriverPropertyInfo(
connectionProperty.getName(),
parseUriProperty(
connectionUri,
connectionProperty.getName(),
connectionProperty.getDefaultValue() == null
? null
: connectionProperty.getDefaultValue().toString()));
result.description = connectionProperty.getDescription();
result.choices =
connectionProperty.getValidValues() == null
? null
: Arrays.stream(connectionProperty.getValidValues())
.map(Object::toString)
.toArray(String[]::new);
return result;
}

public static String getConnectionPropertyName(ConnectionProperty<?> connectionProperty) {
return connectionProperty.getName();
}

private static String parseUriProperty(String uri, String property, String defaultValue) {
Pattern pattern = Pattern.compile(String.format("(?is)(?:;|\\?)%s=(.*?)(?:;|$)", property));
Matcher matcher = pattern.matcher(uri);
if (matcher.find() && matcher.groupCount() == 1) {
return matcher.group(1);
}
return defaultValue;
}

private ConnectionPropertiesHelper() {}
}
24 changes: 6 additions & 18 deletions src/main/java/com/google/cloud/spanner/jdbc/JdbcDriver.java
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,8 @@
import com.google.cloud.spanner.SessionPoolOptionsHelper;
import com.google.cloud.spanner.SpannerException;
import com.google.cloud.spanner.connection.ConnectionOptions;
import com.google.cloud.spanner.connection.ConnectionOptions.ConnectionProperty;
import com.google.cloud.spanner.connection.ConnectionPropertiesHelper;
import com.google.cloud.spanner.connection.ConnectionProperty;
import com.google.rpc.Code;
import io.opentelemetry.api.OpenTelemetry;
import java.sql.Connection;
Expand Down Expand Up @@ -265,29 +266,16 @@ public boolean acceptsURL(String url) {
@Override
public DriverPropertyInfo[] getPropertyInfo(String url, Properties info) {
String connectionUri = appendPropertiesToUrl(url.substring(5), info);
DriverPropertyInfo[] res = new DriverPropertyInfo[ConnectionOptions.VALID_PROPERTIES.size()];
DriverPropertyInfo[] res =
new DriverPropertyInfo[ConnectionPropertiesHelper.VALID_CONNECTION_PROPERTIES.size()];
int i = 0;
for (ConnectionProperty prop : ConnectionOptions.VALID_PROPERTIES) {
res[i] =
new DriverPropertyInfo(
prop.getName(),
parseUriProperty(connectionUri, prop.getName(), prop.getDefaultValue()));
res[i].description = prop.getDescription();
res[i].choices = prop.getValidValues();
for (ConnectionProperty<?> prop : ConnectionPropertiesHelper.VALID_CONNECTION_PROPERTIES) {
res[i] = ConnectionPropertiesHelper.toDriverPropertyInfo(connectionUri, prop);
i++;
}
return res;
}

private String parseUriProperty(String uri, String property, String defaultValue) {
Pattern pattern = Pattern.compile(String.format("(?is)(?:;|\\?)%s=(.*?)(?:;|$)", property));
Matcher matcher = pattern.matcher(uri);
if (matcher.find() && matcher.groupCount() == 1) {
return matcher.group(1);
}
return defaultValue;
}

@Override
public int getMajorVersion() {
return MAJOR_VERSION;
Expand Down
Loading

0 comments on commit c22e654

Please sign in to comment.