• Connection String Options
• Views

Connection String OptionsBack To Top

1. AuthScheme
2. ProjectId
3. DatasetId
4. BillingProjectId
5. AllowLargeResultSets
6. DestinationTable
7. UseQueryCache
8. PageSize
9. PollingInterval
10. AllowUpdatesWithoutKey
11. FilterColumns
12. UseLegacySQL
13. UseStorageAPI
14. UseArrowFormat
15. StorageThreshold
16. StoragePageSize
17. InsertMode
18. WaitForBatchResults
19. GCSBucket
20. GCSBucketFolder
21. TempTableDataset
22. InitiateOAuth
23. OAuthClientId
24. OAuthClientSecret
25. OAuthAccessToken
26. OAuthSettingsLocation
27. Scope
28. OAuthVerifier
29. OAuthRefreshToken
30. OAuthExpiresIn
31. OAuthTokenTimestamp
32. OAuthJWTCert
33. OAuthJWTCertType
34. OAuthJWTCertPassword
35. OAuthJWTCertSubject
36. OAuthJWTIssuer
37. OAuthJWTSubject
38. SSLServerCert
39. FirewallType
40. FirewallServer
41. FirewallPort
42. FirewallUser
43. FirewallPassword
44. ProxyAutoDetect
45. ProxyServer
46. ProxyPort
47. ProxyAuthScheme
48. ProxyUser
49. ProxyPassword
50. ProxySSLType
51. ProxyExceptions
52. Logfile
53. Verbosity
54. LogModules
55. MaxLogFileSize
56. MaxLogFileCount
57. Location
58. BrowsableSchemas
59. Tables
60. Views
61. RefreshViewSchemas
62. ShowTableDescriptions
63. PrimaryKeyIdentifiers
64. AllowedTableTypes
65. AutoCache
66. CacheDriver
67. CacheConnection
68. CacheLocation
69. CacheTolerance
70. Offline
71. CacheMetadata
72. StorageTimeout
73. AllowAggregateParameters
74. AuditLimit
75. AuditMode
76. BigQueryOptions
77. ConnectionLifeTime
78. ConnectOnOpen
79. GenerateSchemaFiles
80. MaximumBillingTier
81. MaximumBytesBilled
82. MaxRows
83. Other
84. PoolIdleTimeout
85. PoolMaxSize
86. PoolMinSize
87. PoolWaitTime
88. PseudoColumns
89. QueryPassthrough
90. Readonly
91. RTK
92. TableSamplePercent
93. Timeout
94. UseConnectionPooling

AuthScheme

Data Type

string

Default Value

"Auto"

Remarks

• Auto: Lets the driver decide automatically based on the other connection properties you have set.
• OAuth: Set this to perform OAuth authentication using a standard user account.
• OAuthJWT: Set this to perform OAuth authentication using an OAuth service account.
• GCPInstanceAccount: Set this to get Access Token from Google Cloud Platform instance.

ProjectId

Data Type

string

Default Value

""

Remarks

When a query refers to a table it can leave the project implicit, or qualify the project directly as the catalog portion of the table:

/* Implicit, resolved against connection string */

SELECT FirstName, LastName FROM `Northwind`.`customers`



/* Explicit, project specified as catalog */

SELECT FirstName, LastName FROM `psychic-valve-137816`.`Northwind`.`customers`

If the query contains unqualified table references then they are resolved this way:

1. If this property is set then the specified project is used.
2. Otherwise, if the BillingProjectId is set then that project is used.
3. Finally, if neither are set then the catalog from the first table in the query is used. In the above example the psychic-valve-137816 project would be used to resolve any other unqualified tables that would appear in the query (for example, if a JOIN were added).

Note that the query is not consulted when QueryPassthrough is enabled. So you either must set the connection ProjectId and DatasetId or qualify each individual table; otherwise the SELECT query fails.

DatasetId

Data Type

string

Default Value

""

Remarks

The DatasetId of the dataset you wish to connect to and view tables of. You can find this value in Google BigQuery, after selecting a project.

After connecting to an initial dataset, you can query the Datasets view to discover the Ids for all datasets in a project. Along with ProjectId.

DatasetId not required when calling INFORMATION_SCHEMA.SCHEMA

You no longer have to specify which dataset to use via the DatasetId property. Also or the schema from the query table to determine which dataset to query. For example, you can connect without the DatasetId specified and write:

SELECT * FROM choff04.TestDataset.join_left INNER JOIN join_right

In this case, the provider assumes that join_right comes from TestDataset. Or you can connect with DatasetId=TestDataset;ProjectId=choff04:

 SELECT * FROM join_left INNER JOIN join_right 

Note that the query is not consulted when QueryPassthrough is enabled. So you either must set the connection ProjectId and DatasetId or qualify each individual table; otherwise the SELECT query fails.

BillingProjectId

Data Type

string

Default Value

""

Remarks

The ProjectId of the billing project for executing jobs. You can obtain the project Id in the Google APIs console: In the main menu, click API Project and copy the Id. For example: psychic-valve-137816. (Note that your domain name is not part of the Id.)

The billing project is determined based on the value of this property, the ProjectId and the query:

1. If this property is set then the specified project will be billed.
2. Otherwise, if the ProjectId is set then that project will be billed.
3. Finally, if neither are set then the catalog from the first table in the query will be used. For example, the below query would bill the psychic-valve-137816 project.

SELECT FirstName, LastName FROM `psychic-valve-137816`.`Northwind`.`customers`

AllowLargeResultSets

Data Type

bool

Default Value

true

Remarks

Whether or not to allow large datasets to be stored in temporary tables for large datasets.

DestinationTable

Data Type

string

Default Value

""

Remarks

Google BigQuery queries have a maximum amount of data they are allowed to return directly. If this limit is exceeded, then queries will fail with an error message like Response too large to return. When this option is enabled the response limit does not apply, because all query responses are stored in a Google BigQuery table before being returned.

This option is set differently depending upon whether your connection is using UseLegacySQL or not. By default this option is set using the standard SQL syntax:

DestinationTable=project-name.dataset-name.table-name

When UseLegacySQL is enabled, this option is set using the legacy table syntax:

DestinationTable=project-name:dataset-name.table-name

When using this option with multiple connections, make sure that each connection has its own destination table. Sharing a table between connections can lead to results getting lost because parallel queries can overwrite each others results.

UseQueryCache

Data Type

bool

Default Value

true

Remarks

Google BigQuery will cache the results of recent queries, and will use this cache for queries by default. Google BigQuery automatically updates the cache when a table is modified, so performance is generally better without any risk of queries returning stale data.

If this is set to false, the query is always run against the table directly.

PageSize

Data Type

string

Default Value

"100000"

Remarks

The pagesize can control the number of results returned per page from Google BigQuery. Setting a higher pagesize will cause more data to come back in a single HTTP request, but may take longer to execute. Setting a smaller pagesize will increase the number of HTTP requests to get all the data, but is generally recommended to ensure timeout exceptions do not occur.

Note that this option does not have an effect if RSBGoogleBigQuery_p_UseStorageAPI is enabled and the queries being executed can be executed on the Storage API. See StoragePageSize for more information.

PollingInterval

Data Type

string

Default Value

"1"

Remarks

This only applies to queries which are stored to a table instead of streamed directly to the driver. This applies in only three cases:

• DestinationTable is set.
• AllowLargeResultSets is true and the query takes longer than Timeout seconds.
• RSBGoogleBigQuery_p_UseStorageAPI is enabled and the query is complex.

This property determines how long to wait between checking whether or not the query's results are ready. Very large resultsets or complex queries may take longer to process, and a low polling interval may result in may unnecessary requests being made to check the query status.

AllowUpdatesWithoutKey

Data Type

bool

Default Value

false

Remarks

Whether or not to allow update without primary keys.

FilterColumns

Data Type

string

Default Value

""

Remarks

Remember setting `AllowUpdatesWithoutKey` to true before you could use this property:

Set the property like this:

`filterColumns=col1[,col2[,col3]];`

UseLegacySQL

Data Type

bool

Default Value

false

Remarks

If set to true, the query will use BigQuery's Legacy SQL dialect to rebuild the query. If set to false, the query will use BigQuery's standard SQL: https://cloud.google.com/bigquery/sql-reference/.
When UseLegacySQL is set to false, the values of AllowLargeResultSets is ignored. The query will be run as if AllowLargeResultSets is true.

UseStorageAPI

Data Type

bool

Default Value

true

Remarks

By default the driver will use the Storage API instead of the default REST API. Depending upon the complexity of the query, the driver may execute the query in one of two ways:

• Simple queries that read all columns from only one table, and have no extra clauses except LIMIT, are executed directly within the Storage API.
• All other queries are executed as a query job which writes to a temporary table. Once the query is complete, the results are read from the temporary table using the Storage API.

The BigQuery Storage API can read data faster and more efficiently than the REST API (accessible by setting this option to false), but is priced differently and requires extra OAuth permissions when using your own OAuth app. It also uses the separate StoragePageSize property instead of PageSize.

The BigQuery REST API requires no extra permissions and uses standard pricing, but is slower than the Storage API.

UseArrowFormat

Data Type

bool

Default Value

false

Remarks

This property only has an effect when RSBGoogleBigQuery_p_UseStorageAPI is enabled. When performing reads against the Storage API, the driver can request data in different formats. By default it uses Avro but enabling this option makes it use Arrow.

This option should be enabled when working with time series data or other datasets that have many date, time, datetime or timestamp fields. For these datasets using Arrow can have noticable improvements over using Avro. Otherwise Avro and Arrow read times are very close and switching between them is unlikely to make a significant difference.

StorageThreshold

Data Type

string

Default Value

"100000"

Remarks

When the driver receives a query too complex to be run directly in the Storage API, it creates a query job and uses the Storage API to read from the query results table. If the query job returns fewer than the number of rows provided in this option, then the results are returned directly and the Storage API is not used.

This value should be set between 1 and 100000. Higher values will use the Storage API only for large resultsets, but will be delayed by reading more results from the query job. Lower values will result in smaller delays but will use the Storage API for more queries.

Note that this option only has an effect if RSBGoogleBigQuery_p_UseStorageAPI is enabled and the queries being executed cannot be executed directly on the Storage API. Queries which run directly on Storage never create query jobs.

StoragePageSize

Data Type

string

Default Value

"10000"

Remarks

When RSBGoogleBigQuery_p_UseStorageAPI is enabled and the query being executed can be run on the Storage API, this option controls how many rows the driver is allowed to buffer on the client.

A higher value will generally make queries faster at the expense of consuming more memory, while lower values will conserve memory but make queries slower.

InsertMode

Data Type

string

Default Value

"Streaming"

Remarks

This section provides only a summary of the mechanisms that each of these modes use. Please see Advanced Integrations for more details on how to use each of these modes.

• Streaming uses the Google BigQuery streaming API (also called insertAll).
• DML uses the Google BigQuery query API to generate INSERT SQL statements which insert individual rows.
• Upload uses the Google BigQuery upload API to create a load job which copies the rows from temporary server-side storage.
• GCSStaging is similar to the Upload mode except that it uses your Google Cloud Storage account instead of public storage.

When UseLegacySQL is true only Streaming and Upload modes are allowed. The Legacy SQL dialect does not support DML statements.

WaitForBatchResults

Data Type

bool

Default Value

true

Remarks

This property determines whether the driver will wait for batch jobs to report their status. By default property is true and INSERT queries will complete only once Google BigQuery has finished executed them. When this property is false the INSERT query will complete as soon as a job is submitted for it.

The default mode is recommended for reliability:

1. Inserts will never fail silently. If the driver does not wait for the job to finish, it will never receive an error if the job failed to execute.
2. If the insert batch size is small enough, the driver may submit jobs quickly enough that it hits Google BigQuery's load job limits. This does not happen when waiting for batch results because the driver will not allow more than one job to execute at the same time on the same connection.

You can disable this option to achieve lower delays when inserting, but you must also make sure to obey the Google BigQuery rate limits and check the status of each job to track their status and determine whether they have succeeded or failed.

GCSBucket

Data Type

string

Default Value

""

Remarks

Only applies when InsertMode is set to GCSStaging, and if that option is set to use staging then this option is required.

GCSBucketFolder

Data Type

string

Default Value

""

Remarks

Only applies when InsertMode is set to GCSStaging. If not set the driver defaults to writing to the root of the bucket.

TempTableDataset

Data Type

string

Default Value

"_CDataTempTableDataset"

Remarks

Internally bulk UPDATE and DELETE use Google BigQuery MERGE queries, which require creating a table to hold all the update operations. This option is used along with the target table's region to determine the name of the dataset where these temporary tables are created. Each region must have its own temporary dataset so that the temporary table and the MERGE table can be stored in the same project/dataset. This avoids unnecessary data transfer charges.

For example, the driver would create a dataset called "_CDataTempTableDataset_US" for tables in the US region and a dataset called "_CDataTempTableDataset_asia_southeast_1" for tables in the Singapore region.

InitiateOAuth

Data Type

string

Default Value

"OFF"

Remarks

The following options are available:

1. OFF: Indicates that the OAuth flow will be handled entirely by the user. An OAuthAccessToken will be required to authenticate.
2. GETANDREFRESH: Indicates that the entire OAuth Flow will be handled by the driver. If no token currently exists, it will be obtained by prompting the user via the browser. If a token exists, it will be refreshed when applicable.
3. REFRESH: Indicates that the driver will only handle refreshing the OAuthAccessToken. The user will never be prompted by the driver to authenticate via the browser. The user must handle obtaining the OAuthAccessToken and OAuthRefreshToken initially.

OAuthClientId

Data Type

string

Default Value

""

Remarks

As part of registering an OAuth application, you will receive the OAuthClientId value, sometimes also called a consumer key, and a client secret, the OAuthClientSecret.

OAuthClientSecret

Data Type

string

Default Value

""

Remarks

As part of registering an OAuth application, you will receive the OAuthClientId, also called a consumer key. You will also receive a client secret, also called a consumer secret. Set the client secret in the OAuthClientSecret property.

OAuthAccessToken

Data Type

string

Default Value

""

Remarks

The OAuthAccessToken property is used to connect using OAuth. The OAuthAccessToken is retrieved from the OAuth server as part of the authentication process. It has a server-dependent timeout and can be reused between requests.

The access token is used in place of your user name and password. The access token protects your credentials by keeping them on the server.

OAuthSettingsLocation

Data Type

string

Default Value

"%APPDATA%\\CData\\GoogleBigQuery Data Provider\\OAuthSettings.txt"

Remarks

When InitiateOAuth is set to GETANDREFRESH or REFRESH, the driver saves OAuth values to avoid requiring the user to manually enter OAuth connection properties and allowing the credentials to be shared across connections or processes.

Alternatively to specifying a file path, memory storage can be used instead. Memory locations are specified by using a value starting with 'memory://' followed by a unique identifier for that set of credentials (ex: memory://user1). The identifier can be anything you choose but should be unique to the user. Unlike with the file based storage, you must manually store the credentials when closing the connection with memory storage to be able to set them in the connection when the process is started again. The OAuth property values can be retrieved with a query to the sys_connection_props system table. If there are multiple connections using the same credentials, the properties should be read from the last connection to be closed.

If left unspecified, the default location is "%APPDATA%\\CData\\GoogleBigQuery Data Provider\\OAuthSettings.txt" with %APPDATA% being set to the user's configuration directory:

Platform	%APPDATA%
Windows	The value of the APPDATA environment variable
Mac	~/Library/Application Support
Linux	~/.config

Scope

Data Type

string

Default Value

""

Remarks

By default the driver requests only the scope https://www.googleapis.com/auth/bigquery, but in some cases more scopes are required. For example, when reading data using an external table connected to Google Drive, the additional scope https://www.googleapis.com/auth/drive is also required.

OAuthVerifier

Data Type

string

Default Value

""

Remarks

The verifier code returned from the OAuth authorization URL. This can be used on systems where a browser cannot be launched such as headless systems.

Authentication on Headless Machines

See Establishing a Connection to obtain the OAuthVerifier value.

Set OAuthSettingsLocation along with OAuthVerifier. When you connect, the driver exchanges the OAuthVerifier for the OAuth authentication tokens and saves them, encrypted, to the specified file. Set InitiateOAuth to GETANDREFRESH automate the exchange.

Once the OAuth settings file has been generated, you can remove OAuthVerifier from the connection properties and connect with OAuthSettingsLocation set.

To automatically refresh the OAuth token values, set OAuthSettingsLocation and additionally set InitiateOAuth to REFRESH.

OAuthRefreshToken

Data Type

string

Default Value

""

Remarks

The OAuthRefreshToken property is used to refresh the OAuthAccessToken when using OAuth authentication.

OAuthExpiresIn

Data Type

string

Default Value

""

Remarks

Pair with OAuthTokenTimestamp to determine when the AccessToken will expire.

OAuthTokenTimestamp

Data Type

string

Default Value

""

Remarks

Pair with OAuthExpiresIn to determine when the AccessToken will expire.

OAuthJWTCert

Data Type

string

Default Value

""

Remarks

The name of the certificate store for the client certificate.

The OAuthJWTCertType field specifies the type of the certificate store specified by OAuthJWTCert. If the store is password protected, specify the password in OAuthJWTCertPassword.

OAuthJWTCert is used in conjunction with the OAuthJWTCertSubject field in order to specify client certificates. If OAuthJWTCert has a value, and OAuthJWTCertSubject is set, a search for a certificate is initiated. Please refer to the OAuthJWTCertSubject field for details.

Designations of certificate stores are platform-dependent.

The following are designations of the most common User and Machine certificate stores in Windows:

MY	A certificate store holding personal certificates with their associated private keys.
CA	Certifying authority certificates.
ROOT	Root certificates.
SPC	Software publisher certificates.

In Java, the certificate store normally is a file containing certificates and optional private keys.

When the certificate store type is PFXFile, this property must be set to the name of the file. When the type is PFXBlob, the property must be set to the binary contents of a PFX file (i.e. PKCS12 certificate store).

OAuthJWTCertType

Data Type

string

Default Value

"GOOGLEJSON"

Remarks

This property can take one of the following values:

USER	For Windows, this specifies that the certificate store is a certificate store owned by the current user. Note: This store type is not available in Java.
MACHINE	For Windows, this specifies that the certificate store is a machine store. Note: this store type is not available in Java.
PFXFILE	The certificate store is the name of a PFX (PKCS12) file containing certificates.
PFXBLOB	The certificate store is a string (base-64-encoded) representing a certificate store in PFX (PKCS12) format.
JKSFILE	The certificate store is the name of a Java key store (JKS) file containing certificates. Note: this store type is only available in Java.
JKSBLOB	The certificate store is a string (base-64-encoded) representing a certificate store in Java key store (JKS) format. Note: this store type is only available in Java.
PEMKEY_FILE	The certificate store is the name of a PEM-encoded file that contains a private key and an optional certificate.
PEMKEY_BLOB	The certificate store is a string (base64-encoded) that contains a private key and an optional certificate.
PUBLIC_KEY_FILE	The certificate store is the name of a file that contains a PEM- or DER-encoded public key certificate.
PUBLIC_KEY_BLOB	The certificate store is a string (base-64-encoded) that contains a PEM- or DER-encoded public key certificate.
SSHPUBLIC_KEY_FILE	The certificate store is the name of a file that contains an SSH-style public key.
SSHPUBLIC_KEY_BLOB	The certificate store is a string (base-64-encoded) that contains an SSH-style public key.
P7BFILE	The certificate store is the name of a PKCS7 file containing certificates.
PPKFILE	The certificate store is the name of a file that contains a PPK (PuTTY Private Key).
XMLFILE	The certificate store is the name of a file that contains a certificate in XML format.
XMLBLOB	The certificate store is a string that contains a certificate in XML format.
GOOGLEJSON	The certificate store is the name of a JSON file containing the service account information. Only valid when connecting to a Google service.
GOOGLEJSONBLOB	The certificate store is a string that contains the service account JSON. Only valid when connecting to a Google service.

OAuthJWTCertPassword

Data Type

string

Default Value

""

Remarks

If the certificate store is of a type that requires a password, this property is used to specify that password in order to open the certificate store.

This is not required when using the GOOGLEJSON OAuthJWTCertType. Google JSON keys are not encrypted.

OAuthJWTCertSubject

Data Type

string

Default Value

"*"

Remarks

When loading a certificate the subject is used to locate the certificate in the store.

If an exact match is not found, the store is searched for subjects containing the value of the property.

If a match is still not found, the property is set to an empty string, and no certificate is selected.

The special value "*" picks the first certificate in the certificate store.

The certificate subject is a comma separated list of distinguished name fields and values. For instance "CN=www.server.com, OU=test, C=US, E=support@cdata.com". Common fields and their meanings are displayed below.

Field	Meaning
CN	Common Name. This is commonly a host name like www.server.com.
O	Organization
OU	Organizational Unit
L	Locality
S	State
C	Country
E	Email Address

If a field value contains a comma it must be quoted.

OAuthJWTIssuer

Data Type

string

Default Value

""

Remarks

The issuer of the Java Web Token. This is typically either the Client Id or Email Address of the OAuth Application.

This is not required when using the GOOGLEJSON OAuthJWTCertType. Google JSON keys contain a copy of the issuer account.

OAuthJWTSubject

Data Type

string

Default Value

""

Remarks

The user subject for which the application is requesting delegated access. Typically, the user account name or email address.

SSLServerCert

Data Type

string

Default Value

""

Remarks

If using a TLS/SSL connection, this property can be used to specify the TLS/SSL certificate to be accepted from the server. Any other certificate that is not trusted by the machine is rejected.

This property can take the following forms:

Description	Example
A full PEM Certificate (example shortened for brevity)	-----BEGIN CERTIFICATE----- MIIChTCCAe4CAQAwDQYJKoZIhv......Qw== -----END CERTIFICATE-----
A path to a local file containing the certificate	C:\cert.cer
The public key (example shortened for brevity)	-----BEGIN RSA PUBLIC KEY----- MIGfMA0GCSq......AQAB -----END RSA PUBLIC KEY-----
The MD5 Thumbprint (hex values can also be either space or colon separated)	ecadbdda5a1529c58a1e9e09828d70e4
The SHA1 Thumbprint (hex values can also be either space or colon separated)	34a929226ae0819f2ec14b4a3d904f801cbb150d

If not specified, any certificate trusted by the machine is accepted.

Certificates are validated as trusted by the machine based on the System's trust store. The trust store used is the 'javax.net.ssl.trustStore' value specified for the system. If no value is specified for this property, Java's default trust store is used (for example, JAVA_HOME\lib\security\cacerts).

Use '*' to signify to accept all certificates. Note that this is not recommended due to security concerns.

FirewallType

Data Type

string

Default Value

"NONE"

Remarks

This property specifies the protocol that the driver will use to tunnel traffic through the FirewallServer proxy. Note that by default, the driver connects to the system proxy; to disable this behavior and connect to one of the following proxy types, set ProxyAutoDetect to false.

Type	Default Port	Description
TUNNEL	80	When this is set, the driver opens a connection to Google BigQuery and traffic flows back and forth through the proxy.
SOCKS4	1080	When this is set, the driver sends data through the SOCKS 4 proxy specified by FirewallServer and FirewallPort and passes the FirewallUser value to the proxy, which determines if the connection request should be granted.
SOCKS5	1080	When this is set, the driver sends data through the SOCKS 5 proxy specified by FirewallServer and FirewallPort. If your proxy requires authentication, set FirewallUser and FirewallPassword to credentials the proxy recognizes.

To connect to HTTP proxies, use ProxyServer and ProxyPort. To authenticate to HTTP proxies, use ProxyAuthScheme, ProxyUser, and ProxyPassword.

FirewallServer

Data Type

string

Default Value

""

Remarks

This property specifies the IP address, DNS name, or host name of a proxy allowing traversal of a firewall. The protocol is specified by FirewallType: Use FirewallServer with this property to connect through SOCKS or do tunneling. Use ProxyServer to connect to an HTTP proxy.

Note that the driver uses the system proxy by default. To use a different proxy, set ProxyAutoDetect to false.

FirewallPort

Data Type

int

Default Value

0

Remarks

This specifies the TCP port for a proxy allowing traversal of a firewall. Use FirewallServer to specify the name or IP address. Specify the protocol with FirewallType.

FirewallUser

Data Type

string

Default Value

""

Remarks

The FirewallUser and FirewallPassword properties are used to authenticate against the proxy specified in FirewallServer and FirewallPort, following the authentication method specified in FirewallType.

FirewallPassword

Data Type

string

Default Value

""

Remarks

This property is passed to the proxy specified by FirewallServer and FirewallPort, following the authentication method specified by FirewallType.

ProxyAutoDetect

Data Type

bool

Default Value

false

Remarks

This takes precedence over other proxy settings, so you'll need to set ProxyAutoDetect to FALSE in order use custom proxy settings.

NOTE: When this property is set to True, the proxy used is determined as follows:

• A search from the JVM properties (http.proxy, https.proxy, socksProxy, etc.) is performed.
• In the case that the JVM properties don't exist, a search from java.home/lib/net.properties is performed.
• In the case that java.net.useSystemProxies is set to True, a search from the SystemProxy is performed.
• In Windows only, an attempt is made to retrieve these properties from the Internet Options in the registry.

To connect to an HTTP proxy, see ProxyServer. For other proxies, such as SOCKS or tunneling, see FirewallType.

ProxyServer

Data Type

string

Default Value

""

Remarks

The hostname or IP address of a proxy to route HTTP traffic through. The driver can use the HTTP, Windows (NTLM), or Kerberos authentication types to authenticate to an HTTP proxy.

If you need to connect through a SOCKS proxy or tunnel the connection, see FirewallType.

By default, the driver uses the system proxy. If you need to use another proxy, set ProxyAutoDetect to false.

ProxyPort

Data Type

int

Default Value

80

Remarks

The port the HTTP proxy is running on that you want to redirect HTTP traffic through. Specify the HTTP proxy in ProxyServer. For other proxy types, see FirewallType.

ProxyAuthScheme

Data Type

string

Default Value

"BASIC"

Remarks

This value specifies the authentication type to use to authenticate to the HTTP proxy specified by ProxyServer and ProxyPort.

Note that the driver will use the system proxy settings by default, without further configuration needed; if you want to connect to another proxy, you will need to set ProxyAutoDetect to false, in addition to ProxyServer and ProxyPort. To authenticate, set ProxyAuthScheme and set ProxyUser and ProxyPassword, if needed.

The authentication type can be one of the following:

• BASIC: The driver performs HTTP BASIC authentication.
• DIGEST: The driver performs HTTP DIGEST authentication.
• NEGOTIATE: The driver retrieves an NTLM or Kerberos token based on the applicable protocol for authentication.
• PROPRIETARY: The driver does not generate an NTLM or Kerberos token. You must supply this token in the Authorization header of the HTTP request.

If you need to use another authentication type, such as SOCKS 5 authentication, see FirewallType.

ProxyUser

Data Type

string

Default Value

""

Remarks

The ProxyUser and ProxyPassword options are used to connect and authenticate against the HTTP proxy specified in ProxyServer.

You can select one of the available authentication types in ProxyAuthScheme. If you are using HTTP authentication, set this to the user name of a user recognized by the HTTP proxy. If you are using Windows or Kerberos authentication, set this property to a user name in one of the following formats:

user@domain

domain\user

ProxyPassword

Data Type

string

Default Value

""

Remarks

This property is used to authenticate to an HTTP proxy server that supports NTLM (Windows), Kerberos, or HTTP authentication. To specify the HTTP proxy, you can set ProxyServer and ProxyPort. To specify the authentication type, set ProxyAuthScheme.

If you are using HTTP authentication, additionally set ProxyUser and ProxyPassword to HTTP proxy.

If you are using NTLM authentication, set ProxyUser and ProxyPassword to your Windows password. You may also need these to complete Kerberos authentication.

For SOCKS 5 authentication or tunneling, see FirewallType.

By default, the driver uses the system proxy. If you want to connect to another proxy, set ProxyAutoDetect to false.

ProxySSLType

Data Type

string

Default Value

"AUTO"

Remarks

This property determines when to use SSL for the connection to an HTTP proxy specified by ProxyServer. This value can be AUTO, ALWAYS, NEVER, or TUNNEL. The applicable values are the following:

AUTO	Default setting. If the URL is an HTTPS URL, the driver will use the TUNNEL option. If the URL is an HTTP URL, the component will use the NEVER option.
ALWAYS	The connection is always SSL enabled.
NEVER	The connection is not SSL enabled.
TUNNEL	The connection is through a tunneling proxy. The proxy server opens a connection to the remote host and traffic flows back and forth through the proxy.

ProxyExceptions

Data Type

string

Default Value

""

Remarks

The ProxyServer is used for all addresses, except for addresses defined in this property. Use semicolons to separate entries.

Note that the driver uses the system proxy settings by default, without further configuration needed; if you want to explicitly configure proxy exceptions for this connection, you need to set ProxyAutoDetect = false, and configure ProxyServer and ProxyPort. To authenticate, set ProxyAuthScheme and set ProxyUser and ProxyPassword, if needed.

Logfile

Data Type

string

Default Value

""

Remarks

Once this property is set, the driver will populate the log file as it carries out various tasks, such as when authentication is performed or queries are executed. If the specified file doesn't already exist, it will be created.

Connection strings and version information are also logged, though connection properties containing sensitive information are masked automatically.

If a relative filepath is supplied, the location of the log file will be resolved based on the path found in the Location connection property.

For more control over what is written to the log file, you can adjust the Verbosity property.

Log contents are categorized into several modules. You can show/hide individual modules using the LogModules property.

To edit the maximum size of a single logfile before a new one is created, see MaxLogFileSize.

If you would like to place a cap on the number of logfiles generated, use MaxLogFileCount.

Java Logging

Java logging is also supported. To enable Java logging, set Logfile to:

Logfile=JAVALOG://myloggername

As in the above sample, JAVALOG:// is a required prefix to use Java logging, and you will substitute your own Logger.

The supplied Logger's getLogger method is then called, using the supplied value to create the Logger instance. If a logging instance already exists, it will reference the existing instance.

When Java logging is enabled, the Verbosity will now correspond to specific logging levels.

Verbosity

Data Type

string

Default Value

"1"

Remarks

The verbosity level determines the amount of detail that the driver reports to the Logfile. Verbosity levels from 1 to 5 are supported. These are detailed in the Logging page.

LogModules

Data Type

string

Default Value

""

Remarks

Only the modules specified (separated by ';') will be included in the log file. By default all modules are included.

See the Logging page for an overview.

MaxLogFileSize

Data Type

string

Default Value

"100MB"

Remarks

When the limit is hit, a new log is created in the same folder with the date and time appended to the end. The default limit is 100 MB. Values lower than 100 kB will use 100 kB as the value instead.

Adjust the maximum number of logfiles generated with MaxLogFileCount.

MaxLogFileCount

Data Type

int

Default Value

-1

Remarks

When the limit is hit, a new log is created in the same folder with the date and time appended to the end and the oldest log file will be deleted.

The minimum supported value is 2. A value of 0 or a negative value indicates no limit on the count.

Adjust the maximum size of the logfiles generated with MaxLogFileSize.

Location

Data Type

string

Default Value

"%APPDATA%\\CData\\GoogleBigQuery Data Provider\\Schema"

Remarks

The path to a directory which contains the schema files for the driver (.rsd files for tables and views, .rsb files for stored procedures). The folder location can be a relative path from the location of the executable. The Location property is only needed if you want to customize definitions (for example, change a column name, ignore a column, and so on) or extend the data model with new tables, views, or stored procedures.

If left unspecified, the default location is "%APPDATA%\\CData\\GoogleBigQuery Data Provider\\Schema" with %APPDATA% being set to the user's configuration directory:

Platform	%APPDATA%
Windows	The value of the APPDATA environment variable
Mac	~/Library/Application Support
Linux	~/.config

BrowsableSchemas

Data Type

string

Default Value

""

Remarks

Listing the schemas from databases can be expensive. Providing a list of schemas in the connection string improves the performance.

Tables

Data Type

string

Default Value

""

Remarks

Listing the tables from some databases can be expensive. Providing a list of tables in the connection string improves the performance of the driver.

This property can also be used as an alternative to automatically listing views if you already know which ones you want to work with and there would otherwise be too many to work with.

Specify the tables you want in a comma-separated list. Each table should be a valid SQL identifier with any special characters escaped using square brackets, double-quotes or backticks. For example, Tables=TableA,[TableB/WithSlash],WithCatalog.WithSchema.`TableC With Space`.

Note that when connecting to a data source with multiple schemas or catalogs, you will need to provide the fully qualified name of the table in this property, as in the last example here, to avoid ambiguity between tables that exist in multiple catalogs or schemas.

Views

Data Type

string

Default Value

""

Remarks

Listing the views from some databases can be expensive. Providing a list of views in the connection string improves the performance of the driver.

This property can also be used as an alternative to automatically listing views if you already know which ones you want to work with and there would otherwise be too many to work with.

Specify the views you want in a comma-separated list. Each view should be a valid SQL identifier with any special characters escaped using square brackets, double-quotes or backticks. For example, Views=ViewA,[ViewB/WithSlash],WithCatalog.WithSchema.`ViewC With Space`.

Note that when connecting to a data source with multiple schemas or catalogs, you will need to provide the fully qualified name of the table in this property, as in the last example here, to avoid ambiguity between tables that exist in multiple catalogs or schemas.

RefreshViewSchemas

Data Type

bool

Default Value

true

Remarks

When using BigQuery views, BigQuery stores a copy of the view schema with the view itself. However, these stored view schemas are not updated when the tables used by the view change. This means that the stored view schema can easily become out of date and cause queries using the view to fail.

By default, the driver will not use the stored view schema and will instead query the view to determine the available columns. This guarantees that the schema will be up to date although it requires the driver to start a query job.

You can disable this option to force the driver to use the stored view schemas. This prevents the driver from running any queries when getting a view schema, but also means that queries using the view will fail if the schema is out of date.

ShowTableDescriptions

Data Type

bool

Default Value

false

Remarks

By default table descriptions are not shown, since the Google BigQuery API requires an extra request beyond what is usually required for reading tables.

Enabling this option will show table descriptions, but will cost an extra API request for every table when a table list is fetched. This can slow down metadata operations on large datasets.

PrimaryKeyIdentifiers

Data Type

string

Default Value

""

Remarks

Google BigQuery does not natively support primary keys, but for certain DML operations or database tools you may need to define them. By default this option is disabled and no tables will have primary keys except for the ones defined in schema files (if you set Location).

Primary keys are defined using a list of rules which match tables and provide a list of key columns. For example, PrimaryKeyIdentifiers="*=key;transactions=tx_date,tx_serial;user_comments=" has three rules separated by semicolons:

1. The first rule *=key means that every table without a more specific rule will have one primary key column called key. Tables that do not have a key column will not have any primary keys.
2. The second rule transactions=tx_date,tx_serial means that the transactions table will have the two primary key columns tx_date and tx_serial. If any of those columns are missing from the table then they will be ignored.
3. The third rule user_comments= means that the user_comments table will have no primary keys. The only use that empty key lists have is in overriding the default rule. If there is no default rule present then the only tables with primary keys would be the ones explicitly listed.

Note that the table names can include just the table, the table and dataset or the table, dataset and project. Both column and table names may be quoted using SQL quotes:

/* Rules with just table names use the connection ProjectId (or DataProjectId) and DatasetId. 

   All these rules refer to the same table with a connection where ProjectId=someProject;DatasetId=someDataset */

someTable=a,b,c

someDataset.someTable=a,b,c

someProject.someDataset.someTable=a,b,c



/* Any table or column name may be quoted */

`someProject`."someDataset".[someTable]=`a`,[b],"c"

AllowedTableTypes

Data Type

string

Default Value

"TABLE,EXTERNAL,VIEW,MATERIALIZED_VIEW"

Remarks

This option is a comma-separated list of the table type values that the driver displays. Any table-like or view-like entity that doesn't have a matching type will not be reported when listing tables.

• TABLE Standard tables
• EXTERNAL Read-only table stored on another service (like GCS or Drive)
• SNAPSHOT A read-only table that preserves the data of another table at a specific point in time
• VIEW Standard views
• MATERIALIZED_VIEW A view that is recalculated and cached each time its base table changes

For example, to restrict the driver to listing only simple tables and views, this option would be set to TABLE,VIEW

AutoCache

Data Type

bool

Default Value

false

Remarks

When AutoCache = true, the driver automatically maintains a cache of your table's data in the database of your choice.

Setting the Caching Database

When AutoCache = true, the driver caches to a simple, file-based cache. You can configure its location or cache to a different database with the following properties:

• CacheLocation: Specifies the path to the file store.
• CacheDriver and CacheConnection: Specifies a driver to a database and the connection string.

See Also

• CacheMetadata: This property reduces the amount of metadata that crosses the network by persisting table schemas retrieved from the Google BigQuery metadata. Metadata then needs to be retrieved only once instead of every connection.
• Explicitly Caching Data: This section provides more examples of using AutoCache in Offline mode.
• CACHE Statements: You can use the CACHE statement to persist any SELECT query, as well as manage the cache; for example, refreshing schemas.

CacheDriver

Data Type

string

Default Value

""

Remarks

You can cache to any database for which you have a JDBC driver, including CData JDBC drivers.

The cache database is determined based on the CacheDriver and CacheConnection properties. The CacheDriver is the name of the JDBC driver class that you want to use to cache data.

Note that you must also add the CacheDriver JAR file to the classpath.

The following examples show how to cache to several major databases. Refer to CacheConnection for more information on the JDBC URL syntax and typical connection properties.

Derby and Java DB

The driver simplifies Derby configuration. Java DB is the Oracle distribution of Derby. The JAR file is shipped in the JDK. You can find the JAR file, derby.jar, in the db subfolder of the JDK installation. In most caching scenarios, you need to specify only the following, after adding derby.jar to the classpath:

jdbc:googlebigquery:CacheLocation='c:/Temp/cachedir';InitiateOAuth=GETANDREFRESH;ProjectId=NameOfProject;DatasetId=NameOfDataset;

To customize the Derby JDBC URL, use CacheDriver and CacheConnection. For example, to cache to an in-memory database, use a JDBC URL like the following:

jdbc:googlebigquery:CacheDriver=org.apache.derby.jdbc.EmbeddedDriver;CacheConnection='jdbc:derby:memory';InitiateOAuth=GETANDREFRESH;ProjectId=NameOfProject;DatasetId=NameOfDataset;

SQLite

The following is a JDBC URL for the SQLite JDBC driver:

jdbc:googlebigquery:CacheDriver=org.sqlite.JDBC;CacheConnection='jdbc:sqlite:C:/Temp/sqlite.db';InitiateOAuth=GETANDREFRESH;ProjectId=NameOfProject;DatasetId=NameOfDataset;

MySQL

The following is a JDBC URL for the included CData JDBC Driver for MySQL:

  jdbc:googlebigquery:Cache Driver=cdata.jdbc.mysql.MySQLDriver;Cache Connection='jdbc:mysql:Server=localhost;Port=3306;Database=cache;User=root;Password=123456';InitiateOAuth=GETANDREFRESH;ProjectId=NameOfProject;DatasetId=NameOfDataset;

  

SQL Server

The following JDBC URL uses the Microsoft JDBC Driver for SQL Server:

jdbc:googlebigquery:Cache Driver=com.microsoft.sqlserver.jdbc.SQLServerDriver;Cache Connection='jdbc:sqlserver://localhost\sqlexpress:7437;user=sa;password=123456;databaseName=Cache';InitiateOAuth=GETANDREFRESH;ProjectId=NameOfProject;DatasetId=NameOfDataset;

Oracle

The following is a JDBC URL for the Oracle Thin Client:

jdbc:googlebigquery:Cache Driver=oracle.jdbc.OracleDriver;CacheConnection='jdbc:oracle:thin:scott/tiger@localhost:1521:orcldb';InitiateOAuth=GETANDREFRESH;ProjectId=NameOfProject;DatasetId=NameOfDataset;

NOTE: If using a version of Oracle older than 9i, the cache driver will instead be oracle.jdbc.driver.OracleDriver .

PostgreSQL

The following JDBC URL uses the official PostgreSQL JDBC driver:

jdbc:googlebigquery:CacheDriver=cdata.jdbc.postgresql.PostgreSQLDriver;CacheConnection='jdbc:postgresql:User=postgres;Password=admin;Database=postgres;Server=localhost;Port=5432;';InitiateOAuth=GETANDREFRESH;ProjectId=NameOfProject;DatasetId=NameOfDataset;

CacheConnection

Data Type

string

Default Value

""

Remarks

The cache database is determined based on the CacheDriver and CacheConnection properties. Both properties are required to use the cache database. Examples of common cache database settings can be found below. For more information on setting the caching database's driver, refer to CacheDriver.

The connection string specified in the CacheConnection property is passed directly to the underlying CacheDriver. Consult the documentation for the specific JDBC driver for more information on the available properties. Make sure to include the JDBC driver in your application's classpath.

Derby and Java DB

The driver simplifies caching to Derby, only requiring you to set the CacheLocation property to make a basic connection.

Alternatively, you can configure the connection to Derby manually using CacheDriver and CacheConnection. The following is the Derby JDBC URL syntax:

jdbc:derby:[subsubprotocol:][databaseName][;attribute=value[;attribute=value] ... ]

For example, to cache to an in-memory database, use the following:

jdbc:derby:memory

SQLite

To cache to SQLite, you can use the SQLite JDBC driver. The following is the syntax of the JDBC URL:

jdbc:sqlite:dataSource

• Data Source: The path to an SQLite database file. Or, use a value of :memory to cache in memory.

MySQL

The installation includes the CData JDBC Driver for MySQL. The following is an example JDBC URL:

jdbc:mysql:User=root;Password=root;Server=localhost;Port=3306;Database=cache

The following are typical connection properties:

• Server: The IP address or domain name of the server you want to connect to.
• Port: The port that the server is running on.
• User: The user name provided for authentication to the database.
• Password: The password provided for authentication to the database.
• Database: The name of the database.

SQL Server

The JDBC URL for the Microsoft JDBC Driver for SQL Server has the following syntax:

jdbc:sqlserver://[serverName[\instance][:port]][;database=databaseName][;property=value[;property=value] ... ]

For example:

jdbc:sqlserver://localhost\sqlexpress:1433;integratedSecurity=true

The following are typical SQL Server connection properties:

• Server: The name or network address of the computer running SQL Server. To connect to a named instance instead of the default instance, this property can be used to specify the host name and the instance, separated by a backslash.
• Port: The port SQL Server is running on.
• Database: The name of the SQL Server database.
• Integrated Security: Set this option to true to use the current Windows account for authentication. Set this option to false if you are setting the User and Password in the connection.

  To use integrated security, you will also need to add sqljdbc_auth.dll to a folder on the Windows system path. This file is located in the auth subfolder of the Microsoft JDBC Driver for SQL Server installation. The bitness of the assembly must match the bitness of your JVM.

• User Id: The user name provided for authentication with SQL Server. This property is only needed if you are not using integrated security.
• Password: The password provided for authentication with SQL Server. This property is only needed if you are not using integrated security.

Oracle

The following is the conventional JDBC URL syntax for the Oracle JDBC Thin driver:

jdbc:oracle:thin:[userId/password]@[//]host[[:port][:sid]]

For example:

jdbc:oracle:thin:scott/tiger@myhost:1521:orcl

The following are typical connection properties:

• Data Source: The connect descriptor that identifies the Oracle database. This can be a TNS connect descriptor, an Oracle Net Services name that resolves to a connect descriptor, or, after version 11g, an Easy Connect naming (the host name of the Oracle server with an optional port and service name).

• Password: The password provided for authentication with the Oracle database.
• User Id: The user Id provided for authentication with the Oracle database.

PostgreSQL

The following is the JDBC URL syntax for the official PostgreSQL JDBC driver:

jdbc:postgresql:[//[host[:port]]/]database[[?option=value][[&option=value][&option=value] ... ]]

For example, the following connection string connects to a database on the default host (localhost) and port (5432):

jdbc:postgresql:postgres

The following are typical connection properties:

• Host: The address of the server hosting the PostgreSQL database.
• Port: The port used to connect to the server hosting the PostgreSQL database.
• Database: The name of the database.
• User name: The user Id provided for authentication with the PostgreSQL database. You can specify this in the JDBC URL with the "user" parameter.
• Password: The password provided for authentication with the PostgreSQL database.

CacheLocation

Data Type

string

Default Value

"%APPDATA%\\CData\\GoogleBigQuery Data Provider"

Remarks

The CacheLocation is a simple, file-based cache. The driver uses Java DB, Oracle's distribution of the Derby database. To cache to Java DB, you will need to add the Java DB JAR file to the classpath. The JAR file, derby.jar, is shipped in the JDK and located in the db subfolder of the JDK installation.

If left unspecified, the default location is "%APPDATA%\\CData\\GoogleBigQuery Data Provider" with %APPDATA% being set to the user's configuration directory:

Platform	%APPDATA%
Windows	The value of the APPDATA environment variable
Mac	~/Library/Application Support
Linux	~/.config

See Also

• AutoCache: Set to implicitly create and maintain a cache for later offline use.
• CacheMetadata: Set to persist the Google BigQuery catalog in CacheLocation.

CacheTolerance

Data Type

int

Default Value

600

Remarks

The tolerance for stale data in the cache specified in seconds. This only applies when AutoCache is used. The driver checks with the data source for newer records after the tolerance interval has expired. Otherwise, it returns the data directly from the cache.

Offline

Data Type

bool

Default Value

false

Remarks

When Offline = true, all queries execute against the cache as opposed to the live data source. In this mode, certain queries like INSERT, UPDATE, DELETE, and CACHE are not allowed.

CacheMetadata

Data Type

bool

Default Value

false

Remarks

As you execute queries with this property set, table metadata in the Google BigQuery catalog are cached to the file store specified by CacheLocation if set or the user's home directory otherwise. A table's metadata will be retrieved only once, when the table is queried for the first time.

When to Use CacheMetadata

The driver automatically persists metadata in memory for up to two hours when you first discover the metadata for a table or view and therefore, CacheMetadata is generally not required. CacheMetadata becomes useful when metadata operations are expensive such as when you are working with large amounts of metadata or when you have many short-lived connections.

When Not to Use CacheMetadata

• When you are working with volatile metadata: Metadata for a table is only retrieved the first time the connection to the table is made. To pick up new, changed, or deleted columns, you would need to delete and rebuild the metadata cache. Therefore, it is best to rely on the in-memory caching for cases where metadata changes often.
• When you are caching to a database: CacheMetadata can only be used with CacheLocation. If you are caching to another database with the CacheDriver and CacheConnection properties, use AutoCache to cache implicitly. Or, use CACHE Statements to cache explicitly.

StorageTimeout

Data Type

string

Default Value

"60"

Remarks

Google BigQuery and many proxies/firewalls restrict the amount of time that idle connections stay alive before they are forcibly closed. This can be a problem when using the Storage API because the driver may stream data faster than it can be consumed. While the consumer is catching up, the driver does not use its connection and it may be closed by the next time the driver uses it.

To avoid this the driver will automatically close and reopen the connection if it has been idle for too long. This property controls how many seconds the connection has to be idle for the driver to reset it. To disable these resets this property can also set to 0 or a negative value.

AllowAggregateParameters

Data Type

bool

Default Value

false

Remarks

This option affects how string parameters are handled when using direct queries through QueryPassthrough. For example, consider this query:

INSERT INTO proj.data.tbl(x) VALUES (@x)

By default, this option is disabled and string parameters are quoted and escaped into SQL strings. That means that any value can be safely used as a string parameter, but it also means that parameters cannot be used as raw aggregate values:

/*

 * If @x is set to: test value ' contains quote

 *

 * Result is a valid query

*/

INSERT INTO proj.data.tbl(x) VALUES ('test value \' contains quote')



/*

 * If @x is set to: ['valid', ('aggregate', 'value')]

 *

 * Result contains string instead of aggregate:

*/

INSERT INTO proj.data.tbl(x) VALUES ('[\'valid\', (\'aggregate\', \'value\')]')

When this option is enabled, string parameters are inserted directly into the query. This means that raw aggregates can be used as parameters, but it also means that all simple strings must be escaped:

/*

 * If @x is set to: test value ' contains quote

 *

 * Result is an invalid query

*/

INSERT INTO proj.data.tbl(x) VALUES (test value ' contains quote)



/*

 * If @x is set to: ['valid', ('aggregate', 'value')]

 *

 * Result is an aggregate

*/

INSERT INTO proj.data.tbl(x) VALUES (['valid', ('aggregate', 'value')])

AuditLimit

Data Type

string

Default Value

"1000"

Remarks

When auditing is enabled with the AuditMode option, this property is used to determine how many rows will be allowed in the audit table at once.

By default this property is 1000, meaning that only the 1000 most recent audit events will be available within the audit table.

This property can also be set to -1, which places no limits on the size of the audit table. In this mode, the audit table should be periodically cleared to prevent the driver from using excessive memory.

DELETE FROM AuditJobs#TEMP

AuditMode

Data Type

string

Default Value

""

Remarks

The driver can record certain internal actions taken when it runs queries. For each of those actions listed in this option, the driver will create a temproary audit table which logs when the action took place, what query caused the action and any other relevant information.

By default this option is set to 'none' and the driver does not record any audit information. This option can also be set to a comma-separated list of the following actions:

Mode Name	Audit Table	Description	Columns
start-jobs	AuditJobs#TEMP	Records all jobs started by the driver	Timestamp,Query,ProjectId,Location,JobId

Refer to AuditLimit for more information on how to limit the size of these tables.

BigQueryOptions

Data Type

string

Default Value

""

Remarks

A list of Google BigQuery options:

Option	Description
gbqoImplicitJoinAsUnion	This option will prevent the driver from converting an IMPLICIT JOIN into a CROSS JOIN as expected by SQL92. Instead, it will leave it as an IMPLICIT JOIN, which Google BigQuery will execute as a UNION ALL.

ConnectionLifeTime

Data Type

int

Default Value

0

Remarks

The maximum lifetime of a connection in seconds. Once the time has elapsed, the connection object is disposed. The default is 0 which indicates there is no limit to the connection lifetime.

ConnectOnOpen

Data Type

bool

Default Value

false

Remarks

When set to true, a connection will be made to Google BigQuery when the connection is opened. This property enables the Test Connection feature available in various database tools.

This feature acts as a NOOP command as it is used to verify a connection can be made to Google BigQuery and nothing from this initial connection is maintained.

Setting this property to false may provide performance improvements (depending upon the number of times a connection is opened).

GenerateSchemaFiles

Data Type

string

Default Value

"Never"

Remarks

This property outputs schemas to .rsd files in the path specified by Location.

Available settings are the following:

• Never: A schema file will never be generated.
• OnUse: A schema file will be generated the first time a table is referenced, provided the schema file for the table does not already exist.
• OnStart: A schema file will be generated at connection time for any tables that do not currently have a schema file.
• OnCreate: A schema file will be generated by when running a CREATE TABLE SQL query.

Note that if you want to regenerate a file, you will first need to delete it.

Generate Schemas with SQL

When you set GenerateSchemaFiles to OnUse, the driver generates schemas as you execute SELECT queries. Schemas are generated for each table referenced in the query.

When you set GenerateSchemaFiles to OnCreate, schemas are only generated when a CREATE TABLE query is executed.

Generate Schemas on Connection

Another way to use this property is to obtain schemas for every table in your database when you connect. To do so, set GenerateSchemaFiles to OnStart and connect.

MaximumBillingTier

Data Type

string

Default Value

""

Remarks

Limits the billing tier for this job. Queries that have resource usage beyond this tier will fail (without incurring a charge). If unspecified, this will be set to your project default. If your query is too compute intensive for BigQuery to complete at the standard per TB pricing tier, BigQuery returns a billingTierLimitExceeded error and an estimate of how much the query would cost. To run the query at a higher pricing tier, pass a new value for maximumBillingTier as part of the query request. The maximumBillingTier is a positive integer that serves as a multiplier of the basic price per TB. For example, if you set maximumBillingTier to 2, the maximum cost for that query will be 2x basic price per TB.

MaximumBytesBilled

Data Type

string

Default Value

""

Remarks

When this value is provided, all jobs will use this value as their default billing cap. If a job uses more than this many bytes, BigQuery will cancel it and it will not be billed. By default there is no cap and all jobs will be billed for however many bytes they consume.

This only has an effect when using DestinationTable or when using the InsertJob stored procedure. BigQuery does not allow standard query jobs to have byte limits.

MaxRows

Data Type

int

Default Value

-1

Remarks

Limits the number of rows returned rows when no aggregation or group by is used in the query. This helps avoid performance issues at design time.

Other

Data Type

string

Default Value

""

Remarks

The properties listed below are available for specific use cases. Normal driver use cases and functionality should not require these properties.

Specify multiple properties in a semicolon-separated list.

Caching Configuration

CachePartial=True	Caches only a subset of columns, which you can specify in your query.
QueryPassthrough=True	Passes the specified query to the cache database instead of using the SQL parser of the driver.

Integration and Formatting

DefaultColumnSize	Sets the default length of string fields when the data source does not provide column length in the metadata. The default value is 2000.
ConvertDateTimeToGMT	Determines whether to convert date-time values to GMT, instead of the local time of the machine.
RecordToFile=filename	Records the underlying socket data transfer to the specified file.

PoolIdleTimeout

Data Type

int

Default Value

60

Remarks

The allowed idle time a connection can remain in the pool until the connection is closed. The default is 60 seconds.

PoolMaxSize

Data Type

int

Default Value

100

Remarks

The maximum connections in the pool. The default is 100. To disable this property, set the property value to 0 or less.

PoolMinSize

Data Type

int

Default Value

1

Remarks

The minimum number of connections in the pool. The default is 1.

PoolWaitTime

Data Type

int

Default Value

60

Remarks

The max seconds to wait for a connection to become available. If a new connection request is waiting for an available connection and exceeds this time, an error is thrown. By default, new requests wait forever for an available connection.

PseudoColumns

Data Type

string

Default Value

""

Remarks

This setting is particularly helpful in Entity Framework, which does not allow you to set a value for a pseudo column unless it is a table column. The value of this connection setting is of the format "Table1=Column1, Table1=Column2, Table2=Column3". You can use the "*" character to include all tables and all columns; for example, "*=*".

QueryPassthrough

Data Type

bool

Default Value

false

Remarks

When this is set, queries are passed through directly to Google BigQuery.

Readonly

Data Type

bool

Default Value

false

Remarks

If this property is set to true, the driver will allow only SELECT queries. INSERT, UPDATE, DELETE, and stored procedure queries will cause an error to be thrown.

RTK

Data Type

string

Default Value

""

Remarks

The RTK property may be used to license a build. See the included licensing file to see how to set this property. The runtime key is only available if you purchased an OEM license.

TableSamplePercent

Data Type

string

Default Value

""

Remarks

This option can be set to make the driver use the TABLESAMPLE for each table referenced by a query. The value determines what percent is provided to the PERCENT clause. That clause will only be generated if this property's value is above zero.

-- Input SQL

SELECT * FROM `tbl`



-- Generated Google BigQuery SQL when TableSamplePercent=10

SELECT * FROM `tbl` TABLESAMPLE SYSTEM (10 PERCENT)

This option is subject to a few limitations:

• It is applied during query converison and has no effect when QueryPassthrough is set.
• More rows may be returned than expected due to how the server implements TABLESAMPLE. Please see the Google BigQuery documentation for more information.
• TABLESAMPLE is not supported on views. If a view is queried in sampling mode, the driver will omit the TABLESAMPLE clause for the view.

Timeout

Data Type

string

Default Value

"300"

Remarks

If Timeout = 0, operations do not time out. The operations run until they complete successfully or until they encounter an error condition.

If Timeout expires and the operation is not yet complete, the driver throws an exception.

UseConnectionPooling

Data Type

bool

Default Value

false

Remarks

This property enables connection pooling. The default is false. See Connection Pooling for information on using connection pools.

ViewsBack To Top

1. Datasets
2. PartitionsList
3. PartitionsValues
4. Projects

Datasets

Lists all the accessible datasets for a given project.

Columns

Name	Type	Description
Id [KEY]	String	The fully qualified, unique, opaque Id of the dataset.
Kind	String	The resource type.
FriendlyName	String	A descriptive name for the dataset
DatasetReference_ProjectId	String	A unique reference to the container project.
DatasetReference_DatasetId	String	A unique reference to the dataset, without the project name.

PartitionsList

Lists the partitioning definitions for tables

Columns

Name	Type	Description
Id [KEY]	String	A unique identifier for the partition.
ProjectId	String	The project that the table belongs to.
DatasetId	String	The dataset that the table belongs to.
TableName	String	The name of the table.
ColumnName	String	The name of the column used for partitioning.
ColumnType	String	The type of the partitioning column.
Kind	String	The type of partitioning used by the table. One of DATE, RANGE or INGESTION.
RequireFilter	Boolean	Whether a filter on the partition column is required to query the table.

PartitionsValues

Lists the partitioning ranges for tables

Columns

Name	Type	Description
Id	String	A unique identifier for the partition.
RangeLow	String	The lowest value of the partition column. Either an integer when Kind is RANGE, or a date otherwise.
RangeHigh	String	The highest value of the partition column. Either an integer when Kind is RANGE, or a date otherwise.
RangeInterval	String	The range of values which are included in each partition. Only valid when Kind is RANGE
DateResolution	String	How much of the date is significant to a TIME or INGESTION partition column. One of DAY, HOUR, MONTH or YEAR.

Projects

Lists all the projects for the authorized user.

Columns

Name	Type	Description
Id [KEY]	String	The unique identifier of the Project
Kind	String	The resource type.
FriendlyName	String	A descriptive name for the project.
NumericId	String	The numeric Id of the project.
ProjectReference_ProjectId	String	A unique reference to the project.