Matillion ETL release notes: v1.53

(6th July 2021)

All platforms

Matillion has enabled transport-layer security (TLS) by default for all Hazelcast communications. This enablement will keep your data and messages secure in transit while operating in a cluster. TLS offers higher security and better performance than the earlier symmetric encryption, which is now deprecated by Hazelcast and which has been removed from Matillion ETL. No user interaction is required with this update, but if you wish to disable this setting (for example, if you wish to run Hazelcast Management Center for monitoring purposes) then you can set the value of variable HAZEL_SECURITY_ENABLED to false in the environment properties.

(27th April 2021)

All platforms

Fixed an issue where an invalid character was causing query profile names to disappear.
We are now distributing Matillion ETL with the enterprise version of Hazelcast. As well as improved support, this change also allows us to offer encrypted network communications by default, helping to keep your data secure.

Snowflake, Redshift, and BigQuery

The Salesforce Incremental Load component has been updated. This update has affected which columns are used by the ETL process for an incremental load on a number of tables (adding the columns 'CreatedDate' and 'LoginTime' to the existing columns 'SystemModstamp', 'LastModifiedDate', which are already used as Incremental Columns, if they are present in the selected source).
Added schema drift to the Salesforce Incremental Load component. By default, this featured is turned off. Users can turn this setting on manually in the Shared Job that is generated by amending the parameter Automatically Update Target Metadata from No to Yes.

(12th April 2021)

Snowflake and Redshift

Fixed an issue where the Manage Change Data Capture (CDC) feature was missing from the Project menu (AWS instances only).

Delta Lake on Databricks

Fixed an issue with listing Databricks clusters in the Create Project setup wizard.

(25th March 2021)

All platforms

Apache Tomcat version updated to 8.5.63
Manage Query Profiles wizard now supports API key=value pairs as a method of authentication.
Removed the "Use Sandbox" property from the Salesforce Marketing Cloud Query component.
Users will receive a popup notification when attempting to delete a job that is in use by other jobs where those other jobs will be affected. Users can then confirm the job deletion or cancel.
Users will receive a popup notification when attempting to rename a job that is in use by other jobs where those other jobs will be affected. Users can then confirm the job renaming or cancel.
Users will receive a popup notification warning that uploading unverified 3rd party libraries may cause Matillion ETL to malfunction. Please always ensure that you only upload trusted JDBC drivers.

Snowflake

New Alter Table component enables users to register one or more tables to Snowflake's search optimization service.
Alter Warehouse component has been updated to include additional Snowflake warehouse sizes.
S3 Load component (AWS only) and Azure Blob Storage Load component (Azure only) now offer support for Snowflake storage integrations.

Snowflake and Redshift

New Oracle Output component enables users to output the contents of a table (or view) from their cloud data warehouse (CDW) to their Oracle database.

Snowflake, Redshift, and BigQuery

New Pardot Extract component, which uses an OAuth authentication method. The original Pardot Extract component has been deprecated, but will persist for existing orchestration jobs. Pardot Authentication Guide has been updated for instructions on authentication.

Snowflake, Redshift, and Synapse

Enable or disable GZip compression on Matillion ETL Query components from within a component's Load Options by setting the Compression Type.

Redshift

Users can now load semi-structured data into Redshift with the S3 Load component. When selecting a target table with the Redshift SUPER data type, the S3 Load component will only offer the following data types: JSON, ORC, and PARQUET. However, if a user selects to include columns of data file types other than SUPER in the Load Columns property, all data file types will be available for selection.
Assert Table component now supports the SUPER data type.
Assert View component now supports the SUPER data type.

Delta Lake on Databricks

Optimise component enables users to optimise the layout of Delta Lake data and optionally optimise a subset of data or co-locate data by column.
S3 Load component enables users to load data into an existing table from objects stored in Amazon Simple Storage Service (AWS only).
S3 Load Generator guides users through a setup wizard where they can create the table to load and view tables on the fly, before loading from S3 (AWS only).
Multi Table Input component reads data from many input tables based on filtering the input tables via a pattern (AWS only).

All platforms

DynamoDB API driver updated (19.0.7417.0 → 20.0.7662.0)
MongoDB driver updated (19.0.7331.0 → 20.0.7662.0)
Salesforce driver updated (19.0.7332.0 → 20.0.7662.0)
Salesforce Marketing Cloud API driver updated (19.0.7354.0 → 20.0.7662.0)
Square API driver updated 20.0.7538.0 to 20.0.7697.0)
YouTube Analytics API driver updated (18.0.6817.0 → 20.0.7628.0)