Fixed an issue where Shopify Query was not fetching all data correctly.
BigQuery
Fixed an issue where the Mailchimp Query component caused an "out of memory" issue with certain data sources.
Fixed an issue where exported credentials via the API were not being encrypted as expected. Warning: Credentials exported prior to 1.47.8 may not be importable in 1.48.7 due to this change. It may be advisable to do an in-place update if this is likely to be an issue for you.
Fixed an issue where the Table Update component would not properly validate when pointing to a table with a RECORD/STRUCT column.
Fixed an issue where the Database Query component would erroneously strip zeros from string fields.
Note: Due to some technical issues with this release, we do not recommend customers manually update to this version. We highly recommend users on 1.47 to update to the latest version of Matillion ETL.
New features and improvements
All platforms
New "Create your own connector" feature. The new API Extract wizard and API Extract component allow you to connect to any service that offers a REST API, allowing you to create your own connectors to load data (does not include Synapse).
New Intercom Extract component lets users connect to their Intercom account and load data into external tables (does not include Synapse).
You can now configure Matillion ETL to authenticate with any Open ID Connect provider. This includes default support for Google, Microsoft, and Okta, plus a "generic" option.
Assert components now auto-populate their properties when connected to another component. For example, connecting Assert Table to an S3 Load component will automatically configure Assert Table to use the same table as the S3 Load component
Individual jobs are now selectable in the Migrate wizard, allowing greater control over what is migrated.
The Environment panel now has a right-click context menu for related options.
Snowflake
Users can now specify a fully qualified name for a stage and format in Snowflake from a different database than that which the user has currently selected as stage name in the format of database.schema.stage and the format name in the format of database.schema.format.
Table data can now be copied to external Parquet files using LZO compression
Load components now have a "Match by Column Name" property that allows loading of semi-structured data directly from a file to a table.
Synapse
New Detect Changes component allows comparison of two similar tables to yield a new column that details whether rows have been inserted, deleted, or changed.
New Pivot component can pivot data from a selected column into multiple columns, partitioned by user-defined criteria.
New Generate Sequence component allows users to create rows whose data are increments of a user-defined numerical sequence.
New Multi Table Input component reads data from multiple tables according to their names matching a user-defined pattern.
Azure Blob Load Generator helps generate compatible "Create Table" and "Azure Blob Storage Load" components by sampling delimited data files on Azure Blob Storage and predicting the layout.
Rewrite Table now allows Partitioning of data by a field.