With the general release of SQL Server 2019 last year, Microsoft introduced Big Data Clusters (BDC) for customers to run exclusively on Linux containers. Supported through Kubernetes, BDC offer a unified platform to discover insights from structured and unstructured data.
Today, the latest cumulative update for SQL Server 2019 has been released. Dubbed CU5, the release brings a bunch of new features, particularly focusing towards expanding the capabilities offered through BDC.
In concise form, these are the changes that have been delivered:
- Support for deploying BDC on Red Hat OpenShift Kubernetes platform.
- Enabled running applications within BDC as non-root users.
- Support for deploying multiple BDCs against the same Active Directory domain.
- Enriched data virtualization experiences.
- Enhanced and open sourced Spark SQL connector.
- Miscellaneous improvements and bug fixes.
The new features have been detailed in a separate blog post. For starters, BDC deployments will be supported on OpenShift version 4.3 and above. The security design of BDC has specifically been upgraded in order to cater to the OpenShift Container Platform. Microsoft"s SQL team has worked closely with Red Hat to make these changes possible. Moving on, all supported platforms will now ensure that container applications running on BDC start as non-root users by default. Pre-CU5 BDC deployments will not be affected by this change.
New data virtualization capabilities, meanwhile, include the introduction of two stored procedures, namely sp_data_source_objects and sp_data_source_columns. These can be used to find tables that need to be virtualized and for schema discovery via T-SQL. Multiple BDC deployments can now also be utilized in enterprise environments in order to "accommodate multiple use cases like development/test, pre-production or production, CI/CD pipelines or HADR".
Finally, the SQL Server and Azure SQL Connector for Apache Spark has been open-sourced under the ApacheV2 license. More updates for the connector are planned to be released when it goes live. Those interested in providing feedback on this update can head over to this forum.
Update: The blog post detailing the release seems to have been taken down for now. Further information in regard to this will be provided as soon as it becomes available.
Update 2: The announcement blog is back up after being removed a few days back. CU5 should now be rolling out to users.