--- title: "What's New in Version 5.7.1" canonical: "https://docs.vaultspeed.com/space/VPD/3857121281/What's%20New%20in%20Version%205.7.1" format: markdown --- **Table of Contents** > Macro (toc) ## New Features ### Database Name Support **Enhanced Database and Schema Management** We have introduced the ability to specify database names in Snowflake. In Databricks, schemas are organized using a `catalog.schema.table/view` hierarchy. This feature enhances flexibility, organization, and I/O resource management, improving data workflows and security. **Key Benefits:** - **Enhanced Flexibility**: This feature enables separate databases for source, intermediate (including extraction and staging), raw, and business Vault schemas, accommodating diverse data management needs. - **Improved Organization**: Facilitates structured and efficient data management by segregating schemas across databases. - **Optimized I/O Resource Management**: Different types of processing have distinct I/O patterns. By segregating schemas, such as staging and business Vault, across databases, we can better manage I/O resources and reduce competition for these resources. This update ensures better data management and organization, enhancing the overall efficiency of your data workflows. > ⚠️ **Limitation**: Currently, Matillion, Talend, and DBT do not support the inclusion of database names in schema parameters. This is a temporary restriction and will be resolved in a future update. The image below illustrates this feature, showing four distinct layers with their respective database designations: > Macro (drawio) ### Implementation in VaultSpeed In VaultSpeed, the syntax for defining the database and schema name is as follows: `.` **Database Names** - **Fixed:** Database names are always fixed and do not change. **Schema Names** - **Dynamic:** Schema names can be dynamic. The keyword PROJECT in the schema name will be replaced by the source name at the source level. #### Example of Dynamic Schema Names using the PROJECT Keyword: - Consider a source named MYSOURCE - **Project Level Definition**: - The Extract Layer schema is defined at the project level as: STAGE_DB.PROJECT_EXT - **Resulting Schema Name**: - When the source is MYSOURCE, the resulting schema name will be: STAGE_DB.MY_SOURCE_EXT. #### Parameters at Project Level You can define database names for different environments and use cases at the project level. This allows for a high-level organization of schemas across various databases, ensuring that each type of processing (Staging, Raw Data Vault, Business Data Vault, Information Marts) can have its own set of databases. - **Configuration**: Within the project settings, specify database names for different environments. The database name is a fixed parameter, as explained in the implementation section. - **Flexibility**: Allows for distinct databases for source, staging, raw, and business Vault schemas at the project level. ![Screenshot 2024-07-11 at 09.45.40.png](media://f424c384-5967-4beb-980d-3056025b2c07) #### Parameters at Source Level At the source level, you can further refine the database configurations for specific data sources. This ensures that each data source can have its own database settings, providing detailed control over data organization and security. - **Configuration**: Define database names for individual data sources. - **Control**: Enables precise management of schemas and databases for each data source, enhancing security and organization. ![Screenshot 2024-07-25 at 14.39.22.png](media://00ff743e-b096-4981-a8ca-b4fd69773543) #### Parameters at Data Vaults Level At the Data Vault level, you can ensure that your Raw and Business Data Vaults loading does not interfere by storing them in separate databases. - **Configuration**: Define database names for the FL = Raw Data Vault and BV = Business Data Vault. - **Control**: Enables precise management of schemas and databases for each storage layer in your Data Vault. ![Screenshot 2024-07-11 at 09.50.52.png](media://e7ba2075-7289-440d-ac5d-8c16d574aab7) By implementing these configurations, VaultSpeed provides a robust framework for managing databases and schemas, ensuring flexibility, organization, and optimized resource management. ### Deterministic Column Ordering for PIT Tables We now ensure consistent sequencing of columns within signature attribute groups specifically for Point-In-Time (PIT) tables. This enhancement maintains the order of columns consistently across all deployments. **Key Benefits** - **Consistency**: Ensures uniform column sequencing within signature attribute groups across all deployments. - **Predictability**: Provides a predictable data structure, enhancing overall data management. - **Ease of Management**: Simplifies maintenance tasks and improves comprehension of data models. ### Action Log Events We have enhanced our action log events to ensure privacy and traceability. On production systems, action log events now display user IDs alongside usernames, ensuring that Personal Identifiable Information (PII) such as usernames or email addresses is still visible to authorized users. However, when customer data is copied to pre-production environments for investigation purposes (e.g., when a customer logs a ticket), support engineers will only see user IDs and not PII data. This change enhances privacy for customer data during support investigations while maintaining effective traceability of user actions. **Key Benefits** - **Enhanced Privacy**: In pre-production environments, action log events display only user IDs, ensuring that Personal Identifiable Information (PII) such as usernames or email addresses remains private. - **Improved Traceability**: Despite privacy enhancements in pre-production, effective traceability of user actions is maintained, providing necessary oversight and accountability. > ℹ️ **Note** > ℹ️ > ℹ️ - In production systems, both user IDs and usernames are displayed to authorized users. > ℹ️ - In pre-production environments, only user IDs are displayed to support engineers to protect PII. ### Agent Authentication Token Refresh Security administrators can now initiate an authentication credentials refresh for the VaultSpeed agent. This new feature enhances security by allowing periodic updates to authentication credentials. **Key Benefits:** - **Enhanced Security:** Enables periodic updates to authentication credentials, reducing the risk associated with static tokens. - **Improved Control:** Security administrators have greater control over authentication processes, ensuring credentials are up-to-date and secure. ![Agent_token_refresh.png](media://70881150-4e60-48d3-b22c-13ab1055c9f2) ### Logon Event Tracking Logon events have been added to the action log, providing improved security monitoring capabilities. Security administrators can now view user login activities with associated timestamps. Additionally, the log now includes user password update events, further enhancing the ability to monitor and secure user authentication activities. **Key Benefits:** - **Enhanced Security Monitoring**: Provides visibility into user logon activities and password updates, enhancing security monitoring capabilities. - **Timestamp Visibility**: Security administrators can track user logins and password updates with precise timestamps, enabling timely response to security incidents. ![Screenshot 2024-07-02 105357.png](media://7f2ab6e8-8bca-4a30-8496-07550a346795) ### Hash Calculation Enhancement Support for the SHA-256 hashing algorithm has been added for Snowflake and Google BigQuery technologies, offering an additional option for hash calculations. **Key Benefits** - **Reduced Collision Rates**: Provides a stronger hashing algorithm (SHA-256) to minimize the risk of hash collisions, ensuring more reliable data integrity. - **Compatibility**: Supports Snowflake, Google BigQuery, Azure Synapse, and SQL Server platforms, ensuring broad applicability across different technologies. --- ## Quality of Life Changes ### Cloudera Parallel Loading - Resolved a technology limitation in Cloudera that previously prevented table updates. - Enables parallel loading of FMC tables, significantly improving performance. ### Databricks Partitioning - Improved filter condition for multi-master hub and link tables in Databricks. - Enables partition pruning and supports parallel loading, enhancing data processing efficiency. ### Airflow on Cloudera Enhancement - Adopted` Livy `batch API endpoint for Airflow on Cloudera. - Enhances support for higher parallel sessions, improving workflow management. ### User Access Management We have implemented a regular process to clean up obsolete user accounts in Jira. This ensures compliance with GDPR rules and enhances the security of our support systems. If users regain access in the future, new accounts will be created as needed. **Key Benefits:** 1. **Enhanced Security Compliance:** Implementation of regular cleanup processes for obsolete user accounts ensures adherence to GDPR regulations, enhancing overall data security and compliance. 2. **Improved System Security:** By removing obsolete accounts promptly, potential security vulnerabilities are mitigated, thereby safeguarding the integrity and confidentiality of data within our support systems. 3. **Efficient Account Management:** Automatic creation of new accounts for users who regain access streamlines the user management process, ensuring continuity of access while maintaining security protocols. --- ## Bug Fixes ### PIT Table Improvements - Addressed the issue of PITs on multi-active satellites without source subsequence not containing the subsequence attribute. - After the fix, the PIT table correctly reflects all columns required to join with satellite tables. ### Hub Business Key Naming - Adjusted 30-character limitation for hub business key naming (applies to Oracle 12.1 or lower). - Improved use of object abbreviated names for hub business keys. ### CDC Systems Full Load Corrected `load_date` field for deleted records to include a timestamp, enhancing data change tracking. ### Satellite Foreign Key Storage **Resolved Issue with STORE_BK_FIELDS_IN_SAT Parameter** **Overview:** An issue was identified where the storage of the foreign key in a **Satellite-only** Signature Object incorrectly depended on the `STORE_BK_FIELDS_IN_SAT` parameter. This dependency should not have existed, and it has now been corrected. **Impact:** - **Scope:** This change applies to objects with the signature "Satellite." > 📝 `STORE_BK_FIELDS_IN_SAT` > 📝 > 📝 **Parameter Limitation:** The `STORE_BK_FIELDS_IN_SAT` parameter is not supported for Satellite-only Signature Attributes. > 📝 > 📝 The Business Key can differ from the Primary Key in the Hub & Satellite Signature Object to which the Satellite-only Signature Object's Foreign Key refers. In such cases, the Business Key will not be copied to the Satellite-only Signature Object, even if `STORE_BK_FIELDS_IN_SAT` is set to `Y`. **Summary:** This update resolves the incorrect dependency on the `STORE_BK_FIELDS_IN_SAT` parameter for foreign key storage in Satellite-only Signature Objects. The limitation regarding the `STORE_BK_FIELDS_IN_SAT` parameter remains, ensuring that Business Keys are not copied in scenarios where they differ from Primary Keys. ### FMC Function Update Issue in Azure Data Factory **Issue Resolution**: - The FMC function issue in Azure Data Factory has been resolved. - FMC function updates are now processed correctly. - Pipeline logic now calls the FMC function with the correct parameters, irrespective of case sensitivity settings.