--- title: "Parameter Descriptions" canonical: "https://docs.vaultspeed.com/space/VPD/3012755470/Parameter%20Descriptions" format: markdown --- This is an overview of all the parameters that are available in VaultSpeed.[https://vaultspeed.atlassian.net/browse/VD-282](https://vaultspeed.atlassian.net/browse/VD-282) Each parameter has its own base setting level. For most parameters this is the System level. Only a few are set on lower levels. Besides that, parameters can be customized and set on the lower levels. More information on how these levels operate can be found here: [https://vaultspeed.atlassian.net/wiki/spaces/VPD/pages/3013574682](https://vaultspeed.atlassian.net/wiki/spaces/VPD/pages/3013574682) > Macro (toc) # Parameters Defined at SYSTEM Level ## Important parameters ### OBJECT_NAMES_CASE_SENSITIVE **Possible Values:** UPPER / LOWER | **Lowest Level:** [🟢 SYSTEM] **Default Value:** LOWER **Full Description** The OBJECT_NAMES_CASE_SENSITIVE parameter controls the case sensitivity of object names, such as tables, views, and indexes, within VaultSpeed. This setting determines whether the application treats object and attribute names as case sensitive or not. ##### Casing Behavior by Setting: - **LOWER**: Recommended for Databricks, Google BigQuery, PostgreSQL, and Greenplum. - **UPPER**: Recommended for Oracle and Snowflake. - **EXACT**: Object names in the Raw Data Vault will match the exact casing defined in the source or application. This parameter is crucial for ensuring consistency and compatibility with your target database's case handling. It is important to decide on the case sensitivity setting early in the development process, as changes after the initial setup can require significant alterations to existing configurations. > ❌ This parameter is locked and cannot be changed once a source system has been added, unless all sources are removed. --- ### USE_SOURCE_UK_AS_BK **Possible Values:** Y / N | **Lowest Level:** [⚪ TAB SRC] **Default Value:** N **Full Description** The USE_SOURCE_UK_AS_BK parameter determines if VaultSpeed should use the unique keys from the source system as business keys in the data warehouse. This parameter influences how Business Key attributes are derived from the source data. **When set to Y (Yes):** - Unique Keys from the source object will be harvested as Business Keys within the data vault. - Enables the use of source unique keys as business keys, which may simplify the mapping process if the source system heavily relies on unique keys. **When set to N (No):** - Primary Keys are used by default for deriving Business Keys. - Defaults to using primary keys for business key assignments, which might be more suitable if unique keys are less prevalent or if there's a preference for stability in primary key usage. > ℹ️ This parameter should only be set at the table level unless all objects have unique keys. Consider setting this parameter if the source system analysis reveals extensive use of unique key constraints. --- ### INTRA_LOADCYCLE_CHANGES **Possible Values:** Y / N | **Lowest Level:** [⚪ TAB SRC] **Default Value:** Y **Requirements:** - When FMC_OVERLAPPING_LOADING_WINDOWS is **Y**, this parameter must be **Y** - When REFERENTIAL_INTEGRITY_VALIDATED is **Y**, this parameter must be **Y** **Full Description** The INTRA_LOADCYCLE_CHANGES parameter determines how changes are recorded during a loading cycle in VaultSpeed. This setting is critical for managing how data is handled, especially regarding timestamps for records in a load cycle. Understanding and setting this parameter correctly is crucial for ensuring your data warehouse reflects the desired level of change detail during each load cycle. **Practical Implementation:** **set to Y (Yes):** When detailed change tracking is required within a single load cycle, providing more granular insights into data modifications. **set to N (No)**: For scenarios where only the final state of data at the end of the cycle is relevant, simplifying the timestamp management. **When set to Y (Yes):** - All changes during a loading cycle are recorded. - Each record receives a unique load date, allowing for detailed tracking of changes. **When set to N (No):** - Only the last change during a loading cycle will be recorded. - A single timestamp is applied to the entire load cycle. > ❌ This parameter should not be set to 'N' for sources with Change Data Capture (CDC) type: 'Incremental Load using Modification Flag & Date'. --- ### PERSIST_DELTA_FIREWALL_VIEWS **Possible Values:** Y / N | **Lowest Level:** [🟣 SOURCE] **Default Value:** N **Full Description** The PERSIST_DELTA_FIREWALL_VIEWS parameter determines whether a table should be created for the delta firewall logic instead of a view. This parameter is crucial for managing how changes in data are tracked and processed within the data vault architecture. This parameter is particularly relevant for VaultSpeed developers and data engineers looking to optimize the handling of delta changes in their data pipelines and ensure efficient data processing and storage management. **Purpose:** - Controls the creation of physical tables for delta firewall logic, which can impact performance and storage considerations. **Default Behavior:** - By default, delta firewall logic is typically implemented using views. Setting this parameter to true will result in the creation of tables instead. **Practical Implications:** - **When to Use**: Opt for tables if your data processing architecture requires faster read performance and you have sufficient storage resources. - **Performance Impact**: Tables may offer better performance for complex queries compared to views, especially in large-scale data environments. > ⚠️ Using tables for delta firewall logic can improve query performance at the cost of increased storage usage. --- ### COLUMN_LENGTH_MULTIPLIER **Possible Values:** Free Text | **Lowest Level:** [🟣 SOURCE] **Default Value:** 2 **Full Description** The COLUMN_LENGTH_MULTIPLIER parameter is crucial for defining the length of Raw Data Vault string attributes. By multiplying the source system string attribute length by this factor, it determines the resulting length in the Raw Data Vault. This approach minimizes the impact of changes in source system attribute lengths and acts as a default in Data Vault data type mappings. **Default Use:** - The parameter serves as a default multiplier in DV data type mappings but can be adjusted if specific customizations are needed. **Practical Implications:** - Adjusting this multiplier can help optimize storage and performance by ensuring that string attributes are sized appropriately for the expected data volume. > ℹ️ This parameter allows for flexibility and adaptability in handling varying source data lengths, ensuring that the target database can accommodate future changes without manual intervention. > ⚠️ While the COLUMN_LENGTH_MULTIPLIER provides a default setting, developers have the ability to override it when specific requirements demand a different configuration. --- ### TIME_ZONE_LOCAL **Possible Values:** Y / N | **Lowest Level:** [🔵 PROJECT] **Default Value:** Y **Dependencies:** - When TIME_ZONE_LOCAL is set to **Y**: - CDC_TIMESTAMP_WITH_TZ must be **N** - LOAD_TIMESTAMP_WITH_TZ must be **N** **Availability:** - When TIME_ZONE_LOCAL is set to **Y**: - LOAD_TIMESTAMP_WITH_TZ is not available - CDC_TIMESTAMP_WITH_TZ is not available **Full Description** The TIME_ZONE_LOCAL parameter determines if a project uses a consistent time zone for all timestamps throughout its data warehouse automation processes. **When set to Y (Yes):** - The project uses a consistent, uniform time zone for all timestamps throughout the data warehouse. - All timestamp values are stored and processed in a single time zone, simplifying time-based operations. - The Load Date and Load End Date attributes use standard timestamp data types without time zone information. - Ensures consistency across all temporal data within the data vault. **When set to N (No):** - The project will not use a uniform time zone for all timestamps. - It enables the use of a timestamp with time zone data type for the Load Date and Load End Date attributes. - Activates the parameters **CDC_TIMESTAMP_WITH_TZ** and **LOAD_TIMESTAMP_WITH_TZ**, allowing for more flexible time zone handling within your data vault patterns. > ℹ️ If your data sources and target databases operate across multiple time zones, consider setting TIME_ZONE_LOCAL to 'N' to ensure accurate timestamp representation across the system. --- ### ORACLE_12_1_OR_HIGHER **Possible Values:** Y / N | **Lowest Level:** [🟢 SYSTEM] **Default Value:** Y **Dependencies:** - When ORACLE_12_1_OR_HIGHER is set to **N**: - ORACLE_12_2_OR_HIGHER must be **N** **Requirements:** - When ORACLE_12_2_OR_HIGHER is **Y**, this parameter must be **Y** **Availability:** - When ORACLE_12_1_OR_HIGHER is set to **N**: - ORACLE_12_2_OR_HIGHER is not available **Full Description** The ORACLE_12_1_OR_HIGHER parameter specifies whether Oracle Database 12c Release 1 or a higher version is being used for the Data Vault. This setting is crucial as it determines the hashing algorithm employed by VaultSpeed. This parameter is significant for ensuring the correct hashing algorithm is utilized, which impacts data integrity and processing efficiency in your data warehouse environment. **When set to Y (Yes):** - This indicates that Oracle Database 12c Release 1 or higher is being used. - It is essential to set this to Y when using other database types as well. **When set to N (No):** - This suggests the use of an older Oracle version, which necessitates a different hashing algorithm. > ℹ️ It is recommended to leave this parameter set to Y for compatibility with Oracle 12c Release 1 and higher, as well as other database systems. --- ### EARLY_ARRIVING_FACTS **Possible Values:** Y / N | **Lowest Level:** [🟣 SOURCE] **Default Value:** N **Full Description** The EARLY_ARRIVING_FACTS parameter determines how VaultSpeed handles the loading of Hub Business Keys from data sources. This parameter is crucial for ensuring that all necessary business keys are available for data processing when there are dependencies between Parent and Child Objects. The EARLY_ARRIVING_FACTS parameter changes how business keys are loaded in the generated code to handle referential integrity issues. Important constraint: This logic only applies when the primary key of the referenced object is also the business key. For other scenarios, you'd use REFERENTIAL_INTEGRITY_VALIDATED instead. The parameter essentially makes your generated ETL more robust by ensuring all related keys are captured even if they don't arrive simultaneously. Business Keys are loaded only from the Parent Object. This setting is appropriate when the business keys are guaranteed to be present in the Parent Object, or when loading from Child Objects is not required. **When set to Y (Yes):** - Generated code loads Hub Business Keys from both the Parent Object AND all related Child Objects - preventing missing key issues when keys arrive in different load cycles Business Keys are loaded from both the Parent Object and all related Child Objects. This setting helps prevent issues that arise when not all keys are present in the Parent Object or are loaded at a later time. It is particularly useful in scenarios where the primary key of the referenced object is also the business key. **When set to N (No):** - Generated code only loads Business Keys from the Parent Object > ❌ Ensure that the referenced object's primary key is also the business key when using EARLY_ARRIVING_FACTS. If not, consider using the REFERENTIAL_INTEGRITY_VALIDATED parameter instead. --- ### STORE_BK_FIELDS_IN_SAT **Possible Values:** Y / N | **Lowest Level:** [⚪ TAB SRC] **Default Value:** N **Full Description** The STORE_BK_FIELDS_IN_SAT parameter determines whether all business keys should be stored as descriptive fields within a Satellite (SAT) in VaultSpeed. This parameter is crucial when dealing with source systems that lack explicit keys, causing all fields in an object to serve as keys. Without this parameter set, such Satellites would remain empty. When configuring data pipelines within VaultSpeed, consider setting this parameter to 'Y' if your source system does not provide explicit keys. This approach ensures all necessary business keys appear in the Satellite, maintaining data integrity and completeness in your data vault architecture. **When set to Y (Yes):** - Business keys are stored within the Satellite, ensuring that the Satellite is populated even when source keys are absent. **When set to N (No):** - Business keys are NOT stored inside the Satellites, which may result in empty Satellites if no keys are available in the source. > ⚠️ This parameter can be overridden in specific situations, such as when using a concatenated business key. In these cases, business keys will always be stored in the Satellite regardless of this parameter's setting. --- ### INSERT_ONLY_LOGIC **Possible Values:** Y / N | **Lowest Level:** [🟣 SOURCE] **Default Value:** N **Requirements:** - When DATABASE_TYPE is **APACHE HIVE**, this parameter must be **N** - When STREAMING_SOURCE is **Y**, this parameter must be **Y** - When DATABASE_TYPE is **SINGLESTORE**, this parameter must be **Y** **Dependency:** - INSERT_ONLY_LOGIC is not available - When DATABASE_TYPE is set to **APACHE HIVE** **Full Description** The INSERT_ONLY_LOGIC parameter defines how VaultSpeed generates loading logic for Satellites, in alignment with Data Vault 2.0 principles. It is essential for managing how historical changes are captured and how data is inserted into Satellite tables. When enabled, this parameter modifies the standard Satellite implementation to follow an insert-only pattern. As a result, current record identification and history tracking are handled without updates, ensuring that all changes are preserved as new entries. **When set to Y (Yes):** - Generated code excludes load end dates from Satellites and uses maximum aggregation on load date to determine current/active records - Implements DV2.0 insert-only logic. - Load end dates are excluded from Satellites. - The current/active record is determined by maximum aggregation on the load date. Use this when: - You want simpler ETL logic and fewer updates. - You are comfortable determining the current record by “latest load date” instead of comparing end dates. **When set to N (No):** - Generated code includes load end dates in Satellites and compares them to active record timestamps Use this when: - You need explicit end dates on records for auditing, reporting, or detailed temporal analysis. - Your existing processes or tools rely on start/end date semantics. **Generated SQL behavior:** The parameter triggers conditional code generation where inline views with MAX aggregations are created for insert-only objects, while traditional end-dating logic is used for non-insert-only objects. This creates more efficient queries for insert-only scenarios by eliminating the need for complex end-dating calculations. - Load end dates are included in Satellites. - The current/active record is determined by comparing load end dates to the active/current record timestamp. > ℹ️ Using the INSERT_ONLY_LOGIC parameter is essential for maintaining the integrity of historical data while ensuring that the most current data is easily accessible. This setting is particularly important in scenarios where maintaining a historical record of changes is critical. --- ### LOAD_TIMESTAMP_WITH_TZ **Possible Values:** Y / N | **Lowest Level:** [⚪ DV] **Default Value:** N **Requirements:** - When TIME_ZONE_LOCAL is **Y**, this parameter must be **N** **Dependency:** - LOAD_TIMESTAMP_WITH_TZ is not available - When TIME_ZONE_LOCAL is set to **Y** **Full Description** The LOAD_TIMESTAMP_WITH_TZ parameter determines if timestamps with time zones are employed for load timestamps within VaultSpeed. This setting is crucial for ensuring that time zone information is appropriately included in your timestamps, which can be pivotal for data accuracy across different geographical locations. Utilizing timestamps with time zones can be particularly beneficial when your data warehouse needs to handle records from multiple time zones, ensuring that all time-related data is standardized and accurate. This setting should be configured based on the specific requirements of your data integration and reporting needs. **When set to Y (Yes):** - Timestamp WITH Time Zones will be used for the LOAD_TIMESTAMP. **When set to N (No):** - Timestamps WITHOUT Time Zones will be utilized for the LOAD_TIMESTAMP. > ⚠️ This parameter requires the TIME_ZONE_LOCAL setting to be set to N to function correctly. --- ### CDC_TIMESTAMP_WITH_TZ **Possible Values:** Y / N | **Lowest Level:** [⚪ DV] **Default Value:** N **Requirements:** - When TIME_ZONE_LOCAL is **Y**, this parameter must be **N** **Dependency:** - CDC_TIMESTAMP_WITH_TZ is not available - When TIME_ZONE_LOCAL is set to **Y** **Full Description** The CDC_TIMESTAMP_WITH_TZ parameter determines the format of timestamps used for Change Data Capture (CDC) in VaultSpeed. This parameter is crucial for ensuring the correct handling of time zones in your data warehouse automation processes. **When set to Y (Yes):** - Timestamps WITH Time Zones are enabled for CDC timestamps. - This setting allows for precise time zone management across different geographical locations, ensuring that the data reflects the exact time of data capture regardless of location. **When set to N (No):** - Timestamps WITHOUT Time Zones are used. - This setting is recommended if you do not need to manage different time zones or if your systems operate within a single time zone. > ❌ This setting requires TIME_ZONE_LOCAL to be set to 'N'. Ensure that this prerequisite is met before enabling Timestamps WITH Time Zones. --- ### CHECK_PARAMETER_NAMES **Possible Values:** Y / N | **Lowest Level:** [🟢 SYSTEM] **Default Value:** Y **Full Description** The CHECK_PARAMETER_NAMES parameter determines if VaultSpeed should verify the uniqueness of parameter names to prevent naming conflicts. **Purpose:** - Ensures that object and attribute names do not conflict, which helps prevent downstream errors in code deployments. **Practical Implications:** - Use this setting to N to enhance performance during initial setup or when confident about the uniqueness of names. - Recommended to set to Y so the system checks for naming conflicts in attribute and parameter names to ensure stability and data integrity. **When set to Y (Yes):** - Parameters are checked for naming conflicts to ensure uniqueness. - Validates that object and attribute names do not conflict across the data vault. - Helps prevent downstream errors in data processing by catching naming issues early. - May slightly increase the loading time of parameter screens due to validation overhead. **When set to N (No):** - Parameters are not checked for naming conflicts. - May improve the loading time of parameter screens. - Increases the risk of encountering errors later if names are not unique. > ❌ Setting CHECK_PARAMETER_NAMES to N can improve performance but may lead to errors due to non-unique names being used in different objects or attributes. --- ### LENGTH_ABBREVIATED_NAMES **Possible Values:** Free Text | **Lowest Level:** [🟢 SYSTEM] **Default Value:** 128 **Full Description** The LENGTH_ABBREVIATED_NAMES parameter defines the maximum length for abbreviated names of source objects and attributes within VaultSpeed. This parameter is crucial when generating key names for various data vault components such as hubs, satellites, and hash keys. It ensures that names remain concise and within database constraints. **General Use:** - This parameter helps in maintaining uniformity and preventing issues related to exceeding name length limits in databases. **Oracle Databases:** - Specifically important for Oracle databases version 12.1 or lower, where the name length is restricted to 14 characters due to database limitations. - Consider adjusting LENGTH_ABBREVIATED_NAMES when working with environments that have specific naming constraints or when a higher version of Oracle is being used. - Ensure compatibility with existing database naming conventions to avoid potential conflicts during the automation process. > ℹ️ When using databases where ORACLE_12_2_OR_HIGHER is set to 'N', the maximum length for abbreviated names is automatically restricted to 14 characters. --- ### LENGTH_SHORT_NAMES **Possible Values:** Free Text | **Lowest Level:** [🟢 SYSTEM] **Default Value:** 128 **Full Description** The LENGTH_SHORT_NAMES parameter specifies the maximum length for the short names of source objects and attributes within VaultSpeed. These short names play a critical role in forming the names of link tables, which are integral components in data vault modeling. **Purpose:** - Set the maximum length for short names used in link table names **Compatibility Note:** - Restrictions apply when using Oracle versions below 12.2 - Adjust this parameter to accommodate naming conventions and database requirements specific to your project's needs, especially when dealing with Oracle databases. Ensuring correct parameter values can prevent issues related to database object name lengths and maintain consistency across your data warehouse models. > ℹ️ When the ORACLE_12_2_OR_HIGHER setting is set to 'N', the LENGTH_SHORT_NAMES parameter is restricted to a maximum value of 4. This ensures compatibility with Oracle databases versions prior to 12.2. --- ### POSTGRES_9_OR_HIGHER **Possible Values:** Y / N | **Lowest Level:** [🟢 SYSTEM] **Default Value:** Y **Full Description** The POSTGRES_9_OR_HIGHER parameter indicates whether the target PostgreSQL database is version 9 or above. This setting is essential to ensure compatibility with features and performance optimizations introduced in PostgreSQL 9 and later versions. Enabling it allows VaultSpeed to generate ELT logic that takes advantage of newer database capabilities, ensuring efficient execution and alignment with the expected behavior of modern PostgreSQL environments. **When set to Y (Yes):** - PostgreSQL version 9 or higher is used. This enables VaultSpeed to utilize features and improvements available in these versions. **When set to N (No):** - An older version of PostgreSQL is used. This might limit certain functionalities within VaultSpeed that rely on newer features. > ℹ️ It is recommended to set this parameter to Y if your system supports PostgreSQL 9 or higher to take advantage of performance enhancements and feature support. --- ### ORACLE_12_2_OR_HIGHER **Possible Values:** Y / N | **Lowest Level:** [🟢 SYSTEM] **Default Value:** Y **Dependencies:** - When ORACLE_12_2_OR_HIGHER is set to **Y**: - ORACLE_12_1_OR_HIGHER must be **Y** **Requirements:** - When ORACLE_12_1_OR_HIGHER is **N**, this parameter must be **N** **Dependency:** - ORACLE_12_2_OR_HIGHER is not available - When ORACLE_12_1_OR_HIGHER is set to **N** **Full Description** The ORACLE_12_2_OR_HIGHER parameter specifies whether Oracle Database 12c Release 2 or higher is being used for the Data Vault implementation. This setting is crucial for determining how VaultSpeed handles object and attribute name lengths. In summary, this parameter should be set to Y for Oracle 12.2 or higher, or when working with non-Oracle databases, to ensure optimal naming flexibility. Set to N only when using Oracle versions prior to 12.2, to adhere to naming restrictions. **When set to Y (Yes):** - Indicates that Oracle Database 12 Release 2 or higher is in use. - Object and attribute names are not restricted by length limitations. - Recommended setting when using other database types. **When set to N (No):** - Indicates that an older version of Oracle is being used. - Enforces length limitations on object and attribute names, which is necessary for compatibility with older Oracle versions. > ❌ Ensure this parameter is set accurately to avoid potential naming conflicts or limitations when generating code and managing database objects in VaultSpeed. --- ### STORE_TEMP_IN_STAGING **Possible Values:** Y / N | **Lowest Level:** [⚪ DV] **Default Value:** N **Requirements:** - When DATABASE_TYPE is **SINGLESTORE**, this parameter must be **N** **Full Description** The STORE_TEMP_IN_STAGING parameter determines where temporary storage tables are placed during the loading process of the raw vault in VaultSpeed. This setting is critical for managing the location of temporary tables and can impact both performance and organization of your data processing tasks. **When set to Y (Yes):** - Temporary tables will be stored in the Staging layer schema, aligning with initial data processing tasks and potentially improving the performance of initial data loads. **When set to N (No):** - Temporary tables will be stored in the Data Vault Layer schema, which can simplify the structure for environments where data vault processing is a priority. > ℹ️ The choice between storing temporary tables in the Staging or Data Vault layer should be made based on your specific data architecture needs and performance considerations. --- ### REFERENTIAL_INTEGRITY_VALIDATED **Possible Values:** Y / N | **Lowest Level:** [⚪ TAB SRC] **Default Value:** N **Dependencies:** - When REFERENTIAL_INTEGRITY_VALIDATED is set to **Y**: - INTRA_LOADCYCLE_CHANGES must be **Y** **Requirements:** - When STREAMING_SOURCE is **Y**, this parameter must be **Y** **Full Description** The REFERENTIAL_INTEGRITY_VALIDATED parameter is designed to enforce and validate the referential integrity of Link records within the Data Vault model. In Data Vault 2.0, Links represent relationships between business entities, and their integrity depends on the existence and correctness of the referenced Hub keys. This parameter ensures that every foreign key in a Link table correctly matches an existing primary key in the associated Hub or Link tables. When enabled, VaultSpeed generates additional error-handling and validation logic as part of the ELT templates. This includes checks for cases where foreign key references rely on surrogate primary keys (typically hash keys) that may not directly correspond to the business keys in the source data. This situation commonly arises in multi-source or multi-primary environments, where the business key used in the source does not always map 1:1 to a primary key in the Data Vault model. By activating this parameter, VaultSpeed ensures that missing or inconsistent relationships are flagged during the load process. This protects the integrity of the model by preventing invalid relationship entries from being inserted and provides a clear audit trail for identifying data quality issues upstream. It is particularly useful in complex integration scenarios involving loosely governed sources or delayed hub loads, where referential mismatches are more likely to occur. **When set to Y (Yes):** it performs the following actions: - Validates the referential integrity of Link records to ensure that all references are valid and correctly linked to their respective primary keys. - If violations are detected, the affected Link records are logged into an error table for further analysis and troubleshooting. - Link records will only be loaded once the reference is validated, ensuring that only accurate and consistent data is integrated unless the REFERENTIAL_INTEGRITY_FORCE_LINK_LOAD parameter is enabled, which allows loading even if validation fails. - Refer to the 'Referential integrity validation' section for detailed information on how this logic is implemented and any additional configurations that might be necessary. **Key code impacts:** - Use this parameter during the initial data loading processes and in ongoing data integration scenarios to ensure data quality. - It is especially useful in environments where maintaining accurate and reliable data links is critical for business operations and analytics. > ℹ️ This parameter is particularly important when dealing with complex data models where the primary key may not directly act as the business key. In such cases, careful validation is essential to prevent data integrity issues. --- ### REFERENTIAL_INTEGRITY_FORCE_LINK_LOAD **Possible Values:** Y / N | **Lowest Level:** [⚪ TAB SRC] **Default Value:** N **Full Description** The REFERENTIAL_INTEGRITY_FORCE_LINK_LOAD parameter allows VaultSpeed to load Link records even when referential integrity violations occur. This is useful in scenarios where some records may not have a corresponding parent record at the time of loading. Here’s how it functions: - **Forces Link Record Loading**: It ensures that Link records are loaded despite referential integrity errors, which might typically prevent this action. - **Handling Missing Parent Records**: Records that lack a parent record will temporarily receive an unknown value. This placeholder ensures the data pipeline can proceed without interruption. - **Subsequent Updates**: Once the reference is resolved and the parent record becomes available, the Link record is updated appropriately with the correct value. Refer to the 'Referential Integrity Validation' documentation for more detailed guidance on managing referential integrity within VaultSpeed. The REFERENTIAL_INTEGRITY_FORCE_LINK_LOAD parameter controls whether Link (LNK) and Link Satellite (LKS) records are loaded when referential integrity errors exist. **When to Use**: - Use this parameter when it's critical to maintain pipeline flow despite missing references. - Ideal for environments where data arrival may be asynchronous or when dealing with legacy systems where referential integrity cannot be immediately enforced. **Key code impacts:** - **Lookup logic modification** - When foreign key lookups fail to find matching records, the generated code includes logic to store unknown replacement values instead of skipping the load - **Error handling branching** - The ETL adds conditional logic that checks the ERROR_CODE values and processes records with unknown references differently - **Initial load behavior** - LNK/LKS and LND/LDS procedures generate code that loads records with unknown linked records rather than filtering them out **What stays unchanged:** - HUB/SAT loading procedures remain the same - they always load regardless of ERROR_CODE values - The error table structure and ERROR_CODE field generation logic doesn't change The parameter essentially adds a code path that allows link records to proceed through the ETL pipeline with placeholder values when referenced hub records are missing, rather than being blocked or skipped. > ❌ Ensure that you understand the implications of using this parameter in your data pipeline, as it may lead to temporary data inconsistencies. --- ### ADD_RECORD_SOURCE_ATTRIBUTE **Possible Values:** Y / N | **Lowest Level:** [⚪ DV] **Default Value:** N **Requirements:** - When ETL_GENERATION_TYPE is **STREAM2VAULT**, this parameter must be **Y** **Full Description** The ADD_RECORD_SOURCE_ATTRIBUTE parameter enables the inclusion of an additional record source attribute in Hubs, Links, and Reference objects. This attribute captures the origin of each record, enhancing traceability without impacting the generation of hash keys. By keeping the hash key logic isolated, the integrity and uniqueness of business keys remain intact, while the source system or process that delivered the data is clearly documented. This added transparency is vital for understanding data lineage and supporting impact analysis during troubleshooting or change management. Activating this parameter is strongly recommended in environments where auditability and data governance are key requirements. Typical use cases include: - Regulatory compliance (e.g. finance, healthcare, government) - Multi-source or multi-primary system integrations - Complex enterprise data platforms with overlapping ownership By incorporating source tracking directly into the model, teams can gain immediate insight into where data originated, making it easier to validate, debug, and certify data throughout the pipeline. **When set to Y (Yes):** - the attribute will be appended to the defined objects. **When set to N (No):** - no additional record source attribute will be added. **Key ETL code changes:** - **Column addition** - All table creation scripts (HUB, SAT, LNK, etc.) include an additional record source attribute column - **ETL mapping logic** - Loading procedures are modified to populate this new column with source system identification - **Hash key exclusion** - The record source attribute is specifically excluded from hash key calculations, so hash generation logic remains unchanged > ℹ️ Consider enabling this parameter if data lineage and provenance are critical for your data governance strategy. --- ### CAST_BINARY_TO_HEX **Possible Values:** Y / N | **Lowest Level:** [⚪ DV] **Default Value:** Y **Full Description** The CAST_BINARY_TO_HEX parameter determines whether binary values should be converted to a string using the hexadecimal approach. This ensures consistent results across different technologies. **Purpose:** - To provide a uniform method for converting binary data into a string representation, leveraging hexadecimal notation for consistency. **Recommended Use:** - Enable this parameter when you need to ensure that binary data is uniformly represented as a hexadecimal string, particularly in environments with multiple database systems or varied technology stacks. > ℹ️ Using the hexadecimal approach can prevent discrepancies when dealing with binary data across various database technologies, which may handle binary-to-string conversions differently. --- ### STREAMING_SOURCE **Possible Values:** Y / N | **Lowest Level:** [🟣 SOURCE] **Default Value:** N **Dependencies:** - When STREAMING_SOURCE is set to **Y**: - REMOTE_JOURNALING_TABLES must be **N** - CDC_UPDATE_RECORD_ALL_ATTRIBUTES must be **Y** - ERROR_RECORD_RETENTION_TIME must be **is not null** - INSERT_ONLY_LOGIC must be **Y** - REFERENTIAL_INTEGRITY_VALIDATED must be **Y** - WATER_MARK_SIZE must be **is not null** **Availability:** - When STREAMING_SOURCE is set to **N**: - ERROR_RECORD_RETENTION_TIME is not available - WATER_MARK_SIZE is not available - GENERAL_TIME_FORMAT_STREAM is not available - GENERAL_DATE_FORMAT_STREAM is not available - GENERAL_TIMESTAMP_FORMAT_STREAM is not available **Full Description** The STREAMING_SOURCE parameter activates support for Spark Structured Streaming as a method for transferring data from source systems to the target database, providing a real-time alternative to traditional batch-based ELT. When enabled, VaultSpeed generates ELT logic optimized for continuous data ingestion, allowing near real-time data integration scenarios to be handled efficiently and at scale. This is particularly valuable in modern architectures where low-latency insights and up-to-date reporting are critical. To use this feature, the source data must be delivered via **Kafka**, a distributed streaming platform widely adopted for high-throughput, real-time pipelines. VaultSpeed can consume Kafka topics directly, leveraging native support for streaming input. This setup is ideal for use cases involving: - Event-driven data processing - IoT or sensor data pipelines - Real-time analytics and monitoring platforms By integrating native streaming capabilities into the automation framework, VaultSpeed ensures scalable and maintainable streaming ELT pipelines without manual intervention. **When set to Y (Yes):** - the generated structures and ETL will be alligned on the usage of spark structured streaming in a databricks setup **Incremental Data with CDC:** - Data from other sources, such as files or databases, can be incrementally captured and streamed into Kafka using Change Data Capture (CDC) systems, allowing for integration with VaultSpeed's streaming capabilities. - Opt for STREAMING_SOURCE when your architecture supports Kafka and there is a need for real-time data processing. - Ensure that your data infrastructure is set up to handle the increased complexity and resource demands of streaming data, compared to batch processing. > ℹ️ Using the STREAMING_SOURCE parameter is ideal in scenarios where real-time data processing and immediate data availability in the target system are crucial. This setting optimizes data ingestion and processing workflows by leveraging Kafka's capabilities to manage and deliver data streams efficiently. --- ### WATER_MARK_SIZE **Possible Values:** Free Text | **Lowest Level:** [⚪ TAB SRC] **Default Value:** 120 **Requirements:** - When STREAMING_SOURCE is **Y**, this parameter must be **is not null** **Dependency:** - WATER_MARK_SIZE is not available - When STREAMING_SOURCE is set to **N** **Full Description** The WATER_MARK_SIZE parameter in VaultSpeed is a crucial setting for managing data latency in Spark Structured Streaming. This parameter leverages watermarking to handle late-arriving data, ensuring data consistency and resolving potential referential integrity issues. **Purpose:** - The primary purpose of WATER_MARK_SIZE is to delay the processing of incoming data to accommodate late data arrivals, which is essential in maintaining the integrity of the data. **When to Use:** - This parameter is particularly useful in scenarios where data may arrive out of order or with delays. By configuring the watermark size, developers can control the duration for which the system waits before processing data, thereby reducing the risk of processing incomplete datasets. **Practical Implications:** - Setting an appropriate WATER_MARK_SIZE is vital for optimizing the balance between data processing latency and accuracy. A larger watermark size may result in delayed processing but can accommodate more late data, while a smaller size might increase the risk of incomplete data processing but reduces latency. By properly configuring the WATER_MARK_SIZE parameter, VaultSpeed developers can effectively manage real-time data streams, ensuring that the target databases are populated with accurate and complete data sets. > ℹ️ Watermarking helps in managing late data arrivals by setting a threshold for data timeliness. This ensures that only complete and consistent data sets are processed, which is critical for maintaining data integrity in real-time analytics. --- ### ERROR_RECORD_RETENTION_TIME **Possible Values:** Free Text | **Lowest Level:** [⚪ TAB SRC] **Default Value:** 120 **Requirements:** - When STREAMING_SOURCE is **Y**, this parameter must be **is not null** **Dependency:** - ERROR_RECORD_RETENTION_TIME is not available - When STREAMING_SOURCE is set to **N** **Full Description** The ERROR_RECORD_RETENTION_TIME parameter specifies the duration, in seconds, for which error records generated during Spark structural streaming are retained. This parameter is crucial for managing the persistence of referential integrity error records, allowing developers to address and resolve validation issues during streaming processes. **Purpose:** - To define how long referential integrity error records are stored during data streaming operations. **Context:** - Utilized in Spark structural streaming to maintain error records that help ensure data consistency and integrity. > ℹ️ This parameter is especially important for scenarios where referential integrity validation is critical, providing a buffer window to troubleshoot and rectify errors without data loss. > ⚠️ Adjust the ERROR_RECORD_RETENTION_TIME based on the volume and velocity of your streaming data, as well as the operational requirements for error resolution. --- ## CDC parameters ### CDC_RELIABLE **Possible Values:** Y / N | **Lowest Level:** [⚪ TAB SRC] **Default Value:** N **Requirements:** - When CDC_PRE_IMAGE_AVAILABLE is **Y**, this parameter must be **Y** - When SRC_CDC is **Modification Date**, this parameter must be **N** - When SRC_CDC is **No Change Data Capture**, this parameter must be **N** - When FMC_OVERLAPPING_LOADING_WINDOWS is **Y**, this parameter must be **N** **Availability:** - When CDC_RELIABLE is set to **N**: - USE_CDC_TS_AS_LOAD_DATES is not available **Full Description** The CDC_RELIABLE parameter specifies the availability and reliability of Change Data Capture (CDC) information, including Insert, Delete, and Update flags. This setting determines how VaultSpeed processes changes in data. **When set to Y (Yes):** - VaultSpeed will treat CDC flags as fully reliable, streamlining processing under the assumption that the data is accurate and complete. **When set to N (No):** - The system will apply more computationally intensive logic to manage changes, as it cannot assume reliability of CDC flags. > ❌ Setting CDC_RELIABLE to Y will alter the FL mappings, making them non-idempotent. This means that rerunning the mappings for the same source records may lead to data duplication or corruption. --- ### REMOTE_JOURNALING_TABLES **Possible Values:** Y / N | **Lowest Level:** [🟣 SOURCE] **Default Value:** N **Dependencies:** - When REMOTE_JOURNALING_TABLES is set to **N**: - USE_REMOTE_CDC_DBLINK must be **N** - CREATE_REMOTE_DELTA_VIEW must be **N** **Requirements:** - When STREAMING_SOURCE is **Y**, this parameter must be **N** **Availability:** - When REMOTE_JOURNALING_TABLES is set to **N**: - USE_REMOTE_CDC_DBLINK is not available - CREATE_REMOTE_DELTA_VIEW is not available **Full Description** The REMOTE_JOURNALING_TABLES parameter specifies the location of Change Data Capture (CDC) tables, determining if they reside within the Data Warehouse or on a remote database. This setting is crucial for configuring how VaultSpeed interacts with CDC data during the data warehouse automation process. Setting the REMOTE_JOURNALING_TABLES parameter correctly is essential for VaultSpeed developers to ensure seamless integration and accurate data processing within pipelines and mappings. Choose 'Y' if your architecture involves a separate database for CDC tables, ensuring you have the necessary database type compatibility, or 'N' if the CDC tables are integrated within your Data Warehouse environment. **When set to Y (Yes):** - Indicates that CDC tables are not in the Data Warehouse but are located on a remote database. - The generated Data Definition Language (DDL) will only function correctly if the database type of the source is compatible with the target database types available in VaultSpeed. **When set to N (No):** - CDC tables are located in the Data Warehouse. > ⚠️ Ensure compatibility between source and target database types to avoid execution issues with the generated DDL. --- ### REMOTE_CDC_DBLINK_NAME **Possible Values:** Free Text | **Lowest Level:** [🟣 SOURCE] **Default Value:** DB_LINK_ACCK_OBI_OWNER **Full Description** The REMOTE_CDC_DBLINK_NAME parameter specifies the name of the database link used to connect to the remote Change Data Capture (CDC) schema. This parameter is crucial for ensuring secure and efficient connectivity between your data warehouse and remote CDC sources. - Use consistent naming conventions to ensure clarity and ease of management. - Consider the security implications of naming conventions in your environment. > ℹ️ Note: This parameter only contains the name of the link and not the actual link itself, in order to safeguard sensitive information. --- ### CDC_UPDATE_RECORD_ALL_ATTRIBUTES **Possible Values:** Y / N | **Lowest Level:** [⚪ TAB SRC] **Default Value:** Y **Dependencies:** - When CDC_UPDATE_RECORD_ALL_ATTRIBUTES is set to **N**: - DATA_QUALITY_BAD must be **N** **Requirements:** - When STREAMING_SOURCE is **Y**, this parameter must be **Y** **Availability:** - When CDC_UPDATE_RECORD_ALL_ATTRIBUTES is set to **Y**: - NO_CHANGE_INDICATION_CHAR is not available - NO_CHANGE_INDICATION_NUMBER is not available - NO_CHANGE_INDICATION_DATE is not available - NO_CHANGE_INDICATION_TIME is not available - NO_CHANGE_INDICATION_TIMESTAMP is not available - NO_CHANGE_INDICATION_OTHER is not available **Full Description** The CDC_UPDATE_RECORD_ALL_ATTRIBUTES parameter controls how the Change Data Capture (CDC) system processes updated records by determining whether it delivers all attributes or only those that have changed. **Purpose:** - This setting influences data capture efficiency and storage use, which can significantly impact pipeline performance and data accuracy. **Practical Implications:** - Use this parameter to balance between comprehensive data capture and performance optimization. - Evaluate the trade-offs between increased data volume and the need for complete historical record snapshots in your data vault patterns. **When set to Y (Yes):** - The CDC system will deliver all attributes of an updated record, regardless of whether they have changed. This approach ensures complete record snapshots, which may be useful for auditing or compliance but can increase data volume. **When set to N (No):** - Only the attributes that have changed will be delivered. This reduces data footprint and may improve processing times, beneficial for systems with high update rates and large tables. > ℹ️ Selecting the appropriate setting depends on your system's specific needs for data completeness vs. efficiency. Consider your data architecture and business requirements when configuring this parameter. --- ### CDC_LOGPOSITION_AVAILABLE **Possible Values:** Y / N | **Lowest Level:** [🟣 SOURCE] **Default Value:** N **Availability:** - When CDC_LOGPOSITION_AVAILABLE is set to **N**: - CDC_LOGPOSITION is not available **Full Description** The CDC_LOGPOSITION_AVAILABLE parameter determines if log position data is available for your source database system and influences how Change Data Capture (CDC) records are uniquely identified and ordered. This parameter is crucial for maintaining the integrity and accuracy of CDC processing by ensuring that each change record is distinct and correctly sequenced. **When set to Y (Yes):** - Enables log position tracking from the source database system. - Combines cdc_timestamp + cdc_logposition (based on parameter CDC_LOGPOSITION) to uniquely identify each CDC record. - Creates an attribute (based on parameter CDC_LOGPOSITION) in signature objects such as SAT, LKS, LDS, NHL, LAS, PIT, BVV. - Captures unique incrementing values like SCN, LSN, or offset for proper change ordering. **When set to N (No):** - No log position tracking. - Relies only on cdc_timestamp for record identification. - May have issues with records having identical timestamps. **Key Impact:** Setting CDC_LOGPOSITION_AVAILABLE to 'Y' provides better CDC record uniqueness and chronological ordering, which is particularly important when multiple changes occur at the same timestamp. This setting is essential for reliable CDC processing when your source system provides log position data. The CDC_LOGPOSITION_AVAILABLE parameter impacts how the generated code handles CDC record uniqueness and ordering. > ❌ Setting this parameter to 'N' may lead to difficulties in distinguishing between records with the same timestamp, potentially affecting data consistency. --- ### CDC_PRE_IMAGE_AVAILABLE **Possible Values:** Y / N | **Lowest Level:** [🟣 SOURCE] **Default Value:** N **Dependencies:** - When CDC_PRE_IMAGE_AVAILABLE is set to **Y**: - CDC_RELIABLE must be **Y** **Availability:** - When CDC_PRE_IMAGE_AVAILABLE is set to **N**: - SATELLITE_RECORD_TYPE is not available - TRANSACTION_TYPE_SATELLITE is not available - CDC_PRE_POST_IMAGE_FLAG is not available - CDC_PRE_POST_IMAGE_UNIQUE_KEY is not available - CDC_PRE_IMAGE_FLAG_VALUE is not available - CDC_POST_IMAGE_FLAG_VALUE is not available - SATELLITE_RECORD_TYPE is not available - TRANSACTION_TYPE_SATELLITE is not available **Full Description** The CDC_PRE_IMAGE_AVAILABLE parameter determines whether the Change Data Capture (CDC) system delivers the previous version of a record (pre-image) before a change occurs. This is particularly relevant when the CDC_UPDATE_RECORD_ALL_ATTRIBUTES parameter is set to 'N'. If a pre-image is not available, the CDC system must use special markers to indicate unchanged values within each attribute. These markers are configured using the literal values parameters: NO_CHANGE_INDICATION_[DATA_TYPE_GROUP]. Understanding and configuring this parameter correctly is crucial for accurately capturing and processing data changes in your data warehouse pipeline, especially in scenarios where only changed data attributes are updated. **When set to Y (Yes):** - The CDC system delivers the previous version of a record (pre-image) before a change occurs. - The complete before-image of the record is available for comparison. - No special markers are needed to indicate unchanged values. - Enables accurate delta detection by comparing pre-image with post-image data. **When set to N (No):** - The system will not provide a pre-image. - Unchanged values must be indicated using predefined special markers within the attributes. > ⚠️ Ensure that the literal values for all data types are properly configured in the NO_CHANGE_INDICATION_[DATA_TYPE_GROUP] parameters to accurately identify unchanged attributes when pre-images are not available. --- ### USE_OBJ_SRC_NAME_FOR_INI_CDC_TABLES **Possible Values:** Y / N | **Lowest Level:** [⚪ TAB SRC] **Default Value:** Y **Availability:** - When USE_OBJ_SRC_NAME_FOR_INI_CDC_TABLES is set to **N**: - USE_ORIGINAL_SOURCE_TABLE_NAME_FOR_INI_CDC_OBJECTS is not available **Full Description** The 'USE_OBJ_SRC_NAME_FOR_INI_CDC_TABLES' parameter determines whether the original source table name is preserved when creating initial and CDC (Change Data Capture) tables in VaultSpeed. This parameter is crucial for maintaining consistency and traceability between your source systems and the data warehouse objects. **Purpose:** - This parameter helps ensure that the initial and CDC tables in your data warehouse mirror the naming conventions of the source tables, facilitating easier mapping and understanding of data lineage. **When to Use:** - Enable this parameter if you require the initial and CDC tables to retain the original source table names, which is particularly useful for data engineers and solution architects who need to maintain clear and consistent mappings between source and target systems. > ℹ️ It is recommended to use this parameter when the source table names are meaningful and align with your organization's data governance policies. --- ### USE_REMOTE_CDC_DBLINK **Possible Values:** Y / N | **Lowest Level:** [🟣 SOURCE] **Default Value:** N **Dependencies:** - When USE_REMOTE_CDC_DBLINK is set to **Y**: - CREATE_REMOTE_DELTA_VIEW must be **Y** **Requirements:** - When REMOTE_JOURNALING_TABLES is **N**, this parameter must be **N** **Availability:** - When USE_REMOTE_CDC_DBLINK is set to **Y**: - CREATE_REMOTE_DELTA_VIEW is not available **Dependency:** - USE_REMOTE_CDC_DBLINK is not available - When REMOTE_JOURNALING_TABLES is set to **N** - USE_REMOTE_CDC_DBLINK is not available - When CREATE_REMOTE_DELTA_VIEW is set to **N** **Full Description** The USE_REMOTE_CDC_DBLINK parameter determines if a database link should be used to connect to the remote Change Data Capture (CDC) schema within the mapping process. This feature is crucial for integrating data from remote CDC sources into your VaultSpeed-managed data warehouse. **Supported Database:** - This parameter is only applicable to Oracle databases. **When to Use:** - Enable this parameter if you need to access a remote CDC schema through a database link, particularly in Oracle environments. > ℹ️ Ensure that your database configuration supports database links if you enable this option. --- ### CREATE_REMOTE_DELTA_VIEW **Possible Values:** Y / N | **Lowest Level:** [🟣 SOURCE] **Default Value:** N **Requirements:** - When REMOTE_JOURNALING_TABLES is **N**, this parameter must be **N** - When USE_REMOTE_CDC_DBLINK is **Y**, this parameter must be **Y** **Availability:** - When CREATE_REMOTE_DELTA_VIEW is set to **N**: - USE_REMOTE_CDC_DBLINK is not available **Dependency:** - CREATE_REMOTE_DELTA_VIEW is not available - When REMOTE_JOURNALING_TABLES is set to **N** - CREATE_REMOTE_DELTA_VIEW is not available - When USE_REMOTE_CDC_DBLINK is set to **Y** **Full Description** The CREATE_REMOTE_DELTA_VIEW parameter determines whether VaultSpeed creates a loading window table and views on the source Change Data Capture (CDC) tables to filter records within the current loading window. This parameter is essential for controlling how data is filtered and transferred from the source to the data vault. Setting this parameter appropriately can improve data transfer efficiency and reduce unnecessary data load on the target database. Consider enabling it when working with large volumes of data or when source system performance is a concern. **When set to Y (Yes):** - VaultSpeed creates a loading window table and views on the source CDC tables. - Records are filtered at the source level to include only those within the current loading window. - Reduces data transfer volume by filtering records before moving to the target database. - Optimizes performance by offloading filtering work to the source system. **When set to N (No):** - The Filtering and Mapping Component (FMC) will handle the filtering of records during data transfer to the target database, optimizing the data loading process by ensuring only relevant records within the current loading window are transferred. - Relies on FMC to manage filtering during data transfer, potentially reducing the load on source systems. > ⚠️ Using CREATE_REMOTE_DELTA_VIEW can optimize data processing by managing data filtering at the source level. This can be crucial for performance in scenarios with large datasets and frequent data changes. --- ### USE_LCI_TIMESTAMP_FOR_INITIAL_LOAD **Possible Values:** Y / N | **Lowest Level:** [🟣 SOURCE] **Default Value:** Y **Full Description** The USE_LCI_TIMESTAMP_FOR_INITIAL_LOAD parameter determines which timestamp is applied during the initial load process in VaultSpeed. This setting is crucial for ensuring the correct timing data is used when data is first ingested into the data warehouse. **When set to Y (Yes):** - The timestamp from the Load_cycle_info table is used for the initial load. - This can be important for maintaining consistency with data cycle information. - Syncs with Load_cycle_info for consistent data cycle tracking. **When set to N (No):** - The system will use the current_timestamp for the initial load. - This option may be preferable in scenarios where the most up-to-date timestamp is required. - Uses the present system time, ensuring the freshest timestamp is applied. > ℹ️ Utilizing the Load_cycle_info timestamp ensures alignment with specific data cycle timings, which can be critical for historical data accuracy and tracking purposes. > ⚠️ Consider your data integration and reporting needs when configuring this parameter to ensure the timestamps align with your project requirements. --- ### ADD_RECORD_TRACKING_SATELLITE **Possible Values:** Y / N | **Lowest Level:** [⚪ TAB SRC] **Default Value:** N **Dependencies:** - When ADD_RECORD_TRACKING_SATELLITE is set to **is null**: - EXISTS_FLAG_POSITIVE_VALUE must be **is not null** **Full Description** The ADD_RECORD_TRACKING_SATELLITE parameter enables the creation of a record tracking satellite in your Data Vault model. This satellite stores metadata about changes or events associated with individual records, such as load timestamps, change indicators, or deletion flags. Its primary purpose is to provide a historical view of data events, which is essential for building robust audit trails and ensuring full data lineage transparency. This parameter is typically used in combination with standard Data Vault components such as Hubs, Links, and Satellites to enhance traceability across the model. It is especially valuable in regulated environments or scenarios requiring detailed tracking of data state changes over time, supporting both compliance and operational debugging needs. **When set to Y (Yes):** - VaultSpeed automatically generates the necessary code to create record tracking satellites during DDL generation and ETL code for the loading. > ℹ️ Record tracking satellites are essential for maintaining a historical record of changes to data entities. They assist data engineers and solution architects in understanding the evolution of data over time and ensuring compliance with data governance policies. --- ### ADD_NH_COUNTER_RECORD_DELETE **Possible Values:** Y / N | **Lowest Level:** [⚪ TAB SRC] **Default Value:** N **Availability:** - When ADD_NH_COUNTER_RECORD_DELETE is set to **N**: - NH_COUNTER is not available - NH_COUNTER_POSITIVE_VALUE is not available - NH_COUNTER_NEGATIVE_VALUE is not available **Dependency:** - ADD_NH_COUNTER_RECORD_DELETE is not available - When SRC_CDC is set to **No CDC Incremental** - ADD_NH_COUNTER_RECORD_DELETE is not available - When SRC_CDC is set to **Modification Date Incremental** - ADD_NH_COUNTER_RECORD_DELETE is not available - When SRC_CDC is set to **Modification Sequence Incremental** **Full Description** The ADD_NH_COUNTER_RECORD_DELETE parameter introduces a COUNTER attribute to a non-historic object within VaultSpeed. This COUNTER attribute serves as an important indicator of whether a record has been deleted from a source system, thereby providing essential tracking for data management and integrity. **Purpose:** - To track the deletion status of records in source systems. **When to Use:** - Implement this parameter when there is a need to monitor and manage records that may be deleted in the source system, ensuring that your data warehouse reflects the most current state. By understanding the deletion status of records, VaultSpeed users can maintain data integrity and make informed decisions based on the most up-to-date data. **Key ETL code changes:** - **Column addition** - Non-historic table creation scripts include an additional COUNTER attribute column - **Delete tracking logic** - ETL procedures populate this COUNTER field to indicate whether records are deleted from source systems - **Non-historic object modification** - Only affects non-historic objects (like non-historized links), not standard Data Vault tables > ℹ️ Ensuring that your data warehouse accurately reflects deleted records can enhance data reliability and support audit processes. --- ### USE_ORIGINAL_SOURCE_TABLE_NAME_FOR_INI_CDC_OBJECTS **Possible Values:** Y / N | **Lowest Level:** [⚪ TAB SRC] **Default Value:** N **Dependency:** - USE_ORIGINAL_SOURCE_TABLE_NAME_FOR_INI_CDC_OBJECTS is not available - When USE_OBJ_SRC_NAME_FOR_INI_CDC_TABLES is set to **N** **Full Description** The USE_ORIGINAL_SOURCE_TABLE_NAME_FOR_INI_CDC_OBJECTS parameter determines the naming convention for Change Data Capture (CDC) and Initial Load (INI) objects when mapping a dropped object to a new incoming object. Instead of adopting the new source table name, this setting ensures that the original source table name continues to be used. **Why This Parameter Matters:** - Maintaining the original source table name can be crucial for consistency in historical data tracking and lineage. - It helps avoid disruptions and potential errors in data processing that could occur due to changes in source table names. **When to Use:** - Set this parameter to Y (Yes) when you need to preserve continuity in data processing and naming conventions despite changes in source systems. - Particularly useful in scenarios where the source system undergoes changes but the historical context of data must remain intact. > ⚠️ Enabling this parameter ensures that the CDC and INI processes continue seamlessly by retaining the original table names, thus maintaining the integrity of the data pipeline. --- ### CDC_TIMESTAMP_FORMAT_MASK **Possible Values:** Free Text | **Lowest Level:** [⚪ TAB SRC] **Full Description** The CDC_TIMESTAMP_FORMAT_MASK parameter enables you to define a specific cast format mask for the CDC (Change Data Capture) timestamp column, allowing for more precise control over how timestamp data is formatted during the data capture process. This parameter is essential when you need to ensure that timestamp data is consistently represented across different database systems. **Example format:** - YYYY-MM-DD HH24:MI:SS > ℹ️ This parameter requires that CAST_SOURCE_ATTRIBUTES is set to 'Y'. --- ## Data quality ### DATA_QUALITY_BAD **Possible Values:** Y / N | **Lowest Level:** [🟣 SOURCE] **Default Value:** N **Requirements:** - When CDC_UPDATE_RECORD_ALL_ATTRIBUTES is **N**, this parameter must be **N** **Availability:** - When DATA_QUALITY_BAD is set to **N**: - DATA_QUALITY_BAD_COLUMN_PREFIX is not available - DATA_QUALITY_BAD_COLUMN_SUFFIX is not available > ❌ Setting this parameter to Y and generating code that is deployed to a Databricks cluster newer than 13.3 will not have impact on the execution of the code, there will not be an automatic attempt to convert the data types and handle the potential errors. **Full Description** The DATA_QUALITY_BAD parameter is used to determine whether the data quality of a source should be considered bad. This parameter is critical in scenarios where data may not align with the suggested data type for an attribute, such as in flat files where attributes lack constraints. The DATA_QUALITY_BAD parameter fundamentally changes how Satellite tables handle attribute columns in the generated code to manage data type conversion failures. **When set to Y (Yes):** - Generated code creates two columns for each attribute - one with the specified data type and another with VARCHAR format to preserve original values This indicates that the data quality is bad. In such cases, Satellites are designed to include two columns for each attribute: - The first column adheres to the specified data type and will display an **UNKNOWN** value if the data does not match the expected type. - The second column uses a **VARCHAR** data type to retain the original value, ensuring data integrity and traceability. **When set to N (No):** - Generated code assumes good data quality and uses single columns with specified data types **Generated behavior:** If a value can't be converted to the specified data type, the first column shows "UNKNOWN" while the VARCHAR column retains the original source value Additional data type validation checks are performed during the loading process Important constraint: Requires CAST_SOURCE_ATTRIBUTES to be set to Y for this logic to work. This is particularly useful for file-based sources or databases that store all data as strings where type conversion might fail. It is assumed that the source data quality is good, and attributes are correctly assigned to their respective data types without discrepancies. > ℹ️ Important: This parameter requires the CAST_SOURCE_ATTRIBUTES parameter to be set to **Y** to function correctly. --- ### BK_CASE_INSENSITIVE_COMPARISON **Possible Values:** Y / N | **Lowest Level:** [🟢 SYSTEM] **Default Value:** Y **Full Description** The BK_CASE_INSENSITIVE_COMPARISON parameter determines the casing behavior of Business Key attributes for a Hub before hash difference calculation, which is crucial for ensuring consistency and avoiding case-related issues in key attributes. This parameter is particularly important when dealing with source systems that use varying case formats for identifiers. The BK_CASE_INSENSITIVE_COMPARISON parameter directly impacts how business key attributes are processed in the generated code for hash calculations. **Parameter Impact on Business Key Handling** The setting of this parameter plays a critical role in ensuring the accuracy and consistency of Business Key management in your data warehouse. Selecting the correct option requires careful consideration of your source system’s casing conventions and the specific Data Vault modeling approach in use. Inconsistent handling of case sensitivity across sources can lead to duplicate records or broken relationships if not addressed properly. **Code Generation Behavior** When this parameter is enabled, VaultSpeed applies UPPER() functions to Business Key fields during code generation. This normalization ensures that hash key values remain consistent regardless of case differences in the source data, which is essential for accurate hub deduplication and reliable linking. Although the business key attributes are still stored in their original case for traceability, enabling this setting mitigates the risk of case-sensitive mismatches that could otherwise compromise referential integrity. **When set to Y (Yes):** - Business key attributes get converted to uppercase before hash calculation using UPPER() function in the generated SQL All Business Key attributes are converted to uppercase before hash calculation. This setting helps eliminate discrepancies caused by different casing in the source data. **When set to N (No):** - Business key attributes remain in their original case during hash processing > ❌ Consider the implications of each setting in relation to your source system's data consistency and any case sensitivity requirements you might have in your data warehouse design. --- ### ATTR_CASE_INSENSITIVE_COMPARISON **Possible Values:** Y / N | **Lowest Level:** [⚪ DV] **Default Value:** N **Full Description** The **ATTR_CASE_INSENSITIVE_COMPARISON** parameter controls how attribute casing is handled during hash difference calculations within a Satellite (SAT) or its parts in VaultSpeed. By converting all attributes to uppercase, it ensures consistency and eliminates case-related issues when determining if record values have changed. This parameter is crucial for ensuring that differences in casing do not affect the detection of changes in data records, which can be particularly important in environments where source systems have inconsistent casing conventions. **When set to Y (Yes):** - All attributes inside a SAT are converted to uppercase before hash difference calculation. **When set to N (No):** - All attributes are kept in the exact case as they appear in the source system. > ⚠️ Ensure that your decision on this setting aligns with the casing conventions and requirements of your specific data environment to prevent unintended data discrepancies. --- ### CAST_SOURCE_ATTRIBUTES **Possible Values:** Y / N | **Lowest Level:** [🟣 SOURCE] **Default Value:** N **Availability:** - When CAST_SOURCE_ATTRIBUTES is set to **N**: - CAST_VS_MANAGED_ATTRIBUTES is not available - CDC_TIMESTAMP_FORMAT_MASK is not available **Full Description** The CAST_SOURCE_ATTRIBUTES parameter determines how VaultSpeed handles the casting of attribute values from the landed data into the data vault. This parameter is essential when dealing with text fields in source table columns, ensuring they are cast to the correct data types during loading. This parameter fundamentally changes how attribute values are handled in the generated code. **Functionality:** - Automatically casts text fields from source tables to the appropriate data type when loading into the data vault. - Ensures consistency and compatibility with the target database schema. **When to Use:** - Use this parameter when your source system includes text fields that need to be cast to specific data types in your target database. - Essential for maintaining data integrity and preventing type mismatches and implicit data type conversions downstream **Example use cases:** - For example, a source system might store **all values as VARCHAR**, including numbers, dates, and timestamps (e.g. `"123.45"` or `"2024-12-31 23:59:59"`). With this parameter enabled, VaultSpeed will **cast these text values to NUMBER, DATE, or TIMESTAMP** in the target data vault. - A CSV‑based source or a JSON API might deliver **numeric identifiers and flags as text** (e.g. `"001234"`, `"0"`, `"1"`). When CAST_SOURCE_ATTRIBUTES is set to Y, these text fields are **automatically converted to integer or boolean‑like types** according to the target schema. - Legacy databases sometimes model **dates and times in CHAR columns** (for example `CHAR(8)` with values like `"20250131"`). Enabling this parameter lets VaultSpeed **convert those string-based dates into proper DATE or TIMESTAMP columns** during loading, avoiding manual casting in every mapping. **When set to Y (Yes):** - Text fields from source tables are **automatically cast to the appropriate target data types** when loading into the data vault. - Ensures **data type consistency** between the source system and the target database schema, reducing type mismatch issues. - **Generated code explicitly handles attribute casting**, so the transformation from generic text to typed attributes is built into the load logic. - **Reduces manual casting effort** in mappings by centralizing the casting behavior in this parameter. - Can be combined with **DATA_QUALITY_BAD = Y** to add **extra data type checks** and preserve original values when conversions fail. > ℹ️ If you also need to perform data type checks, set the parameter DATA_QUALITY_BAD to 'Y'. --- ### CAST_VS_MANAGED_ATTRIBUTES **Possible Values:** Y / N | **Lowest Level:** [⚪ TAB SRC] **Default Value:** Y **Dependency:** - CAST_VS_MANAGED_ATTRIBUTES is not available - When CAST_SOURCE_ATTRIBUTES is set to **N** **Full Description** The CAST_VS_MANAGED_ATTRIBUTES parameter controls the automatic casting of the cdc_timestamp, cdc_sequence, and cdc_logposition attributes in VaultSpeed. This parameter is particularly relevant when using Change Data Capture (CDC) tools, which often land these attributes with the correct data types, rendering additional casting unnecessary. This parameter is essential for ensuring compatibility and preventing errors in environments where data types are already correctly managed by CDC tools. VaultSpeed developers should use this parameter to tailor the casting behavior to their specific data processing needs. **When set to Y (Yes):** - Enables automatic casting of the specified attributes. Be cautious, as this setting may cause firewall views to fail due to NULL casts. **When set to N (No):** - Disables automatic casting of the specified attributes, preventing potential issues with firewall views. **Usage Guidelines:** - This parameter can only be set to N if CAST_SOURCE_ATTRIBUTES is enabled (set to Y). - If real attribute casting is not required, follow these steps: - Set CAST_SOURCE_ATTRIBUTES to Y. - Update CAST_VS_MANAGED_ATTRIBUTES to N. - Revert CAST_SOURCE_ATTRIBUTES to N. > ❌ Setting this parameter to Y may cause firewall views to fail due to NULL casting. --- ## General parameters ### DDL_DROP **Possible Values:** Y / N | **Lowest Level:** [🟢 SYSTEM] **Default Value:** Y **Full Description** The DDL_DROP parameter determines whether the generated Data Definition Language (DDL) statements will include DROP statements for tables before creating them. This setting is crucial in controlling how existing database objects are managed during code generation. The parameter essentially determines whether your generated scripts will clean up existing objects before recreating them, ensuring a fresh deployment without conflicts. **When set to Y (Yes):** - DROP statements are included in the generated DDL code. - Existing tables are automatically dropped before recreating them. - Ensures a clean deployment without conflicts from pre-existing objects. - Essential for delta generation to succeed. - Enables fresh object creation during development and testing phases. **When set to N (No):** - No DROP statements generated - may cause errors if tables already exist - This can lead to errors if the tables already exist in the target database, as the creation attempt will fail without first removing the existing tables. **Practical Implications:** - Use this parameter to control whether existing tables should be dropped and recreated, which may be necessary during development or testing phases. - Be cautious when setting this parameter to N in production environments where existing data should be retained, as it can lead to creation errors without prior object removal. **Key code impacts:** - Has no effect on Spark SQL (Spark doesn't support DROP) - Critical for delta generation - must be set to Y for successful deployment - Only applies to full generations, not incremental updates > ℹ️ Note: The DDL_DROP parameter does not apply to BRIDGE or PIT objects. > ❌ This setting has no effect on Spark SQL, as Spark does not support DROP statements. --- ### USE_CDC_TS_AS_LOAD_DATES **Possible Values:** Y / N | **Lowest Level:** [🟢 SYSTEM] **Default Value:** N **Requirements:** - When SRC_CDC is **Modification Sequence**, this parameter must be **N** **Dependency:** - USE_CDC_TS_AS_LOAD_DATES is not available - When CDC_RELIABLE is set to **N** **Full Description** The USE_CDC_TS_AS_LOAD_DATES parameter controls whether the Change Data Capture (CDC) timestamp, also known as the transaction or modification timestamp, is used as the load date and load end date in your Data Vault model. Instead of applying a system-generated technical timestamp, this setting allows VaultSpeed to use the actual change time from the source system, providing a more accurate timeline of data events. When enabled, the generated ELT code for CDC-enabled objects will assign load dates based on the CDC timestamp. This is particularly useful in scenarios that require precise historical tracking, such as regulatory compliance, auditing, or time-based reporting, where it is important to reflect the true moment a change occurred in the source. **When set to Y (Yes):** - Generated code uses the CDC timestamp (transaction/modification date) for load_date and load_end_date fields instead of technical system timestamps - The system will use the CDC timestamp for both load date and load end date. - This is particularly useful when accurate transaction timing is needed for historical data analysis and reporting. **When set to N (No):** - Standard DV2.0 approach - generated code uses system-generated technical dates Important constraint: This parameter cannot be used together with CDC_RELIABLE=Y - they're mutually exclusive in the generated logic. **Generated code behavior: ** - The parameter affects PIT (Point-in-Time) table creation capabilities and is specifically designed for CDC objects only. - The DVT language includes conditional expressions that check this parameter to determine timestamp handling logic. - This ensures your load dates reflect actual business transaction timing rather than when the ETL process ran. The standard Data Vault 2.0 approach is followed, where a generated technical date is used instead. > ⚠️ This parameter is not needed when insert-only logic is used. > ℹ️ This setting affects the types of PITs (Point-in-Time tables) you can create, as it specifically applies to CDC objects. Ensure your pipeline configuration aligns with your data integration and reporting requirements. --- ### GENERATE_LOGGING **Possible Values:** Y / N | **Lowest Level:** [🟢 SYSTEM] **Default Value:** N **Full Description** The GENERATE_LOGGING parameter controls whether logging is enabled for VaultSpeed's internal processes within the cloud application. This is crucial for tracking and diagnosing issues but can impact performance. This parameter is particularly valuable for VaultSpeed Developers and Data Engineers who need to monitor system behavior and troubleshoot issues. Use logging judiciously, enabling it during development and testing phases, or when specific issues need investigation, and consider disabling it in production environments to maintain optimal performance. **When set to Y (Yes):** - Logging is enabled, and detailed logs can be accessed via the Logbook menu. - This setting is useful for debugging and monitoring but may lead to slower system performance due to the extensive information being stored. **When set to N (No):** - No logs will be generated, which can improve performance but limits visibility into the application's operations. > ❌ Enabling logging may slow performance, as extensive information is stored in the logbook. --- ### EMPTYSTRING_IS_NULL **Possible Values:** Y / N | **Lowest Level:** [🟣 SOURCE] **Default Value:** N **Dependency:** - EMPTYSTRING_IS_NULL is not available - When DATABASE_TYPE is set to **SQLSERVER** - EMPTYSTRING_IS_NULL is not available - When DATABASE_TYPE is set to **ORACLE** - EMPTYSTRING_IS_NULL is not available - When DATABASE_TYPE is set to **APACHE HIVE** - EMPTYSTRING_IS_NULL is not available - When DATABASE_TYPE is set to **AZUREDWH** - EMPTYSTRING_IS_NULL is not available - When DATABASE_TYPE is set to **EXASOL** **Full Description** The EMPTYSTRING_IS_NULL parameter controls whether empty string values are treated as NULL within the VaultSpeed platform, with the exception of Oracle databases where this behavior is inherently set. **When set to Y (Yes):** - Attributes with an empty string value are treated as NULL. **When set to N (No):** - Empty string values are treated as distinct from NULL. - Empty strings are preserved as empty strings in the data vault. - Allows differentiation between NULL (no value) and empty string (explicit empty value). - Standard behavior for most databases except Oracle. > ⚠️ This parameter does not apply to Oracle databases, as Oracle inherently treats empty strings as NULL. --- ### ADD_SOURCE_VIEWS **Possible Values:** Y / N | **Lowest Level:** [🟣 SOURCE] **Default Value:** N **Full Description** The ADD_SOURCE_VIEWS parameter determines whether views are included in addition to tables when loading source data into VaultSpeed. This setting is important for capturing all relevant structures from the source system, especially when views are used to represent pre-aggregated, transformed, or filtered datasets. Enable this parameter if your source models depend on views for delivering business logic or preparing data before ingestion. It ensures that both physical tables and logical views are available during metadata harvesting and ELT generation. This is particularly relevant for VaultSpeed developers who need complete visibility into the source landscape to design accurate and reliable data pipelines. **When set to Y (Yes):** - Both tables and views are loaded from the source system into VaultSpeed. - Includes all data structures (tables + views) in the data pipeline. - Enables data models that rely on views for transformations or aggregations. - Ensures comprehensive data loading from the source system. **When set to N (No):** - Only tables are loaded from the source, excluding views. - This setting is useful when views are not needed for your data processing or when minimizing the data load is a priority. > ℹ️ Consider the implications of loading views on your pipeline performance and storage requirements. Including views may increase the initial load time and require additional resources. --- ### CLOSE_DELETED_RECORDS **Possible Values:** Y / N | **Lowest Level:** [⚪ TAB SRC] **Default Value:** N **Full Description** The CLOSE_DELETED_RECORDS parameter defines how end dates are assigned to deleted records in your Data Vault model. It determines whether the end date should reflect the actual deletion timestamp or follow the default Data Vault 2.0 method, which typically uses the load timestamp of the next arriving record. Enable this parameter when precise tracking of deletions is required, such as in systems where the exact timing of data removal impacts downstream reporting or compliance. Keep in mind that using the delete timestamp may introduce historical gaps if records are later reinserted, so careful consideration is needed to maintain data continuity and integrity. **When set to Y (Yes):** - The end date of a deleted record will be set to the load date, or the CDC timestamp if USE_CDC_TS_AS_LOAD_DATES is enabled. - This approach overrides the default behavior of setting the end date to the CURRENT_RECORD_LOAD_END_DATE. **When set to N (No):** - The standard Data Vault 2.0 approach is followed, maintaining continuity in record history without gaps. > ❌ This setting can lead to inaccurate history if a deleted record is re-added later, as there will be a gap in the history between the delete and subsequent insert. > ⚠️ The CLOSE_DELETED_RECORDS parameter has no effect when INSERT_ONLY_LOGIC = Y or INSERT_ON_DELETE = Y. --- ### TRIM_SPACES_FROM_BUSINESS_KEY **Possible Values:** Y / N | **Lowest Level:** [⚪ TAB SRC] **Default Value:** Y **Full Description** The TRIM_SPACES_FROM_BUSINESS_KEY parameter determines whether leading and trailing spaces are removed from business key fields during data processing in VaultSpeed. This ensures that business keys are consistently formatted, which is essential for generating reliable hash keys and maintaining referential integrity across your Data Vault model. Enable this parameter if your source data includes inconsistent spacing in key fields, which can cause mismatches or unintended duplicates. Setting it to ‘Y’ is recommended in environments where business key consistency is critical, particularly when integrating data from multiple systems with varying formatting standards. **When set to Y (Yes):** - A trim function is applied to the business keys in external mappings, ensuring that leading and trailing spaces do not affect the uniqueness or consistency of the keys. **When set to N (No):** - Spaces are NOT removed from business keys during data processing. - Leading and trailing spaces in key fields are preserved as-is. - Business keys maintain their exact format from the source system. - May lead to mismatches or duplicate entries if source data contains inconsistent spacing. > ⚠️ This parameter is particularly important when dealing with source systems that may include unnecessary spaces in key fields, as it helps maintain data integrity and uniformity in the target database. --- ### CREATE_INI_TABLES **Possible Values:** Y / N | **Lowest Level:** [🟣 SOURCE] **Default Value:** Y **Full Description** The CREATE_INI_TABLES parameter determines if the initial (INI) tables are included in the Data Definition Language (DDL) scripts generated by VaultSpeed. This is crucial for managing the initialization process of data loading in your data warehouse. **When set to Y (Yes):** - VaultSpeed will generate the INI tables in the DDL scripts, which is necessary if your CDC (Change Data Capture) platform does not automatically create these tables. **When set to N (No):** - VaultSpeed will not generate the INI tables, which is advisable if your CDC platform already handles the creation of these tables. > ❌ Ensure that your CDC platform's capabilities align with this setting to avoid duplication or missing table issues. > ⚠️ This parameter primarily affects the initial setup phase of your data ingestion pipeline, so it is essential to configure it correctly before deploying your DDL scripts. --- ### AGENT_PATH **Possible Values:** Free Text | **Lowest Level:** [🟢 SYSTEM] **Default Value:** / **Full Description** The AGENT_PATH parameter specifies the directory location where the VaultSpeed agent is installed on the customer's system. This setting is crucial for ensuring that VaultSpeed can effectively communicate with the agent to perform data warehouse automation tasks. **Purpose:** - Defines the file path for the agent, which acts as a bridge between the VaultSpeed platform and the customer's data environment. **Configuration:** - Set this parameter to the absolute path where the agent is deployed. This path must be accessible and properly configured to allow VaultSpeed operations. **When to Use**: Configure the AGENT_PATH parameter during the initial setup of the VaultSpeed environment or when relocating the agent to a different directory. This setting is critical for maintaining the operational integrity of the data processing workflows. > ⚠️ Ensure that the AGENT_PATH is correctly set to prevent connectivity issues and to facilitate seamless automation processes. --- ### GENERATE_FOREIGN_KEYS **Possible Values:** Y / N | **Lowest Level:** [⚪ DV] **Default Value:** Y **Requirements:** - When GENERATE_PRIMARY_UNIQUE_KEYS is **N**, this parameter must be **N** - When DATABASE_TYPE is **SINGLESTORE**, this parameter must be **N** **Full Description** The GENERATE_FOREIGN_KEYS parameter controls whether foreign keys are created within the raw Data Vault model in the target database. This setting influences the physical enforcement of referential integrity between Hubs, Links, and Satellites during deployment. Use this parameter based on your database environment and access privileges. In cloud-based platforms where superuser rights are often restricted, foreign key creation may not be feasible or allowed. When configuring this setting, consider whether enforcing referential constraints at the database level is necessary for your project, or if logical integrity checks through ELT logic are sufficient for your use case. **When set to Y (Yes):** - Foreign keys will be created, enabling referential integrity checks on the database level. **When set to N (No):** - foreign keys will NOT be generated. - This setting is particularly beneficial when working with PostgreSQL on cloud platforms where users often do not have superuser privileges. - Without superuser access, disabling foreign keys is restricted, making this parameter essential in such scenarios. > ℹ️ This parameter does not impact performance, allowing users to configure it based on their database management and access requirements without concern for performance degradation. --- ### UPDATE_LOAD_CYCLE_ID_ON_UPDATE_DELETE **Possible Values:** Y / N | **Lowest Level:** [⚪ DV] **Default Value:** N **Full Description** The UPDATE_LOAD_CYCLE_ID_ON_UPDATE_DELETE parameter determines if the load cycle ID in a satellite should be updated when a record undergoes an update or deletion, provided that the insert_only_logic is not enabled (insert_only_logic = N). This parameter plays a crucial role in ensuring data consistency and traceability in scenarios where changes to records need to be tracked within the data vault. **When set to Y (Yes):** - And insert_only_logic is set to N, the load cycle ID is updated upon the update or deletion of a record in the satellite. **When set to N (No):** - The load cycle ID remains unchanged during updates or deletions. > ⚠️ This parameter is relevant for maintaining accurate historical data in the data vault by reflecting the latest load cycle ID during non-insert operations. It is particularly useful in scenarios where auditability and data lineage are of high importance. --- ### STORE_HUB_HASH_KEYS_IN_LINK_SAT **Possible Values:** Y / N | **Lowest Level:** [⚪ DV] **Default Value:** N **Full Description** The STORE_HUB_HASH_KEYS_IN_LINK_SAT parameter defines whether the hash keys of related hubs are stored in the satellites of links. Including these hub hash keys can be beneficial for improving referential clarity and simplifying query logic, especially when dealing with complex relationships in large-scale Data Vault models. This setting is particularly valuable for VaultSpeed developers and data engineers who aim to optimize query performance and reduce join complexity. By storing hub hash keys directly in link satellites, data retrieval becomes more efficient, which can lead to noticeable improvements in reporting and downstream processing. **Purpose:** - Ensures that the hash keys of hubs are readily available in link satellites, facilitating faster joins and lookups across data vault layers. **When to Use:** - Enable this parameter when you need to enhance query efficiency by reducing the need for additional joins to hub tables. **When set to Y (Yes):** - Hub hash keys are stored in link satellites, improving access speed at the cost of additional storage. **When set to N (No):** - Hub hash keys are not stored in link satellites, optimizing for storage but potentially increasing query complexity. > ⚠️ Activating this parameter can lead to increased storage requirements in your data warehouse, as hash keys are duplicated in link satellites. Consider your storage capacity and performance needs before enabling this option. --- ### INSERT_ON_DELETE **Possible Values:** Y / N | **Lowest Level:** [🟣 SOURCE] **Default Value:** Y **Full Description** The INSERT_ON_DELETE parameter defines how delete operations are processed in satellite tables. When enabled, VaultSpeed inserts a new record for each deletion event rather than updating an existing row with a delete flag or CDC timestamp. This approach ensures that the original transaction timestamp of the delete is retained, which is critical for maintaining an accurate and auditable change history. This setting is especially important for VaultSpeed developers and data engineers working in environments with frequent delete activity. It supports precise audit trails and strengthens data lineage by allowing the system to distinguish between multiple delete operations, while also helping to prevent duplicate deletes during firewall processing. **When set to Y (Yes):** - each delete action results in a new record, preserving the original transaction timestamp. **When set to N (No):** - the system will only update the existing delete flag and/or CDC timestamp, which may result in omission of precise transactional details. > ⚠️ Enabling this parameter is crucial for maintaining data integrity when dealing with frequent deletes, as it prevents the loss of transactional details that could occur with mere flag updates. --- ### GENERATE_PRIMARY_UNIQUE_KEYS **Possible Values:** Y / N | **Lowest Level:** [⚪ DV] **Default Value:** Y **Dependencies:** - When GENERATE_PRIMARY_UNIQUE_KEYS is set to **N**: - GENERATE_FOREIGN_KEYS must be **N** **Full Description** The GENERATE_PRIMARY_UNIQUE_KEYS parameter controls whether VaultSpeed generates primary keys, unique constraints, or indexes as part of the automated deployment process to the target data warehouse. These constraints are used to enforce data uniqueness and improve query performance, depending on the capabilities and configuration of the underlying database. Enable this parameter when your project requires strict enforcement of uniqueness at the database level or when indexed access to key columns is needed for performance optimization. It is particularly useful in environments where constraint-based integrity checks are preferred over purely logical enforcement in ELT processes. **When set to Y (Yes):** - VaultSpeed generates primary keys, unique keys, and indexes during code generation. - Enforces data integrity and uniqueness constraints at the database level. - Standard behavior for most database platforms. - Enables database-level enforcement of key constraints. **When set to N (No):** - No primary keys, unique keys, or indexes will be generated. - This setting is particularly useful when generating code for Greenplum databases, which may have specific requirements or performance considerations that make the omission of these keys advantageous. **Practical Implications:** - Consider setting this parameter to N if your target database is Greenplum, to optimize the generation process and align with best practices for that system. - Be aware that disabling key generation means that data integrity and uniqueness constraints typically enforced by these keys will need to be managed through alternative means. > ⚠️ This parameter is crucial for developers focusing on performance optimization for specific database systems like Greenplum. --- ### GENERATE_SCHEMA_DEFINITIONS **Possible Values:** Y / N | **Lowest Level:** [⚪ DV] **Default Value:** Y **Full Description** The GENERATE_SCHEMA_DEFINITIONS parameter controls whether VaultSpeed generates Data Definition Language (DDL) scripts that include CREATE SCHEMA statements. This is essential for organizing and managing database schemas as part of the automated data warehouse deployment process. Enable this parameter when setting up new environments or when changes to schema structures are needed. It simplifies the initialization and expansion of your data warehouse by automating schema creation, reducing manual setup tasks. This is especially useful for VaultSpeed developers and data engineers who need to standardize schema management across projects and ensure consistent deployments. **When set to Y (Yes):** - VaultSpeed generates a DDL script that includes instructions to create new schemas, ensuring that all necessary database structures are established as part of the full DDL generation. **When set to N (No):** - Omits the schema creation statements, useful when schema structures are managed externally or already exist. > ℹ️ This parameter is essential for automating the schema creation process, thus reducing manual intervention and potential errors in schema setup. --- ### CREATE_CDC_TABLES **Possible Values:** Y / N | **Lowest Level:** [🟣 SOURCE] **Default Value:** Y **Full Description** The CREATE_CDC_TABLES parameter controls the creation of Change Data Capture (CDC) and initialization (ini) tables within the Data Definition Language (DDL) scripts generated by VaultSpeed. **Functionality**: - This parameter determines whether or not the CDC and ini tables are included in the DDL scripts. **When to Use:** - Set this parameter to 'Y' if you want VaultSpeed to generate CDC and ini tables directly in your DDL scripts. - Set this parameter to 'N' if you are utilizing an external CDC platform that automatically creates these tables for you. **Practical Implications:** - Using the right setting for CREATE_CDC_TABLES ensures that your data pipeline is configured correctly, avoiding redundant table creation and optimizing the integration workflow. > ℹ️ It is important to correctly configure this parameter based on your CDC platform setup, as it affects how the data integration process will be handled. --- ### DETECT_UPDATES_INSERTS_ON_FULL_LOAD **Possible Values:** Y / N | **Lowest Level:** [⚪ TAB SRC] **Default Value:** Y **Full Description** The DETECT_UPDATES_INSERTS_ON_FULL_LOAD parameter determines whether VaultSpeed performs a detailed column-by-column comparison to detect updates during a Full Load when Delete Management is enabled. This allows the system to differentiate between true updates and unchanged records, helping to manage data changes accurately during bulk loading. This parameter is especially valuable in environments where consistent data mapping and standard Data Vault patterns are used, and where precise change tracking is important. Adjust this setting based on your performance requirements and the level of update detection needed for maintaining data integrity during full load operations. **Purpose:** - Determines if updates should be detected through individual column comparison during a Full Load, which can be beneficial for maintaining data accuracy in scenarios involving Delete Management. **When set to Y (Yes):** - updates are accurately detected, allowing for precise data synchronization between source systems and target databases. **When set to N (No):** - may improve performance by reducing computation overhead but risks missing some updates if changes are not captured. > ⚠️ Consider enabling this parameter if data accuracy and update detection are critical for your Full Load operations with Delete Management. However, be aware of the potential impact on performance and adjust based on your system's specific needs. --- ### DELTA_GENERATE_INIT_MAPPINGS **Possible Values:** new / changed / all | **Lowest Level:** [⚪ DV] **Default Value:** changed **Full Description** The DELTA_GENERATE_INIT_MAPPINGS parameter controls the generation of initial load mappings during a delta generation process within VaultSpeed. This parameter determines whether initial load mappings are created for new, modified, or all objects, influencing how data is handled in the data warehouse automation flow. Selecting the correct option is crucial when managing large datasets or frequent schema changes. This parameter allows VaultSpeed developers to efficiently manage data pipeline updates and ensure that the data warehouse reflects the latest source system changes without redundant processing. **“new” → New Objects Only:** - Generates initial load mappings exclusively for objects that are newly added since the last generation. **“changed” → Changed Objects (Default):** - Generates mappings only for objects that have been modified since the last generation. This is the default behavior, optimizing for changes without affecting unchanged data. **“all” → Full Generation:** - Generates initial load mappings for all objects, regardless of their change status. > ℹ️ Using the appropriate setting can optimize performance by reducing unnecessary processing and focusing on relevant data changes. --- ### DELTA_GENERATE_INCR_MAPPINGS **Possible Values:** new / changed / all | **Lowest Level:** [⚪ DV] **Default Value:** changed **Full Description** The DELTA_GENERATE_INCR_MAPPINGS parameter controls which incremental load mappings VaultSpeed generates during a delta generation process. It determines the scope of mappings to be produced based on object changes. This parameter is critical for optimizing the efficiency of data warehouse updates by focusing generation efforts where they are most needed. Choosing the right setting for DELTA_GENERATE_INCR_MAPPINGS can significantly impact the performance and efficiency of your data pipeline operations. It is advisable to regularly review and adjust this parameter in line with changes to your data architecture or business requirements. **“new”:** - Generates mappings only for new objects. Use this setting when you want to minimize generation to just the newly introduced elements. **“changed”:** - Generates mappings only for objects that have undergone changes. This setting is ideal for regular delta generation, ensuring that only modified elements are updated, saving time and resources. **“all”:** - Generates all mappings, regardless of object changes. Opt for this setting when a comprehensive regeneration of all mappings is required, for example, after significant structural updates or debugging needs. > ❌ Ensure that the selected setting aligns with your current data integration needs and project timelines to avoid unnecessary processing. --- ## Hash parameters ### HASHKEY_ESCAPE_CHARACTER **Possible Values:** Free Text | **Lowest Level:** [🟣 SOURCE] **Default Value:** \ **Full Description** The HASHKEY_ESCAPE_CHARACTER parameter specifies the escape character used during the calculation of the hash key and hash difference within VaultSpeed. This parameter is crucial for ensuring the integrity and uniqueness of hash values in your data vault design, particularly when dealing with special characters that may otherwise disrupt the hashing process. For platforms such as Snowflake, BigQuery, Apache Spark, and SingleStore, the escape character '\' will be automatically doubled during code generation, ensuring proper handling of special characters in these environments. > ⚠️ It is important to configure this parameter correctly to avoid potential data inconsistencies or errors in hash key generation. --- ### HASHKEY_DELIMITER **Possible Values:** Free Text | **Lowest Level:** [⚪ DV] **Default Value:** # **Full Description** The HASHKEY_DELIMITER parameter specifies the character used to separate different business keys and attribute values of an object in a data vault. This is crucial for ensuring that the concatenated string representation of these keys and attributes is parsed correctly when generating hash keys. The default delimiter value is automatically updated when working with specific data warehouse types, such as Snowflake or BigQuery, due to their requirement for additional escaping. > ℹ️ Using an appropriate hash key delimiter ensures the uniqueness and integrity of hash keys across different database platforms, which is essential for data consistency and reliability in your data vault. --- ### HASH_ALGORITHM **Possible Values:** MD5 / SHA1 / SHA256 / HASH | **Lowest Level:** [🟢 SYSTEM] **Default Value:** MD5 **Dependencies:** - When HASH_ALGORITHM is set to **NOHASH**: - HASH_STORE_IN_BINARY must be **N** **Full Description** The HASH_ALGORITHM parameter specifies the hash algorithm used to generate the hash key and hash difference fields in your data vault model. This parameter is crucial for ensuring data integrity and optimizing performance, particularly when handling large datasets. The choice of algorithm depends on your platform's compatibility and the size of your tables, considering the likelihood of hash collisions. The algorithms, ranked from least to most likely for collisions, are as follows: SHA-1 < MD5 < HASH. Choosing the right hashing algorithm is essential for balancing performance and collision risk, particularly with large datasets. When using Snowflake, the HASH algorithm can be beneficial, while SHA-256 provides broader compatibility across various platforms. **MD5, SHA-1, SHA-256:** - These are common hashing options supported by multiple platforms. **HASH:** - Available exclusively on Snowflake; this algorithm returns a NUMBER and is optimized for Snowflake environments. **NO_HASH:** - Displays fields in their concatenated form without applying any hashing. However, **SHA-256:** - Supported on Snowflake, Google BigQuery, Azure Synapse, and SQL Server, offering a balance between security and performance. > ❌ NO_HASH is not supported on Oracle databases. > ⚠️ The "HASH" algorithm is only available for Snowflake, while the "SHA256" algorithm is available for Snowflake, BigQuery, Azure Synapse, and SQL Server. --- ### HASHDIFF_NULL_REPLACEMENT **Possible Values:** Free Text | **Lowest Level:** [🔵 PROJECT] **Default Value:** ~ **Full Description** The HASHDIFF_NULL_REPLACEMENT parameter defines the character used to replace null values during hash difference calculations in VaultSpeed. This parameter is crucial for maintaining consistent hash values when new attributes are introduced to your data models. Ensuring that empty attribute values do not affect hash differences helps in preserving data integrity and avoiding unnecessary data vault changes. The replacement value must be a single character, as it is used in conjunction with the trim function. - Use this parameter when you need to ensure that the introduction of new attributes, which may contain nulls, does not alter existing hash differences. - Select a character that is unlikely to appear in your data naturally to avoid unintended replacements. > ℹ️ Choosing an appropriate replacement character helps in minimizing conflicts and maintaining data consistency across your data vault model. --- ### HASH_STORE_IN_BINARY **Possible Values:** Y / N | **Lowest Level:** [🟢 SYSTEM] **Default Value:** N **Requirements:** - When DATABASE_TYPE is **EXASOL**, this parameter must be **N** - When HASH_ALGORITHM is **NOHASH**, this parameter must be **N** **Full Description** The HASH_STORE_IN_BINARY parameter defines the storage format for hash key and hash diff fields in the target database. It determines whether these fields are stored in binary or string format, directly influencing performance, storage efficiency, and compatibility with downstream systems. When enabled, VaultSpeed generates hash fields in binary format, which can reduce storage footprint and improve processing speed, especially in large-scale environments. Choosing the right format is important for optimizing query performance and aligning with your database platform’s capabilities and constraints. Important constraint: Some reporting tools or databases (like SingleStore) don't support binary formats, so you'd need to set this to N for compatibility. The parameter essentially determines whether your hash columns get optimized for storage efficiency (binary) or maximum compatibility (hex strings) in the generated DDL and ETL code. Hashes are stored as hexadecimal character representations. This setting may be necessary for compatibility with specific reporting tools or databases. **When set to Y (Yes):** - Generated code stores hashes in binary format for better performance and reduced storage Hash keys and hash diffs are stored in a binary format. This can lead to improved performance and reduced storage requirements. **When set to N (No):** - Generated code stores hashes as hexadecimal character representations > ⚠️ Some reporting tools or databases (e.g., SingleStore) do not support binary formats. Consider this compatibility aspect when configuring the parameter. --- ## Literal values ### TRANSACTION_TYPE_DELETE **Possible Values:** Free Text | **Lowest Level:** [🟣 SOURCE] **Default Value:** D **Full Description** The TRANSACTION_TYPE_DELETE parameter specifies the value assigned to the TransType attribute, which is used to identify 'delete' transaction types within VaultSpeed. It plays a crucial role in managing how delete operations are recognized and processed in your automated data warehouse environment. **Purpose:** - This parameter is essential for ensuring that delete transactions are correctly identified, allowing for accurate data processing and warehouse management. **Usage:** - Set this parameter to define what value should be used to signal a delete operation in the transaction logs. By correctly configuring this parameter, you can streamline the handling of delete transactions, reduce potential errors, and maintain integrity across data layers and mappings. > ℹ️ Ensure that the value set for TRANSACTION_TYPE_DELETE aligns with your organization's standards for delete transaction identification to maintain consistency across your data processing pipelines. --- ### TRANSACTION_TYPE_INSERT **Possible Values:** Free Text | **Lowest Level:** [🟣 SOURCE] **Default Value:** I **Full Description** The TRANSACTION_TYPE_INSERT parameter specifies the value used for the TransType attribute to identify 'insert' transaction types in VaultSpeed. This parameter is crucial for distinguishing insert operations within your data pipelines and ensuring the correct handling of data as it moves through the layers of your data warehouse. **Functionality:** - Assigns a unique identifier to insert transactions, enabling accurate tracking and processing. **When to Use:** - Configure this parameter when setting up or modifying data pipelines that require precise transaction type differentiation for insert operations. It is especially useful in scenarios where data integrity and accurate record tracking are paramount. By setting the TRANSACTION_TYPE_INSERT parameter, developers can maintain consistency in transaction processing and facilitate troubleshooting by clearly identifying insert transactions in system logs and reports. > ℹ️ This parameter plays a key role in the data vault pattern implementation, ensuring that new records are appropriately flagged and processed as insertions. --- ### CURRENT_RECORD_LOAD_END_DATE **Possible Values:** Free Text | **Lowest Level:** [⚪ DV] **Default Value:** 31/12/2999 23:59:59 **Full Description** The CURRENT_RECORD_LOAD_END_DATE parameter defines the load end date for the current valid or active record(s) in the data warehouse. This parameter is crucial for ensuring that the end date of the current record aligns with the expected data management practices and maintains data consistency across different data vault patterns. The load end date format must match the GENERAL_TIMESTAMP_FORMAT parameter to ensure consistency and compatibility across the platform. Use this parameter to accurately manage data lifecycle and state transitions within your data vault architecture, ensuring that the current record is correctly identified and managed in data processing workflows. > ⚠️ Ensure that the format specified for CURRENT_RECORD_LOAD_END_DATE is compatible with your target database's timestamp format to avoid any discrepancies during data processing. --- ### TRANSACTION_TYPE_UPDATE **Possible Values:** Free Text | **Lowest Level:** [🟣 SOURCE] **Default Value:** U **Full Description** The TRANSACTION_TYPE_UPDATE parameter defines the specific value assigned to the TransType attribute that represents update transactions within your data pipelines. This setting is essential for accurately identifying and processing update events, allowing VaultSpeed to distinguish them from inserts, deletes, or other transaction types. Proper configuration ensures that updates are correctly tracked and handled according to your Data Vault model logic. To maintain consistency across your data models, it is important to use a standardized casing for the TransType values. This helps prevent discrepancies during processing and ensures alignment across environments, especially when integrating data from multiple sources or systems with varying conventions. Make sure the parameter is configured in line with your organization’s transaction handling standards. An incorrect value can result in the misclassification of transactions, which may lead to incomplete history tracking, data integrity issues, or reporting inaccuracies. Careful setup of this parameter supports reliable and auditable data processing throughout your pipelines. > ℹ️ It is important to set the TRANSACTION_TYPE_UPDATE parameter correctly to maintain data integrity and ensure that update operations are processed appropriately within your data vault architecture. --- ### NUMERIC_DECIMAL_CHARACTER_SOURCE **Possible Values:** Free Text | **Lowest Level:** [🟣 SOURCE] **Default Value:** , **Full Description** The NUMERIC_DECIMAL_CHARACTER_SOURCE parameter defines the decimal character used in the source data, such as a period (.) or comma (,). This setting is critical for correctly interpreting numeric values during the ingestion and transformation process within VaultSpeed, especially when working with international or non-standard data formats. By specifying the correct decimal character, VaultSpeed can accurately parse numeric fields and prevent errors or misinterpretations during ELT processing. This is particularly important when numeric values are stored as text or when source systems use different locale-specific formatting. Ensuring that this parameter matches the source system’s formatting conventions allows for precise data mapping and transformation. Misconfiguration may result in incorrect data type conversions or loss of precision, which can impact downstream calculations and data quality. > ⚠️ Ensure that the chosen decimal character matches the source system's configuration to avoid data discrepancies. --- ### NUMERIC_DECIMAL_CHARACTER_TARGET **Possible Values:** Free Text | **Lowest Level:** [⚪ DV] **Default Value:** , **Full Description** The NUMERIC_DECIMAL_CHARACTER_TARGET parameter specifies the decimal character that will be used in the target database. This parameter is crucial for ensuring numeric data is interpreted correctly across different database systems, which may have varying defaults for decimal characters. **Purpose:** - Defines the decimal character for numeric data in the target, ensuring consistency and accuracy in data representation. **Usage:** - Set this parameter to match the expected decimal character of your target database system to avoid potential issues with data interpretation during ETL processes. **Practical Implications:** - When migrating data between systems with different decimal conventions, ensure this parameter aligns with the target system's requirements to maintain data integrity. - Check the target database's documentation to understand its default decimal character and adjust this parameter accordingly if necessary. > ℹ️ This setting is particularly important if your target database uses a different decimal character than your source systems, such as a comma instead of a period. --- ### SOURCE_RECORD_TYPE **Possible Values:** Free Text | **Lowest Level:** [🟣 SOURCE] **Default Value:** S **Full Description** The SOURCE_RECORD_TYPE parameter specifies the value for the RECORD_TYPE attribute, which is crucial for identifying source records in your data pipelines. This parameter helps differentiate between various types of records coming from source systems, ensuring that the data is correctly mapped and processed according to the data vault patterns established in your data warehouse. **Purpose:** - To define a consistent identification attribute for source records, facilitating accurate data mapping and processing. **Usage:** - Set this parameter to the appropriate value that represents the type of source record being processed. This is essential when dealing with multiple source systems or when specific record types need to be distinguished. > ⚠️ Ensure that the SOURCE_RECORD_TYPE aligns with your data governance and record identification strategies to maintain data integrity and consistency across your data warehouse. --- ### NULL_RECORD_TYPE **Possible Values:** Y / N | **Lowest Level:** [🟣 SOURCE] **Default Value:** N **Full Description** The NULL_RECORD_TYPE parameter specifies the value for the RECORD_TYPE attribute used to identify null records within your data warehouse. This parameter is crucial for differentiating null records from other data types, ensuring that data integrity and processing rules are maintained across your data vault model. - Set the parameter to a specific value that your data model recognizes as indicating a null record. - Ensure consistency with other configurations to avoid conflicts in data interpretation. > ℹ️ It is important to choose a NULL_RECORD_TYPE value that does not overlap with any legitimate RECORD_TYPE values in your data model. This prevents misclassification of records and potential data quality issues. --- ### UNKNOWN_RECORD_TYPE **Possible Values:** Free Text | **Lowest Level:** [🟣 SOURCE] **Default Value:** U **Full Description** The UNKNOWN_RECORD_TYPE parameter defines the specific value assigned to the RECORD_TYPE attribute used to identify unknown records in your data warehouse environment. This parameter is crucial for distinguishing records that do not match any known or expected patterns, enabling automated processes to handle data anomalies effectively. **Purpose:** - To set a consistent identifier for unknown records, facilitating error handling and data quality management. **Usage Context:** - Use this parameter when configuring data pipelines or mappings that need to account for records that are not recognized by standard patterns. It helps in maintaining the integrity and reliability of data by ensuring that unknown records are flagged appropriately. > ⚠️ Ensure that the value assigned to the UNKNOWN_RECORD_TYPE parameter is unique and does not conflict with other predefined RECORD_TYPE values within your data vault architecture. --- ### CDC_PRE_IMAGE_FLAG_VALUE **Possible Values:** Free Text | **Lowest Level:** [🟣 SOURCE] **Default Value:** PRE **Dependency:** - CDC_PRE_IMAGE_FLAG_VALUE is not available - When CDC_PRE_IMAGE_AVAILABLE is set to **N** **Full Description** The CDC_PRE_IMAGE_FLAG_VALUE parameter is crucial for distinguishing pre-image records in change data capture (CDC) processes within VaultSpeed. This parameter specifies the value that indicates a record is a pre-image when the CDC_PRE_POST_IMAGE_FLAG attribute contains this value. It is essential in scenarios where tracking data changes over time is necessary, providing a historical snapshot of data before changes occur. By correctly configuring this parameter, VaultSpeed developers can ensure that data pipelines correctly capture and process pre-image data, maintaining data integrity and providing valuable insights into data changes. > ℹ️ Ensure that the CDC_PRE_IMAGE_AVAILABLE parameter is enabled ('Y') to utilize the CDC_PRE_IMAGE_FLAG_VALUE effectively. This allows for accurate data lineage and historical tracking in your data pipelines. --- ### CDC_POST_IMAGE_FLAG_VALUE **Possible Values:** Free Text | **Lowest Level:** [🟣 SOURCE] **Default Value:** POST **Dependency:** - CDC_POST_IMAGE_FLAG_VALUE is not available - When CDC_PRE_IMAGE_AVAILABLE is set to **N** **Full Description** The CDC_POST_IMAGE_FLAG_VALUE parameter determines the value used to identify a record as the post-image when the CDC_PRE_POST_IMAGE_FLAG attribute contains this value. It is crucial for managing Change Data Capture (CDC) processes effectively within VaultSpeed. This parameter is particularly relevant when the CDC_PRE_IMAGE_AVAILABLE is set to Y, indicating that pre-images are available. - Specifies the value marking a record as the post-image in CDC operations. - Used in conjunction with the CDC_PRE_POST_IMAGE_FLAG attribute. - Essential for distinguishing between pre-image and post-image records in data pipelines. > ℹ️ Ensure that the CDC_PRE_IMAGE_AVAILABLE parameter is set to Y to utilize post-image identification effectively. --- ### NO_CHANGE_INDICATION_CHAR **Possible Values:** Free Text | **Lowest Level:** [🟣 SOURCE] **Default Value:** unchanged **Dependency:** - NO_CHANGE_INDICATION_CHAR is not available - When CDC_UPDATE_RECORD_ALL_ATTRIBUTES is set to **Y** **Full Description** The NO_CHANGE_INDICATION_CHAR parameter defines a special placeholder value used by the Change Data Capture (CDC) system to signal that an attribute has not changed, particularly when pre-image data is not available. This setting becomes essential when the CDC_PRE_IMAGE_AVAILABLE parameter is set to ‘N’, meaning the system cannot compare the previous and current values directly. This parameter plays a key role in processing string-based attributes, allowing VaultSpeed to distinguish between changed and unchanged values without relying on historical data. By using a clearly defined indicator, the system can maintain accurate hash diff calculations and ensure correct change detection in satellite tables. When configuring this parameter, it is important to choose a value that does not occur naturally in your source data. Selecting a non-conflicting placeholder helps avoid false interpretations and supports reliable change tracking, preserving the integrity of your data pipelines. > ℹ️ This setting is used when CDC_PRE_IMAGE_AVAILABLE is set to 'N'. Ensure this parameter is properly configured to avoid data inconsistency issues. --- ### NO_CHANGE_INDICATION_NUMBER **Possible Values:** Free Text | **Lowest Level:** [🟣 SOURCE] **Default Value:** -1 **Dependency:** - NO_CHANGE_INDICATION_NUMBER is not available - When CDC_UPDATE_RECORD_ALL_ATTRIBUTES is set to **Y** **Full Description** The NO_CHANGE_INDICATION_NUMBER parameter is used to define a special placeholder value for number-based attributes in situations where pre-image data is not available. It is specifically relevant when the CDC_PRE_IMAGE_AVAILABLE parameter is set to ‘N’, meaning VaultSpeed cannot compare current values to previous ones to detect changes. This parameter helps the system correctly identify attributes that have not changed during a CDC event. By assigning a distinct numeric value to represent “no change,” VaultSpeed ensures consistent handling of unchanged attributes during ELT processing. This approach prevents false positives in change detection and supports accurate hash diff calculations in satellite tables, even when full historical data is not present. To maintain data integrity, it is important to select a value that will not appear in actual business data. Using a clearly defined and non-conflicting numeric indicator allows CDC processes to function reliably, even in the absence of pre-image snapshots. > ℹ️ This parameter plays a key role in scenarios where the detection of unchanged attributes is necessary, but pre-image data is unavailable, thus maintaining efficient data processing and storage. --- ### NO_CHANGE_INDICATION_DATE **Possible Values:** Free Text | **Lowest Level:** [🟣 SOURCE] **Default Value:** 01/01/1970 **Dependency:** - NO_CHANGE_INDICATION_DATE is not available - When CDC_UPDATE_RECORD_ALL_ATTRIBUTES is set to **Y** **Full Description** The NO_CHANGE_INDICATION_DATE parameter defines the special value for a date-based attribute used by the Change Data Capture (CDC) system. This parameter is crucial when there is no pre-image available, enabling the system to indicate unchanged attributes effectively. This parameter is used when the CDC_PRE_IMAGE_AVAILABLE setting is configured to 'N'. > ℹ️ This setting acts as a counterpart to the NO_CHANGE_INDICATION_CHAR, which is used for string-based attributes. Both parameters ensure that the system can properly handle unchanged data when historical images are not present. --- ### NO_CHANGE_INDICATION_TIME **Possible Values:** Free Text | **Lowest Level:** [🟣 SOURCE] **Default Value:** 00:00:00 **Dependency:** - NO_CHANGE_INDICATION_TIME is not available - When CDC_UPDATE_RECORD_ALL_ATTRIBUTES is set to **Y** **Full Description** The NO_CHANGE_INDICATION_TIME parameter specifies the special value of a time-based attribute used by the Change Data Capture (CDC) system to indicate unchanged attributes when there is no pre-image available. This parameter is crucial when the CDC_PRE_IMAGE_AVAILABLE setting is configured to 'N', meaning that pre-image data is not accessible. **Purpose:** - To provide a default value for time-based attributes, allowing the system to distinguish between updated and unchanged data when pre-image data is not present. **Usage Scenario:** - This parameter is particularly useful in scenarios where you are dealing with systems that do not support or provide pre-image data for changes, ensuring the CDC system can still effectively track data changes. By configuring the NO_CHANGE_INDICATION_TIME, developers can maintain the integrity of data tracking and ensure accurate data processing, even in the absence of pre-image data. > ⚠️ Ensure that the NO_CHANGE_INDICATION_TIME value is set appropriately to avoid incorrect data interpretation in your data vault model. --- ### NO_CHANGE_INDICATION_TIMESTAMP **Possible Values:** Free Text | **Lowest Level:** [🟣 SOURCE] **Default Value:** 01/01/1970 00:00:00 **Dependency:** - NO_CHANGE_INDICATION_TIMESTAMP is not available - When CDC_UPDATE_RECORD_ALL_ATTRIBUTES is set to **Y** **Full Description** The NO_CHANGE_INDICATION_TIMESTAMP parameter specifies a special value for timestamp-based attributes, utilized by the Change Data Capture (CDC) system to denote unchanged attributes when there is no pre-image available. This parameter is particularly relevant in scenarios where the pre-image of the data cannot be accessed or is not stored, as indicated by the setting CDC_PRE_IMAGE_AVAILABLE set to 'N'. **Purpose:** - This parameter helps in identifying and managing unchanged attributes in data streams, allowing developers to efficiently handle data updates without pre-image data. **Usage Context:** - This is crucial in environments where minimizing storage or processing overhead is desired, and the system must still differentiate between changed and unchanged data efficiently. > ℹ️ Ensure that you have set the CDC_PRE_IMAGE_AVAILABLE parameter to 'N' to utilize NO_CHANGE_INDICATION_TIMESTAMP effectively. --- ### NO_CHANGE_INDICATION_OTHER **Possible Values:** Free Text | **Lowest Level:** [🟣 SOURCE] **Default Value:** unchanged **Dependency:** - NO_CHANGE_INDICATION_OTHER is not available - When CDC_UPDATE_RECORD_ALL_ATTRIBUTES is set to **Y** **Full Description** The NO_CHANGE_INDICATION_OTHER parameter defines a special value for an attribute with a specific data type. This value is used by the Change Data Capture (CDC) system to flag attributes that remain unchanged when a pre-image is not available. This parameter is particularly relevant when the CDC_PRE_IMAGE_AVAILABLE setting is configured to 'N'. **Purpose:** - To indicate unchanged attributes in the absence of a pre-image. **Context:** - Utilized when CDC_PRE_IMAGE_AVAILABLE is set to 'N'. > ⚠️ This parameter is crucial for ensuring that data integrity is maintained in scenarios where pre-images are not provided by the CDC system. It allows the system to correctly identify and handle unchanged attributes during data processing. --- ### SATELLITE_RECORD_TYPE **Possible Values:** Free Text | **Lowest Level:** [🟣 SOURCE] **Default Value:** SAT **Dependency:** - SATELLITE_RECORD_TYPE is not available - When CDC_PRE_IMAGE_AVAILABLE is set to **N** - SATELLITE_RECORD_TYPE is not available - When CDC_PRE_IMAGE_AVAILABLE is set to **N** **Full Description** The SATELLITE_RECORD_TYPE parameter specifies the value assigned to the RECORD_TYPE attribute, which is crucial for identifying satellite records within the data vault architecture. This parameter plays a key role in distinguishing different types of data records in satellite tables, facilitating accurate data lineage and versioning. **Purpose:** - Defines the RECORD_TYPE attribute, ensuring that satellite records are correctly identified and categorized within the system. **Importance:** - Proper configuration of the SATELLITE_RECORD_TYPE ensures that data lineage and historical tracking are maintained, which is vital for audit trails and data governance. **Recommended Usage:** - Adjust the SATELLITE_RECORD_TYPE to align with your organization's data modeling standards and requirements to maintain consistency across your data vault patterns. By configuring the SATELLITE_RECORD_TYPE parameter appropriately, VaultSpeed developers can ensure that satellite records are accurately processed and managed, thereby enhancing the overall efficiency and reliability of the data warehouse automation process. > ℹ️ Ensure that the value set for SATELLITE_RECORD_TYPE is unique and meaningful within your data vault architecture to prevent conflicts and maintain clear data differentiation. --- ### TRANSACTION_TYPE_SATELLITE **Possible Values:** Free Text | **Lowest Level:** [🟣 SOURCE] **Default Value:** SAT **Dependency:** - TRANSACTION_TYPE_SATELLITE is not available - When CDC_PRE_IMAGE_AVAILABLE is set to **N** - TRANSACTION_TYPE_SATELLITE is not available - When CDC_PRE_IMAGE_AVAILABLE is set to **N** **Full Description** The TRANSACTION_TYPE_SATELLITE parameter is crucial for defining the transaction type attribute within satellite tables in a data vault model. This parameter determines how transaction types are identified and named, which plays a key role in managing Change Data Capture (CDC) operations, specifically update-only CDC. **Purpose:** - Specifies the name for the transaction type column in satellite tables. **Usage:** - Used to calculate and identify satellite transaction types, enabling efficient handling of update-only CDC. **Practical Implications:** - Proper configuration of this parameter is essential for VaultSpeed developers to implement effective CDC strategies and maintain accurate data within their data warehouse automation processes. > ℹ️ Setting the correct value for TRANSACTION_TYPE_SATELLITE ensures that satellite tables accurately reflect transactional changes, improving data consistency and integrity. --- ### DELETE_FLAG_POSITIVE_VALUE **Possible Values:** Free Text | **Lowest Level:** [🔵 PROJECT] **Default Value:** Y **Full Description** The DELETE_FLAG_POSITIVE_VALUE parameter specifies the positive value used in Satellite tables (SATs) to indicate that a record has been deleted in the source system. This parameter is crucial for data warehouse automation processes as it helps manage data lifecycle and ensures that data reflects the current state of the source system. **When set to Y (Yes):** - Satellite tables use the configured **positive value** (for example `Y` or `1`) in the `DELETE_FLAG` column to **mark records that are deleted in the source system**. - The **delete flag column’s data type is aligned with this positive value** (e.g. character or numeric), ensuring consistent storage and interpretation of delete indicators. - Downstream logic and queries can **reliably filter on the positive value** to exclude deleted records from current views while still keeping them available for historical analysis. - This setting supports **clear, explicit delete tracking** in Satellites, making it easier to audit when and where a source record was logically removed. > ℹ️ When configured correctly, DELETE_FLAG_POSITIVE_VALUE allows for efficient tracking of deletions, which is essential for maintaining data integrity and historical accuracy in data vault implementations. --- ### DELETE_FLAG_NEGATIVE_VALUE **Possible Values:** Free Text | **Lowest Level:** [🔵 PROJECT] **Default Value:** N **Full Description** The DELETE_FLAG_NEGATIVE_VALUE parameter specifies the negative value used in the delete flag column within Satellite tables (SATs) to signal that a record has been deleted in the source system. This parameter is crucial for accurately tracking the deletion status of data records in your data warehouse. - Satellite tables use the configured **negative value** (for example `N` or `0`) in the `DELETE_FLAG` column to **mark records that are *****not***** deleted in the source system**. - The **data type of the delete flag column is aligned with this negative value** (e.g. character or numeric), ensuring consistent storage and interpretation of “active” vs “deleted” indicators. - Downstream logic and queries can **reliably filter on the negative value** to select records that are still considered active in the source, while records with the positive value represent logical deletes. - This setting supports **clear, explicit active/delete status tracking** in Satellites, making it easier to distinguish between currently active records and logically removed ones in reporting and auditing. --- ### RECORD_SOURCE_VALUE_TEMPLATE **Possible Values:** Free Text | **Lowest Level:** [⚪ DV] **Full Description** The RECORD_SOURCE_VALUE_TEMPLATE parameter specifies how to store the record source value in VaultSpeed. This can be achieved by using a template or a hardcoded string. The default template is set to `.`, providing a standardized way to reference record sources. - Templates should always be in lower case as follows: - ``: Represents the name of the source record. - ``: Represents the abbreviated name of the object. - Ensure that all templates are lowercase to maintain consistency and avoid errors. - When implementing data vault patterns, consider the use of templates for flexibility and scalability. This will ensure that the record source values are consistent across various database objects and pipelines, simplifying data management and code generation tasks. > ℹ️ Using templates allows for dynamic generation of record source values, which can be particularly useful when dealing with multiple source systems and target databases. This approach enhances automation and reduces manual input errors. --- ### EXISTS_FLAG_POSITIVE_VALUE **Possible Values:** Y / N | **Lowest Level:** [🔵 PROJECT] **Default Value:** Y **Requirements:** - When ADD_RECORD_TRACKING_SATELLITE is **is null**, this parameter must be **is not null** **Full Description** The EXISTS_FLAG_POSITIVE_VALUE parameter specifies the positive value used in Real Time Streams (RTSs) to indicate that a record existed in the source system. This parameter is crucial for ensuring accurate data representation and tracking changes over time in the data vault. - Setting this parameter correctly is essential for the accurate interpretation of data existence, especially in environments with multiple source systems. > ⚠️ Ensure the value set for EXISTS_FLAG_POSITIVE_VALUE is consistent with your source system's data conventions to prevent misinterpretation of record existence. --- ### EXISTS_FLAG_NEGATIVE_VALUE **Possible Values:** Y / N | **Lowest Level:** [🔵 PROJECT] **Default Value:** N **Dependencies:** - When EXISTS_FLAG_NEGATIVE_VALUE is set to **is null**: - RTS_TABLE_PREFIX must be **is not null** **Requirements:** - When RTS_TABLE_PREFIX is **is null**, this parameter must be **is not null** **Full Description** The EXISTS_FLAG_NEGATIVE_VALUE parameter specifies the negative value used in Real-Time Systems (RTSs) to indicate whether a record did not exist in the source system. - This parameter is critical for ensuring accurate data lineage and history tracking in data vault patterns, as it helps distinguish records that were not present in the initial data source. - Use this parameter to configure how non-existent records are flagged in your data warehouse. - Ensuring the correct negative value is set can prevent misinterpretations of data existence across systems. By correctly configuring the EXISTS_FLAG_NEGATIVE_VALUE, you can enhance the reliability and clarity of your data integration processes. > ℹ️ This setting is specifically relevant for developers managing data pipelines where historical data accuracy and integrity are crucial. --- ### NH_COUNTER_POSITIVE_VALUE **Possible Values:** Free Text | **Lowest Level:** [⚪ TAB SRC] **Default Value:** 1 **Dependency:** - NH_COUNTER_POSITIVE_VALUE is not available - When SRC_CDC is set to **No CDC Incremental** - NH_COUNTER_POSITIVE_VALUE is not available - When ADD_NH_COUNTER_RECORD_DELETE is set to **N** - NH_COUNTER_POSITIVE_VALUE is not available - When SRC_CDC is set to **Modification Date Incremental** - NH_COUNTER_POSITIVE_VALUE is not available - When SRC_CDC is set to **Modification Sequence Incremental** **Full Description** The NH_COUNTER_POSITIVE_VALUE parameter is used within VaultSpeed to determine if a record exists in a source system by checking for a positive value in a non-historic object. **Purpose:** - This parameter is crucial for tracking the existence of records in source systems when working with non-historic objects, ensuring accurate data representation in the data warehouse. **Usage:** - By assigning a positive value to this parameter, VaultSpeed can effectively identify existing records, which is essential for maintaining data integrity and consistency across pipelines. This parameter plays a significant role in data transformation processes and helps optimize the handling of source system data in the data vault architecture. Understanding its application can enhance the efficiency of data warehouse operations. > ℹ️ When configuring this parameter, ensure that the non-historic object you are working with correctly reflects the source system's data status, as this directly influences the accuracy of record existence checks. --- ### NH_COUNTER_NEGATIVE_VALUE **Possible Values:** Free Text | **Lowest Level:** [⚪ TAB SRC] **Default Value:** -1 **Dependency:** - NH_COUNTER_NEGATIVE_VALUE is not available - When SRC_CDC is set to **No CDC Incremental** - NH_COUNTER_NEGATIVE_VALUE is not available - When ADD_NH_COUNTER_RECORD_DELETE is set to **N** - NH_COUNTER_NEGATIVE_VALUE is not available - When SRC_CDC is set to **Modification Date Incremental** - NH_COUNTER_NEGATIVE_VALUE is not available - When SRC_CDC is set to **Modification Sequence Incremental** **Full Description** The **NH_COUNTER_NEGATIVE_VALUE** parameter is used within VaultSpeed to specify a negative value for a column in non-historic objects. This negative value is utilized to indicate whether a record exists in the source system. **Purpose:** - To identify the presence of records in source systems by utilizing a distinct negative value in non-historic objects. **Usage Context:** - This parameter is particularly relevant in scenarios where determining the existence of a record in the source system is critical for data processing or validation workflows. **Practical Implications:** - Utilizing the NH_COUNTER_NEGATIVE_VALUE effectively helps in streamlining data validation processes and supports the integrity of data pipelines by clearly marking records that may otherwise be ambiguous in their existence status. > ℹ️ This setting is crucial for ensuring accurate data representation and consistency across data vault layers when dealing with non-historic records. > ⚠️ Ensure that the chosen negative value does not conflict with any valid data values used within your source systems to avoid unintended data interpretation issues. --- ## FMC parameters ### FMC_GENERATE_SRC_LOADING_SCRIPTS **Possible Values:** Y / N | **Lowest Level:** [🟣 SOURCE] **Default Value:** Y **Full Description** The FMC_GENERATE_SRC_LOADING_SCRIPTS parameter determines whether VaultSpeed will generate scripts for the Functional Metadata Component (FMC) involved in loading data from the source systems to the Data Warehouse's Initial (INI) and Change Data Capture (CDC) tables. **Enabled:** - Scripts will be generated, facilitating the automated transfer of data from source systems into the INI and CDC tables, essential for maintaining a consistent data flow. **Disabled:** - No scripts will be generated, which may be suitable for scenarios where data loading processes are managed externally or manually. - Use this parameter to automate the script generation process when setting up pipelines between source systems and the data warehouse. - Ideal for environments where consistent data loading and transformation processes are critical for data integrity and operational efficiency. > ℹ️ The availability of these scripts is crucial for automating the data loading process, ensuring that data from source systems is systematically integrated into the data warehouse environment. --- ### FMC_SKIP_COMPLETED_TASKS **Possible Values:** Y / N | **Lowest Level:** [🔵 PROJECT] **Default Value:** N **Full Description** The FMC_SKIP_COMPLETED_TASKS parameter controls whether tasks in the Functional Module Controller (FMC) are executed only once if they were successful, or if all tasks are rerun regardless of success. This parameter is crucial for optimizing workflow efficiency and execution time. This parameter can be overridden by the FMC_RERUN_ALL_TASKS parameter in Airflow, providing flexibility for specific runs where rerunning all tasks is required. **When set to Y (Yes):** - The parameter generates the necessary logic in the Airflow scripts to skip tasks that have already been completed successfully. - This approach simplifies workflows and saves time by avoiding unnecessary reruns, even when restarting an entire run instead of specific tasks. **When set to N (No):** - All tasks, regardless of their previous success, are rerun. - This can lead to a more complex workflow and potentially slower overall execution due to the additional tasks. > ℹ️ Using FMC_SKIP_COMPLETED_TASKS effectively can lead to significant time savings and workflow optimization by ensuring that only necessary tasks are executed, reducing redundant processing. --- ### FMC_RERUN_ALL_TASKS **Possible Values:** Y / N | **Lowest Level:** [🔵 PROJECT] **Default Value:** N **Full Description** The FMC_RERUN_ALL_TASKS parameter determines if tasks that have previously completed successfully should be rerun when a Directed Acyclic Graph (DAG) run is restarted. This parameter plays a crucial role in managing task execution behavior during retriggered runs, ensuring that only necessary tasks are executed again, which can be vital for optimizing performance and resource usage. **Purpose:** - Control the rerun behavior of successful tasks during a DAG restart. **Storage:** - The parameter is stored in a JSON file, which can be imported as an Airflow variable. **Modification:** - It can be adjusted within Airflow after import, offering flexibility to adapt to changing execution requirements. By understanding and configuring the FMC_RERUN_ALL_TASKS parameter appropriately, VaultSpeed developers can effectively manage task reruns, enhancing the efficiency of data processing workflows. > ℹ️ The FMC_RERUN_ALL_TASKS parameter is particularly useful in scenarios where rerunning all tasks could lead to unnecessary computations or resource consumption. Adjusting this parameter helps in fine-tuning the execution flow based on specific pipeline needs. --- ### FMC_BATCH_SIZE **Possible Values:** Free Text | **Lowest Level:** [🔵 PROJECT] **Default Value:** 1000 **Full Description** The FMC_BATCH_SIZE parameter specifies the number of records that are loaded simultaneously by the Financial Management Console (FMC) loading scripts. This setting is crucial for optimizing data loading processes and is contingent upon the available memory of the FMC workers. - **Usage Condition:** This parameter is utilized only when 'FMC_GENERATE_SRC_LOADING_SCRIPTS' is set to 'Y'. - **Saved to File:** The FMC_BATCH_SIZE value is saved to a JSON file, allowing it to be imported as an Airflow variable. This setup enables users to adjust the parameter within Airflow to fine-tune the batch size and optimize loading speed based on system performance and memory availability. > ⚠️ Adjusting this parameter can significantly impact the performance of the data loading process. Consider the memory capacity of your FMC workers when setting this parameter to avoid potential performance issues. --- ### SCHEMA_PL_PROCEDURES **Possible Values:** Free Text | **Lowest Level:** [🔵 PROJECT] **Default Value:** PROJECT_PL_PROC **Full Description** The SCHEMA_PL_PROCEDURES parameter defines the schema used to store procedures for loading the Presentation Layer (PL) within VaultSpeed. This parameter is crucial for organizing and managing the database objects related to the Presentation Layer, ensuring that they are stored in the correct schema for efficient access and maintenance. This parameter supports templating, allowing adaptability across different projects. Specifically, when creating a project, the placeholder "PROJECT" within this parameter will automatically be replaced with the actual project name, facilitating seamless integration and consistency across various environments. - **Casing Behavior by Setting:** - Supports templating for dynamic schema naming - Use this parameter to systematically organize your PL procedures, enhancing the maintainability and scalability of your data warehouse projects. - Leverage templating to streamline project setup and ensure consistency in schema naming conventions. > ℹ️ Ensure that the schema specified for SCHEMA_PL_PROCEDURES is correctly set up and accessible to avoid runtime errors in procedure execution within the Presentation Layer. --- ### SOURCE_PL_DEP **Possible Values:** Y / N | **Lowest Level:** [🟣 SOURCE] **Default Value:** Y **Full Description** The SOURCE_PL_DEP parameter controls the conditions under which the Presentation Layer (PL) load is allowed to execute, specifically in relation to the success or failure of the source load. **Functionality:** - This parameter determines whether the PL load can proceed only after a successful load of the source or if it can still proceed even if the source load has failed. **Use Case:** - Use this parameter when you need to ensure data integrity in your data warehouse. By setting the SOURCE_PL_DEP parameter, you can control the dependency behavior to either enforce strict load sequence or allow flexibility in processing. **Strict Dependency:** - If set to enforce strict dependency, the PL load will only proceed when the source load is successful. This is crucial for maintaining accurate and consistent data flows, especially in environments where data integrity is paramount. **Flexible Dependency:** - If set to allow the PL load to run despite source load failure, it provides flexibility, useful in environments where partial data availability is acceptable or necessary for continuous processing. **Implications:** - Choosing the appropriate setting for this parameter can significantly impact the robustness and reliability of your data pipelines. Ensure that your choice aligns with your organization's data processing strategy and requirements. > ℹ️ It's important to configure this parameter based on the criticality of data accuracy and the operational requirements of your data warehouse environment. --- ### USE_FMC **Possible Values:** Y / N | **Lowest Level:** [🟢 SYSTEM] **Default Value:** N **Full Description** The USE_FMC parameter determines whether Flow Management Control (FMC) is enabled within VaultSpeed. This feature is available only if the FMC has been purchased. Enabling this parameter impacts both the data definition language (DDL) and extract, transform, load (ETL) processes by creating additional metadata tables and generating extra procedures to manage dependencies between data flows. **FMC Type Selection:** - When USE_FMC is set to 'Y', specify the type of FMC to be generated using the FMC_TYPE parameter. This parameter is crucial for organizations looking to enhance their data management capabilities by automating dependency handling within their data pipelines. Use it to streamline processes and improve the efficiency of your data warehouse operations. > ⚠️ Ensure that your organization has purchased the FMC feature before enabling this parameter, as it will not function otherwise. --- ### FMC_DYNAMIC_LOADING_WINDOW **Possible Values:** Y / N | **Lowest Level:** [🟣 SOURCE] **Default Value:** Y **Requirements:** - When CDC_BASED_LOADING_WINDOW is **Y**, this parameter must be **Y** **Availability:** - When FMC_DYNAMIC_LOADING_WINDOW is set to **N**: - CDC_BASED_LOADING_WINDOW is not available **Full Description** The FMC_DYNAMIC_LOADING_WINDOW parameter controls how the loading window for data processing is determined in VaultSpeed. It allows you to specify whether the start of the loading window is dynamic, based on the end of the last successful run, or if it should be static, waiting for the successful execution of the previous run before starting a new one. **Dynamic Loading Window:** - The start of the loading window is automatically set to follow the end of the last successful run. This approach is beneficial for optimizing processing time and resources. **Static Loading Window:** - The loading process begins only after the previous run has successfully completed, maintaining a consistent schedule. This is useful in scenarios where timing predictability is crucial. **Practical Implications:** - Use the dynamic setting to minimize downtime and improve throughput if your data processing can tolerate variable start times. - Opt for the static setting to ensure predictable scheduling and to avoid potential overlaps in execution, particularly in environments where data consistency and timing are critical. > ℹ️ Choosing between dynamic and static loading windows can significantly impact the efficiency and reliability of your data processing workflows. Consider the nature of your data loads and the criticality of timing when configuring this parameter. --- ### FMC_OVERLAPPING_LOADING_WINDOWS **Possible Values:** Y / N | **Lowest Level:** [🟣 SOURCE] **Default Value:** N **Dependencies:** - When FMC_OVERLAPPING_LOADING_WINDOWS is set to **Y**: - CDC_RELIABLE must be **N** - INTRA_LOADCYCLE_CHANGES must be **Y** **Requirements:** - When SRC_CDC is **Modification Sequence**, this parameter must be **N** **Full Description** The FMC_OVERLAPPING_LOADING_WINDOWS parameter controls whether an overlap is introduced to the loading windows within the Field Mapping Configuration (FMC) of the VaultSpeed platform. This parameter plays a crucial role in managing data consistency and ensuring that no data is missed during transitions between loading windows. The extent of the overlap is dictated by the 'window_overlap' parameter configured in Airflow, which serves as the scheduling and orchestration tool for your data pipelines. - When enabled, this parameter ensures that successive data loads have overlapping windows, which can help in capturing late-arriving data and managing data deduplication effectively. - The overlap size can be adjusted by setting the 'window_overlap' parameter in Airflow, allowing fine-tuning based on the specific requirements of your data pipeline. > ℹ️ It's important to configure your 'window_overlap' in Airflow appropriately, taking into consideration the data arrival patterns and the potential impact on data processing times. --- ### FMC_TYPE **Possible Values:** Free Text | **Lowest Level:** [⚪ DV] **Default Value:** Airflow **Requirements:** - When DATABASE_TYPE is **REDSHIFT**, this parameter must be **Airflow/generic** **Full Description** The FMC_TYPE parameter determines the framework for managing and executing data workflows within VaultSpeed. Selecting the correct FMC_TYPE is crucial for aligning with your organization's orchestration tool and ensuring seamless pipeline execution. **Airflow:** - Choose this option if your organization uses Apache Airflow as the orchestration tool. It allows integration with Airflow to manage and schedule data workflows efficiently. **ADF:** - Select this when Azure Data Factory is used for orchestration. This setting ensures compatibility and smooth operation within the Azure ecosystem. **Generic:** - Opt for this setting to generate a more flexible and adaptable framework that is not specific to Airflow or ADF. This can be useful for testing or custom setups. > ⚠️ Ensure you select the FMC_TYPE that corresponds to your existing orchestration infrastructure to avoid execution issues and maximize automation efficiency. --- ### FMC_WINDOW_OVERLAP_SIZE **Possible Values:** Free Text | **Lowest Level:** [🟣 SOURCE] **Default Value:** 15 **Full Description** The FMC_WINDOW_OVERLAP_SIZE parameter specifies the interval size for the overlap between loading windows when the FMC_OVERLAPPING_LOADING_WINDOWS feature is enabled. It is critical for managing overlapping data loads efficiently. This parameter is used in conjunction with the timestamp attribute to calculate the overlap duration, as shown in the example expression: "timestamp_attribute - FMC_WINDOW_OVERLAP_SIZE * interval '1 FMC_WINDOW_OVERLAP_TYPE'". It ensures that data loading processes have a defined overlap period, which can be crucial for handling late-arriving data or ensuring data consistency across overlapping windows. **FMC_OVERLAPPING_LOADING_WINDOWS = Y:** - Enables the use of FMC_WINDOW_OVERLAP_SIZE to define the overlap interval. By setting an appropriate FMC_WINDOW_OVERLAP_SIZE, you can manage the overlap in loading windows, which helps in maintaining data integrity and consistency, especially in dynamic environments where data is updated frequently or arrives late. > ⚠️ This parameter is only applicable when FMC_OVERLAPPING_LOADING_WINDOWS is set to 'Y'. Ensure this configuration is correctly set to avoid unintended behaviors in your data loading processes. --- ### FMC_WINDOW_OVERLAP_TYPE **Possible Values:** Free Text | **Lowest Level:** [🟣 SOURCE] **Default Value:** minute **Full Description** The FMC_WINDOW_OVERLAP_TYPE parameter defines the interval type used to calculate the overlap between loading windows, a critical aspect when FMC_OVERLAPPING_LOADING_WINDOWS is set to 'Y'. This parameter ensures precise control over data loading intervals by specifying the unit of time (e.g., seconds, minutes, hours, days) for the overlap. **When FMC_OVERLAPPING_LOADING_WINDOWS is 'Y':** - The overlap is calculated using the formula 'timestamp_attribute - FMC_WINDOW_OVERLAP_SIZE * interval '1 FMC_WINDOW_OVERLAP_TYPE''. Using the correct FMC_WINDOW_OVERLAP_TYPE is crucial for ensuring that data loading intervals are managed accurately, preventing data loss or duplication during overlapping window configurations. Choose the interval type based on your specific data loading requirements and frequency. > ⚠️ This parameter is only applicable if FMC_OVERLAPPING_LOADING_WINDOWS is enabled. --- ### OBJECT_SPECIFIC_LOADING_WINDOW **Possible Values:** Y / N | **Lowest Level:** [🟣 SOURCE] **Default Value:** N **Dependencies:** - When OBJECT_SPECIFIC_LOADING_WINDOW is set to **Y**: - OBJECT_NAME must be **is not null** **Requirements:** - When CDC_BASED_LOADING_WINDOW is **Y**, this parameter must be **Y** **Availability:** - When OBJECT_SPECIFIC_LOADING_WINDOW is set to **N**: - CDC_BASED_LOADING_WINDOW is not available - OBJECT_NAME is not available **Full Description** The OBJECT_SPECIFIC_LOADING_WINDOW parameter controls how loading windows are managed within VaultSpeed, offering flexibility in data processing and management. This parameter is essential for VaultSpeed developers and data engineers who need to tailor data loading strategies to specific business needs or technical constraints. **When set to Y (Yes):** - The loading window is defined separately for each source object. - This allows for more granular control, enabling users to log and maintain individual loading windows per object in the FMC loading metadata. **When set to N (No):** - A single loading window is applied to each source as a whole. - This is less granular but can simplify management when individual object loading windows are not required. > ⚠️ Using object-specific loading windows ('Y') is beneficial in cases where different source objects require unique loading schedules or have varying data update patterns. This setting is particularly useful for complex data environments with diverse data sources and requirements. > ❌ Ensure that your data processing infrastructure can handle the increased complexity and potential processing overhead when enabling object-specific loading windows. --- ### CDC_BASED_LOADING_WINDOW **Possible Values:** Y / N | **Lowest Level:** [🟣 SOURCE] **Default Value:** N **Dependencies:** - When CDC_BASED_LOADING_WINDOW is set to **Y**: - OBJECT_SPECIFIC_LOADING_WINDOW must be **Y** - FMC_DYNAMIC_LOADING_WINDOW must be **Y** **Requirements:** - When SRC_CDC is **Modification Sequence**, this parameter must be **Y** **Dependency:** - CDC_BASED_LOADING_WINDOW is not available - When OBJECT_SPECIFIC_LOADING_WINDOW is set to **N** - CDC_BASED_LOADING_WINDOW is not available - When FMC_DYNAMIC_LOADING_WINDOW is set to **N** **Full Description** The CDC_BASED_LOADING_WINDOW parameter controls how the Full Metadata Controller (FMC) determines the end point of the data loading window during ingestion. When set to ‘Y’, the FMC uses the highest Change Data Capture (CDC) timestamp found in the landing table of a specific object to define this boundary. This ensures that all new CDC records since the previous load are included, provided they arrive in chronological order. This setting is most effective when used together with the OBJECT_SPECIFIC_LOADING_WINDOW parameter, allowing for precise and efficient incremental loading. It helps reduce unnecessary data processing by narrowing the window to only include relevant new records. To ensure consistent results, the data ingestion pipeline must be designed to handle CDC records in correct chronological order. Misordered data can lead to partial loads or missed changes, so this configuration is best suited for environments where CDC delivery is reliable and time-ordered. **When set to Y (Yes):** - Utilizes the maximum CDC timestamp for the loading window's end, ensuring precise incremental data loading. **When set to N (No):** - The FMC uses the current system timestamp to define the end of the loading window. - Does not use the maximum CDC timestamp from the landing table. - May load records that have not yet been captured by CDC. - Less precise for incremental data loading based on CDC records. - Default behavior when parameter is not set. > ⚠️ This parameter is particularly useful in scenarios where data needs to be loaded incrementally and precisely, based on the latest available CDC data rather than the current system timestamp. --- ### FMC_FILE_EXTENSION **Possible Values:** Free Text | **Lowest Level:** [🔵 PROJECT] **Default Value:** .sql **Full Description** The FMC_FILE_EXTENSION parameter defines the file extension used for deploying generated code to Databricks. This setting is important because it determines how the code is interpreted and executed within the Databricks environment, whether deployed directly or managed through a GIT integration. To ensure smooth deployment, the parameter should be configured according to your chosen delivery method. Using the correct extension helps Databricks recognize the file type, allowing for proper execution within notebooks or as part of automated workflows. Incorrect configuration may result in integration failures or runtime errors, disrupting your data pipeline. Setting this parameter correctly is essential for ensuring that your generated ELT logic functions as intended within Databricks. **Direct Deployment to Databricks:** - If your mappings are deployed directly to Databricks, maintain the default value, which is '.sql'. **Deployment via GIT Integration:** - If your mappings are deployed to GIT and subsequently imported into Databricks using GIT integration, set this parameter to an empty string. > ℹ️ Selecting the correct file extension ensures seamless integration and execution of your data pipelines within Databricks. --- ### CONVERT_FMC_TIMEZONE **Possible Values:** Y / N | **Lowest Level:** [⚪ DV] **Default Value:** N **Dependencies:** - When CONVERT_FMC_TIMEZONE is set to **Y**: - FMC_TIMEZONE must be **is not null** - TARGET_TIMEZONE must be **is not null** **Availability:** - When CONVERT_FMC_TIMEZONE is set to **N**: - FMC_TIMEZONE is not available - TARGET_TIMEZONE is not available **Full Description** The CONVERT_FMC_TIMEZONE parameter determines whether the Full Metadata Controller (FMC) should convert timestamp values to match the timezone of the target database. This setting is important for maintaining consistency in time-based data across systems that may operate in different time zones. Enabling this parameter ensures that all load timestamps are aligned with the target system’s timezone, supporting accurate historical tracking and consistent reporting. This is particularly relevant in global environments where data sources and targets span multiple regions. Before enabling, consider the implications for historical data and how time conversion may affect existing reports or logic. It is recommended to test the conversion with sample data to ensure that it aligns with your processing requirements and does not introduce discrepancies in time-based analyses. **Purpose**: - Ensures that time data from the FMC source system aligns with the target database's timezone, preventing discrepancies in time-sensitive data processing. **When to Use**: - Enable this parameter if your data pipeline involves multiple timezones and you need to standardize timestamps in the target database layer. > ℹ️ Make sure the target database timezone settings are correctly configured before enabling this parameter to avoid unexpected time offsets. --- ### TARGET_TIMEZONE **Possible Values:** Free Text | **Lowest Level:** [⚪ DV] **Default Value:** UTC **Requirements:** - When CONVERT_FMC_TIMEZONE is **Y**, this parameter must be **is not null** **Dependency:** - TARGET_TIMEZONE is not available - When CONVERT_FMC_TIMEZONE is set to **N** **Full Description** The TARGET_TIMEZONE parameter defines the timezone to be used for the target database within the VaultSpeed platform. This setting ensures that all time-based data, such as load timestamps and change tracking, aligns with the expected time context of the target environment. Accurate configuration of this parameter is essential for consistent data processing and reliable time-based calculations. The value must be provided in the correct format supported by the target database technology to avoid errors or misaligned timestamps. It is recommended to set the timezone to match either the location of your business operations or the data center hosting the target system. This alignment helps maintain clarity in reporting, auditing, and downstream processes that rely on precise time references. > ℹ️ Setting the correct TARGET_TIMEZONE ensures that all datetime operations and timestamps are consistent with the expected time zone settings of your target database, preventing potential errors or mismatches in data reporting. > ⚠️ If the TARGET_TIMEZONE is not correctly set, it may lead to discrepancies in time-based data analysis and reporting. --- ### FMC_TIMEZONE **Possible Values:** Free Text | **Lowest Level:** [⚪ DV] **Default Value:** UTC **Requirements:** - When CONVERT_FMC_TIMEZONE is **Y**, this parameter must be **is not null** **Dependency:** - FMC_TIMEZONE is not available - When CONVERT_FMC_TIMEZONE is set to **N** **Full Description** The FMC_TIMEZONE parameter defines the timezone used by the Functional Metadata Component (FMC) server within VaultSpeed. This setting is essential for aligning all time-based operations—such as data loads, scheduling, and timestamp generation—with the local time of the FMC server. Proper configuration ensures consistency across data pipelines and prevents discrepancies between source, target, and orchestration layers. Setting the correct FMC_TIMEZONE helps avoid common issues such as misaligned load windows, inaccurate reporting timestamps, or inconsistencies in change tracking. It is especially important in distributed environments or global deployments where systems may operate across different timezones. VaultSpeed developers should update this parameter whenever there are changes to server configurations or operational timezones. Keeping it aligned with the FMC server’s actual timezone supports reliable execution and accurate time-based data handling across the platform. **Purpose:** - Aligns the time-related operations of the FMC server with the correct local timezone. **When to use:** - Set this parameter when configuring the FMC server, especially if your operations span multiple time zones or if the server's timezone differs from your primary business operations timezone. > ℹ️ Accurate timezone settings are essential for time-sensitive operations such as data loading schedules, timestamp conversions, and historical data tracking within your data warehouse environment. --- ### CONVERT_SOURCE_TIMEZONE **Possible Values:** Y / N | **Lowest Level:** [🟣 SOURCE] **Default Value:** N **Dependencies:** - When CONVERT_SOURCE_TIMEZONE is set to **Y**: - SOURCE_TIMEZONE must be **is not null** **Availability:** - When CONVERT_SOURCE_TIMEZONE is set to **N**: - SOURCE_TIMEZONE is not available **Full Description** The 'CONVERT_SOURCE_TIMEZONE' parameter determines if the timezone of the source system differs from that of the target database. This parameter is specifically used during the source loading window calculation process. **Purpose:** - Ensures accurate alignment of data load times between differing time zones of source and target systems. **When to use:** - Set (Y) this parameter when your source system's timezone is not the same as the target database's timezone. This is crucial for accurate data ingestion and processing schedules. > ℹ️ Enable this setting if your source data is timestamped in a different timezone compared to your target database. This guarantees that data is processed within the correct time window, maintaining data integrity and consistency. --- ### SOURCE_TIMEZONE **Possible Values:** Free Text | **Lowest Level:** [🟣 SOURCE] **Default Value:** UTC **Requirements:** - When CONVERT_SOURCE_TIMEZONE is **Y**, this parameter must be **is not null** **Dependency:** - SOURCE_TIMEZONE is not available - When CONVERT_SOURCE_TIMEZONE is set to **N** **Full Description** The SOURCE_TIMEZONE parameter defines the timezone of the source system from which data is extracted in the VaultSpeed platform. This setting is essential for accurately interpreting and converting timestamps during data processing, ensuring consistency as data moves from the source to the target data warehouse. When source and target systems operate in different timezones, incorrect or missing timezone alignment can result in data being loaded with inaccurate timestamps. This can impact historical tracking, reporting accuracy, and time-based logic across your data model. Configure the SOURCE_TIMEZONE parameter to match the local timezone of the source system. Doing so ensures proper timestamp conversion, preserves data integrity, and supports reliable analytics in cross-regional or multi-system environments. **Purpose:** - Aligns the source data's timezone with the data warehouse's timezone to maintain consistency in time-based data. **Recommended Usage:** - Set this parameter to match the timezone of your source system to avoid discrepancies in time-sensitive data. > ⚠️ It is important to configure the SOURCE_TIMEZONE accurately to prevent any potential misalignment in data processing that could lead to incorrect data analysis or reporting. --- ## Attribute names ### BK_PREFIX **Possible Values:** Free Text | **Lowest Level:** [🔵 PROJECT] **Full Description** The BK_PREFIX parameter defines the prefix applied to Business Key column names in Hub tables within the VaultSpeed platform. This setting plays a key role in enforcing standardized naming conventions across the data warehouse, making it easier to recognize and manage Business Key attributes. Applying a consistent prefix through this parameter improves readability and supports efficient data mapping, especially when integrating data from multiple sources. It helps developers and data engineers quickly identify Business Key fields during model design, transformation logic, and downstream reporting. This parameter is especially valuable in large-scale or complex environments, where clear and uniform naming conventions are critical for maintaining control over metadata, simplifying governance, and enhancing overall data warehouse maintainability. - **Purpose:** Ensures uniformity in the naming of Business Key columns, aiding in the organization and readability of Hub tables. - **Usage:** Set a prefix that aligns with your organization's naming standards to help differentiate Business Key columns from other types of columns. > ⚠️ It is recommended to choose a prefix that is meaningful and easily recognizable to all team members working with the data warehouse architecture. --- ### BK_SUFFIX **Possible Values:** Free Text | **Lowest Level:** [🔵 PROJECT] **Default Value:** BK **Dependencies:** - When BK_SUFFIX is set to **is null**: - BK_PREFIX must be **is not null** **Requirements:** - When BK_PREFIX is **is null**, this parameter must be **is not null** **Full Description** The BK_SUFFIX parameter specifies the suffix added to Business Key column names in Hub tables within the VaultSpeed platform. This setting is important for maintaining consistent and clear naming conventions across your Data Vault model, supporting both manual identification and automated processing of Business Key fields. Choosing an appropriate and non-conflicting suffix helps avoid naming collisions with existing columns in the source or target databases. It also ensures that Business Keys are easily distinguishable, which simplifies development, debugging, and metadata management. It is recommended to review and validate the BK_SUFFIX setting during initial project setup and whenever schema changes occur. Maintaining alignment between naming conventions and model design helps preserve data integrity and promotes clarity across your data warehouse environment. **Purpose:** - To standardize the naming of Business Key columns, facilitating easier management and integration of data vault layers. **Application:** - Utilize this parameter to customize the suffix according to organizational naming standards or specific project requirements. In combination with the parameter UPPERCASE: - When set to 'upper', the suffix will be appended in uppercase. - When set to 'lower', the suffix will be appended in lowercase. - When set to 'preserve', the suffix will maintain the original casing as provided. > ⚠️ Consistent use of the BK_SUFFIX parameter helps in maintaining a uniform schema, which is vital for large-scale data warehouse automation projects. --- ### CDC_TIMESTAMP **Possible Values:** Free Text | **Lowest Level:** [⚪ TAB SRC] **Default Value:** CDC_TIMESTAMP **Full Description** The CDC_TIMESTAMP parameter defines the column name dedicated to storing the date and time when a change is detected in the source system. This parameter is crucial for capturing Change Data Capture (CDC) events and recording modification dates. **Purpose:** - Facilitates tracking of data changes over time by logging the exact timestamp of detected changes. - Essential for maintaining accurate historical data in data warehousing solutions. **Usage:** - Used primarily in CDC processes to ensure that data modifications are accurately captured and time-stamped. - Assists in aligning data updates across different systems and databases. **Practical Implication:** - Setting this parameter correctly ensures that your data warehouse can accurately reflect changes in the source system, supporting seamless data integration and analysis. > ℹ️ Ensure that the column specified by the CDC_TIMESTAMP parameter is available and correctly formatted in the source system to prevent data synchronization issues. --- ### CDC_FLAG **Possible Values:** Free Text | **Lowest Level:** [⚪ TAB SRC] **Default Value:** JRN_FLAG **Full Description** The CDC_FLAG parameter defines the name of the column or flag used to indicate the type of change that has been detected in the source system, such as 'I' for an Insert or 'U' for an Update. This parameter is essential for change data capture processes as it helps in identifying the nature of changes in the source data, which is critical for maintaining accurate data in your data warehouse. By correctly configuring the CDC_FLAG parameter, VaultSpeed can efficiently automate the handling of changes in your source data, ensuring reliable and up-to-date information is reflected in your data warehouse. This is particularly useful for systems with high volumes of data changes, where manual tracking and updating would be impractical. Specify a clear and consistent naming convention for the CDC flag column to ensure it is easily identifiable and understood across your data pipeline. > ℹ️ Ensure that the CDC_FLAG is adequately mapped to the corresponding source system changes to facilitate correct data processing and transformation in your data warehouse automation workflows. --- ### FK_PREFIX **Possible Values:** Free Text | **Lowest Level:** [🔵 PROJECT] **Full Description** The FK_PREFIX parameter is crucial for defining the naming convention of Foreign Key constraints within all generated tables in VaultSpeed. It specifies the prefix that will be applied to the names of Foreign Key attributes, ensuring consistency and clarity across your data warehouse schema. By setting the FK_PREFIX, developers and data engineers can streamline their processes when managing complex data models, ensuring that all Foreign Key constraints are easily identifiable and consistent with the overall database design. **Purpose:** - To maintain a standardized naming convention for Foreign Key constraints, aiding in the organization and management of database objects. **Usage:** - Set the FK_PREFIX parameter to a desired prefix string that will precede the names of all Foreign Key constraints. This practice helps in quickly identifying and referencing Foreign Key constraints across different database tables. > ℹ️ It's recommended to choose a prefix that aligns with your organization's naming standards to avoid conflicts and enhance readability. --- ### FK_SUFFIX **Possible Values:** Free Text | **Lowest Level:** [🔵 PROJECT] **Default Value:** FK **Dependencies:** - When FK_SUFFIX is set to **is null**: - FK_PREFIX must be **is not null** **Requirements:** - When FK_PREFIX is **is null**, this parameter must be **is not null** **Full Description** The FK_SUFFIX parameter is used to define the suffix applied to the names of Foreign Key constraints for all tables generated within VaultSpeed. This parameter plays a crucial role in ensuring that Foreign Key constraint names are consistent and easily identifiable across different tables. This parameter directly affects the naming conventions of database constraints, which can be critical for maintaining clarity and consistency in large-scale data environments. Using a standardized FK_SUFFIX can facilitate easier database management, streamline troubleshooting processes, and improve collaboration among team members by maintaining uniform naming conventions. This parameter should be configured thoughtfully during the initial setup of your data warehouse automation project to avoid potential renaming efforts later on. > ℹ️ It is recommended to use a concise and descriptive suffix that aligns with your organization's naming conventions. This helps in quickly identifying and managing Foreign Key constraints within the database architecture. --- ### HUB_HASH_KEY_PREFIX **Possible Values:** Free Text | **Lowest Level:** [🔵 PROJECT] **Full Description** The HUB_HASH_KEY_PREFIX parameter defines the prefix applied to Hash Key column names within Hub tables in the Data Vault model. This setting plays a vital role in maintaining consistent, standardized naming across your data warehouse, making Hash Key columns easily identifiable during development, testing, and maintenance. Using a clear and consistent prefix ensures that Hash Key columns stand out from other attributes, reducing confusion and supporting automated processes that rely on column naming patterns. It also helps align with team or organizational naming standards, which is especially important in collaborative or large-scale environments. To minimize disruption and improve maintainability, choose a prefix that fits with your existing conventions. Proper use of this parameter enhances clarity in model design and simplifies navigation through complex schemas in large data vault implementations. > ⚠️ Using a consistent prefix for Hash Key columns streamlines both the development process and future troubleshooting by making these columns immediately recognizable in the schema. --- ### HUB_HASH_KEY_SUFFIX **Possible Values:** Free Text | **Lowest Level:** [🔵 PROJECT] **Default Value:** HKEY **Dependencies:** - When HUB_HASH_KEY_SUFFIX is set to **is null**: - HUB_HASH_KEY_PREFIX must be **is not null** **Requirements:** - When HUB_HASH_KEY_PREFIX is **is null**, this parameter must be **is not null** **Full Description** The HUB_HASH_KEY_SUFFIX parameter defines the suffix appended to the names of the Hash Key columns in the Hub tables within the data vault model. This parameter is crucial for maintaining consistency and readability of the database schema. **Purpose:** - Ensures a standardized naming convention for Hash Key columns, which is important for automated processes and easier schema management. **Usage:** - Set this parameter when configuring your data vault model to ensure all Hub tables' Hash Key columns end with a consistent suffix. **Practical Implications:** - Using a consistent suffix aids in the automated generation of SQL queries and improves maintainability. - Facilitates collaboration among team members by providing a clear and predictable naming pattern. - Helps prevent naming conflicts and enhances the overall organization of the data warehouse. **Recommendation**: Choose a suffix that is short and descriptive, such as '_HK' or '_HASH', to clearly indicate the nature of the column while keeping names concise. > ℹ️ This parameter is particularly useful when integrating multiple source systems, as it helps in distinguishing between similar Hash Key columns originating from different sources. --- ### LNK_HASH_KEY_PREFIX **Possible Values:** Free Text | **Lowest Level:** [🔵 PROJECT] **Default Value:** LNK **Dependencies:** - When LNK_HASH_KEY_PREFIX is set to **is null**: - LNK_HASH_KEY_SUFFIX must be **is not null** **Requirements:** - When LNK_HASH_KEY_SUFFIX is **is null**, this parameter must be **is not null** **Full Description** The LNK_HASH_KEY_PREFIX parameter determines the prefix used for naming the Hash Key columns within Link tables in VaultSpeed. This is a crucial setting for maintaining consistency and clarity in your data warehouse architecture, especially when dealing with multiple link tables. By specifying a prefix, you help ensure that each Link table's Hash Key column can be easily identified, which aids in data management and integration processes. **Purpose:** - Ensures consistent and clear naming for Hash Key columns in Link tables. **When to Use:** - Set this parameter when you need to standardize the naming conventions of your Link tables' Hash Key columns across your data warehouse. > ⚠️ It is recommended to choose a meaningful prefix that aligns with your organization's naming standards to facilitate easier identification and management of database objects. --- ### LNK_HASH_KEY_SUFFIX **Possible Values:** Free Text | **Lowest Level:** [🔵 PROJECT] **Default Value:** HKEY **Dependencies:** - When LNK_HASH_KEY_SUFFIX is set to **is null**: - LNK_HASH_KEY_PREFIX must be **is not null** **Requirements:** - When LNK_HASH_KEY_PREFIX is **is null**, this parameter must be **is not null** **Full Description** The LNK_HASH_KEY_SUFFIX parameter determines the suffix appended to the names of the Hash Key columns within Link tables in your data warehouse. This setting is crucial for maintaining consistent naming conventions across your database objects, which enhances readability and manageability. - The parameter ensures that all Link table Hash Key column names end with a specific suffix, allowing developers to easily identify and differentiate these columns. > ⚠️ It is recommended to choose a suffix that aligns with your organization's naming standards to promote consistency and clarity in your data models. --- ### LOAD_CYCLE_ID **Possible Values:** Free Text | **Lowest Level:** [🔵 PROJECT] **Default Value:** LOAD_CYCLE_ID **Full Description** The LOAD_CYCLE_ID parameter defines the name of the column used to store a unique identifier for each execution of the data loading process. This identifier is critical for tracking load cycles, enabling users to differentiate between individual runs and monitor the progression of data through the pipeline. Proper configuration of this parameter supports traceability, auditability, and error recovery by clearly linking each record to a specific load event. It also plays a key role in incremental loading and change tracking, especially in environments with high data volume or frequent loads. To maintain consistency and avoid naming conflicts, align the LOAD_CYCLE_ID naming convention with your organization’s standards across all systems. This ensures clarity in data lineage and improves the manageability of your data warehouse operations. **Purpose:** - Identifies each execution cycle, facilitating debugging and auditing. **Usage:** - Applicable in scenarios where you need to maintain a historical record of load cycles, ensuring traceability and accountability of data processing activities. > ℹ️ This parameter is crucial for maintaining the integrity and traceability of data processing operations within VaultSpeed. Proper configuration of LOAD_CYCLE_ID can significantly enhance the management and auditability of data workflows. --- ### LOAD_TIMESTAMP **Possible Values:** Free Text | **Lowest Level:** [🔵 PROJECT] **Default Value:** LOAD_DATE **Full Description** The LOAD_TIMESTAMP parameter defines the name of the column used to store the load date for each record in the raw Data Vault. This timestamp is essential for tracking when data was ingested into the warehouse, serving as a key piece of metadata for both operational monitoring and historical analysis. Accurate load timestamps enable reliable auditing and support full data lineage by marking the exact time a record entered the system. This is particularly important in scenarios that require tracing data origins, validating processing timelines, or ensuring compliance with regulatory standards. In addition, the LOAD_TIMESTAMP is critical for processes such as incremental loading, rollback operations, and time-based reporting. Setting this parameter consistently ensures that all records carry a clear and standardized temporal reference across the data warehouse. **Purpose:** - To capture the exact date and time when a record is ingested into the raw data vault. **Usage:** - Set this parameter to ensure that every record in the raw vault includes a timestamp of when it was loaded. > ℹ️ Using a consistent LOAD_TIMESTAMP across all records helps maintain data integrity and supports historical tracking within the data vault architecture. > ⚠️ Make sure the LOAD_TIMESTAMP format is consistent with the target database requirements to avoid any issues during data loading. --- ### LOAD_END_TIMESTAMP **Possible Values:** Free Text | **Lowest Level:** [🔵 PROJECT] **Default Value:** LOAD_END_DATE **Full Description** The LOAD_END_TIMESTAMP parameter defines the name of the column used to store the end date for records in satellite tables within the Data Vault. This column is essential for managing the lifecycle of records, indicating the point at which a record is no longer active or current. By capturing when a change occurs or a new version of the data is introduced, the end timestamp supports accurate historical tracking and enables point-in-time analysis. It ensures that data consumers can distinguish between current and historical values for any given attribute. This parameter plays a key role in auditing, temporal queries, and maintaining the integrity of slowly changing data. Proper configuration ensures consistency in how record validity is handled across all satellite structures in the model. > ℹ️ The LOAD_END_TIMESTAMP is a key component in ensuring that your data vault model accurately reflects data changes over time, aiding in compliance and historical analysis. --- ### PK_PREFIX **Possible Values:** Free Text | **Lowest Level:** [🔵 PROJECT] **Full Description** The PK_PREFIX parameter defines the prefix added to the names of Primary Key constraints in all generated tables within the VaultSpeed platform. This setting helps standardize naming conventions, ensuring consistency across database objects, which is crucial for maintaining organized and easily navigable data structures. If the prefix is set to 'PK_', the primary key for a table named 'CUSTOMER' will be 'PK_CUSTOMER'. This parameter affects how primary key constraints are named, influencing how they are identified and managed within the database. > ℹ️ It's recommended to choose a prefix that aligns with your organization's naming standards to facilitate easier maintenance and integration with existing systems. --- ### PK_SUFFIX **Possible Values:** Free Text | **Lowest Level:** [🔵 PROJECT] **Default Value:** PK **Dependencies:** - When PK_SUFFIX is set to **is null**: - PK_PREFIX must be **is not null** **Requirements:** - When PK_PREFIX is **is null**, this parameter must be **is not null** **Full Description** The PK_SUFFIX parameter is used to define the suffix that will be appended to the names of Primary Key constraints for all tables generated within the VaultSpeed platform. This parameter ensures consistency and clarity in constraint naming across the data warehouse, which is critical for maintaining organized and manageable database structures. **Purpose:** - To establish a uniform naming convention for Primary Key constraints, aiding in database maintenance and readability. **Default Setting:** - Typically, a default suffix is provided, but users can customize this to align with their organization's naming standards. **Usage:** - Set this parameter according to your project's naming conventions to ensure that all Primary Key constraints are easily identifiable and consistent. In combination with parameter UPPERCASE: - If set to lower, all Primary Key suffixes will be generated in lowercase. - If set to upper, all Primary Key suffixes will be generated in uppercase. **Practical Implications:** - Adopting a consistent suffix for Primary Key constraints facilitates smoother communication among team members and eases the management of database objects across multiple environments. > ⚠️ Choosing a meaningful PK_SUFFIX can help in quickly identifying Primary Key constraints in large database schemas. --- ### SRC_NAME_IN_BK **Possible Values:** Free Text | **Lowest Level:** [🔵 PROJECT] **Default Value:** SRC_BK **Full Description** The SRC_NAME_IN_BK parameter defines the name of the column that is used to store the source name within the hub. This column is crucial for the calculation of the hub hash key, which is a unique identifier in the data vault model. **Purpose:** - By specifying this parameter, you ensure consistency and accuracy in the generation of the hub hash key, which is vital for linking related data across various systems and maintaining the integrity of the data vault architecture. **Use Case:** - Use this parameter when defining the structure of your hub tables to guarantee that the source name is appropriately stored and utilized in hash key calculations. > ℹ️ The hub hash key is a critical component in the data vault model, enabling efficient data integration from multiple source systems. --- ### UK_PREFIX **Possible Values:** Free Text | **Lowest Level:** [🔵 PROJECT] **Full Description** The UK_PREFIX parameter is used to define the prefix applied to the names of Unique Key constraints in all generated tables within VaultSpeed. This parameter is crucial for maintaining consistent naming conventions across your data warehouse projects, making it easier to identify and manage Unique Key constraints. **Purpose:** - Establish a consistent prefix for Unique Key constraint names, aiding in organization and clarity. **Usage:** - Set a desired prefix to be automatically added to all Unique Key constraint names generated by VaultSpeed. > ⚠️ Using a standardized prefix helps streamline database management and ensures that Unique Key constraints are easily recognizable across different tables and projects. --- ### UK_SUFFIX **Possible Values:** Free Text | **Lowest Level:** [🔵 PROJECT] **Default Value:** UK **Dependencies:** - When UK_SUFFIX is set to **is null**: - UK_PREFIX must be **is not null** **Requirements:** - When UK_PREFIX is **is null**, this parameter must be **is not null** **Full Description** The UK_SUFFIX parameter is used to define the suffix appended to the names of Unique Key constraints in all generated tables within VaultSpeed. This parameter ensures consistency and clarity in naming conventions across your data warehouse. **Purpose:** - To specify a suffix for Unique Key attributes, aiding in the identification and management of these constraints. **Usage Context:** - Applicable when defining naming conventions for database objects, particularly Unique Key constraints, within generated tables. > ⚠️ Ensuring a consistent naming convention for Unique Key constraints is crucial for maintaining clarity and manageability in large-scale data warehouse environments. --- ### HASH_DIFF **Possible Values:** Free Text | **Lowest Level:** [🔵 PROJECT] **Default Value:** HASH_DIFF **Full Description** The HASH_DIFF parameter defines the name of the column responsible for storing the hash of all historical attributes within a Satellite (SAT) table in a Data Vault model. This parameter is essential for ensuring data integrity and efficient change tracking in your data warehouse. **Uppercase:** - All characters in the column name will be converted to uppercase. **Lowercase:** - All characters in the column name will be converted to lowercase. **Mixed Case:** - The column name will retain the original casing as defined. > ⚠️ It is critical to correctly configure the HASH_DIFF parameter to maintain consistency across your data vault and ensure the integrity of historical data tracking. --- ### DELETE_FLAG **Possible Values:** Free Text | **Lowest Level:** [🔵 PROJECT] **Default Value:** DELETE_FLAG **Full Description** The DELETE_FLAG parameter designates the column name in satellite tables (SATs) that indicates if a record was deleted in the source system. This parameter is crucial for tracking and managing record deletions, ensuring that the data warehouse accurately reflects the source system's state. VaultSpeed developers and data engineers should configure this parameter to align with the source system's deletion logic, facilitating accurate historical tracking and reporting of data changes within the data vault architecture. **Key Details:** - Specifies the column name in SATs for deletion status - Essential for maintaining data integrity and consistency between source and target systems > ⚠️ Ensure that the DELETE_FLAG column is appropriately indexed to improve query performance when checking for deleted records. --- ### LND_HASH_KEY_PREFIX **Possible Values:** Free Text | **Lowest Level:** [🔵 PROJECT] **Default Value:** LND **Dependencies:** - When LND_HASH_KEY_PREFIX is set to **is null**: - LND_HASH_KEY_SUFFIX must be **is not null** **Requirements:** - When LND_HASH_KEY_SUFFIX is **is null**, this parameter must be **is not null** **Full Description** The LND_HASH_KEY_PREFIX parameter is used to specify a prefix for the names of the Hash Key columns in the Object-Based Link tables, specifically for Many-to-Many Link configurations within VaultSpeed. This prefix helps in identifying and managing the Hash Key columns more efficiently across the data warehouse environment. When set, the prefix is applied to all Hash Key columns, ensuring consistency and easier identification. > ℹ️ Using a consistent prefix is crucial for maintaining an organized and easily navigable data structure. This parameter is particularly relevant when dealing with complex data relationships that require Many-to-Many Link tables. --- ### LND_HASH_KEY_SUFFIX **Possible Values:** Free Text | **Lowest Level:** [🔵 PROJECT] **Default Value:** HKEY **Dependencies:** - When LND_HASH_KEY_SUFFIX is set to **is null**: - LND_HASH_KEY_PREFIX must be **is not null** **Requirements:** - When LND_HASH_KEY_PREFIX is **is null**, this parameter must be **is not null** **Full Description** The LND_HASH_KEY_SUFFIX parameter defines the suffix used for naming the Hash Key columns in both Object Based Link tables and Many-to-Many Link tables. This parameter is crucial for ensuring consistent naming conventions across your data warehouse automation processes, facilitating easier identification and management of these columns. When setting this parameter, ensure that the suffix aligns with your organization's naming conventions and is distinct enough to prevent any naming conflicts. > ℹ️ Consistent suffixes help maintain clarity and prevent errors in code generation and data handling. --- ### DATA_QUALITY_BAD_COLUMN_PREFIX **Possible Values:** Free Text | **Lowest Level:** [🔵 PROJECT] **Full Description** The DATA_QUALITY_BAD_COLUMN_PREFIX parameter determines the prefix used for naming secondary attribute columns when data quality is deemed bad. This is essential for managing data quality issues effectively in your data warehouse pipeline. This prefix is applied to VARCHAR attribute columns that hold the original, uncasted source data. The parameter helps to quickly identify columns containing data that did not meet quality standards during the transformation process. - The prefix should be chosen to easily distinguish these columns from standard data columns. This parameter is particularly useful when dealing with heterogeneous data sources or when integrating data from systems with varying data quality standards. It ensures that any issues with data casting or conversion are transparent and can be addressed systematically. > ⚠️ Using a consistent prefix allows for easier querying and troubleshooting of data quality issues within your data vault. --- ### DATA_QUALITY_BAD_COLUMN_SUFFIX **Possible Values:** Free Text | **Lowest Level:** [🔵 PROJECT] **Default Value:** ORG **Dependencies:** - When DATA_QUALITY_BAD_COLUMN_SUFFIX is set to **is null**: - DATA_QUALITY_BAD_COLUMN_PREFIX must be **is not null** **Requirements:** - When DATA_QUALITY_BAD_COLUMN_PREFIX is **is null**, this parameter must be **is not null** **Dependency:** - DATA_QUALITY_BAD_COLUMN_SUFFIX is not available - When DATA_QUALITY_BAD is set to **N** **Full Description** The DATA_QUALITY_BAD_COLUMN_SUFFIX parameter is used to define a specific suffix for the names of attribute columns when the data quality is determined to be bad. This parameter is particularly important in scenarios where maintaining a clear distinction between cleansed data and original source data is crucial. **Purpose:** - This suffix is applied to secondary (VARCHAR) attribute columns that store the original, uncasted source data. - It helps in identifying columns that contain data flagged as having quality issues, ensuring they are easily distinguishable from processed data columns. **When to use it:** - Use this parameter when you need to manage and track data quality issues within your data warehouse. - It is particularly useful in data vault patterns where maintaining the integrity and traceability of source data is essential. **Practical Implications:** - Proper management of this parameter can significantly enhance data traceability and error handling in data pipelines. - By having a consistent suffix for bad quality data columns, teams can streamline their data cleansing and quality monitoring processes. > ⚠️ Ensure the suffix is unique and descriptive enough to avoid confusion with other column names. --- ### SOURCE_IN_TEMP **Possible Values:** Free Text | **Lowest Level:** [🔵 PROJECT] **Default Value:** SOURCE **Full Description** The SOURCE_IN_TEMP parameter determines the name of the column in the temporary table that contains the record source, playing a critical role in loading logic. This parameter is essential for ensuring that data lineage and source tracking are accurately maintained during the data pipeline process. **Purpose:** - By specifying the SOURCE_IN_TEMP, developers can clearly identify where each piece of data originated, which is crucial for both auditing and troubleshooting data issues. **Usage:** - This parameter is used during the transformation phase when data is being staged and prepared for loading into the data vault. **Practical Implications:** - Correctly setting the SOURCE_IN_TEMP parameter helps maintain data integrity and allows for robust data governance practices within the VaultSpeed platform. > ℹ️ Make sure the column name specified in SOURCE_IN_TEMP is consistent with the naming conventions used across your pipelines to avoid any discrepancies in data lineage tracking. --- ### EQUAL_IN_TEMP **Possible Values:** Free Text | **Lowest Level:** [🔵 PROJECT] **Default Value:** EQUAL **Full Description** The EQUAL_IN_TEMP parameter defines the name of the column that stores the 'equal' metadata attribute within temporary tables in VaultSpeed. This parameter is crucial for managing how metadata is handled during the data integration process, ensuring consistency and traceability of data transformations. **Purpose:** - To specify the column name for the 'equal' metadata attribute in temporary tables. **Usage Context:** - Used by VaultSpeed developers to configure the data warehouse automation process, particularly when setting up or managing pipelines that require metadata tracking. > ⚠️ Ensure that the specified column name is unique and does not conflict with existing column names in the temporary tables to avoid data integrity issues. --- ### RECORD_TYPE **Possible Values:** Free Text | **Lowest Level:** [🟣 SOURCE] **Default Value:** RECORD_TYPE **Full Description** The RECORD_TYPE parameter defines the name of the column that holds the record type for the data being processed in the VaultSpeed data warehouse automation platform. This parameter is essential for correctly identifying and categorizing data records as they move through different stages of data processing and storage. **Purpose:** - Specifies the column name that contains the record type information, which is crucial for distinguishing between different types of records. In combination with the UPPERCASE parameter: - Case-Sensitive: Ensure that the column name matches exactly with the source system's column name. - Case-Insensitive: The system will not differentiate between uppercase and lowercase letters in the column name. > ℹ️ Ensure the RECORD_TYPE column is correctly defined to avoid data misclassification in your data vault patterns. --- ### FMC_BEGIN_LW_TIMESTAMP **Possible Values:** Free Text | **Lowest Level:** [🔵 PROJECT] **Default Value:** FMC_BEGIN_LW_TIMESTAMP **Full Description** The FMC_BEGIN_LW_TIMESTAMP parameter defines the name of the column used to store the begin loading window timestamp for the Full Migration Cycle (FMC). This parameter is crucial for tracking when the data loading process starts, providing valuable information for managing and auditing data loads in VaultSpeed. - This parameter must be set according to your database's naming conventions. - Ensure consistency across your data warehouse to avoid discrepancies in data tracking. > ⚠️ Correctly configuring this parameter is essential for maintaining accurate data lineage and loading cycle monitoring. It is recommended to align this parameter with your organization's data governance policies. --- ### FMC_END_LW_TIMESTAMP **Possible Values:** Free Text | **Lowest Level:** [🔵 PROJECT] **Default Value:** FMC_END_LW_TIMESTAMP **Full Description** The FMC_END_LW_TIMESTAMP parameter specifies the name of the column used to store the end loading window timestamp for the Fully Managed Column (FMC) in your data warehouse. This parameter is essential for tracking the completion of data loading processes and ensuring data integrity within your VaultSpeed automation workflows. **Purpose:** - Define the column that records when the data loading window ends, which is crucial for managing data loads and troubleshooting potential issues. **Who Should Use It:** - VaultSpeed Developers and Data Engineers who are configuring and managing the data warehouse pipelines. **Practical Implications:** - This parameter helps in auditing and monitoring the data loading times, allowing for efficient data management and optimization of load schedules. - Proper configuration can facilitate accurate reporting and analysis of the data loading process. > ℹ️ Ensure that the column name defined by this parameter is correctly configured in your target database schema to prevent any data loading issues. --- ### CDC_PRE_POST_IMAGE_FLAG **Possible Values:** Free Text | **Lowest Level:** [⚪ TAB SRC] **Default Value:** CDC_PRE_POST_IMAGE_FLAG **Dependency:** - CDC_PRE_POST_IMAGE_FLAG is not available - When CDC_PRE_IMAGE_AVAILABLE is set to **N** **Full Description** The CDC_PRE_POST_IMAGE_FLAG parameter is used to identify whether an incoming record in a Change Data Capture (CDC) system represents the pre-image or post-image of a source record. This distinction is crucial for accurately processing changes in systems that provide both pre- and post-change states of data. Pre-image: - Refers to the state of the record before the change occurred. Post-image: - Refers to the state of the record after the change has been applied. Understanding whether a record is a pre-image or post-image is essential for maintaining data integrity and accurate historical records in your data vault. By correctly setting the CDC_PRE_POST_IMAGE_FLAG, VaultSpeed can generate the appropriate code to handle data changes efficiently. - Use this parameter when integrating source systems that provide both pre- and post-images for data changes. - Proper configuration of this flag allows for precise data versioning and audit trails in your data vault architecture. > ℹ️ Ensure that your CDC system is appropriately configured to provide both pre- and post-images if you intend to use this parameter. --- ### CDC_PRE_POST_IMAGE_UNIQUE_KEY **Possible Values:** Free Text | **Lowest Level:** [🟣 SOURCE] **Default Value:** CDC_PRE_POST_IMAGE_UNIQUE_KEY **Dependency:** - CDC_PRE_POST_IMAGE_UNIQUE_KEY is not available - When CDC_PRE_IMAGE_AVAILABLE is set to **N** **Full Description** The CDC_PRE_POST_IMAGE_UNIQUE_KEY parameter defines the attribute name of the unique key used to link pre-image and post-image records in Change Data Capture (CDC) tables within VaultSpeed. This key is essential for correctly associating before-and-after versions of a record, enabling precise change tracking across data loads. By identifying a consistent unique identifier, the parameter ensures that CDC events are accurately correlated, allowing VaultSpeed to interpret data changes with full context. This is particularly important for handling updates where both the original and modified values need to be analyzed or stored. Configure this parameter when working with CDC-enabled sources to maintain data consistency and ensure the integrity of historical change tracking. Proper setup supports reliable comparisons and auditability within your Data Vault model. > ℹ️ This parameter is critical for maintaining data integrity in your data warehouse by correctly linking historical data images. --- ### CDC_LOGPOSITION **Possible Values:** Free Text | **Lowest Level:** [🟣 SOURCE] **Default Value:** CDC_LOGPOSITION **Dependency:** - CDC_LOGPOSITION is not available - When CDC_LOGPOSITION_AVAILABLE is set to **N** **Full Description** The CDC_LOGPOSITION parameter specifies the attribute name used to capture the log position from the source database, which represents the chronological sequence of transactions. This value is critical for ordering events accurately and ensuring consistent processing of changes during CDC-based data ingestion. Accurate configuration of this parameter allows VaultSpeed to track and apply changes in the correct sequence, preventing issues such as out-of-order updates or missed transactions. It plays a key role in maintaining data integrity, especially in high-frequency or near-real-time environments. Make sure the attribute name matches the exact casing used in the source system to avoid mismatches during metadata harvesting or ETL execution. Proper use of CDC_LOGPOSITION enhances the reliability and precision of your CDC pipelines. **Purpose:** - Ensures that data extraction processes can accurately follow the order of transactions, which is critical for maintaining data integrity and consistency during data replication or migration processes. **Usage:** - Set this parameter to match the field name in your source database that holds the transactional sequence information. This allows VaultSpeed to correctly read and apply changes in the proper order when processing data pipelines. **Key ETL code impacts:** - **Column addition** - When CDC_LOGPOSITION_AVAILABLE is set to Y, the generated code includes this attribute in SAT, LKS, LDS, NHL, LAS, PIT, and BVV objects - **Casting behavior** - The attribute can be excluded from automatic casting when CAST_VS_MANAGED_ATTRIBUTES is set to N, since most CDC tools already provide correct data types - **Load date logic** - When USE_CDC_TS_AS_LOAD_DATES is Y, the CDC_LOGPOSITION attribute gets incorporated into the generated expressions for chronological ordering - **Uniqueness guarantee** - Combined with cdc_timestamp, it ensures each CDC record has a unique identifier in the generated ETL logic > ℹ️ This parameter is crucial for Change Data Capture (CDC) processes, where maintaining the correct order of operations on data is vital. --- ### SNAPSHOT_TIMESTAMP **Possible Values:** Free Text | **Lowest Level:** [🔵 PROJECT] **Default Value:** SNAPSHOT_TIMESTAMP **Full Description** The SNAPSHOT_TIMESTAMP parameter defines the name of the column used to store the snapshot date in the PIT (Point-In-Time) tables. This parameter is crucial for tracking the exact moment when the data snapshot was taken, allowing for accurate historical data analysis and point-in-time querying. By setting the SNAPSHOT_TIMESTAMP parameter correctly, VaultSpeed developers can ensure that their data warehouse maintains high accuracy and reliability in time-based queries. **Purpose:** - Identifies the designated column for snapshot dates, ensuring consistent data retrieval across PIT tables. **Importance:** - Essential for maintaining data integrity and enabling time-travel queries within the data vault architecture. **Usage Context:** - Utilized in scenarios where precise temporal data tracking is required, particularly in complex data environments involving multiple source systems and frequent updates. > ℹ️ Ensure the column name specified in the SNAPSHOT_TIMESTAMP parameter is unique and does not conflict with other column names in your PIT tables to avoid potential data retrieval issues. --- ### LNA_HASH_KEY_PREFIX **Possible Values:** Free Text | **Lowest Level:** [🔵 PROJECT] **Default Value:** LNA **Full Description** The LNA_HASH_KEY_PREFIX parameter is used to define a specific prefix for the names of the Hash Key columns in the Link Across Source tables. This prefix is crucial for maintaining a consistent naming convention across your data vault implementation. **Purpose:** - Ensures consistent and identifiable naming for Hash Key columns, which are critical in the Link Across Source tables where relationships between different source systems are managed. **Application:** - This parameter is typically set during the initial configuration of your data vault, and it helps in avoiding naming conflicts and ensuring clarity when handling multiple source systems. **Practical Implications:** - Using a consistent prefix aids in the identification and management of link tables across various stages of your data pipeline, facilitating easier troubleshooting and maintenance by VaultSpeed developers and data engineers. - Ensure the prefix aligns with your organization's naming conventions and project requirements. > ⚠️ It is recommended to choose a prefix that clearly differentiates these Hash Key columns from other columns in your data model. --- ### LNA_HASH_KEY_SUFFIX **Possible Values:** Free Text | **Lowest Level:** [🔵 PROJECT] **Default Value:** HKEY **Full Description** The LNA_HASH_KEY_SUFFIX parameter determines the suffix appended to the names of Hash Key columns within Link Across Source (LNA) tables. This parameter is crucial for ensuring consistent naming conventions across the data warehouse, which aids in both development efficiency and data integrity. This parameter only affects the suffix portion of the Hash Key column names, allowing for customization that aligns with organizational naming standards. > ℹ️ Consistent naming conventions help in maintaining clarity and avoid conflicts in column identification, which is especially important when integrating data from multiple source systems. --- ### ERROR_CODE **Possible Values:** Free Text | **Lowest Level:** [🟣 SOURCE] **Default Value:** error_code **Full Description** The ERROR_CODE parameter designates the attribute name for the column that holds error codes. This is particularly relevant when the REFERENTIAL_INTEGRITY_VALIDATED setting is in use. Specifies the column name where error codes are stored. Essential for tracking and referencing errors related to referential integrity validation. > ℹ️ Using the ERROR_CODE parameter ensures that any discrepancies or issues with referential integrity are properly logged and accessible for debugging and resolution. --- ### SUBSEQUENCE_PREFIX **Possible Values:** Free Text | **Lowest Level:** [🔵 PROJECT] **Full Description** The SUBSEQUENCE_PREFIX parameter is used to define a prefix for subsequence source attributes in multi-active satellites within VaultSpeed. This parameter helps in organizing and identifying attributes that are part of a sequence in multi-active satellite models, which are used to manage different versions or states of data that can be active simultaneously. **Purpose:** - To specify a distinct prefix for subsequence attributes, facilitating easier management and recognition of these elements in the data model. **Usage Context:** - Use this parameter when working with multi-active satellite structures where multiple instances of a source attribute can exist concurrently. Setting a consistent prefix aids in the clarity and maintenance of these attributes. **Practical Implications:** - By setting a clear and unique subsequence prefix, developers can streamline the process of handling multi-active data scenarios, improving both the manageability and readability of the data model. - This parameter is particularly relevant in complex data environments where tracking changes over time while retaining multiple active records is critical. > ⚠️ Ensure that the chosen prefix does not conflict with any existing attribute names to avoid confusion and maintain data integrity. --- ### SUBSEQUENCE_SUFFIX **Possible Values:** Free Text | **Lowest Level:** [🔵 PROJECT] **Default Value:** seq **Full Description** The SUBSEQUENCE_SUFFIX parameter is used to define a suffix for subsequence source attributes within multi-active satellites in VaultSpeed. This parameter is crucial for handling scenarios where multiple instances of the same source attribute exist simultaneously in a data vault, ensuring that each is uniquely identifiable. This suffix is appended to source attributes to differentiate between different active occurrences, which is essential for maintaining data integrity and proper tracking of changes within the data vault patterns. **Purpose:** - To uniquely identify and differentiate multiple active instances of the same source attribute in multi-active satellite structures. **Use Case:** - Utilize this parameter when dealing with source systems that provide multiple active records for the same entity, ensuring that each record can be accurately tracked and managed within the data warehouse. > ℹ️ It's important to configure this parameter correctly to prevent data collision and ensure the accuracy of your data vault patterns. --- ### NHL_HASH_KEY_PREFIX **Possible Values:** Free Text | **Lowest Level:** [🔵 PROJECT] **Default Value:** NHL **Full Description** The NHL_HASH_KEY_PREFIX parameter defines the prefix applied to the names of Hash Key columns in Non-Historized Link (NHL) tables. This prefix is crucial for ensuring consistent naming conventions across your data warehouse, facilitating easier identification and management of these columns. **Purpose:** - Provides a standardized naming pattern for Hash Key columns in NHL tables. **Importance:** - Helps in maintaining consistency and clarity in column names within the data model, which is critical for efficient data processing and management. **Practical Implications:** - Use this parameter to define your organization's naming standards for NHL Hash Key columns. - Consider the impact on existing scripts and processes when changing the prefix, as this may require updates to dependent systems or documentation. > ⚠️ It is recommended to choose a prefix that clearly distinguishes NHL Hash Key columns from other types of columns to avoid confusion during data mapping and processing. --- ### NHL_HASH_KEY_SUFFIX **Possible Values:** Free Text | **Lowest Level:** [🔵 PROJECT] **Default Value:** HKEY **Full Description** The NHL_HASH_KEY_SUFFIX parameter defines the suffix used for naming the Hash Key columns within Non-Historized Link (NHL) tables. This parameter is crucial for ensuring consistent naming conventions across your data vault model, specifically for those tables that do not maintain historical records of changes. **Purpose:** - It standardizes the naming of Hash Key columns, which are essential for uniquely identifying records in NHL tables. **Usage:** - This parameter is set during the configuration of your data vault and affects how Hash Key columns are generated by VaultSpeed. **Practical Implications:** - By setting this parameter, you ensure that all Non-Historized Link tables generated by VaultSpeed will have a uniform naming pattern for their Hash Key columns, aiding in both development and maintenance tasks. > ⚠️ It's important to choose a suffix that aligns with your organization's naming standards to maintain clarity and consistency in your data model. --- ### RECORD_SOURCE **Possible Values:** Free Text | **Lowest Level:** [⚪ DV] **Default Value:** RECORD_SOURCE **Full Description** The RECORD_SOURCE parameter determines the attribute name for the column that stores the record source in the HUB. This parameter is crucial for identifying the origin of data records within the data vault model, allowing users to track and audit data lineage effectively. - Specifies the attribute name for the record source column. - Essential for maintaining data provenance in HUB tables. > ℹ️ Ensure that the RECORD_SOURCE parameter is set correctly to accurately capture the source of data records. This is vital for data governance and auditability. --- ### OBJECT_NAME **Possible Values:** Free Text | **Lowest Level:** [🔵 PROJECT] **Default Value:** OBJECT_NAME **Requirements:** - When OBJECT_SPECIFIC_LOADING_WINDOW is **Y**, this parameter must be **is not null** **Dependency:** - OBJECT_NAME is not available - When OBJECT_SPECIFIC_LOADING_WINDOW is set to **N** **Full Description** The OBJECT_NAME parameter determines the column name dedicated to storing the source object name within the FMC metadata tables. It is used in conjunction with the OBJECT_SPECIFIC_LOADING_WINDOW parameter. **Purpose:** - This parameter is critical for mapping source object names to their respective loading windows in the FMC metadata, ensuring accurate and efficient data processing. **Usage Context:** - Utilize the OBJECT_NAME parameter when configuring source systems that require object-specific loading windows. This allows for tailored data loading strategies based on individual source object needs. **Practical Implications:** - Proper configuration of this parameter is essential for VaultSpeed Developers and Data Engineers aiming to optimize their data pipeline efficiency and accuracy. > ⚠️ Ensure that the OBJECT_NAME column is correctly set up in your FMC metadata tables to maintain synchronization with the OBJECT_SPECIFIC_LOADING_WINDOW parameter. --- ### CDC_SEQUENCE **Possible Values:** Free Text | **Lowest Level:** [⚪ TAB SRC] **Default Value:** CDC_SEQUENCE **Full Description** The CDC_SEQUENCE parameter is crucial for managing sequence-based Change Data Capture (CDC) loading. It specifies the name of the column that stores the sequence number of changes detected in the source system. This parameter ensures that data changes are accurately tracked and loaded in the correct order during the CDC process. Ensure the column name matches the exact casing used in the source system to avoid mismatches or errors during data loading. > ℹ️ The CDC_SEQUENCE parameter is essential for maintaining data integrity and consistency in environments where sequence-based change tracking is implemented. --- ### FMC_BEGIN_LW_SEQUENCE **Possible Values:** Free Text | **Lowest Level:** [🔵 PROJECT] **Default Value:** FMC_BEGIN_LW_SEQUENCE **Full Description** The FMC_BEGIN_LW_SEQUENCE parameter specifies the name of the column used to store the beginning loading window number for the FMC. This is particularly relevant for sources of type Modification Sequence. **Purpose:** - This parameter is crucial for tracking the start of a loading window in the Full Metadata Capture (FMC) process, which is essential in ensuring accurate data processing and historical tracking within the data vault. **When to Use:** - Implement this parameter when working with modification sequence source systems to accurately capture and manage the loading window sequence numbers. > ℹ️ Ensure the column name specified is unique and consistent with your database schema to avoid conflicts and ensure seamless data integration. --- ### FMC_END_LW_SEQUENCE **Possible Values:** Free Text | **Lowest Level:** [🔵 PROJECT] **Default Value:** FMC_END_LW_SEQUENCE **Full Description** The FMC_END_LW_SEQUENCE parameter specifies the name of the column used to store the end loading window number for the FMC (Full Modification Cycle). This parameter is particularly relevant for source systems that utilize a Modification Sequence approach. Ensure that the column name adheres to the naming conventions used within your database environment to maintain consistency and avoid potential conflicts. > ℹ️ The end loading window number is critical for tracking the end of a data load cycle, which is essential in data vault environments where accurate data lineage and load tracking are necessary. --- ### EXISTS_FLAG **Possible Values:** Free Text | **Lowest Level:** [🔵 PROJECT] **Default Value:** exists_flag **Full Description** The EXISTS_FLAG parameter specifies the column name used for tracking the existence of records in satellites within a data vault model. This parameter is crucial as it allows VaultSpeed to manage record tracking efficiently, ensuring that the system accurately reflects which records are present in the source data. **Purpose:** - Utilized in satellites to indicate the presence of a record, crucial for maintaining data integrity and consistency. **Usage Scenario:** - When a satellite table is updated, the EXISTS_FLAG helps determine if a record already exists, allowing for the appropriate actions to be taken (e.g., insert, update, or ignore). > ℹ️ Ensure the EXISTS_FLAG column is consistently named across your model to facilitate automated processes and avoid conflicts. --- ### SK_SUFFIX **Possible Values:** Free Text | **Lowest Level:** [⚪ DV] **Default Value:** SK **Full Description** The SK_SUFFIX parameter in VaultSpeed is used to define the suffix applied to the names of the Shard Key constraints for all generated tables within your data warehouse. This parameter is crucial for ensuring consistent naming conventions and for easily identifying Shard Key constraints across different tables. > ℹ️ The Shard Key is an important aspect of data distribution across shards in a distributed database system, which can significantly impact performance and scalability. --- ### SK_PREFIX **Possible Values:** Free Text | **Lowest Level:** [⚪ DV] **Full Description** The SK_PREFIX parameter defines the prefix applied to Shard Key constraint names in all tables generated by VaultSpeed. This setting is important for enforcing consistent naming conventions across your data vault implementation, making shard key constraints easy to recognize and manage within the database. Using a clear and descriptive prefix helps teams quickly identify shard key constraints, which can enhance collaboration and simplify debugging or documentation. It also supports automation efforts, particularly when processes need to programmatically locate or reference shard keys in large or complex models. To ensure clarity and alignment with organizational standards, choose a prefix that reflects your naming strategy and provides immediate context for developers and data engineers working with the generated schema. **Purpose:** - Simplifies the identification and management of shard key constraints across tables. **Usage:** - Define a consistent naming convention to enhance readability and maintainability. > ℹ️ A consistent SK_PREFIX helps differentiate shard key constraints from other constraints, assisting in debugging and database management processes. --- ### NH_COUNTER **Possible Values:** Free Text | **Lowest Level:** [⚪ TAB SRC] **Default Value:** counter **Dependency:** - NH_COUNTER is not available - When ADD_NH_COUNTER_RECORD_DELETE is set to **N** - NH_COUNTER is not available - When SRC_CDC is set to **No CDC Incremental** - NH_COUNTER is not available - When SRC_CDC is set to **Modification Date Incremental** - NH_COUNTER is not available - When SRC_CDC is set to **Modification Sequence Incremental** **Full Description** The NH_COUNTER parameter is used to define the COUNTER attribute name for non-historic objects within the VaultSpeed platform. This parameter is critical for the proper configuration and management of non-historic object types, which do not maintain historical versions of their data. **Purpose:** - Specifies the attribute name that acts as a counter in non-historic objects, ensuring that data integrity and consistency are maintained across the data vault. **Importance:** - Proper configuration of the NH_COUNTER helps in managing data lineage and ensuring that non-historic objects are processed correctly within the data warehouse. > ℹ️ Using the NH_COUNTER parameter correctly is essential for maintaining the integrity of non-historic data objects. Ensure that this attribute is configured appropriately to avoid issues with data processing and lineage. --- ## Schema names > 📝 **Templating Support is active for each of these parameters** > 📝 > 📝 When creating a source, the string 'PROJECT' within this parameter will automatically be replaced by the actual source name. This feature facilitates dynamic schema naming based on your specific project setup. ### SCHEMA_EXT **Possible Values:** Free Text | **Lowest Level:** [🟣 SOURCE] **Default Value:** PROJECT_EXT **Full Description** The SCHEMA_EXT parameter specifies the schema for the Extraction tables within the VaultSpeed platform. This parameter plays a crucial role in defining how and where your extraction tables are stored, ensuring that data is organized and accessible for further processing. **Target Database Compatibility:** - For specific targets such as Snowflake, Databricks, and Microsoft Fabric, you can include the Database or Catalog name within the schema definition. This is done by separating them with a dot, for example, 'database.schema'. This ensures compatibility and proper organization within these systems. - Use SCHEMA_EXT to maintain consistent and clear schema naming conventions across different projects and source systems. - Proper configuration of this parameter is crucial for ensuring that your data pipelines function correctly and efficiently within VaultSpeed. > ℹ️ Keep in mind that the SCHEMA_EXT parameter is essential for the correct generation of Extraction tables, as it directly influences the organization and accessibility of data within your data warehouse architecture. --- ### SCHEMA_STG **Possible Values:** Free Text | **Lowest Level:** [🟣 SOURCE] **Default Value:** PROJECT_STG **Full Description** The SCHEMA_STG parameter defines the schema to be used for the Staging layer in your data warehouse automation process. This schema is crucial for organizing and storing data as it is initially loaded into the system before further processing or transformation. **Target Database Specifics:** For targets such as Snowflake, Databricks, or Microsoft Fabric, the schema definition can include the Database or Catalog name. This should be specified by separating them with a dot, for example, 'database.schema'. This allows for precise identification and management of database objects within these environments. > ℹ️ Ensure that the schema name complies with your database system's naming conventions to avoid errors during deployment. --- ### SCHEMA_FL **Possible Values:** Free Text | **Lowest Level:** [⚪ DV] **Default Value:** PROJECT_FL **Full Description** The SCHEMA_FL parameter specifies the schema used for the Raw Data Vault tables in VaultSpeed. This parameter is crucial for organizing and managing the location where the Raw Data Vault (RDV) tables are stored within your database system. The schema name can include templating, allowing for dynamic naming. When creating a Data Vault (dv), the string "PROJECT" within the schema definition will be replaced by the dv name. **Target Database Specifics:** For databases like Snowflake, Databricks, or Microsoft Fabric, the schema definition can include both the database or catalog name and the schema name, separated by a dot (e.g., 'database.schema'). > ℹ️ Using the SCHEMA_FL parameter effectively helps ensure that your RDV tables are correctly organized according to your project's naming conventions and database management strategies. --- ### SCHEMA_CDC **Possible Values:** Free Text | **Lowest Level:** [🟣 SOURCE] **Default Value:** PROJECT_CDC **Full Description** The SCHEMA_CDC parameter specifies the schema name for incremental load tables within the VaultSpeed platform. This parameter is crucial for organizing and managing data schemas in target databases, particularly when dealing with change data capture (CDC) processes. **Target Database Specifics:** For targets like Snowflake, Databricks, and Microsoft Fabric, users can include the Database or Catalog name within the schema definition. This is done by separating them with a dot, for example, 'database.schema'. This allows for precise schema targeting and management across different platforms. **Practical Implications:** - Setting the SCHEMA_CDC parameter correctly is essential for effective data warehouse automation. - It helps streamline the CDC processes by clearly defining where incremental load tables should reside, thus facilitating efficient data integration and retrieval. - Incorrect configuration may lead to mismanaged schemas and potential data integration issues. > ⚠️ Ensure that the specified schema names align with your organizational standards and naming conventions to maintain consistency across your data infrastructure. --- ### SCHEMA_INI **Possible Values:** Free Text | **Lowest Level:** [🟣 SOURCE] **Default Value:** PROJECT_INI **Full Description** The SCHEMA_INI parameter is used to specify the schema for initial load tables in VaultSpeed, a key aspect in organizing and managing your data warehouse structure. This parameter supports templating, allowing dynamic schema naming by using the string 'PROJECT', which will automatically be replaced by the source name when creating a source. **Target Database Specifics:** For platforms like Snowflake, Databricks, and Microsoft Fabric, you can include the Database/Catalog name within the schema definition by separating them with a dot, such as 'database.schema'. This flexibility ensures that your data is correctly partitioned and accessible per your architectural requirements. > ℹ️ Using templating with 'PROJECT' helps maintain consistent naming conventions across your data warehouse environments. --- ### SCHEMA_CDCR **Possible Values:** Free Text | **Lowest Level:** [🟣 SOURCE] **Default Value:** PROJECT_CDCR **Full Description** The SCHEMA_CDCR parameter defines the schema for the incremental load tables on the source system. This parameter is critical for scenarios where REMOTE_JOURNALING_TABLES is enabled (set to 'Y'), as it dictates where the change data capture (CDC) tables are created in the source database. **Templating Support:** - When setting up a source, you can use templating by including the string 'PROJECT' which will automatically be replaced by the source name. **Target Compatibility:** - For platforms such as Snowflake, Databricks, or Microsoft Fabric, you can specify the database or catalog name in conjunction with the schema name by separating them with a dot (e.g., 'database.schema'). This parameter is particularly important when configuring your data pipelines to ensure that data is effectively captured and transformed according to your business requirements. Always verify that the schema name aligns with the structure of the source database to avoid integration issues. > ℹ️ Ensure that the correct schema is specified, as this can impact the ability to accurately capture and replicate incremental changes from the source system. --- ### SCHEMA_BV **Possible Values:** Free Text | **Lowest Level:** [⚪ DV] **Default Value:** PROJECT_BV **Full Description** The SCHEMA_BV parameter specifies the name of the schema for the Business Vault tables. This parameter is essential for organizing and managing the data structures within the Business Vault layer of your data warehouse. **Templating Support:** - When creating a data vault, the placeholder "PROJECT" within the schema name will be dynamically replaced by the data vault's name, facilitating consistent naming conventions across projects. **Target Compatibility:** - For target databases like Snowflake, Databricks, and Microsoft Fabric, you can include the Database/Catalog name in the schema definition. This is achieved by separating them with a dot, for example, 'database.schema'. This flexibility is particularly useful for managing multi-tenant environments or complex data structures. **Practical Implications:** - Use this parameter to set up a clear and organized schema structure within your Business Vault. - Leverage templating to streamline the creation of consistent and project-specific schemas. > ℹ️ Ensure that the schema name conforms to the naming conventions of your target database to avoid any compatibility issues. --- ### SCHEMA_PROCEDURES **Possible Values:** Free Text | **Lowest Level:** [🔵 PROJECT] **Default Value:** PROJECT_PROC **Full Description** The SCHEMA_PROCEDURES parameter defines the schema location for deploying all procedures within a VaultSpeed project. This parameter is crucial for organizing and managing where your project's procedures reside, facilitating structured deployment and maintenance. **Target Database Specifics:** For targets such as Snowflake, Databricks, and Microsoft Fabric, you can include the Database or Catalog name in the schema definition. This is done by separating the Database or Catalog name from the schema name with a dot (e.g., 'database.schema'). This allows for precise schema targeting in environments that support hierarchical database structures. > ⚠️ Ensure that the schema name correctly reflects the intended organizational structure and that template placeholders are used consistently to avoid deployment issues. --- ### SCHEMA_FMC **Possible Values:** Free Text | **Lowest Level:** [🔵 PROJECT] **Default Value:** PROJECT_FMC **Full Description** The SCHEMA_FMC parameter specifies the schema name for the Flow Management Control layer in VaultSpeed, a critical component in managing data flows and ensuring proper organization within your data warehouse. This parameter supports templating, allowing for dynamic schema naming by replacing the string "PROJECT" with the actual project name during project creation. This feature enhances flexibility and ensures consistency across different projects. **Target Database Specifics:** For Snowflake, Databricks, and Microsoft Fabric targets, you can define the schema name along with the database or catalog name by separating them with a dot. For example, 'database.schema'. --- ### SCHEMA_FL_MIGRATION **Possible Values:** Free Text | **Lowest Level:** [🔵 PROJECT] **Default Value:** PROJECT_FL **Full Description** The SCHEMA_FL_MIGRATION parameter specifies the schema used for migrating Raw Data Vault tables, particularly for older Data Vaults generated via FA3.0. This parameter is primarily relevant when dealing with legacy data vault structures and allows for a smooth transition to the current schema specified by SCHEMA_FL. **Target Database Specifics:** For targets like Snowflake, Databricks, or Microsoft Fabric, you can include the Database/Catalog name in the schema definition, separating them with a dot, e.g., 'database.schema'. Use this parameter when migrating data from older Data Vault models created with FA3.0 to ensure continuity and integrity of data during schema transitions. Proper configuration of this parameter is crucial for maintaining the correct schema references in your data warehouse automation processes. Generally, the SCHEMA_FL_MIGRATION parameter is not needed unless working with legacy systems. --- ### SCHEMA_MTD **Possible Values:** Free Text | **Lowest Level:** [🟣 SOURCE] **Default Value:** PROJECT_MTD **Full Description** The SCHEMA_MTD parameter specifies the schema used for metadata tables within VaultSpeed. This parameter is essential for defining how metadata is organized in your target database environment, impacting how VaultSpeed generates and manages metadata-related database objects. Configuring the SCHEMA_MTD parameter correctly is crucial for the seamless operation of VaultSpeed's data warehouse automation processes. It ensures that metadata is stored in the appropriate schema, which is necessary for maintaining a well-organized and efficient database structure. **Target Database Specifics:** For target platforms such as Snowflake, Databricks, and Microsoft Fabric, you can specify both the database/catalog and schema by separating them with a dot (e.g., 'database.schema'). This allows for precise control over the location of metadata tables across different database systems. > ℹ️ Ensure that the schema name aligns with your organization's naming conventions and database management practices to avoid conflicts and enhance maintainability. --- ### SCHEMA_DTVR **Possible Values:** Free Text | **Lowest Level:** [🟣 SOURCE] **Default Value:** PROJECT_DTVR **Full Description** The SCHEMA_DTVR parameter defines the schema for the remote Delta Layer and is applicable exclusively when the REMOTE_JOURNALING_TABLES parameter is set to 'Y'. This parameter is crucial for configuring the correct schema that VaultSpeed will use for managing the Delta Layer, particularly in distributed environments. Use this parameter when you need to define or change the schema for remote journaling tables, ensuring that your data vault architecture aligns with your organization's data governance and management strategies. **Target Database Specifics:** For Snowflake, Databricks, and Microsoft Fabric targets, users can specify the Database or Catalog name alongside the schema in the format 'database.schema'. This allows for precise schema management across different database environments. --- ### SCHEMA_MTDR **Possible Values:** Free Text | **Lowest Level:** [🟣 SOURCE] **Default Value:** PROJECT_MTDR **Full Description** The SCHEMA_MTDR parameter defines the schema for remote metadata tables and is essential when REMOTE_JOURNALING_TABLES is enabled ('Y'). This parameter allows for templating, which means that when a source is created, the placeholder 'PROJECT' within the schema name will be automatically replaced by the actual source name. - **When to use:** Only applicable when remote journaling is configured, ensuring that metadata tables are properly organized within the specified schema. - **Target Database Specifics:** For Snowflake, Databricks, or Microsoft Fabric environments, you can specify both the database/catalog and schema in the format 'database.schema'. This parameter is crucial for maintaining structured and accessible metadata management across different environments, facilitating efficient data warehousing operations. > ⚠️ Ensure that the schema name is correctly formatted according to your target database's requirements to avoid any connectivity or organizational issues. --- ### SCHEMA_DFV **Possible Values:** Free Text | **Lowest Level:** [🟣 SOURCE] **Default Value:** PROJECT_DFV **Full Description** The SCHEMA_DFV parameter specifies the schema for the Delta Firewall Layer in VaultSpeed, which is crucial for the organization and management of data within the data warehouse. This parameter supports templating, allowing for dynamic schema naming based on the source system. When a source is created, the string "PROJECT" within the schema name will be automatically replaced by the source name. **Target Database Specifics:** For targets such as Snowflake, Databricks, or Microsoft Fabric, you can include the Database or Catalog name within the schema definition by separating them with a dot (e.g., 'database.schema'). > ℹ️ This parameter is essential for defining the organization of the Delta Firewall Layer, ensuring data consistency and integrity across different source systems and target databases. --- ### SCHEMA_DTV **Possible Values:** Free Text | **Lowest Level:** [🟣 SOURCE] **Default Value:** PROJECT_DTV **Full Description** The SCHEMA_DTV parameter specifies the schema used for the Delta Layer within VaultSpeed. This parameter is crucial for defining the organizational structure and management of the Delta Layer in your data warehouse architecture. **Target Database Specifics:** For data warehouse targets such as Snowflake, Databricks, or Microsoft Fabric, the SCHEMA_DTV parameter allows you to include both the Database/Catalog name and schema name. These should be separated by a dot, for example, 'database.schema', to ensure proper schema resolution in these environments. Use the SCHEMA_DTV parameter to control how and where the Delta Layer is implemented, enhancing flexibility and organization in your data warehouse setup. > ℹ️ Note: Proper configuration of the SCHEMA_DTV parameter ensures the correct setup of the Delta Layer's schema, which is integral for data processing and transformation tasks. Ensure the schema names are appropriately defined to avoid conflicts and ensure seamless data operations. --- ### SCHEMA_INI_R **Possible Values:** Free Text | **Lowest Level:** [🟣 SOURCE] **Default Value:** PROJECT **Full Description** The SCHEMA_INI_R parameter specifies the schema for remote initial load tables. This parameter is essential when source loading through the FMC (Flexible Metadata Connector) is enabled, as it determines where the initial load data is retrieved from. **Target Database Specifics:** for Snowflake, Databricks, and Microsoft Fabric targets, you can include the Database/Catalog name by separating it with a dot (e.g., 'database.schema'). > ⚠️ This parameter is crucial for ensuring correct data retrieval paths during the initial load phase, particularly in multi-database or cloud environments. --- ### SCHEMA_CDC_KAFKA **Possible Values:** Free Text | **Lowest Level:** [🟣 SOURCE] **Default Value:** PROJECT_CDC_KAFKA **Full Description** The SCHEMA_CDC_KAFKA parameter defines the schema for the tables used in incremental streaming loads within VaultSpeed. This parameter is crucial for correctly organizing and accessing streaming data tables when utilizing real-time data ingestion into target databases. - This schema is specifically meant for streaming tables that handle Change Data Capture (CDC) operations. - - **Target Database Specifics:** For target environments like Snowflake, Databricks, and Microsoft Fabric, you can specify the Database or Catalog name along with the schema by separating them with a dot (e.g., 'database.schema'). > ℹ️ Ensure that the schema name is appropriately defined to match the structure and organization of your target database. This helps in maintaining consistency and ease of access across your data warehouse. --- ## Object names > ❌ **Attention on handling empty Prefix parameter setting when only working with Suffixes** > ❌ > ❌ You first need to set and save the suffix before you can edit and save the prefix to an empty value. Don’t try the update in 1 go. That will throw an error indicating that the Prefix can’t be empty. > ❌ > ❌ This has to do with the requirements and dependencies that are set between the pre and the suffix. ### HUB_TABLE_PREFIX **Possible Values:** Free Text | **Lowest Level:** [🔵 PROJECT] **Default Value:** HUB **Dependencies:** - When HUB_TABLE_PREFIX is set to **is null**: - HUB_TABLE_SUFFIX must be **is not null** **Requirements:** - When HUB_TABLE_SUFFIX is **is null**, this parameter must be **is not null** **Full Description** The HUB_TABLE_PREFIX parameter defines the prefix added to the names of Hub tables within your data warehouse schema. This is crucial for organizing and identifying tables as part of the Data Vault architecture, ensuring consistency and clarity in your database structure. **Purpose:** - This parameter helps in differentiating Hub tables from other table types and simplifies table management within SCHEMA_FL and SCHEMA_FL_MIGRATION. **Usage:** - By setting a distinct prefix, developers can easily locate and manage Hub tables, which store unique business keys and their associated metadata. Consider the implications of your chosen prefix on readability and maintenance when designing your data warehouse schema. Proper use of this parameter can significantly enhance the organization and efficiency of your data warehouse operations. > ℹ️ Setting an appropriate HUB_TABLE_PREFIX is essential for maintaining a clear and consistent naming convention across your data warehouse layers. --- ### HUB_TABLE_SUFFIX **Possible Values:** Free Text | **Lowest Level:** [🔵 PROJECT] **Full Description** The HUB_TABLE_SUFFIX parameter is used to define the suffix that will be appended to the names of Hub tables within your data warehouse environment. This parameter is crucial for maintaining consistent naming conventions and ensuring that table names are easily identifiable and organized. - **Stores in:** SCHEMA_FL and SCHEMA_FL_MIGRATION - If not specified, default naming conventions will apply. - Ensure the suffix aligns with your organization's naming standards to avoid confusion. Using a well-defined HUB_TABLE_SUFFIX can aid in automated processes by reducing manual intervention when identifying Hub tables, thus enhancing the efficiency of data management in VaultSpeed. > ℹ️ Setting a consistent suffix for Hub tables can help in identifying and differentiating these tables from other types within the data vault architecture, especially when managing complex environments with multiple layers and pipelines. --- ### LDS_TABLE_PREFIX **Possible Values:** Free Text | **Lowest Level:** [🔵 PROJECT] **Default Value:** LDS **Dependencies:** - When LDS_TABLE_PREFIX is set to **is null**: - LDS_TABLE_SUFFIX must be **is not null** **Requirements:** - When LDS_TABLE_SUFFIX is **is null**, this parameter must be **is not null** **Full Description** The LDS_TABLE_PREFIX parameter defines the prefix that is applied to the names of Satellite tables associated with Many-to-Many Link objects within the data vault model. This parameter is crucial for maintaining a consistent naming convention across the database schemas. - The prefix is applied to tables stored in both SCHEMA_FL and SCHEMA_FL_MIGRATION, which are used for managing historical changes in Many-to-Many relationships. > ℹ️ This parameter is particularly important for VaultSpeed developers and data engineers who need to ensure that their naming conventions align with organizational standards and facilitate easy identification and maintenance of database objects related to complex relationships. --- ### LND_TABLE_PREFIX **Possible Values:** Free Text | **Lowest Level:** [🔵 PROJECT] **Default Value:** LND **Dependencies:** - When LND_TABLE_PREFIX is set to **is null**: - LND_TABLE_SUFFIX must be **is not null** **Requirements:** - When LND_TABLE_SUFFIX is **is null**, this parameter must be **is not null** **Full Description** The LND_TABLE_PREFIX parameter is used to specify the prefix for the names of Object Based Link tables, specifically Many-to-Many Link tables, within VaultSpeed. These tables are stored in the SCHEMA_FL and SCHEMA_FL_MIGRATION schemas. - Defines a standard naming convention for Object Based Link tables, ensuring consistency and organization. - Critical for managing and identifying Many-to-Many Link tables within the data vault architecture. > ℹ️ This parameter is essential for maintaining a clear and consistent naming structure across your data warehouse, particularly when handling complex relationships modeled through Many-to-Many links. --- ### LND_TABLE_SUFFIX **Possible Values:** Free Text | **Lowest Level:** [🔵 PROJECT] **Full Description** The LND_TABLE_SUFFIX parameter defines the suffix appended to the names of Many-to-Many Link tables within VaultSpeed. This setting is crucial for distinguishing these specific tables in your data vault architecture, particularly when managing complex relationships between entities. **Casing Behavior by Setting:** - The suffix is applied consistently across all Object Based Link tables. - It is stored in both the SCHEMA_FL and SCHEMA_FL_MIGRATION schemas, ensuring consistency and traceability across different environments. This parameter matters because it helps in organizing and identifying link tables that handle many-to-many relationships, a common scenario in data integration tasks. Set the LND_TABLE_SUFFIX when you need to clearly distinguish these tables from other table types in your architecture. > ⚠️ It's important to choose a suffix that aligns with your organization's naming conventions to maintain clarity and consistency across your data warehouse schema. --- ### LNK_TABLE_PREFIX **Possible Values:** Free Text | **Lowest Level:** [🔵 PROJECT] **Default Value:** LNK **Dependencies:** - When LNK_TABLE_PREFIX is set to **is null**: - LNK_TABLE_SUFFIX must be **is not null** **Requirements:** - When LNK_TABLE_SUFFIX is **is null**, this parameter must be **is not null** **Full Description** The LNK_TABLE_PREFIX parameter is used to set the prefix for the names of Link tables within the VaultSpeed platform. This prefix is applied to the Link tables stored in SCHEMA_FL and SCHEMA_FL_MIGRATION. - Ensure that the prefix aligns with your organization's naming conventions to maintain consistency across the data warehouse. - If using multiple environments or schemas, consider a prefix that reflects the specific environment to avoid confusion. > ℹ️ Link tables are an integral part of the Data Vault architecture, acting as associative entities that facilitate many-to-many relationships between Hubs. Setting a meaningful prefix helps in organizing and identifying these tables efficiently within the database schema. --- ### LNK_TABLE_SUFFIX **Possible Values:** Free Text | **Lowest Level:** [🔵 PROJECT] **Full Description** The LNK_TABLE_SUFFIX parameter determines the suffix appended to the names of Link tables in your data warehouse schema. It is crucial for ensuring the consistency and organization of table names, particularly in environments where multiple schemas or versions are maintained. This suffix is applied to tables stored within both SCHEMA_FL and SCHEMA_FL_MIGRATION schemas. - Ensure the suffix is consistent with your naming conventions to maintain clarity across different layers and pipelines. --- ### LKS_TABLE_PREFIX **Possible Values:** Free Text | **Lowest Level:** [🔵 PROJECT] **Default Value:** LKS **Dependencies:** - When LKS_TABLE_PREFIX is set to **is null**: - LKS_TABLE_SUFFIX must be **is not null** **Requirements:** - When LKS_TABLE_SUFFIX is **is null**, this parameter must be **is not null** **Full Description** The LKS_TABLE_PREFIX parameter defines the prefix applied to the names of Satellite on Link tables within VaultSpeed. This parameter is crucial for maintaining consistent naming conventions across your data warehouse, specifically for Satellite on Link tables, which are stored in SCHEMA_FL and SCHEMA_FL_MIGRATION. - Ensures uniformity in table naming to facilitate easier identification and management. - Important for distinguishing Satellite on Link tables from other tables in your schema. > ⚠️ It is recommended to choose a prefix that aligns with your organization's naming standards and is easily recognizable to all team members. --- ### NHL_TABLE_PREFIX **Possible Values:** Free Text | **Lowest Level:** [🔵 PROJECT] **Default Value:** NHL **Dependencies:** - When NHL_TABLE_PREFIX is set to **is null**: - NHL_TABLE_SUFFIX must be **is not null** **Requirements:** - When NHL_TABLE_SUFFIX is **is null**, this parameter must be **is not null** **Full Description** The NHL_TABLE_PREFIX parameter defines the prefix used for naming Non-Historical Link (NHL) tables within VaultSpeed. These tables are stored in the schemas SCHEMA_FL and SCHEMA_FL_MIGRATION. **Purpose:** - This parameter is used to maintain consistency and clarity in the naming conventions of NHL tables, which are part of the data vault architecture. **Usage Context:** - Set this parameter to a specific prefix to easily identify and manage NHL tables across different schemas. This is particularly useful in environments with multiple pipelines or when integrating with various source systems. **Practical Implications:** - Properly setting the NHL_TABLE_PREFIX helps in organizing your data vault tables, facilitating easier maintenance, and ensuring adherence to your organization's naming standards. > ⚠️ Ensure that the prefix chosen does not conflict with existing table names in the target database to avoid any potential naming collisions. --- ### NHL_TABLE_SUFFIX **Possible Values:** Free Text | **Lowest Level:** [🔵 PROJECT] **Full Description** The NHL_TABLE_SUFFIX parameter defines the suffix appended to the names of Non-Historical Link (NHL) tables in your data warehouse. This parameter is crucial for ensuring consistency and clarity in your database schema, especially when managing multiple environments or migrations. - **Purpose**: To uniquely identify Non-Historical Link tables by appending a specific suffix to their names. **Stored In:** - SCHEMA_FL - SCHEMA_FL_MIGRATION **Practical Implications:** - Use this parameter when setting up or modifying Non-Historical Link tables to ensure they are easily distinguishable from other table types. - It is particularly useful in scenarios where consistent naming conventions are required across different deployment stages or database versions. > ⚠️ Properly setting this parameter helps maintain a clear and organized schema, which is vital for efficient data management and retrieval in complex data warehouse environments. --- ### SAT_TABLE_PREFIX **Possible Values:** Free Text | **Lowest Level:** [🔵 PROJECT] **Default Value:** SAT **Dependencies:** - When SAT_TABLE_PREFIX is set to **is null**: - SAT_TABLE_SUFFIX must be **is not null** **Requirements:** - When SAT_TABLE_SUFFIX is **is null**, this parameter must be **is not null** **Full Description** The SAT_TABLE_PREFIX parameter is crucial for naming Satellite tables in your data vault architecture. This parameter allows you to define a custom prefix for these tables, ensuring consistency and clarity across your data warehouse. **Purpose:** - Specifies the prefix for the names of Satellite tables. **Applies to:** - Tables stored in SCHEMA_FL and SCHEMA_FL_MIGRATION. **Practical Implications:** - Choose a meaningful prefix that aligns with your organization's naming conventions. - This setting is particularly useful when managing multiple data sources or environments, as it provides a clear, standardized approach to table naming. > ℹ️ Using a consistent prefix helps in organizing and managing Satellite tables, making it easier to identify and differentiate them from other tables within your database schema. --- ### SAT_TABLE_SUFFIX **Possible Values:** Free Text | **Lowest Level:** [🔵 PROJECT] **Full Description** The SAT_TABLE_SUFFIX parameter is used to define the suffix that will be appended to the names of Satellite tables. This is crucial for maintaining consistency and clarity in your data warehouse schema. Satellite tables are stored in the SCHEMA_FL and SCHEMA_FL_MIGRATION schemas, and this parameter helps to easily identify and differentiate these tables. **Purpose:** - Specify a consistent naming convention for Satellite tables. **Applies to:** - Only Satellite tables within the SCHEMA_FL and SCHEMA_FL_MIGRATION schemas. **Usage:** - Ideal when you have multiple Satellite tables and need a clear, systematic way to manage their names. This parameter is particularly useful in environments with complex data models, allowing VaultSpeed developers to quickly recognize Satellite tables and streamline their workflow. Consistent use of SAT_TABLE_SUFFIX can enhance the organization of database objects and facilitate easier maintenance and troubleshooting of the data vault architecture. > ℹ️ Ensure that the suffix chosen does not conflict with any existing table names or naming conventions in your database to avoid potential conflicts. --- ### LKS_TABLE_SUFFIX **Possible Values:** Free Text | **Lowest Level:** [🔵 PROJECT] **Full Description** The LKS_TABLE_SUFFIX parameter is used to define the suffix for the names of Satellite on Link tables within the VaultSpeed platform. This parameter ensures consistency and clarity in naming conventions, particularly in the database schemas SCHEMA_FL and SCHEMA_FL_MIGRATION. - Adjusting this parameter allows developers to customize the naming patterns of these tables, which can be critical for maintaining organization and readability in complex data environments. > ℹ️ It is important to set this parameter thoughtfully, as consistent naming conventions aid in the management and navigation of database objects. --- ### LDS_TABLE_SUFFIX **Possible Values:** Free Text | **Lowest Level:** [🔵 PROJECT] **Full Description** The LDS_TABLE_SUFFIX parameter is used to define the suffix appended to the names of Satellite tables associated with Object-Based or Many-to-Many Link tables within the VaultSpeed data warehouse automation platform. - This parameter specifically affects the naming of these tables in both SCHEMA_FL and SCHEMA_FL_MIGRATION schemas. - By setting this parameter, you ensure a consistent naming convention for Satellite tables, which aids in better organization and identification within your data warehouse structure. > ℹ️ This parameter is crucial for maintaining clarity and avoiding naming conflicts in environments where multiple Satellite tables may exist for various link types. --- ### TEMP_TABLE_PREFIX **Possible Values:** Free Text | **Lowest Level:** [🔵 PROJECT] **Full Description** The TEMP_TABLE_PREFIX parameter defines the prefix applied to the names of temporary tables utilized in transformation logic within VaultSpeed. Temporary tables are essential for intermediate data processing and transformations before final loading into target databases. By setting this parameter, you can ensure that all temporary tables follow a consistent naming convention, which aids in organization and debugging. > ℹ️ Using a distinct prefix helps differentiate temporary tables from permanent tables and can prevent naming conflicts in the database. --- ### TEMP_TABLE_SUFFIX **Possible Values:** Free Text | **Lowest Level:** [🔵 PROJECT] **Default Value:** TMP **Dependencies:** - When TEMP_TABLE_SUFFIX is set to **is null**: - TEMP_TABLE_PREFIX must be **is not null** **Requirements:** - When TEMP_TABLE_PREFIX is **is null**, this parameter must be **is not null** **Full Description** The TEMP_TABLE_SUFFIX parameter defines the suffix appended to the names of temporary tables utilized in transformation logic within VaultSpeed. This parameter is crucial for distinguishing these tables from other database objects and ensuring clear identification in complex data processing workflows. - Ensure consistency in table naming conventions by applying a uniform suffix. - Facilitate easier management and troubleshooting of transformation processes by clearly marking temporary tables. - Use this parameter when setting up or adjusting transformation logic to maintain clarity and organization in your data warehouse architecture. - Consider your organization's naming conventions and consistency requirements when determining the appropriate suffix value. > ℹ️ Using a consistent TEMP_TABLE_SUFFIX can aid in maintaining an organized database environment, particularly when multiple temporary tables are generated. --- ### CDC_TABLE_PREFIX **Possible Values:** Free Text | **Lowest Level:** [🟣 SOURCE] **Default Value:** CDC **Full Description** > ℹ️ Ensure that the specified prefix does not conflict with existing table names to prevent any potential naming collisions. --- ### CDC_TABLE_SUFFIX **Possible Values:** Free Text | **Lowest Level:** [🟣 SOURCE] **Full Description** The CDC_TABLE_SUFFIX parameter defines the suffix added to the source table name in the context of Change Data Capture (CDC). This is applicable when the CDC table name is distinct from the imported source table name, ensuring consistent naming conventions across your data pipeline. **Purpose:** - To append a specific suffix to source tables stored within the SCHEMA_CDC schema when their CDC table names differ from their original imported source names. **Usage Context:** - This parameter is essential for maintaining clarity and organization in your data mapping and transformation processes, particularly in environments with complex table naming requirements. > ℹ️ Ensure that the suffix used is consistent with your organization's naming conventions to facilitate easier data management and pipeline operations. --- ### LOAD_CYCLE_INFO_TABLE_NAME **Possible Values:** Free Text | **Lowest Level:** [🔵 PROJECT] **Default Value:** LOAD_CYCLE_INFO **Full Description** The LOAD_CYCLE_INFO_TABLE_NAME parameter defines the name of the table that holds details about the load cycle within your data warehouse. This table is critical for tracking the status and metadata of each load cycle, ensuring transparency and traceability in data processing operations. **Key Features:** - The table is stored in the schemas SCHEMA_MTD and SCHEMA_MTDR. - It serves as a central repository for load cycle information, which is vital for auditing and debugging data load processes. > ℹ️ It is recommended to choose a descriptive and consistent naming convention for this table to maintain clarity and ease of management across different environments. --- ### DFV_TABLE_SUFFIX **Possible Values:** Free Text | **Lowest Level:** [🟣 SOURCE] **Full Description** The DFV_TABLE_SUFFIX parameter defines the suffix appended to the Delta Firewall view within the specified schema (SCHEMA_DFV). This parameter is crucial for maintaining consistent naming conventions across your data warehouse, ensuring that Delta Firewall views are easily identifiable and organized. - **Casing Behavior by Setting:** - The suffix is case-sensitive, so ensure consistency in casing when setting this parameter. **Example:** - If set to 'DFV', the view will be named accordingly (e.g., 'tablename_DFV'). > ℹ️ Using a consistent DFV_TABLE_SUFFIX helps in quickly identifying Delta Firewall views across different schemas and pipelines, aiding in efficient data management and troubleshooting. --- ### DFV_TABLE_PREFIX **Possible Values:** Free Text | **Lowest Level:** [🟣 SOURCE] **Default Value:** VW **Dependencies:** - When DFV_TABLE_PREFIX is set to **is null**: - DFV_TABLE_SUFFIX must be **is not null** **Requirements:** - When DFV_TABLE_SUFFIX is **is null**, this parameter must be **is not null** **Full Description** The DFV_TABLE_PREFIX parameter defines the prefix applied to the Delta Firewall views within the data warehouse schema (SCHEMA_DFV). This prefix is essential for differentiating Delta Firewall views from other database objects, ensuring clarity and organization in the schema. **Key Points for DFV_TABLE_PREFIX:** - Specifies the prefix for Delta Firewall views - Helps in organizing and identifying views within SCHEMA_DFV - Important for maintaining consistent naming conventions across the data warehouse Using a clear and logical prefix for your Delta Firewall views can significantly enhance the manageability of your data warehouse by making it easier to locate and reference these views during development and maintenance activities. > ℹ️ Ensure that the chosen prefix aligns with your organization's naming standards to maintain consistency and avoid conflicts with existing objects. --- ### MTD_EXCEPTION_RECORDS_TABLE_NAME **Possible Values:** Free Text | **Lowest Level:** [🟣 SOURCE] **Default Value:** MTD_EXCEPTION_RECORDS **Full Description** The MTD_EXCEPTION_RECORDS_TABLE_NAME parameter defines the name of the table where exception records are stored within the SCHEMA_MTD. This parameter is crucial for managing data exceptions that occur during the data processing and transformation phases. By specifying a distinct table name for exception records, VaultSpeed allows for organized tracking and resolution of data issues. This aids in maintaining data integrity and ensuring that anomalies can be addressed promptly. **Key Details:** - The table is located within SCHEMA_MTD. - Exception records might include data anomalies, validation errors, or any discrepancies identified during data loading. --- ### FMC_LOADING_WINDOW_TABLE_NAME **Possible Values:** Free Text | **Lowest Level:** [🔵 PROJECT] **Default Value:** FMC_LOADING_WINDOW_TABLE **Full Description** The FMC_LOADING_WINDOW_TABLE_NAME parameter defines the name of the table used to store loading window dates within the VaultSpeed platform. This table is essential for managing and tracking the timeframes in which data loads are executed. **The table is stored in the schemas:** - SCHEMA_MTD and SCHEMA_MTDR. - Ensure that the table name specified in this parameter matches the actual table name in your database schema to avoid data load errors. - Regularly verify the loading window table to ensure it reflects the current data loading schedules and requirements. > ℹ️ This parameter is crucial for ensuring accurate and efficient data loads, as it allows VaultSpeed to determine the correct window of time for executing data load operations. Developers should configure this parameter to align with their system's schema setup to maintain data integrity and consistency. --- ### REF_TABLE_SUFFIX **Possible Values:** Free Text | **Lowest Level:** [🔵 PROJECT] **Full Description** The REF_TABLE_SUFFIX parameter determines the suffix appended to the names of Reference tables within the VaultSpeed platform. - This parameter is crucial for ensuring consistent naming conventions across your data warehouse, particularly in the Reference layer. **These tables are stored in two schemas:** - SCHEMA_FL and SCHEMA_FL_MIGRATION. **Practical Implication:** - Use a meaningful suffix that reflects the role or content of the Reference tables to enhance readability and maintainability of your database objects. > ℹ️ Proper configuration of the REF_TABLE_SUFFIX helps maintain clarity and organization in your data architecture, especially when managing multiple pipelines and mappings. --- ### REF_TABLE_PREFIX **Possible Values:** Free Text | **Lowest Level:** [🔵 PROJECT] **Default Value:** REF **Full Description** The REF_TABLE_PREFIX parameter is critical for defining the naming convention of Reference tables within your data warehouse environment. This parameter sets the prefix applied to Reference table names, ensuring consistency and clarity across different schemas. - **Purpose:** Specifies the prefix for the names of Reference tables, which are stored in SCHEMA_FL and SCHEMA_FL_MIGRATION. - **Importance:** Using a consistent prefix helps in easily identifying and managing Reference tables, particularly when dealing with multiple schemas or environments. - **Application:** This parameter is essential when configuring the setup of Reference tables within VaultSpeed's automated data warehouse generation, providing a systematic approach to table naming. > ⚠️ It is recommended to choose a prefix that reflects the nature or function of the Reference tables to enhance readability and maintenance. --- ### FMC_LOADING_HIST_TABLE_NAME **Possible Values:** Free Text | **Lowest Level:** [🔵 PROJECT] **Default Value:** FMC_LOADING_HISTORY **Full Description** The FMC_LOADING_HIST_TABLE_NAME parameter is crucial for tracking and managing the execution history of loads performed by the Flow Management Control (FMC). This parameter specifies the table that stores comprehensive metadata about every load operation executed using the FMC. **Key metadata stored includes:** - Load cycle ID - Load date - Loading window - Success flag - Directed Acyclic Graph (DAG) name > ℹ️ This parameter is vital for auditing and troubleshooting load operations, allowing developers and data engineers to understand the performance and outcome of each load cycle. It is especially useful when analyzing load patterns, identifying issues, or optimizing load processes. --- ### PIT_TABLE_PREFIX **Possible Values:** Free Text | **Lowest Level:** [🔵 PROJECT] **Default Value:** PIT **Full Description** The PIT_TABLE_PREFIX parameter determines the prefix to be used for the names of the Point-In-Time (PIT) tables within your data vault architecture. By setting this parameter, you provide a consistent naming convention that aids in the organization and identification of PIT tables, which are crucial for efficiently managing historical data snapshots and time-variant data processing. - Ensures a uniform prefix is applied, aiding in the organization and searchability of PIT tables in your database. - Supports clear differentiation of PIT tables from other table types, enhancing data model clarity. > ℹ️ Setting an appropriate PIT_TABLE_PREFIX can simplify database maintenance and improve the clarity of your data warehouse schema. Consider using a descriptive prefix that aligns with your organizational naming standards. --- ### PIT_TABLE_SUFFIX **Possible Values:** Free Text | **Lowest Level:** [🔵 PROJECT] **Full Description** The PIT_TABLE_SUFFIX parameter defines the suffix appended to the names of PIT (Point-In-Time) tables within the VaultSpeed platform, which are a crucial component in data vault modeling for providing a historical view of data states at specific points in time. **Purpose:** - By setting a consistent suffix for PIT tables, it ensures uniformity and easy identification across your data warehouse, facilitating efficient data management and query operations. **Usage:** - This parameter is particularly important when multiple PIT tables are generated, as it helps avoid naming conflicts and supports clear organization within your database schema. This parameter is essential for developers and data engineers who are configuring data vaults and need to maintain a systematic approach to table naming conventions. Adjust the suffix according to your project's naming standards and requirements. > ⚠️ Ensure that the chosen suffix does not conflict with other naming conventions in your database to maintain clarity and prevent potential issues during data retrieval. --- ### FMC_BV_LOADING_WINDOW_TABLE_NAME **Possible Values:** Free Text | **Lowest Level:** [🔵 PROJECT] **Default Value:** FMC_BV_LOADING_WINDOW_TABLE **Full Description** The FMC_BV_LOADING_WINDOW_TABLE_NAME parameter specifies the name of the table containing the BV loading window dates. This parameter is crucial for defining where the loading window dates for Business Vault (BV) are stored within your data warehouse. **Purpose:** - Identifies the exact table that maintains the loading window dates, which are essential for processing and managing data in the Business Vault layer. **Importance:** - Proper configuration ensures that data is loaded within the correct time frames, aligning with the data vault's architectural requirements. **Practical Implications:** - Use this parameter when setting up or modifying the Business Vault structures in your data warehouse. - An incorrect table name can lead to errors in data processing and inaccurate data reporting, making this configuration step vital for maintaining data integrity and consistency. > ⚠️ Ensure that the table name is correctly set to avoid any discrepancies during the data loading process in the BV layer. --- ### BRIDGE_TABLE_PREFIX **Possible Values:** Free Text | **Lowest Level:** [🔵 PROJECT] **Default Value:** BRIDGE **Full Description** The BRIDGE_TABLE_PREFIX parameter determines the prefix that will be applied to the names of all Bridge tables within your VaultSpeed data warehouse projects. This prefix helps in organizing and identifying Bridge tables easily across your database objects. **Purpose:** - By setting a consistent prefix, developers can standardize the naming conventions for Bridge tables, which are critical components in data vault modeling, facilitating easier management and maintenance of these tables. **Recommended Usage:** - Use a prefix that clearly identifies the table as part of the Bridge layer, such as 'BR_' or your organization's specific naming conventions. **Practical Implications:** - Implementing a well-defined prefix strategy improves data warehouse readability and consistency, which is crucial for documentation, troubleshooting, and collaboration among data engineering teams. > ℹ️ Ensure that the chosen prefix does not conflict with other prefixes used in your data warehouse to avoid naming collisions. --- ### BRIDGE_TABLE_SUFFIX **Possible Values:** Free Text | **Lowest Level:** [🔵 PROJECT] **Full Description** The BRIDGE_TABLE_SUFFIX parameter defines the suffix appended to the names of Bridge tables in the data warehouse. This parameter is crucial for ensuring consistent naming conventions across all Bridge tables, facilitating easy identification and management within the VaultSpeed platform. **Purpose:** - To standardize the naming of Bridge tables by adding a specific suffix, which aids in maintaining a clear and organized data model. **When to Use:** - Set this parameter when configuring your data warehouse to ensure all Bridge tables follow your organization's naming standards. Adjusting this parameter is particularly useful when integrating with existing systems or adhering to specific project naming guidelines. > ⚠️ Changing the BRIDGE_TABLE_SUFFIX after tables have been created may require additional steps to update table names and dependencies within the data warehouse. --- ### LNA_TABLE_PREFIX **Possible Values:** Free Text | **Lowest Level:** [🔵 PROJECT] **Default Value:** LNA **Full Description** The LNA_TABLE_PREFIX parameter defines the prefix used for naming Link Across Source tables within VaultSpeed. This prefix helps in organizing and identifying tables that manage link relationships across different source systems. **Purpose:** - To standardize and easily identify tables facilitating link relationships across various sources in your data warehouse. **When to Use:** - Set this parameter when you need a consistent naming convention for Link Across Source tables to ensure clarity and organization. > ⚠️ Adjusting this prefix can impact the readability and management of database objects. Ensure that the prefix aligns with your organization's naming standards and conventions. --- ### LNA_TABLE_SUFFIX **Possible Values:** Free Text | **Lowest Level:** [🔵 PROJECT] **Full Description** The LNA_TABLE_SUFFIX parameter determines the suffix appended to the names of Link Across Source tables within your VaultSpeed project. This parameter is crucial for maintaining consistent naming conventions and ensuring clear identification of these tables across different systems. **Purpose:** - To specify a suffix for Link Across Source tables, aiding in organization and identification. **Usage:** - Utilize this parameter when you need to distinguish Link Across Source tables from other table types, especially in environments with multiple source systems. - Ensures that all Link Across Source tables can be easily identified by their suffix, aiding in both automated processing and manual interventions. - Helps avoid naming conflicts and enhances the readability of the data model for data engineers and architects. > ℹ️ It is recommended to choose a meaningful suffix that reflects the table's role or the source system's identity to enhance clarity and maintainability. --- ### LAS_TABLE_PREFIX **Possible Values:** Free Text | **Lowest Level:** [🔵 PROJECT] **Default Value:** LAS **Full Description** The LAS_TABLE_PREFIX parameter is used to specify the prefix for the names of Satellite on Link Across Source tables in VaultSpeed. This parameter is crucial for maintaining consistent naming conventions across your data warehouse, especially when dealing with multiple source systems. - Use a consistent casing for prefixes to ensure uniformity across database objects. > ⚠️ Adjust the LAS_TABLE_PREFIX to align with your organization's naming standards. This aids in easily identifying and managing tables linked across different source systems. --- ### LAS_TABLE_SUFFIX **Possible Values:** Free Text | **Lowest Level:** [🔵 PROJECT] **Full Description** The LAS_TABLE_SUFFIX parameter determines the suffix added to the names of the 'Satellite on Link Across Source' tables in your data warehouse environment. This parameter plays a crucial role in maintaining consistent naming conventions across your database objects, which is essential for clarity and organization. **Purpose:** - It specifies the suffix that will be appended to Satellite on Link Across Source table names. **Importance:** - Consistent table naming helps in identifying and managing tables, especially in complex data warehouse environments involving multiple data sources. > ⚠️ Ensure that the chosen suffix aligns with your organization's naming standards to facilitate easier table identification and management. --- ### ERR_TABLE_PREFIX **Possible Values:** Free Text | **Lowest Level:** [🟣 SOURCE] **Full Description** The ERR_TABLE_PREFIX parameter defines the prefix used for naming error tables in VaultSpeed. These tables are crucial for temporarily storing records that cannot be resolved when REFERENTIAL_INTEGRITY_VALIDATED is enabled. **Purpose:** - To manage unresolved records efficiently by storing them in designated error tables. **When to Use:** - This parameter is essential when you need to handle referential integrity issues during data processing. Ensure you have a consistent naming convention for easy identification and management of these tables. > ℹ️ Error tables play a critical role in debugging and resolving data integrity issues, providing a mechanism to track and address problematic records without halting the entire data pipeline. --- ### ERR_TABLE_SUFFIX **Possible Values:** Free Text | **Lowest Level:** [🟣 SOURCE] **Default Value:** err **Full Description** The ERR_TABLE_SUFFIX parameter is used to define the suffix for error tables in VaultSpeed. These tables are essential for temporarily storing records that cannot be resolved when referential integrity is validated. - The ERR_TABLE_SUFFIX value is appended to the error table name to distinguish it from other tables. > ℹ️ Referential integrity ensures that relationships between tables in a database remain consistent. When enabled, unresolved records are directed to error tables for further analysis. --- ### USE_IDENTIFIER_PREFIXES **Possible Values:** Y / N | **Lowest Level:** [🔵 PROJECT] **Default Value:** Y **Full Description** The USE_IDENTIFIER_PREFIXES parameter determines the placement of short names for sources and PIT (Point-In-Time) tables within the full names of database objects in VaultSpeed. This setting is crucial for maintaining consistency and clarity in naming conventions across your data warehouse automation workflows. **If USE_IDENTIFIER_PREFIXES = Y:** - The short name of the source or PIT will be inserted immediately after the table type prefix in the full name. **If USE_IDENTIFIER_PREFIXES = N:** - The short name will be appended just before the table type suffix. > ℹ️ This parameter helps in managing the readability and organization of database objects, which is particularly important in environments with multiple source systems or complex data mappings. Adjust this setting based on your organization's naming conventions and the clarity required for data lineage and auditing purposes. --- ### SUBSEQUENCE_SEQ **Possible Values:** Free Text | **Lowest Level:** [🔵 PROJECT] **Default Value:** subsequence_seq **Full Description** The SUBSEQUENCE_SEQ parameter defines the naming convention for the subsequence column in multi-active satellites that lack inherent subsequence attributes from the source system. This parameter is crucial for ensuring that data is accurately sequenced in scenarios where multiple records for the same business entity coexist at the same time period. - When a multi-active satellite does not have a source subsequence attribute, this parameter determines the default name of the subsequence column. - Use this parameter when configuring multi-active satellites where sequence information is not available from the source. - Helps in maintaining data integrity by providing a custom sequence mechanism, thereby enabling correct data lineage and historical tracking in the Data Vault model. > ℹ️ Ensure that the subsequence column name aligns with your organization's naming conventions to maintain consistency across different data structures. --- ### STREAMING_EXT_TABLE_PREFIX **Possible Values:** Free Text | **Lowest Level:** [🟣 SOURCE] **Default Value:** ext **Full Description** The STREAMING_EXT_TABLE_PREFIX parameter defines the prefix used for naming the extraction late table in Spark streaming processes. This prefix is crucial for handling referential integrity issues that may arise during data extraction in streaming environments. **Purpose:** - To resolve referential integrity issues by appropriately prefixing extraction late tables in streaming contexts. **Casing Behavior by Setting:** - Prefixes should be applied consistently to maintain uniformity and prevent conflicts. > ℹ️ This parameter is specifically designed for use with Spark streaming extraction processes, ensuring that table naming conventions align with the requirements for maintaining data integrity. --- ### STREAMING_EXT_TABLE_SUFFIX **Possible Values:** Free Text | **Lowest Level:** [🟣 SOURCE] **Default Value:** late **Full Description** The STREAMING_EXT_TABLE_SUFFIX parameter defines the suffix appended to the table name of the extraction late table used in Spark streaming processes. This suffix is crucial for resolving referential integrity issues within the data warehouse. **Purpose:** - To ensure that table names are unique and to aid in maintaining referential integrity across streaming data operations. **Usage Context:** - Primarily used in environments where Spark streaming is employed to handle real-time data, and where maintaining the consistency and integrity of reference data is critical. > ℹ️ This parameter is particularly important for VaultSpeed developers and data engineers working with streaming data pipelines that involve complex data relationships. --- ### STREAMING_STG_TABLE_PREFIX **Possible Values:** Free Text | **Lowest Level:** [🟣 SOURCE] **Default Value:** stg **Full Description** The STREAMING_STG_TABLE_PREFIX parameter defines the prefix applied to the table name of the staging tables specifically used in streaming processes. This parameter plays a critical role in resolving referential integrity issues that may arise during the streaming data workflows in VaultSpeed. - This prefix is applied uniformly to all streaming staging tables, ensuring consistency across your data pipeline. --- ### STREAMING_STG_TABLE_SUFFIX **Possible Values:** Free Text | **Lowest Level:** [🟣 SOURCE] **Default Value:** fix **Full Description** The STREAMING_STG_TABLE_SUFFIX parameter defines the suffix added to the table name of the staging table when dealing with streaming data. This suffix is crucial for resolving referential integrity issues that may arise during data integration processes. - This parameter is specifically used in the context of Spark streaming operations within VaultSpeed - It ensures that staging tables are uniquely identified and managed, facilitating the maintenance of data integrity across different layers and pipelines > ℹ️ This parameter is particularly important for VaultSpeed Developers and Data Engineers who are implementing streaming data solutions. It helps prevent conflicts and maintains referential integrity in high-volume, real-time data processing environments. --- ### FMC_OBJECT_LOADING_HIST_TABLE_NAME **Possible Values:** Free Text | **Lowest Level:** [🔵 PROJECT] **Default Value:** FMC_OBJECT_LOADING_HISTORY **Full Description** The **FMC_OBJECT_LOADING_HIST_TABLE_NAME** parameter specifies the table that stores the history of all executed loads for a specific object within the Functional Metadata Configuration (FMC). This parameter is crucial for tracking and auditing data loads in the Functional Layer (FL), providing visibility into load operations and their outcomes. When you configure this parameter, you are defining where VaultSpeed will log the historical load data, which is essential for maintaining a detailed audit trail and ensuring data integrity. --- ### RTS_TABLE_PREFIX **Possible Values:** Free Text | **Lowest Level:** [🔵 PROJECT] **Default Value:** RTS **Dependencies:** - When RTS_TABLE_PREFIX is set to **is null**: - EXISTS_FLAG_NEGATIVE_VALUE must be **is not null** **Requirements:** - When EXISTS_FLAG_NEGATIVE_VALUE is **is null**, this parameter must be **is not null** **Full Description** The RTS_TABLE_PREFIX parameter specifies the prefix used for naming Record Tracking Satellite tables within the VaultSpeed platform. This parameter plays a crucial role in the consistent and organized naming conventions of database objects, particularly when dealing with the complexities of data vault implementations. **Purpose:** - To provide a standardized prefix for Record Tracking Satellite tables, ensuring easy identification and management. **Usage:** - Define a unique and meaningful prefix to differentiate Record Tracking Satellite tables from other tables in your data warehouse. This is especially important in environments with multiple pipelines or source systems to avoid naming conflicts. **When configuring this parameter, consider the following Practical Implications:** **Naming Conventions:** - Adopting a systematic approach to prefix names can streamline the process of navigating and managing tables. **Collaboration:** - Agreeing on a naming convention with your team can enhance collaboration and reduce errors. This parameter is essential for VaultSpeed developers, data engineers, and solution architects who are responsible for setting up and managing data vault patterns efficiently. > ℹ️ Setting an appropriate RTS_TABLE_PREFIX helps maintain clarity and consistency across your data vault architecture, aiding in both development and maintenance activities. --- ### RTS_TABLE_SUFFIX **Possible Values:** Free Text | **Lowest Level:** [🔵 PROJECT] **Full Description** The RTS_TABLE_SUFFIX parameter specifies the suffix appended to the names of Record Tracking Satellite tables in your data vault. This parameter is crucial for identifying and managing Record Tracking Satellite tables within your data warehouse architecture. By using a consistent suffix, you can easily distinguish these tables from other types of tables in your database. - **Purpose:** Helps in organizing and maintaining clarity in your data model by clearly differentiating Record Tracking Satellite tables. - **Usage:** Set the RTS_TABLE_SUFFIX when configuring your data vault to ensure uniform naming conventions. This is particularly useful when you have multiple types of tables and need a reliable way to identify Record Tracking Satellites. > ℹ️ Ensure that the suffix used is unique and does not conflict with other table name conventions in your database. --- ### STG_TABLE_CURR_PREFIX **Possible Values:** Free Text | **Lowest Level:** [🟣 SOURCE] **Full Description** The STG_TABLE_CURR_PREFIX parameter defines the prefix to be appended to the names of staging tables designated for storing the current dataset copy. This is particularly applicable in scenarios involving Full Load processes, as well as Full Load using Modification Date/Sequence with Delete Management. **Purpose:** - To ensure the correct naming convention for staging tables, facilitating easy identification and management of current dataset copies within the data warehouse. **Usage Context:** - Use this parameter when implementing Full Load operations or when managing data with modification tracking and deletion capabilities. **Practical Implications:** - Correctly configuring the STG_TABLE_CURR_PREFIX ensures that your staging tables are systematically named, which aids in debugging, monitoring, and managing data flows within your VaultSpeed environment. > ⚠️ Ensuring a consistent prefix helps maintain organization and clarity in the data warehouse, especially when dealing with extensive datasets and complex ETL processes. --- ### STG_TABLE_CURR_SUFFIX **Possible Values:** Free Text | **Lowest Level:** [🟣 SOURCE] **Default Value:** CURR **Dependencies:** - When STG_TABLE_CURR_SUFFIX is set to **is null**: - STG_TABLE_CURR_PREFIX must be **is not null** **Requirements:** - When STG_TABLE_CURR_PREFIX is **is null**, this parameter must be **is not null** **Full Description** The STG_TABLE_CURR_SUFFIX parameter is used to define the suffix that will be appended to the name of the staging table specifically for storing the current dataset copy. This is particularly relevant when utilizing Full Load or Full Load with Modification Date/Sequence in conjunction with Delete Management. By specifying a unique suffix, users can ensure that the staging tables are easily identifiable and managed efficiently. > ⚠️ This parameter is applicable only in scenarios involving Full Load processes, enhancing clarity in table naming conventions and aiding in the streamlined management of staging tables. --- ### STG_TABLE_PREV_PREFIX **Possible Values:** Free Text | **Lowest Level:** [🟣 SOURCE] **Full Description** The STG_TABLE_PREV_PREFIX parameter is used to specify the suffix that will be appended to the name of a staging table designated for storing the previous dataset copy. This is particularly relevant for scenarios involving Full Load operations, including those utilizing Modification Date/Sequence with Delete Management. **Purpose:** - To manage and track previous data versions in staging tables during Full Load processes. **Usage Context:** - Useful when implementing Full Load strategies that require maintaining historical data states for auditing or recovery purposes. - Ensures data integrity and historical tracking by maintaining previous data versions. - Facilitates data recovery and auditing by storing previous datasets effectively. > ⚠️ This parameter is specifically applicable to Full Load and Full Load operations using Modification Date/Sequence with Delete Management. --- ### STG_TABLE_PREV_SUFFIX **Possible Values:** Free Text | **Lowest Level:** [🟣 SOURCE] **Default Value:** PREV **Dependencies:** - When STG_TABLE_PREV_SUFFIX is set to **is null**: - STG_TABLE_PREV_PREFIX must be **is not null** **Requirements:** - When STG_TABLE_PREV_PREFIX is **is null**, this parameter must be **is not null** **Full Description** The STG_TABLE_PREV_SUFFIX parameter defines the suffix appended to the staging table name used for storing a previous dataset copy. This is specifically relevant for Full Load and Full Load operations that utilize Modification Date/Sequence with Delete Management. **Purpose:** - This suffix helps in identifying the staging table intended for holding the previous version of datasets, which is crucial for tracking changes and managing data integrity during load operations. **Use Case:** - • When conducting a Full Load, the parameter ensures that the staging table for the previous dataset is easily distinguishable, facilitating effective data management and rollback capabilities if needed. - In scenarios involving Modification Date/Sequence with Delete Management, this parameter becomes essential for maintaining a historical dataset version and managing deletes efficiently. **Practical Implications:** - Ensure that this parameter is set appropriately to maintain clear naming conventions and avoid potential conflicts or data integrity issues during data processing. > ℹ️ It is crucial to define a meaningful and consistent suffix to avoid confusion and ensure smooth data operations in your data warehouse environment. --- ## Temporary Attribute Names ### DATA_ORDER **Possible Values:** Free Text | **Lowest Level:** [⚪ DV] **Default Value:** DUMMY **Full Description** The DATA_ORDER parameter is a temporary integer attribute crucial for determining the sequence in which data is processed during the loading phase in VaultSpeed. This parameter specifies the attribute name used to order data, ensuring that the data loading process follows a defined and repeatable sequence. **Key Points:** - DATA_ORDER is used within templates to calculate the correct order of data processing. - It is an integer value, which makes it optimal for numerical sorting and sequencing tasks. By using DATA_ORDER, VaultSpeed developers and data engineers can manage and optimize their data pipelines more effectively, ensuring that data dependencies are respected and the overall data processing workflow is efficient and reliable. > ℹ️ The DATA_ORDER parameter is particularly important in scenarios where the sequence of data loading impacts the integrity or performance of the data warehouse. It is typically used when precise control over the order of operations is necessary. --- ### SNAPSHOT_TS **Possible Values:** Free Text | **Lowest Level:** [⚪ DV] **Default Value:** SNAPSHOT_TS **Full Description** The SNAPSHOT_TS parameter specifies the attribute name used for the snapshot timestamp within the PIT (Point-In-Time) templates. This temporary timestamp attribute is crucial for separately storing the SNAPSHOT_TIMESTAMP for individual Satellites in your data vault. **Purpose:** - To define a unique attribute for capturing snapshot timestamps, enabling accurate historical tracking and data versioning in data vault modeling. **Usage in PIT Templates:** - It is utilized within PIT templates to manage and reference snapshot timestamps for Satellites, ensuring temporal consistency and reliability. - When defining the SNAPSHOT_TS, ensure that the attribute name aligns with your organization's naming conventions and casing policies. > ℹ️ Ensure that the SNAPSHOT_TS is correctly configured to avoid potential discrepancies in data retrieval and historical analysis. This parameter is critical for maintaining the integrity of temporal data views within the data vault architecture. --- ### MIN_LOAD_TIMESTAMP **Possible Values:** Free Text | **Lowest Level:** [⚪ TAB SRC] **Default Value:** MIN_LOAD_TIMESTAMP **Full Description** The MIN_LOAD_TIMESTAMP parameter serves a critical role in data warehouse automation by temporarily storing the minimum load timestamp within VaultSpeed templates. This parameter ensures that the earliest point in time for data loading processes is captured and available for further operations, which is essential for tracking data changes accurately. > ℹ️ The MIN_LOAD_TIMESTAMP parameter is crucial for ensuring data integrity by maintaining accurate records of the earliest data load times within your data warehouse. This parameter is particularly important for developers and data engineers focused on optimizing data loading and transformation processes. --- ### MAX_LOAD_TIMESTAMP **Possible Values:** Free Text | **Lowest Level:** [⚪ TAB SRC] **Default Value:** MAX_LOAD_TIMESTAMP **Full Description** The MAX_LOAD_TIMESTAMP parameter is a temporary timestamp attribute used within VaultSpeed templates to store the maximum value for the load timestamp attribute. This parameter is crucial for tracking the most recent data load events, ensuring that only the latest data updates are processed in subsequent workflows. > ℹ️ This parameter is particularly important in scenarios where data is loaded incrementally, as it helps in identifying the most recent updates to be ingested into the data warehouse. --- ### ORIGIN_ID **Possible Values:** Free Text | **Lowest Level:** [⚪ TAB SRC] **Default Value:** ORIGIN_ID **Full Description** The ORIGIN_ID parameter is a temporary integer attribute used in templates to store the origin of a record. This parameter is vital for tracking and managing the source of data as it flows through the data warehouse automation processes. **Purpose:** - Identifies the origin of each record, facilitating tracking and troubleshooting within data pipelines. **Usage:** - Implemented within templates to ensure each record's source is accurately captured and stored. **Type:** - Integer, temporary attribute. **Practical Implications:** - Proper use of the ORIGIN_ID parameter enhances data traceability, making it easier for VaultSpeed Developers and Data Engineers to manage data origins and streamline data audits within the platform. - This parameter is crucial when dealing with complex data integrations and transformations across multiple source systems. > ⚠️ Ensure that the ORIGIN_ID is correctly configured to maintain data lineage and integrity across the data vault. --- ### END_INTERVAL **Possible Values:** Free Text | **Lowest Level:** [⚪ TAB SRC] **Default Value:** END_INTERVAL **Full Description** The END_INTERVAL parameter is crucial for defining the endpoint of a timestamp interval within VaultSpeed's data processing templates. This parameter temporarily holds the timestamp marking the end of data intervals, which is important for effectively managing data timelines and ensuring accurate data processing. **Purpose:** - Store the end of the timestamp interval temporarily in templates. By utilizing the END_INTERVAL parameter, VaultSpeed developers can ensure that data is processed with correct temporal boundaries, enhancing data accuracy and integrity in target databases. > ⚠️ This parameter is typically used in scenarios where precise interval handling is required, such as in data vault pattern implementations or when setting up pipelines that involve time-based data transformations. --- ### GENERAL_ORDER **Possible Values:** Free Text | **Lowest Level:** [⚪ TAB SRC] **Default Value:** GENERAL_ORDER **Full Description** The GENERAL_ORDER parameter is a temporary integer attribute utilized within VaultSpeed templates to establish an ordering column for data processing. This parameter plays a crucial role in determining the sequence in which data is organized and processed within the data warehouse automation processes. **Purpose:** - Used to maintain a specific order of data, which can be critical when the sequence of data processing affects the outcome or when subsequent operations depend on a particular order. **When to Use:** - Implement the GENERAL_ORDER parameter when there is a need to explicitly control the order of data records during the generation of code or within pipelines. > ℹ️ The GENERAL_ORDER parameter does not affect the final data storage but is essential for intermediate processing steps where order matters. --- ### AGG_HASH_DIFF **Possible Values:** Free Text | **Lowest Level:** [⚪ TAB SRC] **Default Value:** agg_hash_diff **Full Description** The AGG_HASH_DIFF parameter is a temporary attribute used in VaultSpeed to store the aggregated hash difference for multi-active satellites. This is particularly relevant when implementing the logic for satellites without subsequence attributes. It plays a crucial role in managing data consistency and integrity within your data vault architecture. - Utilized in multi-active satellites to handle multiple active records per business key. - Ensures data consistency by aggregating hash differences. - Use AGG_HASH_DIFF when working with multi-active satellites to maintain accurate record aggregation. - This parameter is essential for scenarios where subsequence attributes are not applicable, ensuring seamless data processing and storage in your data warehouse. > ℹ️ The AGG_HASH_DIFF parameter is specifically designed for scenarios where subsequence attributes are not used, allowing for efficient handling of multi-active satellite records. --- ### COUNT_MULTI_ACTIVE **Possible Values:** Free Text | **Lowest Level:** [⚪ TAB SRC] **Default Value:** count_multi_active **Full Description** The COUNT_MULTI_ACTIVE parameter is a temporary attribute in VaultSpeed used specifically for multi-active satellites. It functions to store the number of records associated with each key in scenarios where there are no subsequence attributes. This parameter is crucial for managing data within the multi-active satellite pattern, allowing for the efficient tracking of multiple records that share the same business key. **Purpose:** - To count records per key for multi-active satellites **Applicability:** - Used when subsequence attributes are not present > ⚠️ COUNT_MULTI_ACTIVE is only applicable when working with multi-active satellites that do not utilize subsequence attributes. It is important for VaultSpeed developers to understand this context to ensure accurate data processing and satellite management. --- # Parameters Found ONLY at SOURCE Level ## Technology dependent parameters ### DATABASE_OGG **Possible Values:** Free Text | **Lowest Level:** [🟣 SOURCE] **Default Value:** OGG **Full Description** The DATABASE_OGG parameter specifies the name of the Oracle GoldenGate database used in your data warehouse automation process. This parameter is crucial for identifying and integrating the specific GoldenGate instance with VaultSpeed's data pipelines. **Purpose:** - This parameter ensures that data replication and integration tasks are correctly aligned with the designated Oracle GoldenGate database. **When to Use:** - Configure this parameter when setting up or modifying a pipeline that involves Oracle GoldenGate as a source or intermediary system. By correctly setting the DATABASE_OGG parameter, VaultSpeed developers can facilitate seamless data flow and maintain consistency across the data vault architecture. > ℹ️ Ensure that the provided database name matches the GoldenGate instance you're working with to avoid connectivity issues. --- ### GENERATE_OGG_METADATA **Possible Values:** Y / N | **Lowest Level:** [🟣 SOURCE] **Default Value:** N **Full Description** The GENERATE_OGG_METADATA parameter controls whether metadata and functions specific to Oracle GoldenGate (OGG) are generated within VaultSpeed. This parameter is essential for users who integrate Oracle GoldenGate with their data warehouse to enable real-time data replication. By setting this parameter, developers can ensure that the necessary metadata and functions are correctly generated and managed in the VaultSpeed environment, facilitating seamless data flow and consistency across systems. > ℹ️ Oracle GoldenGate (OGG) is a comprehensive software package for real-time data integration and replication. It allows data to be moved, transformed, and synchronized across multiple systems in real-time. --- ### SCHEMA_OGG **Possible Values:** Free Text | **Lowest Level:** [🟣 SOURCE] **Default Value:** PROJECT_OGG **Full Description** The SCHEMA_OGG parameter specifies the name of the Oracle Golden Gate database schema used within VaultSpeed. This parameter is essential when configuring data pipelines involving Oracle Golden Gate, as it determines the schema under which the data operations are executed and managed. **Templating Support:** - This parameter supports templating. When setting up a source, the placeholder "PROJECT" within the schema name will be dynamically replaced by the actual source name, allowing for flexible and automated schema naming conventions. > ℹ️ Ensure that the schema name follows the naming conventions required by both Oracle Golden Gate and your organizational standards to avoid conflicts and ensure smooth operation of data pipelines. --- # Parameters Found ONLY at DATA VAULT Level ## Technology dependent parameters ### ETL_GENERATION_TYPE **Possible Values:** Free Text | **Lowest Level:** [⚪ DATAVAULT] **Default Value:** SQL **Dependencies:** - When ETL_GENERATION_TYPE is set to **DBT**: - USE_MERGE_STATEMENT must be **Y** - When ETL_GENERATION_TYPE is set to **STREAM2VAULT**: - ADD_RECORD_SOURCE_ATTRIBUTE must be **Y** **Requirements:** - When DATABASE_TYPE is **SINGLESTORE**, this parameter must be **SQL** - When DATABASE_TYPE is **SNOWFLAKE**, this parameter must be **SQL/STREAM2VAULT/MATILLION/DBT** - When DATABASE_TYPE is **SQLSERVER**, this parameter must be **SQL/STREAM2VAULT** - When DATABASE_TYPE is **GREENPLUM**, this parameter must be **SQL/TALEND/STREAM2VAULT** - When DATABASE_TYPE is **POSTGRESQL**, this parameter must be **SQL/TALEND/STREAM2VAULT** - When DATABASE_TYPE is **APACHE HIVE**, this parameter must be **SQL/TALEND/STREAM2VAULT** - When DATABASE_TYPE is **ORACLE**, this parameter must be **SQL/ODI/TALEND/STREAM2VAULT** - When DATABASE_TYPE is **BIGQUERY**, this parameter must be **SQL/STREAM2VAULT** - When DATABASE_TYPE is **EXASOL**, this parameter must be **SQL/STREAM2VAULT** - When DATABASE_TYPE is **AZURESYNAPSE**, this parameter must be **SQL/STREAM2VAULT/MATILLION** - When DATABASE_TYPE is **APACHE SPARK**, this parameter must be **SQL/STREAM2VAULT/DBT** - When DATABASE_TYPE is **FABRIC**, this parameter must be **SQL** - When DATABASE_TYPE is **REDSHIFT**, this parameter must be **SQL** - When ETL_GENERATION_TYPE is set to **STREAM2VAULT**: - ADD_RECORD_SOURCE_ATTRIBUTE must be **Y** **Full Description** The ETL_GENERATION_TYPE parameter specifies the framework used to generate ETL (Extract, Transform, Load) code within the VaultSpeed platform. This parameter is crucial for determining the methodology and structural rules applied when building and managing the data processing pipelines. **Purpose:** - Sets the ETL framework, impacting how data is extracted, transformed, and loaded into the target database. **Importance:** - Directly influences the efficiency and scalability of data pipelines, ensuring that the generated code aligns with the specific requirements of the project's architecture. > ℹ️ Ensure the selected ETL generation type is compatible with your organization's data processing standards and target database capabilities. --- ### FOLDER_SPLIT **Possible Values:** Free Text | **Lowest Level:** [⚪ DATAVAULT] **Default Value:** 100 **Full Description** The FOLDER_SPLIT parameter determines the maximum number of mappings that can be included in a single folder within Oracle Data Integrator (ODI). This parameter is crucial for managing and organizing mappings efficiently, ensuring that folders do not become overly populated and remain manageable. - Specifies the limit of mappings per folder in ODI - A smaller value results in more folders with fewer mappings, which can simplify navigation and management. - A larger value allows more mappings per folder, reducing the total number of folders but potentially increasing complexity within each folder. > ℹ️ Setting an appropriate FOLDER_SPLIT value helps maintain optimal organization and performance in your data integration tasks by preventing folders from becoming too large and unwieldy. --- ### GENERAL_DATE_FORMAT **Possible Values:** Free Text | **Lowest Level:** [⚪ DATAVAULT] **Default Value:** DD/MM/YYYY **Dependency:** - GENERAL_DATE_FORMAT is not available - When DATABASE_TYPE is set to **SQLSERVER** - GENERAL_DATE_FORMAT is not available - When DATABASE_TYPE is set to **AZUREDWH** **Full Description** The GENERAL_DATE_FORMAT parameter defines the format in which system dates will be represented within VaultSpeed. This is critical for ensuring consistency and accuracy across all date-related data processing and transformations within your data warehouse environment. - The format is specified as a string, such as 'DD/MM/YYYY'. - **DD/MM/YYYY**: Day-Month-Year format, widely used in many regions. - **MM/DD/YYYY**: Month-Day-Year format, commonly used in the United States. - **YYYY/MM/DD**: Year-Month-Day format, often used in databases for consistency. **GENERAL_DATE_FORMAT_ID ** - This parameter specifies the format ID for dates, particularly for SQL Server implementations. It ensures that dates are correctly formatted according to the predefined schema. - Consistent date formatting is essential for accurate data mapping and integration across different layers and systems. - Incorrect date formats can lead to data mismatches, processing delays, and potential errors in data reporting. > ℹ️ Make sure to choose a date format that aligns with your organization's data standards to prevent any data discrepancies or processing errors. > ❌ Changing the date format parameter after initial setup may require additional data validation and testing to ensure all systems interpret the dates correctly. --- ### GENERAL_NUMBER_FORMAT **Possible Values:** Free Text | **Lowest Level:** [⚪ DATAVAULT] **Default Value:** 9999999999999D9999999999 **Dependency:** - GENERAL_NUMBER_FORMAT is not available - When DATABASE_TYPE is set to **SQLSERVER** - GENERAL_NUMBER_FORMAT is not available - When DATABASE_TYPE is set to **AZUREDWH** **Full Description** The GENERAL_NUMBER_FORMAT parameter specifies the format that will be used for representing numbers within VaultSpeed. This parameter ensures consistency in how numbers are displayed and processed across different components of the data warehouse automation platform. **Example format:** - 9999999999D9999999999 This setting is particularly important when dealing with numerical data from various source systems, as it helps standardize the data before it is loaded into the target database. By defining a consistent number format, users can avoid discrepancies and potential errors in data handling and transformation. Ensure that the format specified in GENERAL_NUMBER_FORMAT aligns with your organization’s data standards and the requirements of any downstream systems that will consume the data. Adjustments to this parameter should be made cautiously, as changes can impact how numbers are interpreted and processed throughout the pipeline. > ⚠️ This parameter is crucial for maintaining uniformity in number representation, which can affect data processing, reporting, and integration with other systems. --- ### GENERAL_TIMESTAMP_FORMAT **Possible Values:** Free Text | **Lowest Level:** [⚪ DATAVAULT] **Default Value:** DD/MM/YYYY HH24:MI:SS **Dependency:** - GENERAL_TIMESTAMP_FORMAT is not available - When DATABASE_TYPE is set to **SQLSERVER** - GENERAL_TIMESTAMP_FORMAT is not available - When DATABASE_TYPE is set to **AZUREDWH** **Full Description** The GENERAL_TIMESTAMP_FORMAT parameter defines the format used for all timestamps within VaultSpeed, ensuring consistency across data pipelines, mappings, and layers. By setting this parameter, you determine how date and time data is formatted when managed by the platform. **Format Example:** - DD/MM/YYYY HH:MI:SS.FF AM This setting is crucial for maintaining uniformity in timestamp representation, facilitating data integration across different source systems and target databases. It plays a vital role in data warehouse automation by standardizing timestamp data, which is essential for accurate data tracking and reporting. - Consistent timestamp formatting aids in data quality and integrity. - Misalignment with database requirements can lead to data import/export issues. - It's generally recommended to confirm the format with the team responsible for database management and data integration processes. > ⚠️ Ensure that the chosen format aligns with the requirements of your target database system and any data governance policies in place. --- ### USE_MERGE_STATEMENT **Possible Values:** Y / N | **Lowest Level:** [⚪ DATAVAULT] **Default Value:** N **Requirements:** - When DATABASE_TYPE is **POSTGRESQL**, this parameter must be **N** - When DATABASE_TYPE is **GREENPLUM**, this parameter must be **N** - When DATABASE_TYPE is **APACHE HIVE**, this parameter must be **N** - When ETL_GENERATION_TYPE is **DBT**, this parameter must be **Y** - When DATABASE_TYPE is **SINGLESTORE**, this parameter must be **N** - When DATABASE_TYPE is **FABRIC**, this parameter must be **N** - When DATABASE_TYPE is **REDSHIFT**, this parameter must be **N** **Dependency:** - USE_MERGE_STATEMENT is not available - When DATABASE_TYPE is set to **POSTGRESQL** - USE_MERGE_STATEMENT is not available - When DATABASE_TYPE is set to **APACHE HIVE** - USE_MERGE_STATEMENT is not available - When DATABASE_TYPE is set to **AZUREDWH** - USE_MERGE_STATEMENT is not available - When DATABASE_TYPE is set to **17** - USE_MERGE_STATEMENT is not available - When DATABASE_TYPE is set to **GREENPLUM** - USE_MERGE_STATEMENT is not available - When DATABASE_TYPE is set to **REDSHIFT** **Full Description** The USE_MERGE_STATEMENT parameter determines whether Merge SQL statements are utilized in the ETL logic instead of traditional updates. This influences how data is managed during the loading process in VaultSpeed, affecting performance and compatibility with different database systems. **Functionality:** - This parameter controls the use of Merge SQL statements in the data warehouse automation process. **When to Use:** - Consider enabling this option if your target database supports Merge statements, as it can optimize the data integration process by combining insert and update operations. - **Casing Behavior by Setting:** **When set to Y (Yes):** - Merge SQL statements will be used for data integration. **When set to N (No):** - Separate Insert SQL statements will be utilized, which may be necessary for some databases that do not support Merge. > ❌ This parameter may not be available for all target database types. Verify compatibility with your specific database system before enabling Merge statements. --- ### GENERAL_CHAR_FORMAT **Possible Values:** Free Text | **Lowest Level:** [⚪ DATAVAULT] **Default Value:** CHAR **Full Description** The GENERAL_CHAR_FORMAT parameter determines the format type for character data and is specifically applicable to Oracle databases. - This parameter is crucial for ensuring that character data is stored and processed correctly in Oracle environments, aligning with the specific character handling requirements of the database system. Understanding and setting the GENERAL_CHAR_FORMAT appropriately helps maintain data integrity and optimizes performance when handling character data in Oracle databases. Utilize this parameter when you are working with Oracle to ensure that all character data adheres to the specified format, mitigating potential issues related to character storage and retrieval. > ℹ️ This parameter is only relevant for Oracle databases. Ensure that your database type matches this requirement before configuring the parameter. --- ### GENERAL_TIME_FORMAT **Possible Values:** Free Text | **Lowest Level:** [⚪ DATAVAULT] **Default Value:** HH24:MI:SS **Dependency:** - GENERAL_TIME_FORMAT is not available - When DATABASE_TYPE is set to **SQLSERVER** - GENERAL_TIME_FORMAT is not available - When DATABASE_TYPE is set to **AZUREDWH** **Full Description** The GENERAL_TIME_FORMAT parameter specifies the format used for representing time values across the VaultSpeed platform. This parameter is critical for ensuring consistent time representation in generated code, data handling, and database object management. **Purpose:** - Defines the formatting standard for time values, ensuring uniformity across different layers and processes within VaultSpeed. **Usage:** - Set this parameter to match the expected time format of your target database or to adhere to organizational standards for time representation. **Impact:** - Consistency in time format helps avoid errors in data processing and improves the readability of generated scripts. > ⚠️ Ensure that the format specified is compatible with your target database's expected time format. --- ### TRIM_FUNCTION_AVAILABLE **Possible Values:** Y / N | **Lowest Level:** [⚪ DATAVAULT] **Default Value:** N **Full Description** The TRIM_FUNCTION_AVAILABLE parameter determines the availability of the TRIM function in the Data Vault (DV) database, specifically for SQL Server environments. This parameter is crucial for ensuring code compatibility and functionality, as certain SQL Server versions may lack native support for the TRIM function. By accurately configuring this parameter, VaultSpeed can generate appropriate Data Definition Language (DDL) scripts that either utilize the native TRIM function or create a custom implementation using LTRIM and RTRIM functions. **When set to Y (Yes):** - Assumes the TRIM function is available natively in the DV database, and no additional function is generated. **When set to N (No):** - Generates a custom TRIM function during DDL generation, combining LTRIM and RTRIM to mimic the TRIM functionality. > ⚠️ This parameter is only relevant for SQL Server. Ensure your SQL Server version's capabilities are checked before setting this parameter. --- ### CAST_NUMBER_WITH_FORMAT_FOR_HASH **Possible Values:** Y / N | **Lowest Level:** [⚪ DATAVAULT] **Default Value:** Y **Full Description** The CAST_NUMBER_WITH_FORMAT_FOR_HASH parameter determines if a specific format should be used when converting numbers to strings in Oracle databases. This parameter is crucial for ensuring the consistency of string generation from numerical data, which is vital for maintaining data integrity in hash-based processes. **Purpose:** - This parameter controls the formatting applied during the conversion of numbers to strings, affecting how numerical data is represented as strings in Oracle environments. **Functionality:** - When enabled, a specified format is applied, impacting string generation and ensuring uniformity across data transformations. **Use Case:** - Activate this setting when precise numerical formatting is required for string conversions, particularly in scenarios where hashing processes depend on predictable string outputs. > ℹ️ Enabling this parameter is recommended if your data warehouse processes rely heavily on hash keys for data integrity and consistency. --- ### CAST_TO_NVARCHAR_IN_HASH **Possible Values:** Y / N | **Lowest Level:** [⚪ DATAVAULT] **Default Value:** N **Dependencies:** - When CAST_TO_NVARCHAR_IN_HASH is set to **Y**: - CAST_TO_VARCHAR_IN_HASH must be **N** **Requirements:** - When CAST_TO_VARCHAR_IN_HASH is **Y**, this parameter must be **N** **Full Description** The CAST_TO_NVARCHAR_IN_HASH parameter determines whether NVARCHAR data types are used in hash key calculations, which can lead to discrepancies in hash values between SQL Server/Synapse and other database technologies. This setting is particularly relevant when dealing with NVARCHAR business keys. **Setting Details:** **When set to Y (Yes):** - NVARCHAR data types are used, which may be necessary for certain business keys. - However, this will result in different hash values when comparing SQL Server/Synapse with other platforms. **When set to N (No):** - standard data types are used, ensuring consistent hash values across different platforms. - Use this parameter when you have NVARCHAR business keys and require their specific handling in SQL Server/Synapse. - Consider the impact on cross-platform consistency and test thoroughly to ensure data integrity is maintained. > ❌ Setting this parameter to 'Y' can cause hashes to be different between SQL Server/Synapse and other technologies, potentially impacting data consistency and integration. --- ### CAST_TO_VARCHAR_IN_HASH **Possible Values:** Y / N | **Lowest Level:** [⚪ DATAVAULT] **Default Value:** Y **Dependencies:** - When CAST_TO_VARCHAR_IN_HASH is set to **Y**: - CAST_TO_NVARCHAR_IN_HASH must be **N** **Requirements:** - When CAST_TO_NVARCHAR_IN_HASH is **Y**, this parameter must be **N** **Full Description** The CAST_TO_VARCHAR_IN_HASH parameter ensures consistent hash calculation across different database technologies, such as SQL Server/Synapse and others. When this parameter is set to 'Y', it forces the system to cast data to VARCHAR before hashing, thereby standardizing the hash values generated across various platforms. This is crucial for maintaining data integrity and consistency in environments where multiple database systems are used. **Default Value:** - N (No) **Recommended Setting:** - Set to 'Y' if you are working with multiple database technologies to ensure hash value consistency. > ℹ️ Setting CAST_TO_VARCHAR_IN_HASH to 'Y' is particularly important in data vault implementations where consistent hash values are critical for linking records across different systems. --- ### DENODO_DATASOURCE_NAME **Possible Values:** Free Text | **Lowest Level:** [⚪ DATAVAULT] **Full Description** The **DENODO_DATASOURCE_NAME** parameter specifies the name of the datasource within Denodo where data vault objects, including both data vault and business vault objects, are stored. This parameter is crucial for ensuring that VaultSpeed can accurately connect to and manage the data vault structures within your Denodo environment. **Purpose:** - Establishes the link between VaultSpeed and the Denodo datasource, enabling the automation platform to interact with and manipulate the data vault objects. **Usage:** - Set this parameter to the exact name of the datasource as configured in your Denodo environment to ensure seamless integration and data flow management. **Practical Implications:** - Proper configuration of this parameter allows VaultSpeed developers to automate and streamline the management of data vaults, facilitating efficient data processing and storage within the Denodo platform. > ⚠️ Ensure the datasource name matches exactly with the Denodo configuration to avoid connectivity issues. --- ### GENERATE_DENODO_WRAPPERS_AND_BASE_VIEWS **Possible Values:** Y / N | **Lowest Level:** [⚪ DATAVAULT] **Default Value:** N **Full Description** The 'GENERATE_DENODO_WRAPPERS_AND_BASE_VIEWS' parameter in VaultSpeed is designed to automatically create wrappers and base views for each Data Vault and Business Vault object within your data warehouse in Denodo. This feature ensures seamless integration and representation of your data vault structures in Denodo, facilitating efficient data virtualization and access. **Purpose:** - Automates the creation of Denodo views for Data Vault objects. **Scope:** - Applies to both Data Vault and Business Vault layers. **Outcome:** - Enhances data accessibility and management in Denodo by generating wrappers and base views automatically. > ℹ️ This parameter is essential for users looking to leverage Denodo's data virtualization capabilities with their Data Vault structures, ensuring that all necessary views are created without manual intervention. --- ### MAX_QUERY_SIZE **Possible Values:** Free Text | **Lowest Level:** [⚪ DATAVAULT] **Default Value:** 15 **Full Description** The MAX_QUERY_SIZE parameter in VaultSpeed defines the maximum number of Common Table Expressions (CTEs) that can be added to a single transformation job in Matillion. This setting is crucial for managing and optimizing the performance of your data transformation processes. **Purpose:** - To prevent SQL compilation errors related to memory exhaustion by limiting the number of CTEs processed at once. **Recommended Action:** **If you encounter errors such as 'SQL compilation error:** - Compilation memory exhausted,' consider reducing the MAX_QUERY_SIZE value. **Practical Implications:** - By setting an appropriate MAX_QUERY_SIZE, you can ensure your data transformation jobs run smoothly without overwhelming system resources, thus avoiding potential errors and improving overall data processing performance. > ℹ️ Adjusting the MAX_QUERY_SIZE can help maintain stability and efficiency in your data transformation jobs, especially when dealing with complex queries or large datasets. --- ### GENERAL_BINARY_FORMAT **Possible Values:** Free Text | **Lowest Level:** [⚪ DATAVAULT] **Default Value:** HEX **Full Description** The GENERAL_BINARY_FORMAT parameter specifies the default format used for conversions to binary data within VaultSpeed. This parameter is critical for ensuring consistent and accurate data transformations when handling binary data types. **Purpose:** - To set a standardized format for converting data into binary, ensuring uniformity across different data processes and systems. **When to use:** - Use this parameter when you need to define or change the default binary format for your data conversions. It's particularly useful in scenarios where consistent data representation is crucial, such as in multi-source integration projects. > ℹ️ Ensure that the format defined by this parameter aligns with your source and target database requirements to avoid data conversion errors. ---