--- title: "2019" canonical: "https://docs.vaultspeed.com/space/VPD/3163914431/2019" format: markdown --- > Macro (excerpt) > > > Macro (toc) ### Version 4.1.10 Release date: 14/11/19 **Fixes and improvements in this release require new source and data vault releases, and if you have not set a production date yet before today should do a full generation to have the correct complete version.** **A pdf is available to give more information on how to tackle the migration: **[VaultSpeed migration 4.1.10.pdf](https://vaultspeed.atlassian.net/wiki/download/attachments/2826829826/Vaultspeed%20migration%204.1.10.pdf?version=1&modificationDate=1640174423736&cacheVersion=1&api=v2) - Features: - Added Support for **PIT** tables to the **delta generation.** - Added support for the **Snowflake HASH** algorithm, which produces numbers instead of binary strings. This means that it uses less disk an memory space and can be more easily interpreted by BI tools. Note that this has a higher collision probability than MD5. - When creating a source release, you can now choose whether to read new data from the source, or continue working with the existing data. - Some changes to the generated code due to bug fixes or improvements to VaultSpeed, will require **migrations** of existing data vaults. To support this we added a new screen in the DDL/ETL page, named migrations. Here you can see which migrations that you need. In you next delta generations there will then be an extra folder MIGR which contains migration scripts (always sql) (numbered in the execution order!!!). In that folder you can also find the migrations that are needed due to certain deltas (instead of being mixed with the other code as before). After you have successfully executed the migrations you can mark them as completed in the migrations screen, which means that they wont be generated anymore. These migrations are only activated if you already have a DV release in production! Note that they might take a while to run, since some of them require recalculation of hash keys. - Important changes: - The **structure** of the zip-file containing the **ETL code **has been improved, it now contains sub-directories for load types and layers. - Improved the performance of the data vault parameter overview screen, it should no longer hang when loading large amounts of parameters. - Enabled viewing of a source in the graphical editor if the source release is locked. - The bk's that are stored in the SAT's will now be stored as a regular attribute, meaning that they retain their original datatype and will no longer have the bk prefix/suffix (Requires migration). - Removed the extra separator at the end of concatenated business keys (requires migration). This was mainly an annoyance in cases where you only have 1 bk in the concatenated_bk. - The hub hash keys will no longer be stored in link satellites, instead we will store the primary and foreign key values. This decreases storage needed and provides more information. - Improved migration scripts for changing hubs to and from single master. - quality of life changes: - Added loading indicators to the source graphical editor. - Added a button for re-centering the source graphical editor. - Added the ability to select and drag multiple objects by using shift-click in the source graphical editor. - Added the mass update button to the source graphical editor. - Improved the naming of the short name attribute in the data vault-lnk name management screen. - VaultSpeed user invitations now get automatically resend if they are not activated withing their lifetime. Since the invitations are only valid for 24h (and we are unable to increase that), we will now send new invitations every day, up until a week has passed, after that you will have to manually request a new one. - Added shadows to the source graphical editor. - The default object type in the source editors is now "HUB & SAT" instead of null. - In the source graphical editor, when u click the add relation button, the back button is at the same location, so if u accidentally click twice it cancels it immediately. The back button is now disabled for a second to prevent this. - Bug fixes: - The order of the business key when they are concatenated was alphabetic instead of following the order of the fields in the source, this is now fixed (requires migration). - Fixed the unknown hash key in the link not matching the ghost record in the case of concatenated bks and BK<>PK (requires migration), which meant that inner joins would not work properly. - Fixed issues with loading foreign keys from MySQL sources. - The insert only delete inserts a record without a hashdiff. The hashdiff should not be null, so the all null fields value will now be inserted. - Fixed DV overview search showing an incorrect amount of results. - If a deleted record was added again with the same attributes, it would not trigger a change in the satellites because the hashdiff was the same, this is now fixed. - Clicking of a self reference in the source graphical editor will now also hide the other relations. - Fixed an overlapping element issue with editing FMC configurations on small resolution screens. - DV overview search returned unexpected results because it was also searching on hidden fields. - In PostgreSQL based databases, the DFV record type attribute needs to be cast when a union is used in the view. - Fixed a few issues with creating relations in the source graphical editor. - After fixing problems for erroneous red object on the source graphical editor canvas, the error was not always cleared, this is now fixed. - DV and src release names are now cleaned such that they do not contain single quotes, this caused errors in the creation of table comments. - Fixed the bottom attribute not showing up when scrolling in the source graphical editor. ### Version 4.1.9 Release date: 30/09/19 - Features: - Added an option to the **FMC** to **group tasks**, this will change the DAG structure such that for every layer there are only concurrency number of tasks, and inside every task multiple mapping are executed in series, within the same connection. This reduces the overhead caused by opening connections, creating transactions, and scheduling tasks. This option should be enabled for optimal performance in mini batch loading. - Added the ability to **update source attributes across an entire source. **On the source objects screen there is a new button "mass update", this will open a screen where you can enter a pattern to select attributes from all objects in the source. Then you can select a field to update and set it to a certain value for all selected attributes. This can be useful when, for example, all objects have some technical attributes which you want to set to non historic. - Important changes: - Grouping single master HUBs caused them to become multi master (adding source bk attribute), this is no longer the case. - quality of life changes: - Improved times displayed on in the application, they should now all be in your local timezone. All times are now also in the same format. - Improved shape of self referencing relations in the graphical editor. - Improved data type icons on the graphical editor. - In the graphical editor, if u select the keys screen for an object, it will show the detailed connections for all relations of that object. - Bug fixes: - Fixed sat split overview not showing all splits of an attribute based split. - Fixed PIT loading all keys instead of only the keys that existed at a certain point in time. - Fixed inconsistent attribute order between the layers in the generated DDL. - Fixed parameters buttons not fitting on screen for small resolutions. - Fixed multiple errors on a source object causing doubles in the object list. - Fixed tool tips not showing after sorting a table. - Fixed issues with deleting and re-adding relations in the graphical editor. - Fixed an issue with the prepare delete mappings causing errors in the LNK mappings due to nulls. - Fixed reference tables missing from the business views. - Fixed source cdc selection dropdown being hidden. - Fixed PIT interval unit dropdown being hidden. - Fixed not being able to scroll while drawing a relation in the graphical editor. - Fixed positioning of the hide/show release icon/button on smaller screens. - Fixed an ordering issue in the releases views on the release number column. - Disabled edit mode in the name management screens for locked DV objects, instead of only disabling the save button. - Fixed the prepare delete mappings marking the ghost records as deleted. ### Version 4.1.8 Release date: 08/09/19 - Features: - Added a **graphical editor** for configuring the **source! **This editor can be opened by clicking the "Graphical editor" button on the top right of the sources-objects screen. This editor allows you to configure your source from an ERD diagram. The editor makes creating reference relationships a lot easier (especially relationships on multiple attributes), and makes the entire source configuration more fluent. Almost all actions can be done in this editor, only the creation of primary keys and unique keys is not possible. Take a look at the following wiki page for a short introduction: [https://vaultspeed.atlassian.net/wiki/spaces/VPD/pages/3011051563](https://vaultspeed.atlassian.net/wiki/spaces/VPD/pages/3011051563) . - Important changes: - Added a warning in the source configuration when there are foreign keys to a reference table. - Created a workaround for the Snowflake JDBC driver performance issues, obtaining metadata should be much faster now. - quality of life changes: - The BV releases selection screen no longer shown non locked data vault releases. - Added tool tips for long names in the PIT objects screens. - Renamed the TRIPWIRE_LOGIC parameters and improved the descriptions to make their purpose clearer. END_DATE_TRIPWIRE_LOGIC is now USE_CDC_TS_AS_LOAD_DATES and DELETE_FLAG_TRIPWIRE_LOGIC is now CLOSE_DELETED_RECORDS. - Added a warning when creating a DV release if there are no locked source releases. - Removed the datavault_info table from the generate Data Vault since it is no longer used. - Bug fixes: - Fixed a bunch of issued with the business vault graphical overview. - Fixed database link screen url changing color. - Fixed table overflows in the UI on screens with a small resolution. - In the DV name management screens, edit mode could be activated even if the name is locked (although save was disabled), this is no longer possible. - Fixed tool-tip and popup if the source object ignore button. - Fixed an issue when adding a key for a source table with a name that already exists resulting in a success message along with error 'duplicate key value violates unique constraint' message. - Fixed issue with attribute based sat splitting where incorrect splits where created if  ‘split only sats with all the attribute …’ is used. - Fixed numbering of files in the DDL of a delta generation. - Fixed an issue where source parameter saving was not working. - Fixed unknown in foreign keys between some SATs and HUBs in the DDL. ### Version 4.1.7 Release date: 25/08/19 - Features: - Added the ability to define and generate code for** PIT tables, ** both detail pits and snapshot pits with a certain frequency are supported. For now only the load date and cdc/transaction timestamps can be used, support for using other date attributes will be coming later. Take a look at the Cookbook for a detailed walkthrough of how you can define PIT tables. Delta generation support for PIT tables will be added in the next release, so for now you will have to do a full generation while selecting "BV". - A new menu item has been added namely "Business Vault", the B**usiness Vault** has its own releases. In this menu you can find the business views (instead of in the Data Vault menu) and the PIT tables + more to come e.g. Bridge tables. - We will now check the number of active sessions. This means that the number of users that can use the application at the same time is limited by the number of seats in your subscription. - Temp tables can be stored in the Staging schema's instead of the raw vault schema. This behavior is controlled by the STORE_TEMP_IN_STAGING DV parameter (N by default). - Important changes: - You can now choose to not generate a certain business view, or a certain attribute of a business view. - Added support for MySQL sources. - Setting the CHAR_LENGTH_SEMANTICS parameter for oracle will change the length definitions in the generated DDL. - The business views are now included in the ETL generation instead of the DDL generation. - quality of life changes: - When a parameter has limited possible values because of an other parameter, it will now show the reason in a tooltip, instead of having to go and look at the documentation. - Added extra information about the generation to all the generated files. - In the source objects screen, the source validity checks are now rerun after adding new constraints. So there is no longer a need to refresh manually after fixing key issues. - The HUB management tab in the Data Vault menu is now split into 2, a tab under management-HUB for setting the concatenation of Business keys and a Naming conventions tab for changing the object names. - Disabled the USE_MERGE parameter for Greenplum Data Vaults. - Added a search bar for the attributes to the SAT splitting screen. - Updated the logos and text on the application to the VaultSpeed logo. - Bug fixes: - Fixed deleted attributes not being included in the Hashdiff calculation. This meant that after deleting an attribute, all records where seen as changed even if the attribute was empty before the removal. - Fixes to the logic needed for cdc systems which only deliver the updated records. - Fixed data selector in FMC creation not scaling properly. - Fixed an issue where the source attributes view for locked releases did not show the correct constraint info. - Fixed overflowing text in Sources -> Releases & Builds -> builds view. - Fixed the connection string in ODI settings and generation not showing the updated value. - Fixed an issue where you could go to the HUBS&SATS screen before refreshing you release after saving the source model, this could cause issues when trying to create an attribute based split. - Fixed generated UK constraints not getting deployed in the correct schema for Oracle databases when deploying the DDL using a user different from the DV schema user. - The generated FMC code will no longer include empty source loading logic when using remote journalling with a CDC source, and forgetting to disable the FMC_GENERATE_SRC_LOADING_SCRIPTS parameter. - Added the source name to the reference table names, to prevent naming conflicts when multiple sources contain the same reference table (the Reference table grouping feature is in our backlog). - Fixed Unauthenticated users getting stuck on a white screen after logging in and being unable to log out to try to log in with another account. - Fixed the Intra_load_cycle load_timestamp ordering. - Fixed the header positioning while scrolling of the Sats view in Sources-HUBS&SATS. - Fixed naming conventions not being editable for new objects in a Data Vault with a production release. - Fixed some source checks not showing up as errors when editing the source, but only when saving it. This could cause the objects that failed those checks to be disabled in the release without warning. - Fixed errors in the source keys and relations, when creating a pk on a column that already has a disabled pk on it. - Fixed doubles showing up in the Data vault naming conventions after setting a production release. - Made sure that the agent can connect through a proxy. ### Version 4.1.6 Release date: 15/07/19 - Features: - Added the ability to **hide** source and Data Vault **releases**, this enables the user to remove clutter by hiding failed or intermediate releases. Hidden releases also do not show up in other places where you need to select a release (e.g. creating a DV release or generating ETL and DDL). - Added support for **overlapping loading windows** to the **FMC,** the size of the overlap can be set by using an Airflow variable. This is only available if intra loadcycle changes are enabled and cdc reliable disabled. - Added data quality bad support to reference tables. - Important changes: - the CURRENT_RECORD_LOAD_END_DATE parameter was moved from the system level to the Data Vault level. - Auto-deploy is now available for all generation types if the AGENT_CUSTOM_DEPLOY_SCRIPT parameter is enabled. - The source bk is now stored in the FMC load history table. - Improved the dynamic loading window logic for the FMC, if the first incremental load fails, the next one will now scale its loading window based on the initial load date. - Security improvements to client authorization. - Show an error on the source objects if they have multiple unique keys with the use_uk_as_bk parameter enabled, since then we do not know which key to use, in the past all columns which had a uk on them where used as business keys which could cause issues due to wrongly defined business keys. - When a primary key in the source has a separate unique index defined on it (with a different name, not auto generated), then this index will no longer be loaded in VaultSpeed as a unique key. VaultSpeed can only detect indices not actual unique keys, this is a limitation of the JDBC standard, so the unique keys get defined based on the unique indices. - quality of life changes: - Improved speed of generating the ETL metadata when using the FMC. - Added a warning to the FMC screen to warn the user that they have not yet enabled the FMC parameter. That parameter is needed because it makes sure that extra metadata is generated with the ETL and DDL. - Changed the color of the current number of generated mappings in the mapping count graph on the dashboard. - Store the business keys in the satellite when the user enables concatenated business keys for a HUB, even if they forgot to set the STORE_BK_IN_SAT to Y. - Bug fixes: - multi-master link tables no longer get truncated in the initial load, instead they will be deleted based on the source bk. - Fixed a scaling issue on the source excludes screen when entering into edit mode. - Fixed an issue where hub groupings did not propagate to link groups due to the user locking the DV to fast. Opening the Lock DV tab now runs extra checks to make sure this does not occur, this means that it might take a bit longer before you are able to click the lock button. - Added an extra connection to the FMC definition which defines the connection to an ETL tool. - Added extra casting of formats in casts functions for Postgres and Greenplum. - Fixed an issue with the generated ext layer code for direct links with update only cdc logic. ### Version 4.1.5 Release date: 17/06/19 - Features: - Changes to the **mapping count calculation:** As of this release, every incremental mapping of a data vault object that is generated is counted once for the first generation and 1/4'th for subsequent generations. Initial load mappings no longer count, so for most customers this change will decrease the total count. - **Reference constraints **between two objects **which share an attribute **are now supported e.g. fk1(col1,col2) references pk1(col1,col2) & fk2(col3,col2) references pk1(col1,col2). Important to note is that the reference constraints still need to have at least one unique attribute. - Added the ability to **load data** from **Kafka** to the **FMC.** This feature is still in the proof of concept phase so expect more fixes and improvements in the coming releases. - Important changes: - Improved authentication such that you do not get the unauthorized messages every so often, requiring you to log back in. - Improved the FMC airflow plugin such that it no longer requires the installation of packages that where needed for certain operators, even when they where not used e.g. Snowflake, BigQuery and Kafka. - quality of life changes: - Added dropdowns for selecting attributes to the sources objects draw constrains screen, this makes it easier to create reference constraints between objects with a large amount of attributes. - Added a notification to the get objects screen that indicates that new objects are being loaded when the get objects task is running. It used to still show the old data while new data was being loaded which could be confusing. - Disabled the ability to create a new release for a source while the get objects task for that source is still running. - Added hover text to the business key symbol in the source object attributes. - Improved the data vault top navigation bar on smaller screens by adding dropdowns containing the similar screens e.g. "overview" contains the parameters, DV overview and graphical overview. - Bug fixes: - Fixed resolution scaling in the create DB link screen. - Fixed the mapping chart on the dashboard not showing the maximum in some cases. - Fixed the HUB names in the HUB grouping screen getting cut of if they are too long. - Fixed incorrect syntax for default values in Oracle alter table statements. - Fixed the disable physics button on the graphical overviews not being able to be turned back on and blocking the generation of a new graph. - Fixed manually adding a second column to an existing reference constraint. his could, in some circumstances cause problems after creating a new release. - Fixed an issue with the agent where it would not upload the select objects CSV file. ### Version 4.1.4 Release date: 23/05/19 - Features: - Added **code highlighting** to the DDL/ETL code preview. - Updated the icons for the data vault **graphical overview** to the **official icons**. - **Preview** of **Google BigQuery** support, this target and source requires more testing. Feel free to try it out and send us issues/improvements. - Important changes: - Replaced unique constraint with indices for Greenplum, since unique keys cannot be disabled and thus slowed down loading. - quality of life changes: - Added a link to the documentation on the dashboard. - Added a tooltip to the sources - objects - short name column. - Bug fixes: - Fixed default and null values for a bunch of data types. - Fixed BV names not being kept from the previous dv release but only the previous production release. - Fixed the multi-master switch in sources, hubs&sats, the slider is now no longer clickable for locked releases. - Removed the unsaved changes popup after successfully editing a source and switching screens. - Fixed an issue where changing the schema of a source had no effect on the get objects task until the page was refreshed. - Fixed an issue where some fields in the sources screen could overlap. - After creating an attribute based sat split, it is no longer needed to return to the sat overview, the new split will be immediately visible. - The database link dropdown, when editing a source, will no longer show links of a different database type. - The delete flag value parameters are moved to their correct parameter group, the literal values. The parameters are now case sensitive so they will no longer be cased based on the casing parameter. - Fixed the sat mapping for satellite objects with insert only logic. - Fixed an issue where the extraction table DDL for direct links was missing the fk_bk columns. - Fixed missing lnk mappings for multi-master links within one source. - Fixed incorrect dependency between CDC_UPDATE_RECORD_ALL_ATTRIBUTES and STORE_BK_FIELDS_IN_SAT, this should be STORE_FK_FIELDS_IN_SAT. - Fixed an issue with the minimum data length for postgres varchars (varchar attributes should not have a length in postgres). - Disabled the autodeploy for Snowflake due to issues with the Snowflake JDBC driver (it does not support bulk DDL). - Fixed an issue where some tasks gave a "something went wrong" error message instead of a useful message. - Added casting around the delete flag values in the satellite mappings since some database types are unable to implicit cast them in some cases. ### Version 4.1.3 Release date: 07/05/19 - Features: - **Informatica** support: A simple implementation of Informatica is now available as a target language. This implementation is focused on ELT, the generated mappings will not execute any logic but instead trigger a SQL procedure containing the logic. Full informatica support is on the road map for the end of the year/beginning of next year. - Hash keys and hash diffs can now be stored in binary or character representation based on parameter **HASH_STORE_IN_BINARY.** - Added options for **delete flag values.** Based on the values entered for positive and negative values in the parameters(true & false, 1 & 0, 1 & -1,Y & N,...), the column will get an appropriate data type (boolean, bit, integer, character,...). - Added a parameter for **disabling** the creation of **foreign key constraints** in the FL. This is needed for Postgres based databases if the user cannot obtain superuser access, since that is needed for disabling the constraints. This is the case for most cloud databases. - quality of life changes: - The source attributes view now has a key icon next to the business key, this takes into account all parameters such as use unique key as business key. - Changed the name of the source lock button to "save source model" to make its function more clear. - Bug fixes: - Fixed an issue with the delta firewall logic for reference tables. - Hash difference calculations had a different format for the initial and incremental loads (but the same result) this is now the same. - Added missing grant from the cdc tables to the delta firewall schema (for Oracle). - Improved the reference table incremental load logic. If multiple changes came in with the same cdc timestamp, a duplicate key error occurred. - Fixed the generated isdatatype function for Greenplum (replaced the case statement with if statements). - Fixed the default length scaling of the NVARCHAR2 data type for Oracle, this was set to no scaling. This change only affects new data vaults, for existing data vaults, change the length value in the data vault/data types/data type mapping screen if u want this change. - Fixed an issue where the special values for user defined data types would not be properly added to new data vaults of the same database type. - Fixed the prepare delete mappings for direct links with no driving keys. - Fixed an issue with link tables, when 2 links of 1 source are grouped and if they share a hub a bug could occur that causes the generation of incorrect mappings. - Fixed an issue with the delete flag logic for insert only logic, a duplicate key error could occur when a delete and an update occur in the same load. ### Version 4.1.2 Release date: 17/04/19 - Features: - **New source** type: the accelerator can now import metadata from** Apache Kafka schema registry**. Support for loading data from Kafka streams in batches will be coming to the FMC in a future release, support for near real time loading will be coming later this year. - Important changes: - When enabling the use unique key as business key parameter, the source object will no longer show errors if no unique keys are found for an object. It will now correctly default to using the primary key (if no business keys are defined). - The data type screen is now visible for locked data vault releases. - There is a new button next to data vault releases that shows which source releases were used to make that data vault release. - The agent will no longer transfer all source metadata files when executing the putting generated CSV files task, now it will only transfer the files for that source. - The delete flag attribute now has data type CHAR(1). - When trying to generate ETL for ODI or Talend without filling in the corresponding settings an error will be thrown. This prevents the generation of useless code. - Improved performance of the task system and made preparations for automatic refresh (coming soon). - Fixed an issue with the default numeric format and the null exception value for numeric (the exception value was to large for the format) this is fixed by increasing the size of the format. - Fixed an issue with multi active hubs which could cause duplicate key violations during the initial load. - changed the default values for the length of abbreviated and short names to 128 instead of 16 and 4 respectively. This has no impact for existing customers, the overwrite lock will be enabled for these parameters such that they wont be overwritten with the new defaults. - Other: - Removed the deprecated business key concatenated parameter. - Fixed an error that occurred when creating a release if the comment field was filled in and then emptied. - Cleaned up the delta firewall logic for direct links (the current logic contained cte's that had no purpose). - Fixed an issue where source objects would no longer be visible when entering edit mode on an object and then editing one of its attributes. - Fixed update statement for Greenplum ELT, the cte's are now replaced by inline views. - Fixed create foreign key statement for Greenplum. - Added a check to the source objects that checks for foreign keys that point to only a part of a primary key, since this can cause problems in the ELT logic. - Fixed some bugs with hub grouping and its effect on link objects. - Fixed the length of business keyss for single master hubs ### Version 4.1.1 Release date: 09/04/19 - Features: - Added the ability to load **reference tables with cdc or modification date**, this is without history so just with inserts and updates(if not insert only logic). **This requires a new source release and datavault release for all source with reference tables (even if no cdc is used)!!!!** - **Added Snowflake db support** as a source and target. - Important changes: - Improved formatting of generated SQL code. - Improved the syntax of the update statements for PostgreSQL and Greenplum. - Added disabled foreign keys for PostgreSQL. - Other: - Object parameters are no longer editable after locking that source release, this is to prevent confusion and bugs. - Improvement to the Hub grouping screen. - Changed the generated isdatatype function for PostgreSQL such that it uses if instead of case, this makes sure that it also works on older versions of Postgres. - Improved the performance of ETL metadata generation. ### Version 4.1.0 Release date: 28/03/19 - Features: - **Automated sat split for audit satellites**. Possibility to split for all tables certain columns at once. - **Data types screen** to define datatypes, datatype mappings between source and target and exception values management - **Multi-master within a source**. Possibility to group tables within a source - Important changes: - Improved speed of locking a final version of a source. - Rework hub lnk lnd management in the datavault (drag and drop) - Data quality bad default = 'N' - Rework of the ddl settings screen, reworked with feedback by customers. - Possibility to store connection data of ODI next to the local agent instead of in the cloud - Rework of ODI mappings as they now can be split to avoid the errors caused by the limitation of file size which can be processed by Java. - Other: - Added extra logging to the etl generator - Extended testing for sql server - Fixed issue with concat business keys - Added search box to the business datavault screen - Improved task polling system - Fixed visual bugs for the datavault overview/ODI Settings and Generation/Sources/DV Releases/FMC/System Parameters - Fixed a bug for deleting foreign keys in the graphical overview in the source - Fixed a Talend settings bug so no source has to be chosen when adding a target connection - Fixed a bug in the No change data capture logic for the delta firewalls - Removed unused logic from the templates - Graphical overview mapping counter rework, now scrollable in time - Negative release number are not possible anymore - Fixed an issue with casing in the sat splits screen. - Extension of the mappings supported with Talend ### Version 4.0.10 Release date: 11/03/19 - Features: / - Important changes: - Changed the way the system parameters work, you can now edit and save multiple parameters at the same time, the other parameter screens will follow soon. - Changed the way delta generation works, now the SRC, EXT and STG layers will just be dropped and recreated, this will greatly reduce the number of bugs related to delta generation. - Added an extra button to ODI Settings & Generation, which will automatically create the Topology & Models based on the parameters. - Other: - Improved performance of locking sources. - Back end preparations for supporting user defined data types and data type mappings (the screen for managing this will be added next release). - Fixed an issue with ordering of tables in the source objects and keys and relations. - Fixed an issue where the unknown record had the same exception values as the null exception record which could cause duplicate key violations in direct links (manual changing of these values will be possible in the next release). - Fixed some issues with business key concatenated and store business key ion SAT. - Decreased the size of the News feed. - Improved column sizes for source parameters. - Add hover text to the headers in data tables, such that the header is still readable even for wide tables such as source object attributes. ### Version 4.0.9 Release date: 26/02/19 - Features: - Added a new **source object type: "Satellite"**, these are objects that have a relationship to a Business Element (which will become a HUB) but that are themselves NOT Business Elements, these will become additional Satellites on the Business Element (HUB) they are related to - Added support for **Informix** as a source. - Added a source object parameter STORE_FK_FIELDS_IN_SAT, which enables the **storage of foreign keys in the satellite.** - Important changes: - Hub unlinking is now done with a cross next to the hub instead of in a separate tab. - For large tables, some of the groovy scripts for generating mappings became to large for Java to execute, these will now be split into separate scripts. - The ODI topology settings are now automatically generated based on the information in the source parameters (set the ETL language parameter in dv paramaters to ODI). - Fixed an issue with end dating if intra loadcycle changes is on, make sure to regenerate the satellite type mappings if you are using this. - Other: - Performance improvements. - The single loadcycle parameter is removed, since it was entirely dependent on the intra loadcycle changes parameter. - The SRC_MODIF_TIMESTAMP_IN_SAT parameter is removed because it was not used anywhere. - Parameter dependency error improvements. - Improvements to the dv overview screen for unlocked data vaults to make it more accurate. - Removed a glitch with table loading where sometimes the records would be duplicated. - Fixed unavailable source parameters not being grayed out. - Improved the loading of the graphical overview, now it will remain in the loading state until the graph is rendered instead of only until it was generated. - Tables with the chevrons where more options can be accessed by clicking on them, can now also be closed instead of only opened. - Removed the driving key column from the source object attributes, since this property is set in keys and relations and served no purpose on the attributes screen. - Improvements to the data type mappings for integers and floats. - Numeric data types will no longer keep the scale and precision of the source, this is to eliminate issues with changing precision and scales in the source (not for SQL server, since their the scale and precision is less flexible). - Fixed an issue with the data vault shortest path query when the start hub was a multi master hub. - Fixed an issue with the generations list on the automatic deployment screen, where it would show that the source was "all" even when the generation was for one specific source. - Improved scaling of tables on a lot of the screens and improved the transition to edit mode. ### Version 4.0.8 Release date: 07/02/19 - Features: / - Important changes: - Added a mass storage flag to the source attributes, this can be used to indicate that a source field contains a lot of data, this will make it such that the target data type will become CLOB, VARCHAR(MAX), text, etc, this will also set this attribute to non historic, meaning that it is not included in the hashdiff (Not all databases support hashing large values, and this also improves performance). - When using intra loadcycle changes, the current_timestamp was used as a load date, which caused pk violations in the hub, changed this to make it a unique timestamp per record. - When using intra loadcycle changes, changed the distinct in the hub to a partition on load date, such that when multiple records for the same key are added in one load, only the one with the lowest load date will get inserted in the hub. - Other: - Fixed issues with link tables if the first data vault release was not locked. - Fixed production date input for Firefox - Added extra options for remote journalling (mostly to use in conjunction with the FMC), namely: - src cdc table → cdc (only data in window will be loaded) (FMC). - src delta view → target delta table (requires the creation of views on the cdc tables & the creation of a table to store the delta window on the source system) (FMC). - src delta view → target delta table through a dblink in a mapping (if sql then it will select from a table through a dblink, if ELT tool then mapping from src to target) (also requires creation of views) (no FMC). - Added a searchable dropdown to the sources draw relations screen - Added restrictions on the name lengths to the datavault name management screens(based on the parameter values). - Moved the full DDL/ETL Generation and delta generation to 1 screen with 2 tabs. - Data Vaults - GPR management, the hub and link sats screens are now tabs instead of buttons to switch between them. - Improved scaling of the Data vault overview columns screen. - The news feed is now sorted on date by default. - Added chevrons to tables where you can click on a record to show extra buttons. - Added a check to the source objects which shows whether one of the object parameters has an error. - Fixed an issue on the FMC screen with the start date, where editing an FMC workflow and not changing the start date would cause the start date to increase with the difference between the user timezone and the server timezone. - Fixed the filters in the task window disappearing after refreshing the tasks. - Fixed an issue where the graphical overview wont generate if the Show all button was not pressed. - Improved scaling of the Data vault menu bar such that all menu item are still visible on smaller screens (some of them will still disappear on very small screens, we are working on reducing the number of tabs to improve this further). - Fixed the data type mapping for booleans for postgresql and greenplum. ### Version 4.0.7 Release date: 28/01/19 - Features: - The **Flow Management Control application** is now in beta. The accelerator will generate code for Apache Airflow to schedule and execute the generated ELT code. You can also generate code to execute presentation layer (or other) loads, you can define which presentation layer mappings have to be executed and in which order in a json file (see documentation). Documentation: [https://vaultspeed.atlassian.net/wiki/spaces/VPD/pages/3013672982](https://vaultspeed.atlassian.net/wiki/spaces/VPD/pages/3013672982) - Added support for **CDC with only updated attributes** (with or without pre image). - Added support for **CDC with log position.** - Important changes: - Added an extra mapping for determining deletes if there is no cdc. - Added CDC tables to the generated DDL (this can be disabled with a parameter). - Changed it such that the name of CDC and INI tables is the same as in the source instead of also having the patterns removed. - Foreign Business keys short name has been updated: When you have tables with multiple foreign keys to the same table and selfreferences, redeploy DDL and ETL of the extraction and staging layer for those tables - Other: - Changed the load cycle id to a negative number for exception records such that they can easily be identified and filtered out. - Fixed issues with deleted constraints. - Fixed some issues with delta generation. - Adjusted the way mappings are counted to make it a bit fairer. - Added a refresh button to the automatic deployment screen. - Fixed some issues with dropdownboxes opening underneath ui element on Firefox. - Fixed DV overview columns not being scrollable. - Fixed DV parameters changing positions after being edited. - The source table parameters screen now has a refresh button to clear solved errors. ### Version 4.0.6 Release date: 10/01/19 - Features: / - Important changes: / - Other: - Creating a final source build now consists of 2 tasks instead of 1 big one to improve performance. - Dropdown selection boxes are now searchable. - Added search boxes to the GDPR Management screen and ODI Settings screens. - Select objects screen is now sorted by table name by default. - Fixed missing primary keys in ext and stg when using USE_SOURCE_UK_AS_BK. - Fixed some issues with Data Vault Shortest path queries. - Fixed an issue where source constraint conflicts caused the create source release task to fail, now they will just show up as errors in the source objects view. - Fixed some issues with dropdown boxes and scrollbars in Firefox. - Improved loading times for a bunch of screens. - Fixed an issue with the source select objects screen where it would keep loading when selecting all objects if there are no objects loaded yet. - Fixed an issue with the switches on the sources attributes screen where clicking on them would not activate edit mode - When creating a new release with new columns, there is now a check to make sure these new columns are assigned to a split when a sat is splitted, this check occurs when trying to lock a source release in the sources HUBs & SATs screen. - Made a lot of progress on the flow management control application (FMC), this will probably be released in beta next release (24/01/19).