--- title: "Generic FMC" canonical: "https://docs.vaultspeed.com/space/VPD/3364388956/Generic%20FMC" format: markdown --- The generic FMC is a VaultSpeed FMC type that generates scheduler-independent metadata, enabling easy setup of custom integrations with any scheduling tool. The Generic FMC provides all the logic and code needed for a proper FMC implementation. The resulting code includes mappings to manage metadata and JSON files that describe the execution order of the mappings. As with other FMC types, the generated DDL includes all necessary metadata tables, such as the FMC_LOADING_HISTORY table. ETL generation creates additional procedures; see the FMC Mappings section for an overview. Finally, FMC generation produces a set of JSON files with the required metadata. > Macro (toc) # JSON files ![image](media://88564c59-bf5d-44ba-b757-ef1c4aa951c4) ## Description VaultSpeed generates these files for your FMC flows (where is the flow name you defined in VaultSpeed): - FL Flows (the Raw data vault) - FMC info: `FMC_info_.json` - source metadata (only for FL flows with source loading enabled): `src_load_mtd_.json` - mapping metadata: `mappings_.json` - object metadata: `FL_mtd_.json` - BV flows (the Business data vault) - FMC info: `FMC_info_.json` - mapping metadata: `BV_mappings_.json` - object metadata: `BV_mtd_.json` > ℹ️ Note that the mentioned file names are the base names without generation id and timestamp. These are the names after deploying to GIT or downloading with the SDK (which is coming soon). ## Info file This file contains all the information about the FMC generation: - logo: The VaultSpeed ASCII logo - vaultspeed_version: the version of VaultSpeed used to generate the code - generation_date: when this code was generated - comment: the generation comment contains data about which versions (source, data vault, business vault, …) were used for this generation - dag_name: the name of the flow (this is used as an input for some mappings) - dag_description: the flow description - start_date: the start date is the date at which the initial load should be executed. This should become the load_date input variable in the initial load mappings. This date could be in the past. - concurrency: how many mappings should be executed in parallel - schedule_interval: the interval between loads. This free text field can be filled in however works best for your FMC application. - dv_connection_name: the connection name for the target DB - etl_connection_name: the connection name for the ETL tool - flow_type: the type of flow, this is `FL` or `BV` - load_type: the load type, this is `INIT` or `INCR` - group_tasks: whether task grouping is enabled, this changes the schema of the mapping metadata JSON - map_mtd_file_name: the name of the JSON file containing the mapping metadata - map_mtd_base_file_name: the base file name for the mapping metadata, the is the name without the loading generation id and training generation timestamp. This is the name of the file after a git deploy or an SDK generation download. - mtd_file_name: the name of the object metadata JSON file - mtd_base_file_name: the base name of the object metadata JSON file (see map_mtd_base_file_name) - etl_language: The ETL language for the generated ETL mappings, e.g., `SQL`, `GROOVY`, `JOBSCRIPT`, `DBT`, `MATILLION`. - has_procedures: Indicates whether the mappings are in some form of procedure that can be called through a connection to the target or if they are SQL files that need to be read out and have their content executed through a database connection. This would be true for a target like Snowflake or Databricks but false for Google Bigquery. - dv_database_type: the target database type - dv_code: the code of the DV of this flow. This can be found in the ‘Data Vaults - Releases' screen in the VaultSpeed. - fmc_file_extention: the value of the `FMC_FILE_EXTENTION` parameter - batch_size: the value of the `FMC_BATCH_SIZE` parameter Source-related fields only present in the info file of an FL flow: - src_load_base_file_name: the name of the source metadata JSON file - src_load_file_name: the base name of the source metadata JSON file (see map_mtd_base_file_name) - do_source_loading: the value of the `FMC_GENERATE_SRC_LOADING_SCRIPTS` parameter - dynamic_loading_window: the value of the `FMC_DYNAMIC_LOADING_WINDOW` parameter - object_based_window: the value of the `OBJECT_SPECIFIC_LOADING_WINDOW` parameter - cdc_based_window: the value of the `CDC_BASED_LOADING_WINDOW` parameter - src_database_type: The database type of the source - src_connection_name: the connection name for the source - src_name: the name of the source - src_short_name: the short name of the source - src_name_in_bk: the source business key for this source ## Source metadata This file contains the metadata about the source objects for the source and the target side. This metadata can be used to create a process that transfers data from the source system to the target system. The structure is as follows: ```json { "": { "source":{ "schema_name": "String", "table_name": "String", "column_names": ["String"], "window_column_name": "String" }, "target":{ "schema_name": "String", "table_name": "String", "column_names": ["String"], "window_column_name": "String", "fmc_window_col": "String" "modification_type": "String = sequence | date" } } } ``` There is a record for each source object. The key is the source object’s abbreviated name. This name will be referenced from the mapping metadata file. Note that for objects that are split in the source editor, this is the abbreviated name of the main object since splitting only happens in the delta firewall layer, which comes after the source loading. Each record contains two objects: a source and a target definition. Both contain the table definitions (schema name, table name, and column names). The target object also contains some extra fields which can be used to identify how to filter the data based on the loading window. - The `window_column_name` is the name of the attribute of the source object which contains the window data, so the data should be filtered based on the contents of this attribute. - The `fmc_window_col` is the name of the attribute in the FMC history (or object history) table that contains the beginning of the loading window data. This can be `FMC_BEGIN_LW_TIMESTAMP` or `FMC_BEGIN_LW_SEQUENCE.` - The `modification_type` is the type of modification field used to filter the data. It can be `sequence` or `date`. To determine which data to load using the loading window, use one of these queries: ```sql select {src_mtd.fmc_window_col} as fmc_begin_lw from SCHEMA_FMC.FMC_LOADING_HIST_TABLE_NAME where LOAD_TIMESTAMP = {load_date} and DAG_NAME = {dag_name} ``` When using `OBJECT_SPECIFIC_LOADING_WINDOW`: ```sql select MAX({src_mtd.fmc_window_col}) as fmc_begin_lw from SCHEMA_FMC.FMC_OBJECT_LOADING_HIST_TABLE_NAME where SRC_NAME_IN_BK = '{fmc_info.src_name_in_bk}' and SUCCESS_FLAG = 1 and LOAD_TIMESTAMP < {load_date} and OBJECT_NAME = '{src_mtd.src_object_name}' ``` All source records with a date or sequence larger than the value retrieved by these queries should be transferred to the target. ## Mapping metadata This file contains the metadata about all the mappings, in which order they need to be executed, and which input parameters they have. The structure of this file depends on the “group tasks” option. If this is disabled, the file will contain a list of mappings and, for each of them, a list of dependencies. These dependencies are the mappings that must be completed before the mapping is executed. If the group tasks option is enabled, the file will contain a list of layers, where each layer contains a list of the mappings for that layer. The layers should be executed one by one in order. All the mappings within a layer can be executed in parallel. #### Non-Grouped Tasks The structure of the JSON when tasks are not grouped is as follows. ```json { "" : { "dependencies": ["", ...], "src_objects":[ { "src_object_name": "", "has_trans_indicator": bool, "modification_type":"sequence | date", "table_name": "", "schema_name": "" }, ... ], "map_schema": "", "input":["", ...], "map_type":"" }, ... } ``` The `map_name` is the name of the mapping to be executed. How this should be done depends on the technology and is described in the chapter “Target Technologies” on this page. Each mapping has the following properties: - `dependencies`: This is a list of `map_name`s that must be executed successfully before this one. - `src_objects`: This is a list of the source objects whose data is loaded by this mapping. These are the same names as the keys in the source load metadata, namely the source object abbreviated names (before splitting). Most mappings will only have one source object, but some, like the HUB, could have multiple. The source object has the following properties: - `src_object_name`: This is the same name as the keys in the source load metadata, namely the source object abbreviated names (before splitting). - `has_trans_indicator`: Indicates whether this source object has a field that indicates when a change happened, so when there is a modification date or a modification sequence attribute. - `modification_type`: This is the type of modification field used to filter the data. It can be `sequence` or `date`. - `table_name`: This is the name of the landing table. This object is the start of the VaultSpeed loading process and is referenced in the firewall view. Depending on the configuration, this could be the INI table, the CDC table, or the DTV table. - `schema_name`: The schema name of the landing table, can be: `SCHEMA_INI`, `SCHEMA_DTV,` or `SCHEMA_CDC`. - `map_schema`: The schema where the mappings are stored. - `input`: A list of input parameters for the mapping. What data should be fed into each of these is explained in the “Input Variables” below on this page. - `map_type`: This contains the mapping type and, as such, influences how the mappings should be executed. See the “Target Technologies” section on this page for a breakdown of each available target technology. A flow can consist of multiple types of mappings. There are three possible values: - `sql_mapping`: This is a normal SQL mapping. Depending on the target technology, this could be a procedure, a notebook, or a SQL file. - `etl_mapping`: This is a mapping in an ETL tool. Depending on the ETL tool, this could take different forms, such as a tag for DBT, an orchestration job for Matillion, etc. - `src_load`: This indicates that this mapping is for loading source data. These mappings are **not **generated by VaultSpeed. These tasks should trigger an externally built data-loading process, which can be fed with metadata about the source and target objects from the source-loading JSON file. #### Grouped Tasks The structure of the JSON when tasks are not grouped is as follows. All the fields describing the mappings are the same as in the ungrouped case. ```json [ { "layer": "", "mappings": [ { "map":"", "src_objects":[ { "src_object_name": "", "has_trans_indicator": bool, "modification_type":"sequence | date", "table_name": "", "schema_name": "" } ], "map_schema": "", "input": ["", ...], "map_type": "" }, ... ] }, ... ] ``` ## Object metadata This file contains data about which data vault objects are loaded by which mappings. This could be used, for example, to add tasks that analyze the raw vault tables after they are loaded. In that case, one could add a task that does the analysis of the object when all the mappings mentioned in the dependencies are completed. ```json { "" : { "dependencies": ["", ...], "schema": "" }, ... } ``` # JSON schemas ## Structure
FL FMC info ```json { "title": "FLWorkflowInformation", "description": "DV specific properties. Mostly source loading related", "type": "object", "properties": { "logo": { "title": "Logo", "type": "string" }, "vaultspeed_version": { "title": "Vaultspeed Version", "type": "string" }, "generation_date": { "title": "Generation Date", "type": "string", "format": "date-time" }, "comment": { "title": "Comment", "type": "string" }, "dag_name": { "title": "Dag Name", "type": "string" }, "dag_description": { "title": "Dag Description", "type": "string" }, "start_date": { "title": "Start Date", "type": "string", "format": "date-time" }, "concurrency": { "title": "Concurrency", "type": "integer" }, "schedule_interval": { "title": "Schedule Interval", "type": "string" }, "src_connection_name": { "title": "Src Connection Name", "type": "string" }, "dv_connection_name": { "title": "Dv Connection Name", "type": "string" }, "etl_connection_name": { "title": "Etl Connection Name", "type": "string" }, "flow_type": { "title": "Flow Type", "type": "string" }, "load_type": { "$ref": "#/definitions/LoadTypes" }, "group_tasks": { "title": "Group Tasks", "type": "boolean" }, "map_mtd_file_name": { "title": "Map Mtd File Name", "type": "string", "format": "path" }, "map_mtd_base_file_name": { "title": "Map Mtd Base File Name", "type": "string", "format": "path" }, "mtd_base_file_name": { "title": "Mtd Base File Name", "type": "string", "format": "path" }, "mtd_file_name": { "title": "Mtd File Name", "type": "string", "format": "path" }, "etl_language": { "title": "Etl Language", "type": "string" }, "has_procedures": { "title": "Has Procedures", "type": "boolean" }, "dv_database_type": { "title": "Dv Database Type", "type": "string" }, "dv_code": { "title": "Dv Code", "type": "string" }, "fmc_file_extention": { "title": "Fmc File Extention", "type": "string" }, "batch_size": { "title": "Batch Size", "type": "integer" }, "src_load_base_file_name": { "title": "Src Load Base File Name", "type": "string", "format": "path" }, "src_load_file_name": { "title": "Src Load File Name", "type": "string", "format": "path" }, "do_source_loading": { "title": "Do Source Loading", "type": "boolean" }, "dynamic_loading_window": { "title": "Dynamic Loading Window", "type": "boolean" }, "object_based_window": { "title": "Object Based Window", "type": "boolean" }, "cdc_based_window": { "title": "Cdc Based Window", "type": "boolean" }, "src_database_type": { "title": "Src Database Type", "type": "string" }, "src_name": { "title": "Src Name", "type": "string" }, "src_short_name": { "title": "Src Short Name", "type": "string" }, "src_name_in_bk": { "title": "Src Name In Bk", "type": "string" } }, "required": [ "logo", "vaultspeed_version", "generation_date", "comment", "dag_name", "dag_description", "start_date", "concurrency", "schedule_interval", "dv_connection_name", "etl_connection_name", "flow_type", "load_type", "group_tasks", "map_mtd_file_name", "map_mtd_base_file_name", "mtd_base_file_name", "mtd_file_name", "etl_language", "has_procedures", "dv_database_type", "dv_code", "fmc_file_extention", "batch_size", "do_source_loading", "dynamic_loading_window", "object_based_window", "cdc_based_window", "src_database_type", "src_name", "src_short_name", "src_name_in_bk" ], "additionalProperties": false, "definitions": { "LoadTypes": { "title": "LoadTypes", "description": "FMC load types", "enum": [ "INIT", "INCR" ], "type": "string" } } } ```
BV FMC info ```json { "title": "BVWorkflowInformation", "description": "BV specific properties", "type": "object", "properties": { "logo": { "title": "Logo", "type": "string" }, "vaultspeed_version": { "title": "Vaultspeed Version", "type": "string" }, "generation_date": { "title": "Generation Date", "type": "string", "format": "date-time" }, "comment": { "title": "Comment", "type": "string" }, "dag_name": { "title": "Dag Name", "type": "string" }, "dag_description": { "title": "Dag Description", "type": "string" }, "start_date": { "title": "Start Date", "type": "string", "format": "date-time" }, "concurrency": { "title": "Concurrency", "type": "integer" }, "schedule_interval": { "title": "Schedule Interval", "type": "string" }, "src_connection_name": { "title": "Src Connection Name", "type": "string" }, "dv_connection_name": { "title": "Dv Connection Name", "type": "string" }, "etl_connection_name": { "title": "Etl Connection Name", "type": "string" }, "flow_type": { "title": "Flow Type", "type": "string" }, "load_type": { "$ref": "#/definitions/LoadTypes" }, "group_tasks": { "title": "Group Tasks", "type": "boolean" }, "map_mtd_file_name": { "title": "Map Mtd File Name", "type": "string", "format": "path" }, "map_mtd_base_file_name": { "title": "Map Mtd Base File Name", "type": "string", "format": "path" }, "mtd_base_file_name": { "title": "Mtd Base File Name", "type": "string", "format": "path" }, "mtd_file_name": { "title": "Mtd File Name", "type": "string", "format": "path" }, "etl_language": { "title": "Etl Language", "type": "string" }, "has_procedures": { "title": "Has Procedures", "type": "boolean" }, "dv_database_type": { "title": "Dv Database Type", "type": "string" }, "dv_code": { "title": "Dv Code", "type": "string" }, "fmc_file_extention": { "title": "Fmc File Extention", "type": "string" }, "batch_size": { "title": "Batch Size", "type": "integer" } }, "required": [ "logo", "vaultspeed_version", "generation_date", "comment", "dag_name", "dag_description", "start_date", "concurrency", "schedule_interval", "dv_connection_name", "etl_connection_name", "flow_type", "load_type", "group_tasks", "map_mtd_file_name", "map_mtd_base_file_name", "mtd_base_file_name", "mtd_file_name", "etl_language", "has_procedures", "dv_database_type", "dv_code", "fmc_file_extention", "batch_size" ], "additionalProperties": false, "definitions": { "LoadTypes": { "title": "LoadTypes", "description": "FMC load types", "enum": [ "INIT", "INCR" ], "type": "string" } } } ```
source metadata ```json { "title": "WorkflowSourceLoads", "type": "object", "properties": { "src_object_name": { "title": "Source object abbreviated name", "type": "object", "additionalProperties": { "$ref": "#/definitions/SourceLoadMapping" } } }, "required": [ "mappings" ], "definitions": { "SourceLoadObject": { "title": "SourceLoadObject", "type": "object", "properties": { "schema_name": { "title": "Schema Name", "type": "string" }, "table_name": { "title": "Table Name", "type": "string" }, "column_names": { "title": "Column Names", "type": "array", "items": { "type": "string" } }, "window_column_name": { "title": "Window column Name", "type": "string" }, "fmc_window_col": { "title": "FMC_END_LW_ attribute", "type": "string" }, "modification_type": { "title": "Type of the window column", "type": "string" } }, "required": [ "schema_name", "table_name", "column_names" ] }, "SourceLoadMapping": { "title": "SourceLoadMapping", "type": "object", "properties": { "source": { "$ref": "#/definitions/SourceLoadObject" }, "target": { "$ref": "#/definitions/SourceLoadObject" } }, "required": [ "source", "target" ] } } } ```
ungrouped flows ```json { "title": "WorkflowUngroupedMappings", "type": "object", "properties": { "mappings": { "title": "Mapping Name", "type": "object", "additionalProperties": { "$ref": "#/definitions/UngroupedMapping" } } }, "required": [ "mappings" ], "definitions": { "SourceObject": { "title": "SourceObject", "type": "object", "properties": { "src_object_name": { "title": "Source object abbreviated name", "type": "string" }, "has_trans_indicator": { "title": "Has Trans Indicator", "type": "boolean" }, "trans_indicator": { "title": "Trans Indicator attribute", "type": "boolean" }, "modification_type": { "title": "Modification Type", "type": "string" }, "table_name": { "title": "Table Name", "type": "string" }, "schema_name": { "title": "Schema Name", "type": "string" } }, "required": [ "src_object_name", "has_trans_indicator", "modification_type", "table_name", "schema_name" ] }, "UngroupedMapping": { "title": "UngroupedMapping", "type": "object", "properties": { "src_objects": { "title": "Source Objects", "type": "array", "items": { "$ref": "#/definitions/SourceObject" } }, "map_schema": { "title": "Map Schema", "type": "string" }, "input": { "title": "Input", "type": "array", "items": { "type": "string" } }, "map_type": { "title": "Map Type", "type": "string" }, "dependencies": { "title": "Dependencies", "type": "array", "items": { "type": "string" } } }, "required": [ "map_schema", "input", "map_type", "dependencies" ] } } } ```
grouped flows ```json { "title": "WorkflowLayeredMappings", "type": "object", "properties": { "layers": { "title": "Layers", "type": "array", "items": { "$ref": "#/definitions/Layer" } } }, "required": [ "layers" ], "definitions": { "SourceObject": { "title": "SourceObject", "type": "object", "properties": { "src_object_name": { "title": "Src Object Name", "type": "string" }, "has_trans_indicator": { "title": "Has Trans Indicator", "type": "boolean" }, "trans_indicator": { "title": "Trans Indicator attribute", "type": "boolean" }, "modification_type": { "title": "Modification Type", "type": "string" }, "table_name": { "title": "Table Name", "type": "string" }, "schema_name": { "title": "Schema Name", "type": "string" } }, "required": [ "src_object_name", "has_trans_indicator", "modification_type", "table_name", "schema_name" ] }, "GroupedMapping": { "title": "GroupedMapping", "type": "object", "properties": { "src_objects": { "title": "Source Objects", "type": "array", "items": { "$ref": "#/definitions/SourceObject" } }, "map_schema": { "title": "Map Schema", "type": "string" }, "input": { "title": "Input", "type": "array", "items": { "type": "string" } }, "map_type": { "title": "Map Type", "type": "string" }, "map": { "title": "Map", "type": "string" } }, "required": [ "map_schema", "input", "map_type", "map" ] }, "Layer": { "title": "Layer", "type": "object", "properties": { "layer": { "title": "Layer", "type": "string" }, "mappings": { "title": "Mappings", "type": "array", "items": { "$ref": "#/definitions/GroupedMapping" } } }, "required": [ "layer", "mappings" ] } } } ```
object metadata ```json { "title": "ObjectMetadata", "type": "object", "properties": { "objects": { "title": "Objects", "type": "object", "additionalProperties": { "$ref": "#/definitions/DataVaultObject" } } }, "required": [ "objects" ], "definitions": { "DataVaultObject": { "title": "DataVaultObject", "type": "object", "properties": { "dependencies": { "title": "Mapping Dependencies", "type": "array", "items": { "type": "string" } }, "schema": { "title": "Schema", "type": "string" } }, "required": [ "dependencies", "schema" ] } } } ```
## Examples
FL FMC info ```json { "logo": " __ __ _ _ _ __ ___ __ __ \n \\ \\ / /_ _ _ _| | |_ ___ ____ ___ ___ __| | \\ \\/ _ \\/ / /_/ \n \\ \\ / / _` | | | | | __/ __| _ \\ / _ \\/ _ \\/ _` | \\/ / \\ \\/ /\\ \n \\ V / (_| | |_| | | |_\\__ \\ |_) | __/ __/ (_| | / / \\/\\ \\/ / \n \\_/ \\__,_|\\__,_|_|\\__|___/ .__/ \\___|\\___|\\__,_| /_/ \\/_/\\__/ \n |_| \n", "comment": "DV_NAME: moto_sf - Release: 11(11) - Comment: - Release date: 2022/12/05 12:23:07, \nSRC_NAME: moto_sales - Release: moto_sales(5) - Comment: - Release date: 2022/09/15 09:11:02", "dv_code": "moto", "dag_name": "moto_sales_incr", "src_name": "moto_sales", "flow_type": "FL", "load_type": "INCR", "batch_size": 1000, "start_date": "2021-07-11T22:00:00", "concurrency": 4, "group_tasks": true, "etl_language": "SQL", "mtd_file_name": "299_FL_mtd_moto_sales_incr_20221205_124239.json", "has_procedures": true, "src_name_in_bk": "moto_sales", "src_short_name": "ms", "dag_description": "moto_sales_incr", "generation_date": "2022-12-05T12:42:39.752205+00:00", "cdc_based_window": true, "dv_database_type": "SNOWFLAKE", "do_source_loading": true, "map_mtd_file_name": "299_mappings_moto_sales_incr_20221205_124239.json", "schedule_interval": "\"@hourly\"", "src_database_type": "POSTGRESQL", "dv_connection_name": "sf", "fmc_file_extention": ".sql", "mtd_base_file_name": "FL_mtd_moto_sales_incr.json", "src_load_file_name": "299_src_load_mtd_moto_sales_incr_20221205_124239.json", "vaultspeed_version": "5.2.1.7", "etl_connection_name": "matillion", "object_based_window": true, "src_connection_name": "test_src", "dynamic_loading_window": true, "map_mtd_base_file_name": "mappings_moto_sales_incr.json", "src_load_base_file_name": "src_load_mtd_moto_sales_incr.json" } ```
BV FMC info ```json { "logo": " __ __ _ _ _ __ ___ __ __ \n \\ \\ / /_ _ _ _| | |_ ___ ____ ___ ___ __| | \\ \\/ _ \\/ / /_/ \n \\ \\ / / _` | | | | | __/ __| _ \\ / _ \\/ _ \\/ _` | \\/ / \\ \\/ /\\ \n \\ V / (_| | |_| | | |_\\__ \\ |_) | __/ __/ (_| | / / \\/\\ \\/ / \n \\_/ \\__,_|\\__,_|_|\\__|___/ .__/ \\___|\\___|\\__,_| /_/ \\/_/\\__/ \n |_| \n", "comment": "DV_NAME: moto_sf - Release: 11(11) - Comment: - Release date: 2022/12/05 12:23:07, \nBV release: init(1) - Comment: initial release - Release date: 2022/12/05 12:24:28", "dv_code": "moto", "dag_name": "moto_bv_incr", "flow_type": "BV", "load_type": "INCR", "batch_size": 1000, "start_date": "2022-12-03T23:00:00", "concurrency": 4, "group_tasks": false, "etl_language": "SQL", "mtd_file_name": "300_BV_mtd_moto_bv_incr_20221205_124725.json", "has_procedures": true, "dag_description": "moto_sf_bv_incr", "generation_date": "2022-12-05T12:47:25.81667+00:00", "dv_database_type": "SNOWFLAKE", "map_mtd_file_name": "300_BV_mappings_moto_bv_incr_20221205_124725.json", "schedule_interval": "\"@daily\"", "dv_connection_name": "sf", "fmc_file_extention": ".sql", "mtd_base_file_name": "BV_mtd_moto_bv_incr.json", "vaultspeed_version": "5.2.1.7", "etl_connection_name": null, "map_mtd_base_file_name": "BV_mappings_moto_bv_incr.json" } ```
Source metadata ```json { "addresses": { "source":{"schema_name":"moto_sales_cdcr", "table_name":"jrn_addresses", "column_names":["coordinates", "trans_timestamp", "operation", "address_number", "update_timestamp", "street_number", "city", "street_name", "postal_code"], "window_column_name":"trans_timestamp"}, "target":{"schema_name":"\"moto_sales_dtv\"", "table_name":"\"addresses\"", "column_names":["\"coordinates\"", "\"trans_timestamp\"", "\"operation\"", "\"address_number\"", "\"update_timestamp\"", "\"street_number\"", "\"city\"", "\"street_name\"", "\"postal_code\""], "window_column_name":"trans_timestamp","fmc_window_col":"\"fmc_end_lw_timestamp\"","modification_type":"date"}} , "codes_to_language": { "source":{"schema_name":"moto_sales_cdcr", "table_name":"jrn_codes_to_language", "column_names":["trans_timestamp", "operation", "description", "language_code", "code", "update_timestamp"], "window_column_name":"trans_timestamp"}, "target":{"schema_name":"\"moto_sales_dtv\"", "table_name":"\"codes_to_language\"", "column_names":["\"trans_timestamp\"", "\"operation\"", "\"description\"", "\"language_code\"", "\"code\"", "\"update_timestamp\""], "window_column_name":"trans_timestamp","fmc_window_col":"\"fmc_end_lw_timestamp\"","modification_type":"date"}} , "customers": { "source":{"schema_name":"moto_sales_cdcr", "table_name":"jrn_customers", "column_names":["trans_timestamp", "customer_ship_to_address_id", "operation", "update_timestamp", "birthdate", "gender", "national_person_id", "first_name", "customer_number", "customer_invoice_address_id", "last_name"], "window_column_name":"trans_timestamp"}, "target":{"schema_name":"\"moto_sales_dtv\"", "table_name":"\"customers\"", "column_names":["\"trans_timestamp\"", "\"customer_ship_to_address_id\"", "\"operation\"", "\"update_timestamp\"", "\"birthdate\"", "\"gender\"", "\"national_person_id\"", "\"first_name\"", "\"customer_number\"", "\"customer_invoice_address_id\"", "\"last_name\""], "window_column_name":"trans_timestamp","fmc_window_col":"\"fmc_end_lw_timestamp\"","modification_type":"date"}} , "invoice_lines": { "source":{"schema_name":"moto_sales_cdcr", "table_name":"jrn_invoice_lines", "column_names":["operation", "trans_timestamp", "amount", "product_id", "unit_price", "part_id", "quantity", "update_timestamp", "invoice_line_number", "invoice_number"], "window_column_name":"trans_timestamp"}, "target":{"schema_name":"\"moto_sales_dtv\"", "table_name":"\"invoice_lines\"", "column_names":["\"operation\"", "\"trans_timestamp\"", "\"amount\"", "\"product_id\"", "\"unit_price\"", "\"part_id\"", "\"quantity\"", "\"update_timestamp\"", "\"invoice_line_number\"", "\"invoice_number\""], "window_column_name":"trans_timestamp","fmc_window_col":"\"fmc_end_lw_timestamp\"","modification_type":"date"}} , "invoices": { "source":{"schema_name":"moto_sales_cdcr", "table_name":"jrn_invoices", "column_names":["operation", "discount", "update_timestamp", "trans_timestamp", "amount", "invoice_date", "invoice_number", "invoice_customer_id"], "window_column_name":"trans_timestamp"}, "target":{"schema_name":"\"moto_sales_dtv\"", "table_name":"\"invoices\"", "column_names":["\"operation\"", "\"discount\"", "\"update_timestamp\"", "\"trans_timestamp\"", "\"amount\"", "\"invoice_date\"", "\"invoice_number\"", "\"invoice_customer_id\""], "window_column_name":"trans_timestamp","fmc_window_col":"\"fmc_end_lw_timestamp\"","modification_type":"date"}} , "parts": { "source":{"schema_name":"moto_sales_cdcr", "table_name":"jrn_moto_parts", "column_names":["part_language_code", "part_number", "part_id", "trans_timestamp", "operation", "update_timestamp"], "window_column_name":"trans_timestamp"}, "target":{"schema_name":"\"moto_sales_dtv\"", "table_name":"\"parts\"", "column_names":["\"part_language_code\"", "\"part_number\"", "\"part_id\"", "\"trans_timestamp\"", "\"operation\"", "\"update_timestamp\""], "window_column_name":"trans_timestamp","fmc_window_col":"\"fmc_end_lw_timestamp\"","modification_type":"date"}} , "products": { "source":{"schema_name":"moto_sales_cdcr", "table_name":"jrn_moto_products", "column_names":["product_intro_date", "replacement_product_id", "product_name", "operation", "update_timestamp", "product_et_code", "product_cc", "product_part_code", "product_id", "trans_timestamp"], "window_column_name":"trans_timestamp"}, "target":{"schema_name":"\"moto_sales_dtv\"", "table_name":"\"products\"", "column_names":["\"product_intro_date\"", "\"replacement_product_id\"", "\"product_name\"", "\"operation\"", "\"update_timestamp\"", "\"product_et_code\"", "\"product_cc\"", "\"product_part_code\"", "\"product_id\"", "\"trans_timestamp\""], "window_column_name":"trans_timestamp","fmc_window_col":"\"fmc_end_lw_timestamp\"","modification_type":"date"}} , "payments": { "source":{"schema_name":"moto_sales_cdcr", "table_name":"jrn_payments", "column_names":["invoice_number", "customer_number", "transaction_id", "operation", "update_timestamp", "date_time", "trans_timestamp", "amount"], "window_column_name":"trans_timestamp"}, "target":{"schema_name":"\"moto_sales_dtv\"", "table_name":"\"payments\"", "column_names":["\"invoice_number\"", "\"customer_number\"", "\"transaction_id\"", "\"operation\"", "\"update_timestamp\"", "\"date_time\"", "\"trans_timestamp\"", "\"amount\""], "window_column_name":"trans_timestamp","fmc_window_col":"\"fmc_end_lw_timestamp\"","modification_type":"date"}} , "product_feat_class_rel": { "source":{"schema_name":"moto_sales_cdcr", "table_name":"jrn_product_feat_class_rel", "column_names":["update_timestamp", "product_feature_id", "trans_timestamp", "product_feature_class_id", "product_id", "operation"], "window_column_name":"trans_timestamp"}, "target":{"schema_name":"\"moto_sales_dtv\"", "table_name":"\"product_feat_class_rel\"", "column_names":["\"update_timestamp\"", "\"product_feature_id\"", "\"trans_timestamp\"", "\"product_feature_class_id\"", "\"product_id\"", "\"operation\""], "window_column_name":"trans_timestamp","fmc_window_col":"\"fmc_end_lw_timestamp\"","modification_type":"date"}} , "product_feature_cat": { "source":{"schema_name":"moto_sales_cdcr", "table_name":"jrn_product_feature_cat", "column_names":["prod_feat_cat_language_code", "prod_feat_cat_description", "product_feature_category_code", "update_timestamp", "operation", "product_feature_category_id", "trans_timestamp"], "window_column_name":"trans_timestamp"}, "target":{"schema_name":"\"moto_sales_dtv\"", "table_name":"\"product_feature_cat\"", "column_names":["\"prod_feat_cat_language_code\"", "\"prod_feat_cat_description\"", "\"product_feature_category_code\"", "\"update_timestamp\"", "\"operation\"", "\"product_feature_category_id\"", "\"trans_timestamp\""], "window_column_name":"trans_timestamp","fmc_window_col":"\"fmc_end_lw_timestamp\"","modification_type":"date"}} , "product_feature_class": { "source":{"schema_name":"moto_sales_cdcr", "table_name":"jrn_product_feature_class", "column_names":["product_feature_class_desc", "update_timestamp", "operation", "product_feature_class_code", "product_feature_class_id", "trans_timestamp"], "window_column_name":"trans_timestamp"}, "target":{"schema_name":"\"moto_sales_dtv\"", "table_name":"\"product_feature_class\"", "column_names":["\"product_feature_class_desc\"", "\"update_timestamp\"", "\"operation\"", "\"product_feature_class_code\"", "\"product_feature_class_id\"", "\"trans_timestamp\""], "window_column_name":"trans_timestamp","fmc_window_col":"\"fmc_end_lw_timestamp\"","modification_type":"date"}} , "product_features": { "source":{"schema_name":"moto_sales_cdcr", "table_name":"jrn_product_features", "column_names":["product_feature_id", "trans_timestamp", "update_timestamp", "product_feature_code", "product_feature_description", "operation", "product_feature_language_code", "product_feature_cat_id"], "window_column_name":"trans_timestamp"}, "target":{"schema_name":"\"moto_sales_dtv\"", "table_name":"\"product_features\"", "column_names":["\"product_feature_id\"", "\"trans_timestamp\"", "\"update_timestamp\"", "\"product_feature_code\"", "\"product_feature_description\"", "\"operation\"", "\"product_feature_language_code\"", "\"product_feature_cat_id\""], "window_column_name":"trans_timestamp","fmc_window_col":"\"fmc_end_lw_timestamp\"","modification_type":"date"}} } ```
ungrouped flow ```json { "set_fmc_mtd_fl_incr_ms" : {"dependencies": [], "src_objects":[], "map_schema": "\"moto_proc\"", "input":["dag_name", "load_cycle_id", "load_date"], "map_type":"sql_mapping"}, "src_load_incr_ms_addresses" : {"dependencies": ["set_fmc_mtd_fl_incr_ms"], "src_objects":[{"src_object_name": "addresses", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"addresses\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":["load_cycle_id"], "map_type":"src_load"}, "src_load_incr_ms_codes_to_language" : {"dependencies": ["set_fmc_mtd_fl_incr_ms"], "src_objects":[{"src_object_name": "codes_to_language", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"codes_to_language\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":["load_cycle_id"], "map_type":"src_load"}, "src_load_incr_ms_customers" : {"dependencies": ["set_fmc_mtd_fl_incr_ms"], "src_objects":[{"src_object_name": "customers", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"customers\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":["load_cycle_id"], "map_type":"src_load"}, "src_load_incr_ms_invoice_lines" : {"dependencies": ["set_fmc_mtd_fl_incr_ms"], "src_objects":[{"src_object_name": "invoice_lines", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"invoice_lines\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":["load_cycle_id"], "map_type":"src_load"}, "src_load_incr_ms_invoices" : {"dependencies": ["set_fmc_mtd_fl_incr_ms"], "src_objects":[{"src_object_name": "invoices", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"invoices\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":["load_cycle_id"], "map_type":"src_load"}, "src_load_incr_ms_parts" : {"dependencies": ["set_fmc_mtd_fl_incr_ms"], "src_objects":[{"src_object_name": "parts", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"parts\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":["load_cycle_id"], "map_type":"src_load"}, "src_load_incr_ms_payments" : {"dependencies": ["set_fmc_mtd_fl_incr_ms"], "src_objects":[{"src_object_name": "payments", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"payments\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":["load_cycle_id"], "map_type":"src_load"}, "src_load_incr_ms_product_feat_class_rel" : {"dependencies": ["set_fmc_mtd_fl_incr_ms"], "src_objects":[{"src_object_name": "product_feat_class_rel", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"product_feat_class_rel\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":["load_cycle_id"], "map_type":"src_load"}, "src_load_incr_ms_product_feature_cat" : {"dependencies": ["set_fmc_mtd_fl_incr_ms"], "src_objects":[{"src_object_name": "product_feature_cat", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"product_feature_cat\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":["load_cycle_id"], "map_type":"src_load"}, "src_load_incr_ms_product_feature_class" : {"dependencies": ["set_fmc_mtd_fl_incr_ms"], "src_objects":[{"src_object_name": "product_feature_class", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"product_feature_class\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":["load_cycle_id"], "map_type":"src_load"}, "src_load_incr_ms_product_features" : {"dependencies": ["set_fmc_mtd_fl_incr_ms"], "src_objects":[{"src_object_name": "product_features", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"product_features\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":["load_cycle_id"], "map_type":"src_load"}, "src_load_incr_ms_products" : {"dependencies": ["set_fmc_mtd_fl_incr_ms"], "src_objects":[{"src_object_name": "products", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"products\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":["load_cycle_id"], "map_type":"src_load"}, "set_fmc_object_mtd_incr_ms_addresses" : {"dependencies": ["src_load_incr_ms_addresses"], "src_objects":[{"src_object_name": "addresses", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"addresses\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":["load_cycle_id", "load_date"], "map_type":"sql_mapping"}, "set_fmc_object_mtd_incr_ms_codes_to_language" : {"dependencies": ["src_load_incr_ms_codes_to_language"], "src_objects":[{"src_object_name": "codes_to_language", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"codes_to_language\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":["load_cycle_id", "load_date"], "map_type":"sql_mapping"}, "set_fmc_object_mtd_incr_ms_customers" : {"dependencies": ["src_load_incr_ms_customers"], "src_objects":[{"src_object_name": "customers", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"customers\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":["load_cycle_id", "load_date"], "map_type":"sql_mapping"}, "set_fmc_object_mtd_incr_ms_invoice_lines" : {"dependencies": ["src_load_incr_ms_invoice_lines"], "src_objects":[{"src_object_name": "invoice_lines", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"invoice_lines\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":["load_cycle_id", "load_date"], "map_type":"sql_mapping"}, "set_fmc_object_mtd_incr_ms_invoices" : {"dependencies": ["src_load_incr_ms_invoices"], "src_objects":[{"src_object_name": "invoices", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"invoices\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":["load_cycle_id", "load_date"], "map_type":"sql_mapping"}, "set_fmc_object_mtd_incr_ms_parts" : {"dependencies": ["src_load_incr_ms_parts"], "src_objects":[{"src_object_name": "parts", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"parts\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":["load_cycle_id", "load_date"], "map_type":"sql_mapping"}, "set_fmc_object_mtd_incr_ms_payments" : {"dependencies": ["src_load_incr_ms_payments"], "src_objects":[{"src_object_name": "payments", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"payments\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":["load_cycle_id", "load_date"], "map_type":"sql_mapping"}, "set_fmc_object_mtd_incr_ms_product_feat_class_rel" : {"dependencies": ["src_load_incr_ms_product_feat_class_rel"], "src_objects":[{"src_object_name": "product_feat_class_rel", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"product_feat_class_rel\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":["load_cycle_id", "load_date"], "map_type":"sql_mapping"}, "set_fmc_object_mtd_incr_ms_product_feature_cat" : {"dependencies": ["src_load_incr_ms_product_feature_cat"], "src_objects":[{"src_object_name": "product_feature_cat", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"product_feature_cat\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":["load_cycle_id", "load_date"], "map_type":"sql_mapping"}, "set_fmc_object_mtd_incr_ms_product_feature_class" : {"dependencies": ["src_load_incr_ms_product_feature_class"], "src_objects":[{"src_object_name": "product_feature_class", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"product_feature_class\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":["load_cycle_id", "load_date"], "map_type":"sql_mapping"}, "set_fmc_object_mtd_incr_ms_product_features" : {"dependencies": ["src_load_incr_ms_product_features"], "src_objects":[{"src_object_name": "product_features", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"product_features\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":["load_cycle_id", "load_date"], "map_type":"sql_mapping"}, "set_fmc_object_mtd_incr_ms_products" : {"dependencies": ["src_load_incr_ms_products"], "src_objects":[{"src_object_name": "products", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"products\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":["load_cycle_id", "load_date"], "map_type":"sql_mapping"}, "ext_ms_addresses_incr" : {"dependencies": ["set_fmc_object_mtd_incr_ms_addresses"], "src_objects":[{"src_object_name": "addresses", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"addresses\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, "ext_ms_altaddresses_incr" : {"dependencies": ["set_fmc_object_mtd_incr_ms_addresses"], "src_objects":[{"src_object_name": "addresses", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"addresses\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, "ext_ms_customers_incr" : {"dependencies": ["set_fmc_object_mtd_incr_ms_customers"], "src_objects":[{"src_object_name": "customers", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"customers\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, "ext_ms_invoicelines_incr" : {"dependencies": ["set_fmc_object_mtd_incr_ms_invoice_lines"], "src_objects":[{"src_object_name": "invoice_lines", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"invoice_lines\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, "ext_ms_invoices_incr" : {"dependencies": ["set_fmc_object_mtd_incr_ms_invoices"], "src_objects":[{"src_object_name": "invoices", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"invoices\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, "ext_ms_parts_incr" : {"dependencies": ["set_fmc_object_mtd_incr_ms_parts"], "src_objects":[{"src_object_name": "parts", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"parts\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, "ext_ms_payments_incr" : {"dependencies": ["set_fmc_object_mtd_incr_ms_payments"], "src_objects":[{"src_object_name": "payments", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"payments\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, "ext_ms_productfeatclassrel_incr" : {"dependencies": ["set_fmc_object_mtd_incr_ms_product_feat_class_rel"], "src_objects":[{"src_object_name": "product_feat_class_rel", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"product_feat_class_rel\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, "ext_ms_productfeaturecat_incr" : {"dependencies": ["set_fmc_object_mtd_incr_ms_product_feature_cat"], "src_objects":[{"src_object_name": "product_feature_cat", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"product_feature_cat\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, "ext_ms_productfeatureclass_incr" : {"dependencies": ["set_fmc_object_mtd_incr_ms_product_feature_class"], "src_objects":[{"src_object_name": "product_feature_class", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"product_feature_class\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, "ext_ms_productfeatures_incr" : {"dependencies": ["set_fmc_object_mtd_incr_ms_product_features"], "src_objects":[{"src_object_name": "product_features", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"product_features\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, "ext_ms_products_incr" : {"dependencies": ["set_fmc_object_mtd_incr_ms_products"], "src_objects":[{"src_object_name": "products", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"products\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, "stg_ms_addresses_incr" : {"dependencies": ["ext_ms_addresses_incr"], "src_objects":[{"src_object_name": "addresses", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"addresses\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, "stg_ms_altaddresses_incr" : {"dependencies": ["ext_ms_altaddresses_incr"], "src_objects":[{"src_object_name": "addresses", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"addresses\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, "stg_ms_customers_incr" : {"dependencies": ["ext_ms_customers_incr", "ext_ms_addresses_incr"], "src_objects":[{"src_object_name": "customers", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"customers\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, "stg_ms_invoicelines_incr" : {"dependencies": ["ext_ms_invoicelines_incr", "ext_ms_parts_incr", "ext_ms_products_incr"], "src_objects":[{"src_object_name": "invoice_lines", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"invoice_lines\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, "stg_ms_invoices_incr" : {"dependencies": ["ext_ms_invoices_incr", "ext_ms_customers_incr"], "src_objects":[{"src_object_name": "invoices", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"invoices\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, "stg_ms_parts_incr" : {"dependencies": ["ext_ms_parts_incr"], "src_objects":[{"src_object_name": "parts", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"parts\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, "stg_ms_productfeaturecat_incr" : {"dependencies": ["ext_ms_productfeaturecat_incr"], "src_objects":[{"src_object_name": "product_feature_cat", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"product_feature_cat\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, "stg_ms_productfeatureclass_incr" : {"dependencies": ["ext_ms_productfeatureclass_incr"], "src_objects":[{"src_object_name": "product_feature_class", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"product_feature_class\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, "stg_ms_productfeatures_incr" : {"dependencies": ["ext_ms_productfeatures_incr", "ext_ms_productfeaturecat_incr"], "src_objects":[{"src_object_name": "product_features", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"product_features\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, "stg_ms_products_incr" : {"dependencies": ["ext_ms_products_incr"], "src_objects":[{"src_object_name": "products", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"products\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, "stg_dl_ms_payments_incr" : {"dependencies": ["ext_ms_payments_incr", "ext_ms_customers_incr"], "src_objects":[{"src_object_name": "payments", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"payments\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, "stg_dl_ms_productfeatclassrel_incr" : {"dependencies": ["ext_ms_productfeatclassrel_incr", "ext_ms_productfeatureclass_incr", "ext_ms_productfeatures_incr", "ext_ms_products_incr"], "src_objects":[{"src_object_name": "product_feat_class_rel", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"product_feat_class_rel\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, "hub_ms_invoice_lines_incr" : {"dependencies": ["stg_ms_invoicelines_incr"], "src_objects":[{"src_object_name": "invoice_lines", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"invoice_lines\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, "hub_ms_invoices_incr" : {"dependencies": ["stg_ms_invoices_incr"], "src_objects":[{"src_object_name": "invoices", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"invoices\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, "hub_ms_customers_incr" : {"dependencies": ["stg_ms_customers_incr"], "src_objects":[{"src_object_name": "customers", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"customers\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, "hub_ms_product_feature_cat_incr" : {"dependencies": ["stg_ms_productfeaturecat_incr"], "src_objects":[{"src_object_name": "product_feature_cat", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"product_feature_cat\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, "hub_ms_product_feature_class_incr" : {"dependencies": ["stg_ms_productfeatureclass_incr"], "src_objects":[{"src_object_name": "product_feature_class", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"product_feature_class\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, "hub_ms_product_features_incr" : {"dependencies": ["stg_ms_productfeatures_incr"], "src_objects":[{"src_object_name": "product_features", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"product_features\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, "hub_ms_addresses_incr" : {"dependencies": ["stg_ms_addresses_incr", "stg_ms_altaddresses_incr"], "src_objects":[{"src_object_name": "addresses", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"addresses\"", "schema_name": "\"moto_sales_dtv\""}, {"src_object_name": "addresses", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"addresses\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, "hub_ms_products_incr" : {"dependencies": ["stg_ms_products_incr"], "src_objects":[{"src_object_name": "products", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"products\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, "hub_ms_parts_incr" : {"dependencies": ["stg_ms_parts_incr"], "src_objects":[{"src_object_name": "parts", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"parts\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, "sat_ms_parts_incr" : {"dependencies": ["stg_ms_parts_incr"], "src_objects":[{"src_object_name": "parts", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"parts\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, "sat_ms_productfeatures_incr" : {"dependencies": ["stg_ms_productfeatures_incr"], "src_objects":[{"src_object_name": "product_features", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"product_features\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, "sat_ms_invoicelines_incr" : {"dependencies": ["stg_ms_invoicelines_incr"], "src_objects":[{"src_object_name": "invoice_lines", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"invoice_lines\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, "sat_ms_customers_birth_incr" : {"dependencies": ["stg_ms_customers_incr"], "src_objects":[{"src_object_name": "customers", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"customers\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, "sat_ms_altaddresses_incr" : {"dependencies": ["stg_ms_altaddresses_incr"], "src_objects":[{"src_object_name": "addresses", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"addresses\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, "sat_ms_customers_name_incr" : {"dependencies": ["stg_ms_customers_incr"], "src_objects":[{"src_object_name": "customers", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"customers\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, "sat_ms_productfeaturecat_incr" : {"dependencies": ["stg_ms_productfeaturecat_incr"], "src_objects":[{"src_object_name": "product_feature_cat", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"product_feature_cat\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, "sat_ms_products_incr" : {"dependencies": ["stg_ms_products_incr"], "src_objects":[{"src_object_name": "products", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"products\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, "sat_ms_addresses_incr" : {"dependencies": ["stg_ms_addresses_incr"], "src_objects":[{"src_object_name": "addresses", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"addresses\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, "sat_ms_invoices_incr" : {"dependencies": ["stg_ms_invoices_incr"], "src_objects":[{"src_object_name": "invoices", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"invoices\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, "sat_ms_productfeatureclass_incr" : {"dependencies": ["stg_ms_productfeatureclass_incr"], "src_objects":[{"src_object_name": "product_feature_class", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"product_feature_class\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, "lnk_ms_invoicelines_invoices_incr" : {"dependencies": ["stg_ms_invoicelines_incr"], "src_objects":[{"src_object_name": "invoice_lines", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"invoice_lines\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, "lnk_ms_customers_addresses_customerinvoiceaddressid_incr" : {"dependencies": ["stg_ms_customers_incr"], "src_objects":[{"src_object_name": "customers", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"customers\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, "lnk_ms_productfeatures_productfeaturecat_incr" : {"dependencies": ["stg_ms_productfeatures_incr"], "src_objects":[{"src_object_name": "product_features", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"product_features\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, "lnk_ms_invoices_customers_incr" : {"dependencies": ["stg_ms_invoices_incr"], "src_objects":[{"src_object_name": "invoices", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"invoices\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, "lnk_ms_invoicelines_parts_incr" : {"dependencies": ["stg_ms_invoicelines_incr"], "src_objects":[{"src_object_name": "invoice_lines", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"invoice_lines\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, "lnk_ms_customers_addresses_customershiptoaddressid_incr" : {"dependencies": ["stg_ms_customers_incr"], "src_objects":[{"src_object_name": "customers", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"customers\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, "lnk_ms_invoicelines_products_incr" : {"dependencies": ["stg_ms_invoicelines_incr"], "src_objects":[{"src_object_name": "invoice_lines", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"invoice_lines\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, "lnk_ms_products_products_replacementproductid_incr" : {"dependencies": ["stg_ms_products_incr"], "src_objects":[{"src_object_name": "products", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"products\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, "lks_ms_customers_addresses_customerinvoiceaddressid_incr" : {"dependencies": ["stg_ms_customers_incr"], "src_objects":[{"src_object_name": "customers", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"customers\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, "lks_ms_invoicelines_products_incr" : {"dependencies": ["stg_ms_invoicelines_incr"], "src_objects":[{"src_object_name": "invoice_lines", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"invoice_lines\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, "lks_ms_invoices_customers_incr" : {"dependencies": ["stg_ms_invoices_incr"], "src_objects":[{"src_object_name": "invoices", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"invoices\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, "lks_ms_invoicelines_parts_incr" : {"dependencies": ["stg_ms_invoicelines_incr"], "src_objects":[{"src_object_name": "invoice_lines", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"invoice_lines\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, "lks_ms_invoicelines_invoices_incr" : {"dependencies": ["stg_ms_invoicelines_incr"], "src_objects":[{"src_object_name": "invoice_lines", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"invoice_lines\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, "lks_ms_products_products_replacementproductid_incr" : {"dependencies": ["stg_ms_products_incr"], "src_objects":[{"src_object_name": "products", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"products\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, "lks_ms_customers_addresses_customershiptoaddressid_incr" : {"dependencies": ["stg_ms_customers_incr"], "src_objects":[{"src_object_name": "customers", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"customers\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, "lks_ms_productfeatures_productfeaturecat_incr" : {"dependencies": ["stg_ms_productfeatures_incr"], "src_objects":[{"src_object_name": "product_features", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"product_features\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, "lnd_ms_product_feat_class_rel_incr" : {"dependencies": ["stg_dl_ms_productfeatclassrel_incr"], "src_objects":[{"src_object_name": "product_feat_class_rel", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"product_feat_class_rel\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, "nhl_ms_payments_incr" : {"dependencies": ["stg_dl_ms_payments_incr"], "src_objects":[{"src_object_name": "payments", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"payments\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, "lds_ms_productfeatclassrel_incr" : {"dependencies": ["stg_dl_ms_productfeatclassrel_incr"], "src_objects":[{"src_object_name": "product_feat_class_rel", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"product_feat_class_rel\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, "ref_ms_codestolanguage_incr" : {"dependencies": ["set_fmc_object_mtd_incr_ms_codes_to_language"], "src_objects":[{"src_object_name": "codes_to_language", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"codes_to_language\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, "fmc_upd_object_status_ms_product_features" : {"dependencies": ["hub_ms_product_features_incr", "lks_ms_productfeatures_productfeaturecat_incr", "lnk_ms_productfeatures_productfeaturecat_incr", "sat_ms_productfeatures_incr"], "src_objects":[{"src_object_name": "product_features", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"product_features\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":["load_cycle_id", "success_flag"], "map_type":"sql_mapping"}, "fmc_upd_object_status_ms_products" : {"dependencies": ["sat_ms_products_incr", "lks_ms_products_products_replacementproductid_incr", "hub_ms_products_incr", "lnk_ms_products_products_replacementproductid_incr"], "src_objects":[{"src_object_name": "products", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"products\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":["load_cycle_id", "success_flag"], "map_type":"sql_mapping"}, "fmc_upd_object_status_ms_addresses" : {"dependencies": ["sat_ms_altaddresses_incr", "hub_ms_addresses_incr", "hub_ms_addresses_incr", "sat_ms_addresses_incr"], "src_objects":[{"src_object_name": "addresses", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"addresses\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":["load_cycle_id", "success_flag"], "map_type":"sql_mapping"}, "fmc_upd_object_status_ms_codes_to_language" : {"dependencies": ["ref_ms_codestolanguage_incr"], "src_objects":[{"src_object_name": "codes_to_language", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"codes_to_language\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":["load_cycle_id", "success_flag"], "map_type":"sql_mapping"}, "fmc_upd_object_status_ms_customers" : {"dependencies": ["sat_ms_customers_name_incr", "lks_ms_customers_addresses_customerinvoiceaddressid_incr", "sat_ms_customers_birth_incr", "lnk_ms_customers_addresses_customerinvoiceaddressid_incr", "hub_ms_customers_incr", "lks_ms_customers_addresses_customershiptoaddressid_incr", "lnk_ms_customers_addresses_customershiptoaddressid_incr"], "src_objects":[{"src_object_name": "customers", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"customers\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":["load_cycle_id", "success_flag"], "map_type":"sql_mapping"}, "fmc_upd_object_status_ms_invoice_lines" : {"dependencies": ["lks_ms_invoicelines_products_incr", "hub_ms_invoice_lines_incr", "lnk_ms_invoicelines_products_incr", "lnk_ms_invoicelines_parts_incr", "lks_ms_invoicelines_parts_incr", "lks_ms_invoicelines_invoices_incr", "lnk_ms_invoicelines_invoices_incr", "sat_ms_invoicelines_incr"], "src_objects":[{"src_object_name": "invoice_lines", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"invoice_lines\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":["load_cycle_id", "success_flag"], "map_type":"sql_mapping"}, "fmc_upd_object_status_ms_invoices" : {"dependencies": ["lks_ms_invoices_customers_incr", "lnk_ms_invoices_customers_incr", "sat_ms_invoices_incr", "hub_ms_invoices_incr"], "src_objects":[{"src_object_name": "invoices", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"invoices\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":["load_cycle_id", "success_flag"], "map_type":"sql_mapping"}, "fmc_upd_object_status_ms_parts" : {"dependencies": ["hub_ms_parts_incr", "sat_ms_parts_incr"], "src_objects":[{"src_object_name": "parts", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"parts\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":["load_cycle_id", "success_flag"], "map_type":"sql_mapping"}, "fmc_upd_object_status_ms_payments" : {"dependencies": ["nhl_ms_payments_incr"], "src_objects":[{"src_object_name": "payments", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"payments\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":["load_cycle_id", "success_flag"], "map_type":"sql_mapping"}, "fmc_upd_object_status_ms_product_feat_class_rel" : {"dependencies": ["lnd_ms_product_feat_class_rel_incr", "lds_ms_productfeatclassrel_incr"], "src_objects":[{"src_object_name": "product_feat_class_rel", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"product_feat_class_rel\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":["load_cycle_id", "success_flag"], "map_type":"sql_mapping"}, "fmc_upd_object_status_ms_product_feature_cat" : {"dependencies": ["hub_ms_product_feature_cat_incr", "sat_ms_productfeaturecat_incr"], "src_objects":[{"src_object_name": "product_feature_cat", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"product_feature_cat\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":["load_cycle_id", "success_flag"], "map_type":"sql_mapping"}, "fmc_upd_object_status_ms_product_feature_class" : {"dependencies": ["sat_ms_productfeatureclass_incr", "hub_ms_product_feature_class_incr"], "src_objects":[{"src_object_name": "product_feature_class", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"product_feature_class\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":["load_cycle_id", "success_flag"], "map_type":"sql_mapping"}, "fmc_upd_run_status_fl_ms" : {"dependencies": ["nhl_ms_payments_incr", "ref_ms_codestolanguage_incr", "hub_ms_invoice_lines_incr", "hub_ms_invoices_incr", "hub_ms_customers_incr", "hub_ms_product_feature_cat_incr", "hub_ms_product_feature_class_incr", "hub_ms_product_features_incr", "hub_ms_addresses_incr", "hub_ms_addresses_incr", "hub_ms_products_incr", "hub_ms_parts_incr", "lks_ms_customers_addresses_customerinvoiceaddressid_incr", "lks_ms_invoicelines_products_incr", "lks_ms_invoices_customers_incr", "lks_ms_invoicelines_parts_incr", "lks_ms_invoicelines_invoices_incr", "lks_ms_products_products_replacementproductid_incr", "lks_ms_customers_addresses_customershiptoaddressid_incr", "lks_ms_productfeatures_productfeaturecat_incr", "sat_ms_parts_incr", "sat_ms_productfeatures_incr", "sat_ms_invoicelines_incr", "sat_ms_customers_birth_incr", "sat_ms_altaddresses_incr", "sat_ms_customers_name_incr", "sat_ms_productfeaturecat_incr", "sat_ms_products_incr", "sat_ms_addresses_incr", "sat_ms_invoices_incr", "sat_ms_productfeatureclass_incr", "lnk_ms_invoicelines_invoices_incr", "lnk_ms_customers_addresses_customerinvoiceaddressid_incr", "lnk_ms_productfeatures_productfeaturecat_incr", "lnk_ms_invoices_customers_incr", "lnk_ms_invoicelines_parts_incr", "lnk_ms_customers_addresses_customershiptoaddressid_incr", "lnk_ms_invoicelines_products_incr", "lnk_ms_products_products_replacementproductid_incr", "lds_ms_productfeatclassrel_incr", "lnd_ms_product_feat_class_rel_incr"], "src_objects": [], "map_schema": "\"moto_proc\"", "input":["load_cycle_id", "success_flag", "upd_all_objects"], "map_type":"sql_mapping"} } ```
grouped flow ```json [ { "layer": "start_fmc_metadata", "mappings": [ {"map":"set_fmc_mtd_fl_incr_ms", "src_objects":[], "map_schema": "\"moto_proc\"", "input":["dag_name", "load_cycle_id", "load_date"], "map_type":"sql_mapping"} ] }, { "layer": "source_loading", "mappings": [ {"map":"src_load_incr_ms_addresses", "src_objects":[{"src_object_name": "addresses", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"addresses\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":["load_cycle_id"], "map_type":"src_load"}, {"map":"src_load_incr_ms_codes_to_language", "src_objects":[{"src_object_name": "codes_to_language", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"codes_to_language\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":["load_cycle_id"], "map_type":"src_load"}, {"map":"src_load_incr_ms_customers", "src_objects":[{"src_object_name": "customers", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"customers\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":["load_cycle_id"], "map_type":"src_load"}, {"map":"src_load_incr_ms_invoice_lines", "src_objects":[{"src_object_name": "invoice_lines", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"invoice_lines\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":["load_cycle_id"], "map_type":"src_load"}, {"map":"src_load_incr_ms_invoices", "src_objects":[{"src_object_name": "invoices", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"invoices\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":["load_cycle_id"], "map_type":"src_load"}, {"map":"src_load_incr_ms_parts", "src_objects":[{"src_object_name": "parts", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"parts\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":["load_cycle_id"], "map_type":"src_load"}, {"map":"src_load_incr_ms_payments", "src_objects":[{"src_object_name": "payments", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"payments\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":["load_cycle_id"], "map_type":"src_load"}, {"map":"src_load_incr_ms_product_feat_class_rel", "src_objects":[{"src_object_name": "product_feat_class_rel", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"product_feat_class_rel\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":["load_cycle_id"], "map_type":"src_load"}, {"map":"src_load_incr_ms_product_feature_cat", "src_objects":[{"src_object_name": "product_feature_cat", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"product_feature_cat\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":["load_cycle_id"], "map_type":"src_load"}, {"map":"src_load_incr_ms_product_feature_class", "src_objects":[{"src_object_name": "product_feature_class", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"product_feature_class\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":["load_cycle_id"], "map_type":"src_load"}, {"map":"src_load_incr_ms_product_features", "src_objects":[{"src_object_name": "product_features", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"product_features\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":["load_cycle_id"], "map_type":"src_load"}, {"map":"src_load_incr_ms_products", "src_objects":[{"src_object_name": "products", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"products\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":["load_cycle_id"], "map_type":"src_load"} ] }, { "layer": "object_fmc_metadata", "mappings": [ {"map":"set_fmc_object_mtd_incr_ms_addresses", "src_objects":[{"src_object_name": "addresses", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"addresses\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":["load_cycle_id", "load_date"], "map_type":"sql_mapping"}, {"map":"set_fmc_object_mtd_incr_ms_codes_to_language", "src_objects":[{"src_object_name": "codes_to_language", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"codes_to_language\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":["load_cycle_id", "load_date"], "map_type":"sql_mapping"}, {"map":"set_fmc_object_mtd_incr_ms_customers", "src_objects":[{"src_object_name": "customers", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"customers\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":["load_cycle_id", "load_date"], "map_type":"sql_mapping"}, {"map":"set_fmc_object_mtd_incr_ms_invoice_lines", "src_objects":[{"src_object_name": "invoice_lines", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"invoice_lines\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":["load_cycle_id", "load_date"], "map_type":"sql_mapping"}, {"map":"set_fmc_object_mtd_incr_ms_invoices", "src_objects":[{"src_object_name": "invoices", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"invoices\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":["load_cycle_id", "load_date"], "map_type":"sql_mapping"}, {"map":"set_fmc_object_mtd_incr_ms_parts", "src_objects":[{"src_object_name": "parts", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"parts\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":["load_cycle_id", "load_date"], "map_type":"sql_mapping"}, {"map":"set_fmc_object_mtd_incr_ms_payments", "src_objects":[{"src_object_name": "payments", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"payments\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":["load_cycle_id", "load_date"], "map_type":"sql_mapping"}, {"map":"set_fmc_object_mtd_incr_ms_product_feat_class_rel", "src_objects":[{"src_object_name": "product_feat_class_rel", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"product_feat_class_rel\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":["load_cycle_id", "load_date"], "map_type":"sql_mapping"}, {"map":"set_fmc_object_mtd_incr_ms_product_feature_cat", "src_objects":[{"src_object_name": "product_feature_cat", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"product_feature_cat\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":["load_cycle_id", "load_date"], "map_type":"sql_mapping"}, {"map":"set_fmc_object_mtd_incr_ms_product_feature_class", "src_objects":[{"src_object_name": "product_feature_class", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"product_feature_class\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":["load_cycle_id", "load_date"], "map_type":"sql_mapping"}, {"map":"set_fmc_object_mtd_incr_ms_product_features", "src_objects":[{"src_object_name": "product_features", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"product_features\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":["load_cycle_id", "load_date"], "map_type":"sql_mapping"}, {"map":"set_fmc_object_mtd_incr_ms_products", "src_objects":[{"src_object_name": "products", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"products\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":["load_cycle_id", "load_date"], "map_type":"sql_mapping"} ] }, { "layer": "extraction_layer", "mappings": [ {"map":"ext_ms_customers_incr", "src_objects":[{"src_object_name": "customers", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"customers\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, {"map":"ext_ms_invoicelines_incr", "src_objects":[{"src_object_name": "invoice_lines", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"invoice_lines\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, {"map":"ext_ms_invoices_incr", "src_objects":[{"src_object_name": "invoices", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"invoices\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, {"map":"ext_ms_parts_incr", "src_objects":[{"src_object_name": "parts", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"parts\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, {"map":"ext_ms_payments_incr", "src_objects":[{"src_object_name": "payments", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"payments\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, {"map":"ext_ms_productfeatclassrel_incr", "src_objects":[{"src_object_name": "product_feat_class_rel", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"product_feat_class_rel\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, {"map":"ext_ms_productfeaturecat_incr", "src_objects":[{"src_object_name": "product_feature_cat", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"product_feature_cat\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, {"map":"ext_ms_productfeatureclass_incr", "src_objects":[{"src_object_name": "product_feature_class", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"product_feature_class\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, {"map":"ext_ms_productfeatures_incr", "src_objects":[{"src_object_name": "product_features", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"product_features\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, {"map":"ext_ms_products_incr", "src_objects":[{"src_object_name": "products", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"products\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, {"map":"ext_ms_addresses_incr", "src_objects":[{"src_object_name": "addresses", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"addresses\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, {"map":"ext_ms_altaddresses_incr", "src_objects":[{"src_object_name": "addresses", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"addresses\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"} ] }, { "layer": "staging_layer", "mappings": [ {"map":"stg_ms_customers_incr", "src_objects":[{"src_object_name": "customers", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"customers\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, {"map":"stg_ms_invoicelines_incr", "src_objects":[{"src_object_name": "invoice_lines", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"invoice_lines\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, {"map":"stg_ms_invoices_incr", "src_objects":[{"src_object_name": "invoices", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"invoices\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, {"map":"stg_ms_parts_incr", "src_objects":[{"src_object_name": "parts", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"parts\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, {"map":"stg_dl_ms_payments_incr", "src_objects":[{"src_object_name": "payments", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"payments\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, {"map":"stg_dl_ms_productfeatclassrel_incr", "src_objects":[{"src_object_name": "product_feat_class_rel", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"product_feat_class_rel\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, {"map":"stg_ms_productfeaturecat_incr", "src_objects":[{"src_object_name": "product_feature_cat", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"product_feature_cat\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, {"map":"stg_ms_productfeatureclass_incr", "src_objects":[{"src_object_name": "product_feature_class", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"product_feature_class\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, {"map":"stg_ms_productfeatures_incr", "src_objects":[{"src_object_name": "product_features", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"product_features\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, {"map":"stg_ms_products_incr", "src_objects":[{"src_object_name": "products", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"products\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, {"map":"stg_ms_addresses_incr", "src_objects":[{"src_object_name": "addresses", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"addresses\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, {"map":"stg_ms_altaddresses_incr", "src_objects":[{"src_object_name": "addresses", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"addresses\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"} ] }, { "layer": "foundation_layer", "mappings": [ {"map":"hub_ms_customers_incr", "src_objects":[{"src_object_name": "customers", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"customers\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, {"map":"hub_ms_invoice_lines_incr", "src_objects":[{"src_object_name": "invoice_lines", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"invoice_lines\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, {"map":"hub_ms_invoices_incr", "src_objects":[{"src_object_name": "invoices", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"invoices\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, {"map":"hub_ms_parts_incr", "src_objects":[{"src_object_name": "parts", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"parts\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, {"map":"hub_ms_product_feature_cat_incr", "src_objects":[{"src_object_name": "product_feature_cat", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"product_feature_cat\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, {"map":"hub_ms_product_feature_class_incr", "src_objects":[{"src_object_name": "product_feature_class", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"product_feature_class\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, {"map":"hub_ms_products_incr", "src_objects":[{"src_object_name": "products", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"products\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, {"map":"hub_ms_product_features_incr", "src_objects":[{"src_object_name": "product_features", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"product_features\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, {"map":"hub_ms_addresses_incr", "src_objects":[{"src_object_name": "addresses", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"addresses\"", "schema_name": "\"moto_sales_dtv\""}, {"src_object_name": "addresses", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"addresses\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, {"map":"sat_ms_customers_name_incr", "src_objects":[{"src_object_name": "customers", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"customers\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, {"map":"sat_ms_customers_birth_incr", "src_objects":[{"src_object_name": "customers", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"customers\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, {"map":"sat_ms_invoicelines_incr", "src_objects":[{"src_object_name": "invoice_lines", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"invoice_lines\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, {"map":"sat_ms_invoices_incr", "src_objects":[{"src_object_name": "invoices", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"invoices\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, {"map":"sat_ms_parts_incr", "src_objects":[{"src_object_name": "parts", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"parts\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, {"map":"sat_ms_productfeaturecat_incr", "src_objects":[{"src_object_name": "product_feature_cat", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"product_feature_cat\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, {"map":"sat_ms_productfeatureclass_incr", "src_objects":[{"src_object_name": "product_feature_class", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"product_feature_class\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, {"map":"sat_ms_products_incr", "src_objects":[{"src_object_name": "products", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"products\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, {"map":"sat_ms_productfeatures_incr", "src_objects":[{"src_object_name": "product_features", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"product_features\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, {"map":"sat_ms_altaddresses_incr", "src_objects":[{"src_object_name": "addresses", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"addresses\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, {"map":"sat_ms_addresses_incr", "src_objects":[{"src_object_name": "addresses", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"addresses\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, {"map":"lnk_ms_productfeatures_productfeaturecat_incr", "src_objects":[{"src_object_name": "product_features", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"product_features\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, {"map":"lnk_ms_products_products_replacementproductid_incr", "src_objects":[{"src_object_name": "products", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"products\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, {"map":"lnk_ms_invoices_customers_incr", "src_objects":[{"src_object_name": "invoices", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"invoices\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, {"map":"lnk_ms_invoicelines_products_incr", "src_objects":[{"src_object_name": "invoice_lines", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"invoice_lines\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, {"map":"lnk_ms_invoicelines_parts_incr", "src_objects":[{"src_object_name": "invoice_lines", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"invoice_lines\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, {"map":"lnk_ms_invoicelines_invoices_incr", "src_objects":[{"src_object_name": "invoice_lines", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"invoice_lines\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, {"map":"lnk_ms_customers_addresses_customershiptoaddressid_incr", "src_objects":[{"src_object_name": "customers", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"customers\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, {"map":"lnk_ms_customers_addresses_customerinvoiceaddressid_incr", "src_objects":[{"src_object_name": "customers", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"customers\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, {"map":"lks_ms_productfeatures_productfeaturecat_incr", "src_objects":[{"src_object_name": "product_features", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"product_features\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, {"map":"lks_ms_products_products_replacementproductid_incr", "src_objects":[{"src_object_name": "products", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"products\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, {"map":"lks_ms_invoices_customers_incr", "src_objects":[{"src_object_name": "invoices", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"invoices\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, {"map":"lks_ms_invoicelines_products_incr", "src_objects":[{"src_object_name": "invoice_lines", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"invoice_lines\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, {"map":"lks_ms_invoicelines_parts_incr", "src_objects":[{"src_object_name": "invoice_lines", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"invoice_lines\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, {"map":"lks_ms_invoicelines_invoices_incr", "src_objects":[{"src_object_name": "invoice_lines", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"invoice_lines\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, {"map":"lks_ms_customers_addresses_customershiptoaddressid_incr", "src_objects":[{"src_object_name": "customers", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"customers\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, {"map":"lks_ms_customers_addresses_customerinvoiceaddressid_incr", "src_objects":[{"src_object_name": "customers", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"customers\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, {"map":"lnd_ms_product_feat_class_rel_incr", "src_objects":[{"src_object_name": "product_feat_class_rel", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"product_feat_class_rel\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, {"map":"lds_ms_productfeatclassrel_incr", "src_objects":[{"src_object_name": "product_feat_class_rel", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"product_feat_class_rel\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, {"map":"nhl_ms_payments_incr", "src_objects":[{"src_object_name": "payments", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"payments\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"}, {"map":"ref_ms_codestolanguage_incr", "src_objects":[{"src_object_name": "codes_to_language", "has_trans_indicator": true, "trans_indicator": "trans_timestamp", "modification_type":"date", "table_name": "\"codes_to_language\"", "schema_name": "\"moto_sales_dtv\""}], "map_schema": "\"moto_proc\"", "input":[], "map_type":"sql_mapping"} ] }, { "layer": "end_fmc_metadata", "mappings": [ {"map":"fmc_upd_run_status_fl_ms", "src_objects":[], "map_schema": "\"moto_proc\"", "input":["load_cycle_id", "success_flag", "upd_all_objects"], "map_type":"sql_mapping"} ] } ] ```
object metadata ```json { "hub_invoice_lines" : {"dependencies": ["hub_ms_invoice_lines_incr"], "schema": "moto_fl"}, "hub_invoices" : {"dependencies": ["hub_ms_invoices_incr"], "schema": "moto_fl"}, "hub_customers" : {"dependencies": ["hub_ms_customers_incr"], "schema": "moto_fl"}, "hub_product_feature_cat" : {"dependencies": ["hub_ms_product_feature_cat_incr"], "schema": "moto_fl"}, "hub_product_feature_class" : {"dependencies": ["hub_ms_product_feature_class_incr"], "schema": "moto_fl"}, "hub_product_features" : {"dependencies": ["hub_ms_product_features_incr"], "schema": "moto_fl"}, "hub_addresses" : {"dependencies": ["hub_ms_addresses_incr"], "schema": "moto_fl"}, "hub_products" : {"dependencies": ["hub_ms_products_incr"], "schema": "moto_fl"}, "hub_parts" : {"dependencies": ["hub_ms_parts_incr"], "schema": "moto_fl"}, "sat_ms_parts" : {"dependencies": ["sat_ms_parts_incr"], "schema": "moto_fl"}, "sat_ms_product_features" : {"dependencies": ["sat_ms_productfeatures_incr"], "schema": "moto_fl"}, "sat_ms_invoice_lines" : {"dependencies": ["sat_ms_invoicelines_incr"], "schema": "moto_fl"}, "sat_ms_customers_birth" : {"dependencies": ["sat_ms_customers_birth_incr"], "schema": "moto_fl"}, "sat_ms_altaddresses_addresses" : {"dependencies": ["sat_ms_altaddresses_incr"], "schema": "moto_fl"}, "sat_ms_customers_name" : {"dependencies": ["sat_ms_customers_name_incr"], "schema": "moto_fl"}, "sat_ms_product_feature_cat" : {"dependencies": ["sat_ms_productfeaturecat_incr"], "schema": "moto_fl"}, "sat_ms_products" : {"dependencies": ["sat_ms_products_incr"], "schema": "moto_fl"}, "sat_ms_addresses_addresses" : {"dependencies": ["sat_ms_addresses_incr"], "schema": "moto_fl"}, "sat_ms_invoices" : {"dependencies": ["sat_ms_invoices_incr"], "schema": "moto_fl"}, "sat_ms_product_feature_class" : {"dependencies": ["sat_ms_productfeatureclass_incr"], "schema": "moto_fl"}, "lnk_invoicelines_invoices" : {"dependencies": ["lnk_ms_invoicelines_invoices_incr"], "schema": "moto_fl"}, "lnk_customers_addresses_customerinvoiceaddressid" : {"dependencies": ["lnk_ms_customers_addresses_customerinvoiceaddressid_incr"], "schema": "moto_fl"}, "lnk_productfeatures_productfeaturecat" : {"dependencies": ["lnk_ms_productfeatures_productfeaturecat_incr"], "schema": "moto_fl"}, "lnk_invoices_customers" : {"dependencies": ["lnk_ms_invoices_customers_incr"], "schema": "moto_fl"}, "lnk_invoicelines_parts" : {"dependencies": ["lnk_ms_invoicelines_parts_incr"], "schema": "moto_fl"}, "lnk_customers_addresses_customershiptoaddressid" : {"dependencies": ["lnk_ms_customers_addresses_customershiptoaddressid_incr"], "schema": "moto_fl"}, "lnk_invoicelines_products" : {"dependencies": ["lnk_ms_invoicelines_products_incr"], "schema": "moto_fl"}, "lnk_products_products_replacementproductid" : {"dependencies": ["lnk_ms_products_products_replacementproductid_incr"], "schema": "moto_fl"}, "lks_ms_customers_addresses_customerinvoiceaddressid" : {"dependencies": ["lks_ms_customers_addresses_customerinvoiceaddressid_incr"], "schema": "moto_fl"}, "lks_ms_invoicelines_products" : {"dependencies": ["lks_ms_invoicelines_products_incr"], "schema": "moto_fl"}, "lks_ms_invoices_customers" : {"dependencies": ["lks_ms_invoices_customers_incr"], "schema": "moto_fl"}, "lks_ms_invoicelines_parts" : {"dependencies": ["lks_ms_invoicelines_parts_incr"], "schema": "moto_fl"}, "lks_ms_invoicelines_invoices" : {"dependencies": ["lks_ms_invoicelines_invoices_incr"], "schema": "moto_fl"}, "lks_ms_products_products_replacementproductid" : {"dependencies": ["lks_ms_products_products_replacementproductid_incr"], "schema": "moto_fl"}, "lks_ms_customers_addresses_customershiptoaddressid" : {"dependencies": ["lks_ms_customers_addresses_customershiptoaddressid_incr"], "schema": "moto_fl"}, "lks_ms_productfeatures_productfeaturecat" : {"dependencies": ["lks_ms_productfeatures_productfeaturecat_incr"], "schema": "moto_fl"}, "lnd_product_feat_class_rel" : {"dependencies": ["lnd_ms_product_feat_class_rel_incr"], "schema": "moto_fl"}, "nhl_ms_payments" : {"dependencies": ["nhl_ms_payments_incr"], "schema": "moto_fl"}, "lds_ms_product_feat_class_rel" : {"dependencies": ["lds_ms_productfeatclassrel_incr"], "schema": "moto_fl"}, "ref_ms_codes_to_language" : {"dependencies": ["ref_ms_codestolanguage_incr"], "schema": "moto_fl"} } ```
# Mappings ## Input Variables The following input variables are required by some of the FMC mappings. When a mapping has one of these variables in its `input` field in the JSON, the FMC solution should provide these variables to the mappings. - `dag_name`: The name of the flow, which can be found in the info file. - `load_cycle_id`: A unique number identifying the execution of a flow. This is calculated automatically when inserting into the FMC loading history table. - `load_date`: The date at which the load is scheduled to be executed. This does not have to be the same date as when the flow actually starts to run. This timestamp should be passed as a string in the following format: `%Y-%m-%d %H:%M:%S.%f`. If the parameter `LOAD_TIMESTAMP_WITH_TZ` is enabled, you should add a time zone to this format. With the `CONVERT_FMC_TIMEZONE` parameter enabled, this timestamp should be from the timezone entered in the `FMC_TIMEZONE` parameter. The combination of load_date and dag_name should be unique. - `FMC_BEGIN_LW_TIMESTAMP`: The beginning of the loading window. This is only used when the dynamic loading window is disabled. This timestamp should be passed as a string in the following format: `%Y-%m-%d %H:%M:%S.%f`. With the `CONVERT_FMC_TIMEZONE` parameter enabled, this timestamp should be from the timezone entered in the `FMC_TIMEZONE` parameter. - `success_flag`: 0 or 1, this indicates the state of the flow. - `UPD_ALL_OBJECTS`: This determines whether the `FMC_UPD_RUN_STATUS` mapping should also update the status of specific objects. For grouped flows, this should be set to `Y` since, in that case, there are no `FMC_UPD_OBJECT_STATUS` mappings. For ungrouped flows, there are mappings that set the state for a specific object. So, in that case, `UPD_ALL_OBJECTS` should be set to `N`. How these parameters should be passed to the mapping will depend on the target technology and will be described in the chapter “Target Technologies” below. ## FMC Mappings VaultSpeed will generate the following Mappings for the FMC. Note that these mappings will always be SQL mappings, meaning that, even when generating code for an ETL tool (Matillion, dbt, etc.), these mappings will always be target Database SQL mappings. | **mapping** | **description** | **input** | | --- | --- | --- | | SET_FMC_MTD_FL_INI | This mapping inserts all the FMC metadata for the initial load of the raw vault for a certain source. | - flow_name/dag_name: String (same value as entered in VaultSpeed when creating the flow) - load_date: String (timestamp value in format `"%Y-%m-%d %H:%M:%S.%f"`) | | SET_FMC_MTD_FL_INCR | This mapping inserts all the FMC metadata for the incremental load of the raw vault for a certain source. | - flow_name/dag_name: String (same value as entered in VaultSpeed when creating the flow) - load_date: String (timestamp value in format `"%Y-%m-%d %H:%M:%S.%f"`) | | SET_FMC_MTD_BV_INI | This mapping inserts all the FMC metadata for the initial load of the business vault. | - flow_name/dag_name: String (same value as entered in VaultSpeed when creating the flow) - load_date: String (timestamp value in format `"%Y-%m-%d %H:%M:%S.%f"`) | | SET_FMC_MTD_BV_INCR | This mapping inserts all the FMC metadata for the incremental load of the raw vault for a certain source. | - flow_name/dag_name: String (same value as entered in VaultSpeed when creating the flow) - load_date: String (timestamp value in format `"%Y-%m-%d %H:%M:%S.%f"`) | | SET_FMC_OBJECT_MTD_INCR | This mapping inserts the FMC metadata for the initial load of a specific object.
This mapping is used when `OBJECT_SPECIFIC_LOADING_WINDOW = Y` and `FMC_GENERATE_SRC_LOADING_SCRIPTS = Y` and `CDC_BASED_LOADING_WINDOW = Y`
In those cases the flow is SET_FMC_MTD_FL_INCR for getting the start of the loading window, then loading the source data (all data after window start date), and then SET_FMC_OBJECT_MTD_INCR for determining the end of the loading window. Since the end of the loading window calculation requires the data to be loaded from the source. | - load_date: String (timestamp value in format `"%Y-%m-%d %H:%M:%S.%f"`) | | SET_FMC_OBJECT_MTD_INIT | This mapping inserts the FMC metadata for the incrementalload of a specific object.
This mapping is used when `OBJECT_SPECIFIC_LOADING_WINDOW = Y` and `FMC_GENERATE_SRC_LOADING_SCRIPTS = Y` and `CDC_BASED_LOADING_WINDOW = Y` | - load_date: String (timestamp value in format `"%Y-%m-%d %H:%M:%S.%f"`) | | FMC_UPD_RUN_STATUS_FL | This mappings sets the status of a raw vault load (success or failure) and its end time. It uses the content of the Load Cycle Info table to determine what the current load cycle id is to update the correct record in the history table. | - success_flag: String (“1“ if the load completes successfully, “0” if it failed) | | FMC_UPD_OBJECT_STATUS | This mappings sets the status of a raw vault load (success or failure) and its end time for a specific object. This is used with `OBJECT_SPECIFIC_LOADING_WINDOW = Y` and with non-grouped flows.
It uses the content of the Load Cycle Info table to determine what the current load cycle id is to update the correct record in the history table. | - success_flag: String (“1“ if the load successes, “0” if it failed) - object_name: String (Name of the object that failed, this name will be in the JSON metadata for each mapping) | | FMC_UPD_RUN_STATUS_BV | This mappings sets the status of a business vault load (success or failure) and its end time. It uses the content of the Load Cycle Info table to determine what the current load cycle id is to update the correct record in the history table. | - success_flag: String (“1“ if the load successes, “0” if it failed) | ## Target Technologies ### Databricks Databricks mapping names in the JSON files are relative paths to the notebooks, so to execute them, they should be prefixed by the base directory where the notebooks are deployed (this is one of the parameters in the agent connection). Depending on the deployment method, you might also have to add a file extension, “.sql”. The info JSON provides the fmc_file_extention property, which could be used for this purpose. The input variables can be passed to notebooks via the `base_parameters` property. The exact method will depend on how the notebook execution happens. Consult the relevant Databricks Documentation. ### Apache Spark For Cloudera Spark SQL, the mapping names are relative paths to SQL files, these files will have to be read, and their content executed in Spark SQL. The input variables are in the files with curly brackets around them, for example: `{load_cycle_id}`. If you are using python, you could then, for example, use the format function to fill in the variables before executing the SQL. ### Snowflake In Snowflake, the mappings are stored as procedures, so executing them is straightforward. ``` CALL .""(); ``` Do make sure that the raw vault mappings are executed in a transaction (auto-commit off). If insert-only logic is disabled. ### Bigquery Bigquery mappings are generated as SQL files, so in order to run them, the files should be read out and executed. The mapping names are relative paths to the files. The input variables are in the files with curly brackets around them, for example: `{load_cycle_id}`. If you are using python, you could then, for example, use the format function to fill in the variables before executing the SQL. ### Singlestore In Singlestore, the mappings are stored as procedures, so executing them is straightforward. ``` CALL .""(); ``` ### Groovy (ODI) ODI mappings (`map_type=etl_mapping`) come in the form of scenarios. The name of the scenario that should be executed in ODI to execute a mapping is `SCEN_`. There are no ODI mappings with input variables. ### Matillion For Matillion mappings(`map_type=etl_mapping`), the name of the mapping is equal to the name of the orchestration job in Matillion. There are no Matillion mappings with input variables. ### Talend For Talend mappings(`map_type=etl_mapping`), the name of the mapping is equal to the name of the job in Talend. There are no Talend mappings with input variables. ### DBT With dbt, the mapping name (`map_type=etl_mapping`) references a tag in dbt, since one mapping can consist of multiple models (e.g. the satellite mappings consists of a model for the temp table and a model for the SAT table. To execute dbt models, you have to use the mapping name as a model selection tag and add a variable “load_type”, which contains the load type (“INIT” or “INCR”), and for FL flows, also add a variable “source“ which contains the source name. The load type and the source name can be found in the info JSON file as “load_type” and “src_name”, respectively. ``` dbt run --select tag: --vars "{load_type: , source: }" ``` ### Oracle In Oracle, the mappings are stored as procedures, so executing them is straightforward. ``` BEGIN .""(); END; ``` ### PostgreSQL In PostgreSQL, the mappings are stored as procedures, so executing them is straightforward. ``` select .""(); ``` Do make sure that the raw vault mappings are executed in a transaction (auto-commit off). If insert-only logic is disabled. ### Greenplum In Greenplum, the mappings are stored as procedures, so executing them is straightforward. ``` select.""(); ``` Do make sure that the raw vault mappings are executed in a transaction (auto-commit off). If insert-only logic is disabled. ### Azure Synapse In Azure Synapse, the mappings are stored as procedures, so executing them is straightforward. ``` EXEC ."" ; ``` Do make sure that the raw vault mappings are executed in a transaction (auto-commit off). If insert-only logic is disabled. All other mappings should not be executed in a transaction since they could contain truncate statements, which are not supported inside transactions. ### SQL Server In SQL Server, the mappings are stored as procedures, so executing them is straightforward. ``` EXEC ."" ; ``` Do make sure that the raw vault mappings are executed in a transaction (auto-commit off). If insert-only logic is disabled. Markdown version of this document for LLM’s: