Skip to main content

Source configurations

Available configurations

Sources configurations support enabled, event_time, and meta

General configurations

General configurations provide broader operational settings applicable across multiple resource types. Like resource-specific configurations, these can also be set in the project file, property files, or within resource-specific files.

dbt_project.yml
sources:
<resource-path>:
+enabled: true | false
+event_time: my_time_field
+meta:
key: value

Configuring sources

Sources can be configured via a config: block within their .yml definitions, or from the dbt_project.yml file under the sources: key. This configuration is most useful for configuring sources imported from a package.

You can disable sources imported from a package to prevent them from rendering in the documentation, or to prevent source freshness checks from running on source tables imported from packages.

  • Note: To disable a source table nested in a YAML file in a subfolder, you will need to supply the subfolder(s) within the path to that YAML file, as well as the source name and the table name in the dbt_project.yml file.

    The following example shows how to disable a source table nested in a YAML file in a subfolder:

    dbt_project.yml
    sources:
    your_project_name:
    subdirectory_name:
    source_name:
    source_table_name:
    +enabled: false
    +event_time: my_time_field

Examples

The following examples show how to configure sources in your dbt project.

Disable all sources imported from a package
Conditionally enable a single source
Disable a single source from a package
Configure a source with an event_time
Configure meta to a source

Disable all sources imported from a package

To apply a configuration to all sources included from a package, state your configuration under the project name in the sources: config as a part of the resource path.

dbt_project.yml
sources:
events:
+enabled: false

Conditionally enable a single source

When defining a source, you can disable the entire source, or specific source tables, using the inline config property:

models/sources.yml
version: 2

sources:
- name: my_source
config:
enabled: true
tables:
- name: my_source_table # enabled
- name: ignore_this_one # not enabled
config:
enabled: false

You can configure specific source tables, and use variables as the input to that configuration:

models/sources.yml
version: 2

sources:
- name: my_source
tables:
- name: my_source_table
config:
enabled: "{{ var('my_source_table_enabled', false) }}"

Disable a single source from a package

To disable a specific source from another package, qualify the resource path for your configuration with both a package name and a source name. In this case, we're disabling the clickstream source from the events package.

dbt_project.yml
sources:
events:
clickstream:
+enabled: false

Similarly, you can disable a specific table from a source by qualifying the resource path with a package name, source name, and table name:

dbt_project.yml
sources:
events:
clickstream:
pageviews:
+enabled: false

Configure a source with an event_time

To configure a source with an event_time, specify the event_time field in the source configuration. This field is used to represent the actual timestamp of the event, rather than something like a loading date.

For example, if you had a source table called clickstream in the events source, you can use the timestamp for each event in the event_timestamp column as follows:

dbt_project.yml
sources:
events:
clickstream:
+event_time: event_timestamp

In this example, the event_time is set to event_timestamp, which has the exact time each clickstream event happened. Not only is this required for the incremental microbatching strategy, but when you compare data across CI and production environments, dbt will use event_timestamp to filter and match data by this event-based timeframe, ensuring that only overlapping timeframes are compared.

Configure meta to a source

Use the meta field to assign metadata information to sources. This is useful for tracking additional context, documentation, logging, and more.

For example, you can add meta information to a clickstream source to include information about the data source system:

dbt_project.yml
sources:
events:
clickstream:
+meta:
source_system: "Google analytics"
data_owner: "marketing_team"

Example source configuration

The following is a valid source configuration for a project with:

  • name: jaffle_shop
  • A package called events containing multiple source tables
dbt_project.yml
name: jaffle_shop
config-version: 2
...
sources:
# project names
jaffle_shop:
+enabled: true

events:
# source names
clickstream:
# table names
pageviews:
+enabled: false
link_clicks:
+enabled: true
0
Loading