Enabling Materialization

Overview

Materialization is the process of computing features from source data and persisting them in the feature store, for later use in training or inference.

Batch Feature Views (such as user_credit_card_issuer) and Stream Feature Views support materialization. These Feature Views materialize data by running the transformation that is defined in the Feature View.

On-Demand Feature Views do not support materialization.

info

The materialization concepts discussed on the remainder of this page apply to Batch Feature Views (such as user_credit_card_issuer) and Stream Feature Views.

The user_credit_card_issuer Feature View is used as specific example.

Feature View materialization

A materialization run produces feature values for a specific time range. This time range is known as the “materialization window”. The materialization window is different for backfills and incremental runs:

• During the initial backfill of feature data to the Online and Offline Feature Store, the materialization time window starts with feature_start_time and ends with Tecton’s “current wall clock time” at which the feature’s materialization is enabled.

  The user_credit_card_issuer Feature View specifies feature_start_time=datetime(2016, 1, 1).

• On incremental runs, the materialization time window starts with the previous run's end_time, and ends with the current run's start_time + batch_schedule. The user_credit_card_issuer Feature View specifies batch_schedule=timedelta(days=1), meaning the materialization job for the Feature View will run once a day.

For more information on backfill and incremental materialization, see this section.

Enabling materialization

For a Feature View to be materialized, it must be applied to a live workspace. Additionally, parameters in the Feature View must be set to materialize to the online and/or offline store, as appropriate.

Creating and using a live workspace

So far, you have been using a development workspace, that you created in the tutorial setup. You now need a live workspace, which you can create by running the following in your terminal:

tecton workspace create <live workspace name> --live

Output:

Created workspace "<live workspace name>".
Switched to workspace "<live workspace name>".

You have created a new, empty workspace. Workspaces let
you create and manage an isolated feature repository.
Running "tecton plan" will compare your local repository
against the remote repository, which is initially empty.

info

To create a live workspace, you must have the Admin role. If you do not have this role, ask your adminstrator to create the live workspace.

The name of your live workspace must be different than the name of your development workspace.

caution

Live workspaces incur materialization costs.

Should you decide to not complete this tutorial, you should delete your live workspace to avoid compute costs that are incurred during scheduled materialization jobs; see the end of this tutorial for instructions on deleting your live workspace.

In your terminal, run tecton apply to apply your local feature repository to the live workspace.

Setting materialization parameters in a Feature View

To materialize feature data generated by a Feature View to the online store, set the Feature View's online parameter to True. To materialize feature data generated by a Feature View to the offline store, set the Feature View's offline parameter to True.

The definition of all batch features we've built in this tutorial (user_credit_card_issuer, user_transaction_counts,user_home_location) already have online=True and offline=True, so no change is required to these parameters.

Monitoring Materialization

To monitor the materialization jobs, select Features on the left navigation bar in the Web UI. On the Feature Views and Tables tab, select a feature, such as user_credit_card_issuer, to monitor. Then, select the Materialization tab, and scroll down to see the materialization jobs for the Feature View. Job status is listed under the Job column and will initally have the status Pending, followed by Running, and finally Success (assuming the job completes successfully). Each job has a different materialization window. Because the user_credit_card_issuer Feature View has just been created, the jobs listed are backfill jobs.

You can monitor feature freshness by looking at the Freshness Status section at the top of the Materialization tab and the feature freshness graph on the Monitoring tab.

You can also configure alerts to be triggered in case of incidents.