How to manage Tecton versions and safely upgrade

Tecton releases new features, product improvements, bug fixes, and security updates with each new version (see Versioning Scheme for more details). This page outlines upgrade guidance to minimize breaking changes or unexpected issues when upgrading to a new release.

1) Read the Release Notes & Upgrade Guide

For each new release, Tecton publishes Release Notes and an Upgrade Guide which contain details about new features, breaking / non-breaking changes, and changes to SDK functionality. Read these pages prior to upgrading to prevent any unexpected issues during the upgrade process.

2) Pin a Tecton minor version

Tecton recommends pinning minor versions, but allowing for automatic upgrades for patch versions (e.g. 0.6.*) This ensures that customers receive low-risk bug fixes automatically, but can adopt new minor versions safely. Generally, this means pinning the Tecton version (pip install tecton==0.6.*, pinning Tecton Spark UDFs) and pinning Python versions (e.g. to Python 3.8).

note

If you will be running unit tests, use one of the following commands, instead of pip install tecton==0.6.*, to pin the Tecton version:

• To install with Pyspark 3.1: pip install 'tecton[pyspark]==0.6'
• To install with Pyspark 3.2: pip install 'tecton[pyspark3.2]==0.6'
• To install with Pyspark 3.3: pip install tecton==0.6 pyspark==3.3

The table below describes the areas where Tecton is versioned and recommended action items (if any) for staying up-to-date with the latest release.

Type	Description	Action items
CLI	The main place to update is in CI/CD (e.g. to use tecton apply, to rotate API keys, run tests)	Pin the Tecton pip version and Python versions (e.g. in the Docker image used for CI/CD)
Offline feature retrieval (including notebook environments)	This is how users connect to Tecton for offline feature retrieval. This can be for training data generation or batch scoring. With Tecton on Spark, users use a Spark-based notebook cluster which needs to be updated. In Tecton on Snowflake, users can retrieve features in any Python environment.	Tecton on Spark: As per initial setup for Databricks or EMR, users should pin versions of Tecton libraries + jars at the cluster level. Customers may also want to pin the Python version too. For Databricks, this can be achieved using a specific Databricks Runtime (which installs a specific Python version). For EMR, customers can specify a custom AMI. EMR notebook users can also select a specific version via a %%configure cell. Tecton on Spark (Athena-based feature retrieval): Pin any production inference or training jobs that leverage Tecton’s Athena-based feature retrieval. Tecton on Snowflake: Ensure that production inference or training jobs pin dependencies (e.g. pip install 'tecton[snowflake]==0.5.*' and Python version = 3.8). If you use a Jupyter notebook server like Sagemaker notebooks, make sure you pin Tecton versions in the servers.
[Tecton Managed] Materialization jobs	Tecton launches jobs to materialize feature data to the offline and online store.	No action.
[Tecton Managed] Tecton internal Spark cluster	Tecton uses this internal cluster to validate Feature Views and Data Sources.	No action. Tecton automatically upgrades this cluster.

3) Test and upgrade interactive environments

Tecton 0.6+ supports a new development workflow, which allows users to quickly create, retrieve, and experiment with Tecton objects in any Python environment (outside of a Tecton workspace).

With each minor release of Tecton (e.g. 0.6.0), users should upgrade these interactive environments (e.g. local Python notebooks) in addition to updating CI/CD and production environments. This ensures that these environments function as expected when interacting with objects applied using a Tecton workspace.

4) Test and upgrade in a staging workspace

note

To ensure a safe upgrade, Tecton disallows any destructive changes (i.e. Recreates) to your Feature Repository while upgrading. For example, tecton apply will prevent changes to a Feature View's Offline Store format while upgrading as this would normally cause re-materialization.

Upgrade your Feature Repository first and run tecton apply before making any destructive changes or Recreates.

As users go to production, they typically maintain different workspaces for different stages of a launch. A good practice is to test changes in a staging workspace that mimics the live production workspace. Below, we walk through how to test a new Tecton version on such a staging workspace. Note that it is OK for a Tecton cluster to have different Tecton versions on different workspaces.

There are two options for upgrading:

a) Use tecton plan to do an in-place upgrade on a staging workspace. This is sufficient for most releases. b) A/B test using two different staging workspaces to validate that feature values are the same across Tecton versions. This is a slower and more expensive process but is the more safe option.

This process aims to upgrade an existing staging workspace while looking out for any unexpected issues. Follow any version-specific upgrade guide with the below recommendations in mind.

a) In-place upgrade

1. Upgrade your local machine’s version of Tecton.
2. Run tecton plan on an existing staging live workspace that is equivalent to the production workspace. Follow the version upgrade guide until tecton plan produces an expected diff (i.e. no re-materialization, no errors).
   1. This should usually indicate no need to re-materialize features (which can be costly). If it does, consult the version specific upgrade guide or contact support.
   2. If there are any deprecated fields or breaking changes, follow the version specific upgrade guide to resolve
   3. Tecton may upgrade some definitions to be compatible with the latest version, as described in the plan output.
   4. (if necessary) run tecton plan --suppress-recreates when Tecton’s upgrade guide recommends it
3. (Optional) If there are a lot of changes in the above, consider using the slower A/B test approach to be safe.
4. On the staging workspace, run tecton apply of the final plan id from step 2. (i.e. tecton apply --plan-id <plan-id>) when you’re confident you’ve made all the right changes.
5. Upgrade training / batch inference jobs’ Tecton versions
6. Validate that there are no potential production impacting issues, especially with batch inference jobs, training jobs, or (rare) re-materialization jobs for feature views.
7. If not already done, update your staging notebook environments to use the latest version of Tecton (as per above table).

b) A/B test using two workspaces

One way to do this is to have two different Tecton workspaces that have the same feature definitions. Upgrade the version on one workspace to the latest Tecton version. Also, create a new notebook cluster that uses the Tecton version. You can then A/B test the new Tecton SDK version and verify the outputs of get_historical_featuresare the same for a feature service.

Example test on Spark:

• Control environment (existing)
  • Staging workspace applied using Tecton 0.5.7
  • Notebook cluster using Tecton 0.5.7, which calls get_historical_features
• Treatment environment (upgrading)
  • Workspace applied using Tecton 0.6
  • Notebook cluster using Tecton 0.6, which calls get_historical_features
• Expected outcome: The result of get_historical_features on the same spine should be identical. Be careful that the spine is deterministically generated (e.g read from S3) otherwise the results may differ.

5) Upgrade production Tecton clusters and inference pipelines

1. Follow the previous instructions for all production workspaces.
2. Update your CI/CD pipelines to use the latest version of Tecton.
3. Update your production training / inference jobs to use the latest version of Tecton. This likely requires updating your notebook environments as well (per above table).
4. Validate that there are no potential production impacting issues, especially with batch inference jobs, training jobs, or (rare) re-materialization jobs for feature views.