Connect Redis as an Online Store

All Tecton deployments on AWS include support for a DynamoDB Online Store by default. You may optionally configure Redis to use as the Online Store with Tecton. If you're wondering which store is best for you, read this Tecton blog post.

In order to use a Redis Online Store, follow this guide to connect Tecton to your Redis Enterprise Cloud or Amazon ElastiCache for Redis cluster.

Configuring ElastiCache for Tecton

Cluster Requirements

The Redis cluster you connect to Tecton should only be for use with Tecton. Tecton requires specific cluster configuration, depending on which managed version of Redis you're using.

When creating your ElastiCache cluster, ensure your cluster meets the following requirements:

• Redis Cluster Mode Enabled
• Multi AZ Enabled
• At least one replica per shard
• Redis Version 6.x

Setting up your ElastiCache Cluster

You can set up an ElastiCache cluster either using the AWS Console or through terraform. This sample terraform script creates a cluster based on Tecton's requirements.

resource "aws_elasticache_subnet_group" "tecton_<customer_name>_redis_cluster_subnet_group" {
  name       = "tecton-<customer_name>-redis-cluster-cache-subnet"
  subnet_ids = XXXX
}

resource "aws_elasticache_replication_group" "tecton_<customer_name>_redis_cluster" {
  replication_group_id          = "tecton_<customer_name>_redis_cluster"
  replication_group_description = "tecton-<customer_name>-redis-replication-group"
  node_type                     = "cache.m5.xlarge"
  port                          = 6379
  automatic_failover_enabled    = true
  engine                        = "redis"
  engine_version                = "6.x"
  multi_az_enabled              = true
  parameter_group_name          = "default.redis6.x.cluster.on"
  # Eight shards. One replica per shard.
  cluster_mode {
    replicas_per_node_group = 1
    num_node_groups         = 8
  }
  security_group_ids         = [XXXX]
  subnet_group_name          = "tecton-<customer_name>-redis-cluster-cache-subnet"
  transit_encryption_enabled = true
  depends_on                 = [aws_elasticache_subnet_group.tecton_<customer_name>_redis_cluster_subnet_group]
}

Note that the node_type, number_of_shards, and replicas_per_node_group values can be tuned based on your workload.

Optional Backups

You may want to configure automatic or manual backups for faster data recovery in case of a cluster-wide failure.

Connecting Tecton to your ElastiCache cluster with VPC peering

Once your cluster has been configured, Tecton Customer Success will help complete the connection with VPC peering.

Create a Tecton Support ticket the following information:

1. VPC's account ID
2. VPC ID
3. AWS Region
4. Relevant CIDR blocks containing the ElastiCache cluster

Tecton will then send a VPC peering request, and provide instructions on how to finish setting up networking.

Once Tecton has completed connecting to your ElastiCache cluster, you should test writing and reading feature data to validate the connection to Tecton.

After validating the connection, see how to manage your ElasticCache Redis Cluster.

Configuring Redis Enterprise Cloud for Tecton

Cluster Requirements

Skip this section if you are using ElasticCache. The Redis cluster you connect to Tecton should only be for use with Tecton. Tecton requires specific cluster configuration, depending on which managed version of Redis you're using.

When creating your Redis Enterprise cluster, ensure your cluster meets the following requirements:

• Multi-AZ Enabled
• Use the same region as the rest of the deployments
• Have High availability(Replication) enabled
• Redis Version 6.x
• OSS Cluster API should be enabled and use external endpoint
• TLS should be enabled but with TLS Client Authentication unchecked.
• maxmemory-policy should be set to noeviction
• CIDR allow list should be set to allow Tecton's IP addresses. Please contact Tecton Customer Success for the list of IP addresses.

Setting up your Redis Enterprise Cluster

You can set up a Redis Enterprise cluster either using their console or through Terraform. Once your Redis Enterprise cluster is set up, you should validate the connection to Tecton by writing and reading feature data.

After validating the connection, see how to manage your Redis Enterprise Cluster.

Validating the connection

To materialize a feature view to Redis, simply specify the RedisConfig in your feature view definition by adding the online_store=RedisConfig() in the Feature View declaration.

@batch_feature_view(
    online_store=RedisConfig(
        primary_endpoint="my_redis_primary_endpoint", authentication_token=os.getenv(MY_REDIS_AUTH_TOKEN)
    )
)
def test_feature_view():
    ...

Once the materialization jobs have been completed, you can use the FeatureView.get_online_features() command to test reading features from your Redis Cluster.