WORK/W-02/CLOUD & KUBERNETES

Native container workloads within the data perimeter.

Architected end-to-end deployment solutions hosted entirely within Snowflake using Snowpark Container Services (SPCS), driving modern data-app integration and native container workloads.

PLATFORM ENGINEER / QUANTIPHI
SPCS
Native container hosting in Snowflake
SNOWFLAKESPCSDOCKERNETWORKINGCOMPUTE POOLS
01 / PROBLEM & CONTEXT

Compute next to the data.

Traditional financial analytics applications frequently required transporting large datasets out of enterprise data warehouses into external cloud container clusters, creating governance overhead, egress latency, and compliance complexity.

The objective was to run intensive containerized variance calculation workloads directly within the Snowflake governance boundary without external network data egress.

02 / MY ROLE & OWNERSHIP

Platform Architecture & Ingress Design

Led the platform transition to Snowpark Container Services (SPCS) for native container application hosting within Snowflake.

Designed secure ingress configurations, API gateways, container lifecycle policies, and internal service-to-service communication pathways.

Led platform transition to Snowpark Container Services (SPCS) for native app hosting within Snowflake
Designed secure ingress configurations, API gateways, and internal networking between services
Managed scalable Snowflake compute warehouses and optimized container resource allocations
Established security boundaries and secure application access patterns within the Snowflake perimeter
Defined best practices for Snowflake image repositories and container lifecycles
04 / APPROACH & ARCHITECTURE

Containerized execution inside the data boundary.

01 / PERIMETER SECURITY

Zero external egress

Execution stays inside Snowflake compute pools, eliminating data transit risks and keeping compliance within existing data governance policies.

02 / RESOURCE EFFICIENCY

Compute pool scaling

Dedicated compute pools are right-sized for intensive financial calculations, shutting down when idle to eliminate cloud compute waste.

03 / INTERNAL ROUTING

Isolated service ingress

Controlled API gateways and ingress endpoints route communication between microservices within Snowflake without exposing internal endpoints.

Native Snowpark Container Services (SPCS)

Packaged backend financial calculation services into OCI-compliant container images hosted in Snowflake internal image repositories.

Service networking and API ingress

Configured secure ingress rules and endpoints enabling front-end interfaces to query backend calculation containers seamlessly.

Resource optimization and compute pools

Balanced CPU and memory limits across compute pools, preventing noisy neighbor contention across parallel financial simulation runs.

05 / KEY DECISIONS & TRADE-OFFS

The choices behind the architecture.

Native SPCS containers over external VM hosting
Kept sensitive financial data inside the Snowflake governance boundary without external network egress.
Required adapting CI/CD workflows to push to Snowflake image registries rather than standard container registries.
Dedicated compute pools over multi-tenant warehouses
Guaranteed predictable compute resources for intensive financial variance calculations.
Required explicit capacity provisioning and idle lifecycle management.
Controlled internal ingress gateways
Restricted inter-service communication to verified internal pathways.
Routing configuration required explicit ingress specifications inside Snowflake definitions.
06 / RESULT
SPCS Adoption
Native data-app integration

Secure financial workloads without external compute hops.

Successfully established modern container deployment patterns within Snowflake, reducing data transit latency and maintaining enterprise governance.

Pioneered SPCS adoption across enterprise workloads
Optimized compute costs through right-sized compute pools
Seamless internal integrations between data pipelines and services
More systems. Same intent.