Feast: Setup your own ML Feature store on Kubernetes
A promising cloud-based open-source ML Feature store solution!
History: Feast has been through several revisions in the past year. With the current version (0.9), its possible to setup end-to-end on a barebones k8s cluster.
Feast team is currently working on version 0.10 to be released in April 2021 (which is expected to further simplify the architecture and the setup). There are companies around the world that are already using Feast or in the process of integrating.
Background (From Feast website): Feast (Feature Store) is an operational data system for managing and serving machine learning features to models in production.
As you can see below, even with the basic use-case, there is lot happening with in Feast in terms of technology stack.
Below Architecture diagram provides additional technologies you could plug-in based on your requirements.
Setup :
I used a 6 node GKE cluster (version 1.18.16-gke.302) with GS bucket and Spark k8s operator.
I also tried using a barebones k8s cluster with Spark in standalone mode using local tmp staging/work folders attached to a k8s node.
After you create features, save them in the data-lake and fetch the features using a Jupyter notebook, you can check Prometheus metrics and Grafana dashboards!
As an optional setup, Feast integrates with Prometheus and Grafana to provide visualizations
Summary and next steps:
Feast came a long way from the initial versions in 2019. Till version 0.9 came along, it was closely tied to a particular cloud provider (GCP) and/or set of a technologies.
Feast team has been working hard to simplify a very complex use-case, de-couple technologies and make it cloud-provider agnostic. Their RFC for version 0.10 looks very promising.
