Improved UX, UI, Security and support for hybrid-cloud

Background: I tried Kubeflow 1.0 in May 2020, with a narrow focus of Cloud native ML pipelines.
With the latest Kubeflow 1.3 release, they streamlined the setup process, improved security and user experience. Even with these updates, there is still a learning curve for non-technical/non-engineering users. Another improvement is the ability to pick and choose the components you want to install.
IMO, the ideal use-case is a cross-functional Data Science team with a mix of Platform Engineers, ML Engineers and Data Scientists.

  1. Setup and Improved UX/UI:
    Relatively easy to setup compared to version 1.0, easy to understand UX and responsive…

A promising cloud-based open-source ML Feature store solution!

History: Feast has been through several revisions in the past year. With the current version (0.9), its possible to setup end-to-end on a barebones k8s cluster.
Feast team is currently working on version 0.10 to be released in April 2021 (which is expected to further simplify the architecture and the setup). There are companies around the world that are already using Feast or in the process of integrating.

Background (From Feast website): Feast (Feature Store) is an operational data system for managing and serving machine learning features to models in production.

As you can see below, even with the basic…

RayML + Kubernetes = Finally, a truly scalable Distributed ML solution

Background (From Ray website): Ray is an open-source distributed execution framework that makes it easy to scale your applications and to leverage state of the art machine learning libraries. Ray provides a simple, universal API for building distributed applications (supports Python and Java API).

I first came across RayML on Software Engineering daily podcast in July 2020.
In September 2020, I attended Ray Summit 2020 organized by Anyscale (A startup founded by the creators of Ray from the UC Berkeley RISELab, the successor to the AMPLab, that created Apache…

Kubeflow pipelines + Argo workflows on Kubernetes

Kubeflow is now GA with certain components are stable, while others are still beta. Based on my experience trying some of the features so far, Kubeflow could provide a full-fledged cloud native ML solution.

Most cloud providers (GCP, Azure, AWS, IBM etc.) are actively contributing to Kubeflow.

Coming to the Kubeflow components, the central dashboard covers only some of the components and it’s still a work in progress.

Kubeflow pipelines + Argo has the potential to be a game changer

For me, the most interesting component is the pipelines. The pipelines component uses Agro as the workflow Orchestrator. Argo itself has four options Argo Workflows…

How Agile transformed Tech industry and super charged software delivery

Agile practices in the wild

Background : I first came across Agile methodologies during my graduate school program (Distributed and Multimedia Information Systems at Heriot-Watt University in Edinburgh, UK) . My first thoughts after learning the concepts such as Agile unified process (AUP), Extreme programming (XP), Test-driven development (TDD) and Pair programming was, why would companies/teams use anything other than these new software development practices.
Later, when I worked at PayPal, Yodlee and CareFirst BCBS, I could clearly see the benefits of using Agile methodologies in the real world. …

Basic mis-understandings of each others work

Background: For the past 3 years, I have been working at the intersection of Cloud (AWS, OpenShift/Kubernetes, Docker, Snowflake), Software Engineering (UI and Microservices), and Data Science (Python, Spark, H2O, R, SAS). Its been a great learning experience working with talented engineers in building AIR9 Data Science Platform.

This post is about my observations working with Data Engineers and Data Scientists/Analysts, and their blind-spots when it comes to Machine learning projects.
Side note #1: There is an interesting back story to the word “data scientist”.

Setup Kubernetes on a AWS EC2 instance

Kubernetes
Kubernetes

What? : https://microk8s.io/

How? :

  1. Create your own EC2 : I used t3.2xlarge with Ubuntu 18.04 LTS
  2. Login to your EC2 and run the below commands to install microk8s
sudo snap list
sudo snap install microk8s --classic --channel=stable

3. Check if the cluster is up and running

sudo microk8s.kubectl cluster-info

4. Enable DNS, Storage, Dashboard, Istio and Prometheus

sudo microk8s.enable dns storage dashboard ingress istio prometheus

5. Setup kubectl

sudo snap alias microk8s.kubectl kubectl
sudo usermod -a -G microk8s ubuntu

6. Setup kube config

sudo chown -f -R ubuntu ~/.kube
sudo microk8s.kubectl config view…

Prasad Paravatha

My personal blog | Not a real ninja | https://github.com/paravatha

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store