Nights and weekend projects have been a staple of my professional development since the beginning of my career. I learned early on that texts, blog posts, and tutorials are good at getting started with a new skill. Having a safe environment to push the limits of those skills is critical. I need an environment with sufficient complexity to try new stuff out, and challenge the old ideas I have as well.

The Implementation

I write code that’s gonna be deployed to the cloud. One of the issues of having an environment that is sufficiently complex becomes expensive on the cloud providers. I have really wanted to have an environment that mimiced the cloud providers without the cost. Which led me to build out a workstation that I could devote wholly to being configured to be as close to a cloud provider as possible.

The Services…

• Frontend services VueJS just because I’ve done Angular and React, just gonna try somethin’ else.
• API services in Go, probably going to put an API gateway in front of it so I can minimize routing complexity and middleware.
• services for Logistics, high rate transactions, persistence etc are gRPC Go or Rust k8s services with independent repos and deployments.
• Logging we’re using an Elastic Stack, or ELK (Elastic Search, Log Stash, and Kibana). I may extend my use of Elastic search however into the persistence realm. That’s something that I’ve entertained but I’ve been afraid to do that in my day job.
• Persistence/databases and what not. Well, lets to Couchbase, Redis and …mmm.. PostgreSQL, we’ll pull in the old PSQL because you know someone at a big ass company will demand it.
• Lets go ahead and have some pub/sub goodness. Ahhh…Kafka…bless you.
• We’ll use a combination of Helm, terraform to automate deployments, and ArgoCD to automate the rollouts. Argo because we have a ton of services dependent on each other and we’ll need to monitor each services health during the deployment to turn it green when everyone’s happy. It’s an App of Apps thing.

The Plan…

• Phase 1 Overton BAC A fictional Big Ass Company. I’m building out a machine with all the stuff I need to pretend like I’m deploying to a cloud provider. The goal is to get as far as I can into devloping a “non-trivial” enterprise application without checking my work on AWS, GCP or Azure.
• Phase 2 Sandy Ridge RBAC You thought RBAC was role based access control, nope Really Big Ass Company In this phase I will try to deploy this application to AWS, GCP and Azure to see how well I was able to design this to be provider agnostic.
• Phase 3 Prestige World Wide Once I get it working on all the usual suspects I’d like to go one more level and split services between GCP, AWS and Azure and get them talking to each other.

The Tech stack…so far…

• Go and Rust. Go because its the primary language I use each day and I need to continually push the limits of what I know and challenge patterns that I’ve fallen into. Rust because I’ve been using it for less than a year and I need some time building application components with some complexity.
• Rust and Python for Analytics – Python is hard to get away from in analytics, mainly because of NumPy. Rust has a growing number of analysts are using Rust. It’s super fast on ripping through data for ETL.
• VueJS – I’ve been out of the front end game for about five years now. I’ve done Angular and React so I’d like to write the front end services in Vue to learn that.
• Elastic Stack – Luckily I can install Elastic Search, Logstash and Kibana locally. This means I can go ahead and wire in logging like I would deploying to a cloud provider.
• Couchbase, Redis, and PostgreSQL – Couchbase because I haven’t worked with it, but also because the locally installed version seems to have 100% fidelity with the cloud based service. Redis, because that’s a staple memory cache for a huge number of solutions. PSQL, because you know someone in a big ass company is going to ask for some data to be put in an RDBMS. I will also mention that with Elastic Search running, that I plan on leveraging data accessible by Elastic Search. I want to explore the boundaries between Couchbase and Elastic.
• Polyrepo Structure – Normally I would build out of a single large monorepo with the services built out of there. I want to explore a more granular approach with each service in a repo and making more use of private packages for shared code.
• Decouple Deployment – I want to try to keep each service separate from its deployment details. I want a single deployment repo that contains the deployment for each service.
• gRPC for service messaging – each service will implement a protocol buffer for gRPC
• Kafka for pub/sub services – some services will be pub sub I’m sure. I want to push the boundaries of both gRPC and pub sub service implementations. I hope this is a good test platform to test both approaches against each other in varying contexts.
• K8s – Docker and K8s are the defacto for the cloud providers. I am trying Lens Desktop Kube for seeing the results of my deployments and profiling the application running in clusters.
• Helm and Terraform – one of the principal goals of this experiment are to be able to deploy to any cloud platform.

Checkout the video series on YouTube https://youtu.be/fZH1QecTUYI