Problem
So i was just wandering around a student org's workshop. this one was the first workshop of the semester under the new administration at The AI Society at ASU. we'd prepared a lot beforehand -- making sure we were speaking on point, the slides looked good, and the timing was right for the qna -- but the biggest thing we overlooked, in hindsight, was the logistics of actually running things during the workshop.
We began normally, around 50-100 attendees, good looking GBM so far -- until we got to the practical part. that's where attendees were supposed to get things running on their own laptops. the workshop was about building LLMs from scratch, which means heavy compute. people instantly ran into problems cloning the repo and matching their dependencies, and mid-workshop a dependency had shipped some BS breaking updates to the HF library, which ate up basically the whole session. it really was downhill from that point on.
Shared compute
this is where i saw the problem, the problem of software/OS logistics -- linux, macos, windows -- all of them have their own BS reasons to fail. i reasoned about it a bit and landed on just using github codespaces, but then people wouldn't have enough compute to train an entire LLM. google colab? nice, but not really. i wanted to build something in house, something that AIS owns and anyone could run compute on: a shared compute platform for workshops specifically, or anywhere collaboration matters. mentors could get on a dashboard and spin up pods with whatever GPU they want (that obviously comes from the club's operating funds, within what we could afford).
But yeah, that was the vision -- everything authorized with discord Oauth, so it's super easy to just SSH into the pod your mentor is running for the current session.
and especially, it's FREE for students who attend our workshops. some people might never otherwise get their hands on an RTX 69420x.
Building it out was its own challenge. i had never spun up custom docker sessions before, so i learnt how VMs worked, how images were built, how security worked in them, how NOT to do certain things in them -- but i got it to work!
It now serves as a ready-to-go solution for anyone looking to connect discord -- workshop -- compute -- attendees for their ML-heavy sessions.
I called it "Godfather" because it was the godly father of all the smaller pods lol.