Senior Software Engineer – ModelVault
ModelVault is building the world’s first fully-integrated, on-prem AI appliance — a plug-and-play AI server preloaded with software, models, and tools that let businesses (and eventually consumers) run powerful AI models locally, privately, and securely.
We're looking for a Senior Software Engineer to help us build the core software stack that powers our on-prem systems — from model integration and system orchestration to user interfaces and remote updates. You’ll be a key technical partner in defining our architecture, tools, and technical culture from day one.
🚀 What You’ll Do
-
Design and build the software stack that ships with ModelVault’s on-prem AI systems — everything from containerized inference engines to local dashboards and command-line tools.
-
Set up and manage integration with modern LLMs and vision models (e.g. LLaMA, Mistral, SDXL), including support for fine-tuning and local deployment.
-
Build and maintain user-facing tooling: CLIs, lightweight GUIs, and APIs that make the device intuitive for developers and AI teams.
-
Make architecture decisions for system orchestration, hardware interfacing, remote update pipelines, and telemetry — all optimized for stability and security.
-
Help set up GPU drivers, OS configuration (e.g., Ubuntu, NVIDIA stack), and optimized inference runtimes as needed.
-
Collaborate with the founding team to test and iterate with early business customers.
-
Contribute to hiring and scaling the engineering team over the next 12 months.
🧠 You Might Be a Fit If You
-
Have 5+ years of software engineering experience, preferably at early-stage startups or in fast-paced environments.
-
Are comfortable working across the stack — from OS-level setup and deployment to user-facing tools.
-
Have experience with containerization, system packaging, deployment tooling, and managing model runtimes.
-
Have worked with, or integrated, LLMs and ML models in real-world systems (e.g., inference, fine-tuning, embeddings).
-
Can independently drive architecture decisions and own large pieces of product infrastructure.
-
Thrive in ambiguity, learn fast, and are passionate about building great tools from scratch.
⭐️ Bonus Points
-
Experience with GPU computing, NVIDIA drivers, CUDA, or model acceleration tools.
-
Familiarity with ML tooling: vLLM, Ollama, Triton Inference Server, LangChain, etc.
-
Comfort with Linux system internals, remote access tooling (e.g., SSH, Ansible), or auto-update agents.
-
Previous experience with AI developer tooling, embedded systems, or secure local apps.
🌍 Why Join ModelVault?
-
Help define a new category: secure, local AI for everyone — starting with businesses, expanding to the world.
-
Work on real, shipped hardware + software that users will touch.
-
Be part of the founding engineering team with high autonomy and deep ownership.
-
Fully remote, async-friendly, with strong product velocity and clear vision.
Originally posted on Himalayas