sharpbyte.dev
← Interview ready
Interview ready · Design

LLM system design

Scenario-style questions for senior architect interviews—answered in plain English, with examples, recap tables, and Mermaid diagrams. All seventeen sections are published below (Q1–265). Pair with the theory Q&A bank for vocabulary depth.

§1

Core architecture

Layers, gateways, streaming, state, multi-team platforms, mixed inputs.

Open →
§2

Scale & performance

Cost envelopes, cache, queues, batching, K8s, routing, autoscaling.

Open →
§3

RAG

Ingestion, hybrid retrieval ACLs, GraphRAG, citations, SQL+RAG.

Open →
§4

Agents

Tools, sandboxes, memory, workflows, HITL, observability, injection.

Open →
§5

Data & knowledge

Realtime ingest, versioning, CDC, lineage, graph + vector.

Open →
§6

Cost

Governance, routing, caching, ROI, tiers, shared GPU clusters.

Open →
§7

Reliability

Breakers, idempotency, DR, 429s, validation, canaries, degradation.

Open →
§8

Security

Injection, PII, GDPR/HIPAA, RBAC, audits, threat modeling.

Open →
§9

Observability & evaluation

Tracing, privacy-safe logs, offline/online quality, probes, A/B, human labels, red-team CI.

Open →
§10

Multi-tenancy & platform

Fair-share GPUs, quotas, isolation, CMK, residency cells, tier migrations, SLAs.

Open →
§11

Fine-tuning & lifecycle

RAG vs FT, multi-LoRA serving, data governance, DPO/RLHF, registry, rollback, synthetic data.

Open →
§12

Advanced retrieval

Hybrid fusion, learned sparse, late interaction, rerank economics, multi-index, session state.

Open →
§13

Edge & on-device

Mobile/WebGPU, quantization, OTA models, offline RAG, thermals, air-gap, weight protection.

Open →
§14

Vertical & domain

Domain packs, legal/health/finance, dev tools, support, IoT, gov, locale, partners.

Open →
§15

Roadmap & GTM

Metrics, pricing, differentiation, phased rollout, SE motion, change management, build vs buy, deprecation, international GTM.

Open →
§16

Governance & RAI

Enterprise controls, model risk, harm IR, fairness, carbon, IP, vendor diligence, explainability, board reporting, exam readiness.

Open →
§17

Capstone

Interview narrative, when not to use LLMs, launch checklist, ADRs, M&A diligence, senior signals, socio-technical thesis.

Open →