Resources
A reference lookup tables for fitting variable parameters (4B to 70B) across graphic cards.
Hardware Sizing Guide
This dynamic chart defines size limits:
- 8GB VRAM (RTX 4060): Safely fits 8B models quantized to Q4_K_M formats with standard 4K context scopes.
- 16GB VRAM (Macbook Air 16G): Fits 14B Q4 formats or 8B Q8 formats comfortably.
- 24GB VRAM (RTX 4090): Accommodates 32B models in Q4 configurations or 70B models aggressively quantized.