OpenAI API Interview Questions for Senior (7+ Years)

Cost reduction: (1) Use gpt-3.5-turbo for classification/simple tasks, gpt-4 only for reasoning, (2) Semantic caching — cache responses by embedding the query and finding similar cached queries, (3) Prompt compression — use LLMLingua to compress prompts by 3–10x, (4) Streaming — faster perceived response, (5) Batch API — 50% discount for non-realtime tasks, (6) Fine-tuning — smaller fine-tuned model can outperform GPT-4 at 10x lower cost for specific tasks.