What AWS services are most important for AI/ML deployments?
Key AI/ML services: (1) SageMaker — fully managed ML platform for training and deployment, (2) EC2 with GPU instances (p3, g4dn, g5) — for custom training, (3) S3 — for storing datasets and model artifacts, (4) Lambda — for serverless inference on small models, (5) ECS/EKS — for containerized AI API deployments, (6) Bedrock — managed LLM APIs (Claude, Llama), (7) OpenSearch — for vector search at scale.
How do you deploy a FastAPI AI application on AWS ECS?
Steps: (1) Push Docker image to ECR (Elastic Container Registry), (2) Create ECS Task Definition with container config, CPU/memory, env vars from Secrets Manager, (3) Create ECS Service with desired count and health checks, (4) Place behind Application Load Balancer for traffic distribution, (5) Set up CloudWatch for logs and alarms, (6) Use auto-scaling policies based on CPU/request count. Total time: ~2 hours for first deployment.
Explain AWS IAM roles and why they matter for AI applications.
IAM roles define permissions for AWS services. For AI apps: (1) Never use root account credentials, (2) Give EC2/ECS tasks an IAM role with only needed permissions (least privilege), (3) For SageMaker: role needs S3 read/write + CloudWatch logs, (4) Use IAM policies to restrict which S3 buckets, Secrets Manager secrets, and Bedrock models each service can access. IAM mistakes cause security breaches — always review with IAM Access Analyzer.
What is S3 and how do you use it for ML datasets and model artifacts?
S3 is object storage with 99.999999999% durability. For ML: store raw datasets, processed features, trained model weights, and predictions. Best practices: (1) Use versioning for model artifacts, (2) Lifecycle policies to move old models to Glacier (cheaper), (3) S3 Transfer Acceleration for large dataset uploads, (4) VPC endpoints to avoid egress costs, (5) S3 Select to query large CSV/Parquet files without downloading. For cost: use intelligent tiering for infrequently accessed model weights.
How do you implement auto-scaling for an AI inference API on AWS?
ECS auto-scaling: (1) Set up CloudWatch alarms on CPUUtilization and RequestCountPerTarget, (2) Define scaling policies: scale out when CPU > 70% for 2 minutes, scale in when CPU < 30% for 5 minutes, (3) Set minimum (2) and maximum (20) task counts, (4) Use Application Auto Scaling with step scaling or target tracking. For spiky AI workloads, add SQS queue between API and workers — scale workers based on queue depth, not just CPU.
Unlock 3,000+ Interview Questions
Get full access to all interview questions with detailed answers, explanations, and real company context
Unlock Full Access →