War Story: We Replaced AWS IAM with Vault 1.16 and Cut Our Permission Error Rate by 60% for 500+ Developers

At 2:17 AM on a Tuesday in Q3 2023, our on-call rotation got 14 PagerDuty alerts in 12 minutes. All were the same error: AccessDenied from AWS IAM for developers trying to deploy to our EKS clusters. By the end of that quarter, we’d replaced AWS IAM’s static role-based access for 500+ engineers with