Multi-Model Failover In Your AI Gateway

Think about two scenarios that are pretty common. 1) You hit a rate limit or run out of tokens, so you have to "downgrade" to a small/less powerful Model. 2) An LLM provider is down or having intermittent issues. In these two cases, what do you do if you only have one Model set up for your Gateway t