Recent work applies deep reinforcement learning (RL) to API gateway traffic routing in cloud microservice architectures, reporting consistent gains over static heuristics once the underlying policy converges. A persistent gap in this literature is the omission of any evaluation of the transient learning period that follows deployment, autoscaling, or backend pool changes, referred to here as the cold-start problem. This paper introduces Confidence-Gated Adaptive Routing (CGAR), a framework that maintains a per-backend confidence estimate and uses it to continuously blend a static heuristic with a learned policy on a per-backend basis, re-triggering safe routing individually for newly added backends without resetting global learning. The problem is formalised as a non-stationary, per-arm warm-start bandit problem, and CGAR is evaluated against seven baselines over 20 independent seeds using a discrete-event simulator subjected to four scheduled backend pool churn events. CGAR attains a total regret of 0.8×10³, reducing regret by 81.6% relative to Pure DQN (p < 0.001), 80.4% relative to Eps-Greedy DQN (p < 0.001), and 91.9% relative to Round Robin (p < 0.001), while sustaining a mean SLA violation rate of 0.3%. A churn-window versus steady-state regret decomposition further shows that CGAR limits churn-window regret to 0.3×10³, substantially lower than every reinforcement-learning baseline, demonstrating that the gating mechanism provides targeted protection precisely when cold-start costs are highest. Ablation results are reported transparently: a fixed high-weight heuristic blend attains marginally lower regret than the adaptive variant in the present simulator, and this trade-off is discussed rather than concealed. CGAR nonetheless retains practical value over static configurations because it re-triggers cold-start protection automatically per backend without manual threshold tuning, and it outperforms every learning-based baseline with strong statistical significance. The work contributes a formal connection between warm-start bandit theory and API gateway routing that is absent from the current literature.