A production carrier integration isn't one system. It's five, stacked.
Most teams that set out to build carrier connectivity in-house picture the integration as one layer - the API client. That's the layer that shows up in the architecture diagram on the kickoff deck. It's not the layer that wakes someone up at 3am. The real shape is five layers, each absorbing a different class of failure, each capable of degrading silently for weeks before anything obvious moves on the dashboard.

The reason in-house builds fall over in production isn't that any single one of these layers is hard to build. It's that the thinnest layer in your stack defines the failure mode you'll spend the next year debugging.
Layer 1 — Authentication and session management
What this layer absorbs: every way a carrier proves you're allowed to talk to them.
Across our portfolio we've integrated against OAuth2, custom token exchanges, mutual TLS, IP allowlists, portal-cookie hybrids, and broker-code provisioning models that range from "self-served via API" to "manual carrier-side approval per agency." The variance isn't the cost. The lack of warning when one of those variances applies to your next integration is the cost.
Failures here don't look like outages. They look like a quote that mysteriously doesn't quote. A few weeks back our auth adapter on a Nationwide BOP submission built the request headers but didn't attach the bearer token. Nationwide did the right thing — a clean 401 with a descriptive body. Layer 1 is almost always the problem, until it isn't.
Layer 2 — Request transformation
What this layer absorbs: turning one canonical application into thirty different carrier dialects.
We covered this in depth in Part 4 - the canonical schema, NAICS↔NCCI mappings, payroll conventions, decline taxonomies, the recurring discovery that one carrier's "restaurant" is another carrier's hard decline. The mapper is one layer of the stack, not the whole stack. But it touches every layer above and below it. If the mapper drops a field, Layer 1 sends a request the carrier won't accept, Layer 3 retries it, Layer 4 normalizes the rejection into the wrong error class, Layer 5 audits the misfire. One thin layer compounds through four others.
Layer 3 — Transport, retry, and idempotency
What this layer absorbs: the network being a network, and the carrier's API being a moving target.
This is where the post earns its title. Idempotency is the whole game. The transport layer has to assume that any single request can be sent more than once — by retry logic on our side, by an HTTP client doing automatic redirects, by an operator replaying a failed request, by a webhook redelivery on the carrier side. The question isn't whether a retry will happen. It's whether your retry layer understands what it's actually retrying.
A few months back we shipped a bind through Coterie. Coterie runs payment through Stripe, which means the integration pulls a single-use Stripe token before calling bind. The bind call timed out. Our retry layer did what we'd built it to do — fired a second attempt with the same payload. Stripe, doing what it's built to do, returned token_already_used. Our retry layer had no idea what a single-use token was; it saw a 4xx and surfaced it.
The fix took twenty minutes. The root cause was older than the bug. Our retry layer was idempotent in the way we meant it — same payload, same outcome — not in the way Stripe meant it, which is that some fields in that payload expire after a single use. Carriers that ship modern payment APIs are giving you a gift here. They force you to build a retry layer that understands which fields in a request are replayable and which aren't. If your stack treats every retry as "send the same bytes again," you'll find this bug. The only question is which carrier finds it for you first.
The bind side of this layer needs the same discipline at a higher level. Our BIND_IN_PROGRESS lock prevents a duplicate bind request from firing while a previous bind is mid-flight. Without it, a slow carrier and a refreshed browser tab is a double policy. With it, a slow carrier is just a slow carrier.
A few patterns we've made peace with at this layer:
- Some carriers mutate their own quote ID on every response, so you cannot key idempotency on the carrier's identifier.
- Some carriers expose tens of thousands of class-code-and-state combinations behind an API that crashes its own server midway through a full pull. Retry logic at this layer has to detect partial failure and resume from the gap.
- Some carriers' quote endpoints run slow enough under load that the retry layer has to distinguish a slow success from a real timeout — a thirty-second response that eventually returns 200 is not a candidate for retry.
None of these are anyone's fault. They're the shape of the world. Layer 3 has to absorb them.
Layer 4 — Response normalization and error classification
What this layer absorbs: every way thirty carriers say "no."
The temptation here is to treat error classification as a UI problem. It isn't. It's an alerting problem. The difference between a legitimate decline (the carrier did its job, the risk is out of appetite) and an internal failure (something in our stack is broken) is the difference between an INFO log and a page to oncall. Conflate the two and you either drown oncall in noise or you go silent on real production incidents. Both have a half-life.
Our internal contract is strict. Legitimate declines log at INFO. ERROR is reserved for unhandled failures. The error-taxonomy mapping itself lives in a database-backed model — carrier, status code, carrier error conditions, our internal quote type, our error message, whether to retry. We keep it in a table, not in code, so we can add a new error class for a carrier's mid-integration change without a redeploy.
The hard cases are the carriers whose decline taxonomy isn't structured to begin with. USLI returns decline reasons across three different response fields in different combinations. GAIG requires iterating through every answer in the request to figure out which one tripped a rule. Each of these forces Layer 4 to carry weight that the carrier's API hasn't carried for itself.
Layer 5 — Audit logging and replay
What this layer absorbs: time.
Every request/response pair needs to be replayable for years. Carrier disputes — our records show no such quote — get resolved by replaying the exact payload we sent and the exact response we received. Compliance audits, post-incident root cause analysis, and retroactive bug fixes all live in the same store.
The reason this layer matters more than it looks: it's the only layer that can save you when the layer above it fails. A Hiscox quote had been sitting in a customer's session for three weeks. Hiscox returned a 422 with an empty body — the quote had expired, which is reasonable carrier behavior. Layer 4 saw the 422 and threw. The only reason the customer didn't see a hard error is that Layer 5 had already stored the bridgingLink from the original quote response. We fell back to the stored link. The customer kept moving. Layer 5 caught what Layer 4 dropped.
Coalition deserves a positive callout here. Their webhook contract documents idempotency, defines the event lifecycle explicitly, and ships a replay endpoint that re-consumes any event by ID. That's the kind of carrier-side design that lets the integrator's audit layer stay simple.
Observability across the stack
Each of these layers needs its own observability hooks. Auth-failure rate at Layer 1. Retry exhaustion and idempotency-key collisions at Layer 3. ERROR-rate vs. INFO-rate divergence at Layer 4. Audit-write success rate at Layer 5. Aggregate "quote success rate" alone tells you nothing — by the time it moves, three layers have already silently degraded.
We'll go deep on monitoring in Part 9. The framing here is that monitoring isn't a sixth layer bolted on top of the stack. It's a property each layer ships with. If a layer can't tell you whether it's healthy, it isn't a layer. It's an outage waiting to be discovered.
The thinnest layer wins
A stack is only as strong as its thinnest layer. Most in-house carrier integrations ship four solid layers and one thin one. The thin one is the only one anyone remembers — because it's where every production incident eventually traces back to.
The reason this matters for the build-vs-buy decision isn't that any single one of these layers is hard. It's that building all five well, at the same time, while adding new carriers, and keeping the layers consistent across the integrations you already shipped — that's the work. It doesn't end. It compounds.
For the full lifecycle, see Anatomy of a Carrier Integration.

.png)
.png)

.png)
