How we built the IoT infrastructure for one of Japan's largest car OEMs

Most software teams, if they're honest, have never met real scale. They've met busy - a launch spike, a seasonal peak, a marketing campaign that landed - and busy passes. A connected vehicle fleet is a different animal entirely, because every vehicle is a device that never logs off, telemetry arrives whether you're ready or not, and the graph only ever climbs. When we took on the IoT infrastructure for one of Japan's largest car OEMs, the first thing the work taught us was that at this scale, engineering stops being about features and starts being about physics.

This piece is about what that actually means in practice - the three layers the platform stands on, and the habits a team has to grow to run them. None of it is exotic, which is in some ways the point: automotive scale doesn't reward cleverness, it rewards discipline applied without exceptions.

The device layer: designing for the worst road

A vehicle is close to the hardest environment you can ask a connected device to live in. It moves at highway speed through cellular dead zones, it powers down without warning, it sits in an underground car park for a fortnight and then expects to sync, and its onboard compute has to be shared politely with systems that matter considerably more than telemetry does. The device layer has to treat all of that as normal operation rather than as an error state.

The design principles that fall out are old, hard-won distributed-systems ideas wearing overalls. Buffer locally and transmit opportunistically, because the network is a sometimes-thing. Make every message idempotent and every batch replayable, because the same data will arrive twice and out of order and occasionally three weeks late, and the platform's maths has to come out identical regardless. Version every schema from the very first day, because a vehicle fleet doesn't take software updates the way a web app does - populations of devices run old firmware for years, and the pipeline has to speak every dialect it has ever spoken, simultaneously, forever.

The fleet speaks every dialect it has ever spoken. The pipeline has to answer all of them.

The ingestion pipeline: backpressure is the product

Ingestion at this scale is a river management problem, and the cardinal rule is that the river doesn't care about your release schedule. The pipeline's job is to accept sustained, unrelenting flow - with bursts arriving whenever a chunk of the fleet reconnects after an outage or a firmware rollout - without ever losing a message or falling over in a way that loses the next one.

The unglamorous heroes are queues with real backpressure, consumers that degrade gracefully instead of heroically, and dead-letter paths for every malformed message so one bad payload can't poison a partition. We load-test against simulated fleets rather than optimistic averages, because averages lie - the pipeline that handles the mean beautifully and dies on the reconnection storm has failed at its actual job. Capacity planning at this scale is less about the traffic you have and more about the traffic the worst plausible Tuesday can generate, and the honest answer is always higher than the dashboard suggests.

Idempotency deserves its own sentence here too, because at fleet scale 'exactly once' is a bedtime story: the same telemetry will be delivered more than once, and the only question is whether the platform's aggregates flinch. Designing every write so that replays are harmless is the kind of decision that costs a little thought early and saves an unquantifiable amount of 2am later.

The data layer: where telemetry becomes an asset

Raw telemetry is exhaust; the data layer is where it becomes fuel. The work here is partly structural (time-series storage tiered by age and access pattern, aggregates computed once rather than per-query, retention policies that respect both cost and regulation) and partly about trust, because a data platform is only as valuable as the confidence people have in its answers.

Trust is engineered the same way throughput is: with monitoring on data quality, not just system health. Completeness checks that notice when a slice of the fleet goes quiet, distribution checks that notice when a sensor starts lying, lineage that can answer 'where did this number come from' all the way back to the device that sent it. A pipeline that's green while the data inside it quietly rots is the most dangerous state an IoT platform has - everything looks fine right up until a business decision gets made on it.

What it asks of the team

The technical patterns transfer between projects; the culture is the harder build. Running infrastructure at automotive scale asks a team to internalise that production is the only environment whose opinion matters, that every change ships behind the assumption it might need reversing, and that observability isn't a feature of the platform but a precondition for touching it. It also asks for a particular kind of calm - the fleet doesn't pause while you debug, so the response to incidents has to be rehearsed procedure rather than adrenaline.

Working alongside the engineering standards of a major Japanese OEM sharpened all of this in ways we've carried into every platform since. There's a quality bar in that industry - a seriousness about failure modes, a respect for the boring disciplines - that recalibrates what a team considers normal, and once recalibrated, you don't really go back. Scale, it turns out, is a teacher: not because it's glamorous, but because it refuses every shortcut, and a team that has built for it carries the habits into everything smaller.