Vedico: realtime fleet tracking for telecom field operations

Challenge

Vedico maintains telecom infrastructure across regions ("circles"). Technicians move between towers all day; operations managers in a NOC need to know where everyone is, right now, whether a technician is actually at the site they claim, and whether an SLA is about to be missed.

The previous setup polled a REST endpoint every 30 seconds from every dashboard. It was slow, expensive, and blind to the two things that matter: anomalies and escalations.

Approach

I treated it as three systems:

1. A realtime broadcast service in Go, because fan-out of small messages to many subscribers is exactly what goroutines and channels are good at.
2. A spatial rules engine in the FastAPI backend, because the business rules change often and Python keeps them readable.
3. An escalation state machine with push notifications, because a missed SLA needs a human within minutes, not a report tomorrow.

Realtime broadcast (Go)

• One goroutine event loop per circle; connections live in a sync.RWMutex-guarded map keyed by circle so a broadcast never blocks on an unrelated region.
• Channel multiplexing merges GPS updates from the mobile ingest path with system events (escalations, geofence breaches) into a single ordered stream per subscriber.
• Back-pressure: slow dashboards get coalesced updates instead of unbounded queues.

Result: sub-millisecond broadcast from ingest to every dashboard subscribed to that circle.

Spatial rules (Python + PostGIS)

• Haversine distance checks on every ping, with PostGIS for the heavier "which work orders are within N metres" queries.
• A 5-minute grace window for offline flushes: mobile clients buffer pings in an isolate when the network drops and flush later, so the engine reorders by device timestamp before judging.
• Theft / proximity anomaly detector: a 50 m geofence around active work-order sites plus the technician's current work-order context. A technician 2 km from a site with no open order there is an anomaly; the same technician en route to an assigned order is not. False positives dropped to near zero after adding the context check.
• SELECT … FOR UPDATE pessimistic locking on the anomaly row so two concurrent pings can't open two incidents.

SLA escalation

A 4-tier state machine: Technician → Supervisor → Cluster In-charge → Manager. Each tier has a timer; on expiry the incident moves up and the next person gets an FCM push. Transient FCM failures retry with backoff; the state machine is idempotent so a replayed event never double-escalates. Hierarchical RBAC scopes what each tier can see, which doubles as multi-tenant isolation between regions.

Architecture

What shipped

• Live map with per-circle filtering and technician trails.
• Anomaly feed with one-click "explain" showing the geofence and work-order context that triggered it.
• Escalation timeline per incident, with who was notified when.
• Offline-tolerant mobile ingest with server-side reordering.

Results

• Dashboards went from 30-second polling to sub-millisecond push.
• Anomaly false positives became rare enough that managers actually read the feed.
• SLA breaches now escalate automatically; the "nobody knew" failure mode is gone.

What I'd do differently

• Ship the Go service behind a thin gateway from day one so auth, rate limits and tracing are uniform across the WebSocket and REST paths.
• Store trails in a time-series layout (partitioned by day) earlier; the first month of pings in a flat table made analytics queries slow.
• The ideas here connect to the event-driven refactors I describe in the infra write-up only indirectly, but the next version will run on Kubernetes with the same GitOps pipeline.