Gateway & Edge Documentation

Source-based gateway contract

From Bluetooth collection to acknowledged cloud commit

Based on the current gateway firmware and mobile collector sources. Gateway & Edge keeps collection, local retention, upload, and server acknowledgement separate: a sample is synchronized only when HiveServer accepts the submitted batch.

01Collector

Scans BLE, identifies the device type, and reads current values or replay history.

02Journal

Retains every captured sample locally and advances the replay cursor only after storage.

03Sender

Uploads queued batches when network transport is available and retries failures safely.

04Server commit

Removes data from the pending queue only after the server protocol acknowledges it.

Source projects: firmware/hivegateway2, firmware/meshgateway, firmware/hivegateway, and apps/hivegateway-maui. The detailed firmware modules, mobile flows, API routes, and installation evidence follow below.

Edge interaction model

How the gateway coordinates devices, local storage, and the cloud

The gateway layer exists because real deployments are messy: devices sleep, coverage drops, service staff travel, and uploads do not always happen in the same moment as collection. RoboMeshOS already includes an active mobile gateway architecture that separates collection from upload and preserves data integrity when the environment is unreliable.

BLE advertisements

Gateway scans nearby devices and detects supported sensor families from advertisements.

Type recognitionRSSI

Connection and replay

For devices that store history, gateway connects and requests replay or delta replay from the last local cursor.

Replay characteristicDelta sync

Local persistence

Collected rows are stored locally with replay cursor state, outbox batching, and retry metadata.

Local storeJournal

Upload transport

Queued data is sent in batches to the backend over the compact binary protocol.

TCP/TLS uploadCommit

Diagnostics and operator feedback

The app exposes runtime behavior, retry state, and collection context useful during field service.

Diagnostics UITroubleshooting
Discovery
Replay recovery
Offline-safe queue
Verified delivery

Detailed runtime sequence

01

Scan and classify

The gateway listens for BLE advertisements, identifies the device family, and decides whether the current task is passive sampling, service inspection, or backlog recovery.

02

Connect and request history

If history exists, the gateway enables notifications, requests replay, and resumes from the last known cursor instead of downloading the same data forever.

03

Store before upload

Downloaded rows are first preserved locally, then converted into pending outbox work. This protects the data path from app restarts, coverage gaps, and temporary backend failures.

04

Upload and retry

The sender pushes batches to the cloud and tracks success or retry conditions. Upload can be paused, resumed, or delayed without losing the original collected rows.

Gateway interfaces and extension points

BLE edge contracts

New device support usually starts with advertisement recognition, replay characteristic handling, and payload decoding. That is a contained and valuable extension surface.

Persistence and outbox logic

Projects that need stronger offline behavior, exports, or replay auditing can extend the local store, cursor handling, and upload batching without breaking the public app surface.

Transport and diagnostics

Because the upload path is explicit, gateway work is a strong place to add telemetry, observability, service tools, and partner-specific synchronization rules.

Collector pseudocode

This pattern works for smart farming, meter walks, campus servicing, and other field-heavy deployments.

for each advertisement in scan():
  if supported(advertisement):
    device = resolveDeviceType(advertisement)
    cursor = loadReplayCursor(device)
    session = connect(device)
    rows = replay(session, from=cursor)
    storeLocally(rows)
    enqueueOutbox(rows)

while internetAvailable():
  batch = getOldestPendingBatch()
  if not batch: break
  upload(batch)
  markCommitted(batch)

Why different audiences care

User

The user benefits from reliable collection that keeps data flowing even when mobile coverage is poor or visits are infrequent.

Integrator

The integrator benefits from a reusable offline-first field architecture that works across many verticals with only moderate adaptation.

Programmer

The programmer benefits from an explicit runtime model with active code paths already documented in the MAUI gateway repository.

Smart farming and smart metering

Great fit for remote sites, periodic collection, and backlog recovery where devices are near a collector only occasionally.

Smart city, school, and facility deployments

Useful when a mobile service team, maintenance crew, or local device hub needs to bridge isolated sensors into a central cloud without permanent device connectivity.

Offline-first state machine

Connectivity can fail without losing the measurement

Collection, replay, persistence, and upload are separate states. The gateway keeps working in a field, basement, vehicle, or remote site while the cloud is unreachable.

Gateway synchronization loop

The loop advances only after local persistence and server acknowledgement protect the sample.

LocalQueuedCommittedRetry
1

Discover

Scan BLE advertisements, identify device family, RSSI, live payload, and replay capability.

2

Recover

Connect when needed and request history after the last confirmed device cursor.

3

Journal

Store samples locally, create an outbox batch, and preserve retry metadata.

4

Commit

Upload framed records through TLS and clear the batch only after success.

BLE

Radio adapter

Device recognition and replay are isolated from cloud transport.

Q

Durable outbox

Queued rows survive process restarts and temporary network loss.

!

Diagnostics

Operators can distinguish discovery, device replay, local queue, and upload failures.

Example field session

Live collection continues while connectivity drops; queued records drain after recovery.

OnlineWeakOfflineReplayUploadSynced

Source-based contract

Gateway & Edge: the durable path from sensor to HiveServer

The edge layer discovers nearby low-power devices over Bluetooth, collects current values or history, stores captured samples locally, and sends pending batches once a network is available. BLE connection, GSM connection, and cloud commit are separate states. Synchronization is successful only after the server acknowledges the submitted batch.

CollectorDiscovery, device classification, current reads, diagnostics, and replay requests.
JournalLocal durable storage and a cursor that advances only after retention.
SenderIndependent network worker that retries from pending journal data.
CommitPositive proof that the submitted batch was accepted by HiveServer.

Firmware repositories

firmware/hivegateway2 is the modular edge gateway, firmware/meshgateway is the mesh variant, and firmware/hivegateway is the GSM BGM13 implementation.

Mobile collector

apps/hivegateway-maui is a phone-based field collector. It requires proximity, Bluetooth permission, and mobile-platform-compatible background behaviour; it is not a permanent hardware gateway.

Source boundary

This documentation describes the source structure and intended flow. It does not prove a particular board has connected, retained data, or uploaded successfully in production.

Firmware responsibilities

How collection, replay, durability, and sending work together

01Discover and classify

Collector scans advertisements, identifies the device family, then selects the appropriate parser, live read, configuration, or replay flow.

02Replay safely

A replay starts at a known timestamp. Every next page begins after the last safely received sample, avoiding duplicate pages and gaps after disconnects.

03Journal first

Logger and storage retain collected rows locally. A full page is not a completion signal; continue until an end marker or short final page.

04Send and acknowledge

Sender uploads pending batches, waits for the server protocol response, and removes data only after acknowledgement. Failures leave the batch queued.

GSM BGM13 module areaResponsibility
advertise, bluetooth, gatt.xmlBluetooth advertising, services, and GATT database.
collector, devices, packetsDevice discovery, device-specific handling, and protocol packets.
journal_ids, pressureJournal, logFlash-backed records, pending samples, and operational evidence.
sender, sim800GSM modem control, upload transport, retry, and server response handling.
upgrader, vbat, board/HALFirmware update, battery measurement, board setup, and peripheral configuration.
Security: bootloader signing and encryption material is deployment-only secret data. Never place it in documentation, logs, screenshots, source control, or distributable build directories.

Mobile, API, and diagnostics

Field collection and operational verification

Platform identity

Android normally supplies a BLE MAC address. iOS uses a CoreBluetooth identifier and can require Device Information/System ID discovery before a hardware address is known. Keep BLE byte parsing and upload-side conversion as distinct stages.

Device paths

Hi and Hy use the internal-device path; Hc uses the counter-device path; other supported codes use the external-device path. Resolve type before parsing or replay.

History completion

The first history session without a cursor starts from time zero. Later pages start after the last sample. A complete page is a transport boundary, not proof that device history ended.

MethodHiveServer routeUse
GET/api/gatewaysList project gateways.
GET/api/gateways/locationsReturn gateway locations.
PUT/api/gateways/{gatewayId}/relay-settingsUpdate relay settings.
POST/api/hives/{id}/sample-requestCreate a requested device sample operation.
EvidenceProvesDoes not prove
BLE advertisementDevice is discoverable.A connection or upload occurred.
BLE connection openedGateway and device connected.A sample was retained.
Sample receivedA reading reached the collector.It persisted in cloud.
GSM or Internet connectedTransport is available.Server accepted a batch.
Server commit acknowledgedThe submitted batch was accepted.Future collection will succeed.

Installation checklist

Verify power, board configuration, and intended transport; identify an advertising sensor; record scan, select, connect, discover, read/replay, queue, upload, and acknowledgement; interrupt connectivity to confirm local retention; restore it and verify the acknowledged server commit. For replay, validate both the requested start time and the final-page or end-marker condition.

Gateway operational reference

Source-confirmed collection and retry rules

These rules define the boundary between a device reading, durable edge data, and a committed cloud batch. Each boundary is independently observable during field diagnostics.

Collector and BLE

Advertising may be sufficient for a quick latest-value read when a sensor exposes manufacturer data. A BLE connection is required for configuration, live diagnostics, historical replay, and device-specific service functions. Resolve the device family before choosing a parser or protocol timing.

Journal and cursor

Write a sample to persistent storage before treating it as available for upload. Advance the local replay cursor only after that write succeeds. This prevents missing records if Bluetooth disconnects in the middle of transfer.

Sender and acknowledgement

The sender consumes pending journal data, not transient collector memory. It establishes network transport, sends a prepared batch, waits for the server protocol response, and clears a batch only after acknowledgement. Failed batches remain available for a bounded retry cycle.

Scheduler and power

Scheduler keeps collection, sending, time-related work, and sleep as separate tasks. Timed wakeups avoid leaving the modem and BLE stack active unnecessarily. Retry intervals must be bounded so an unavailable server cannot drain battery capacity or a data plan in a tight loop.

Shared firmware layer

The firmware variants reuse shared components for radio/Bluetooth, GSM, flash-backed NVRAM, sensor and sensor-base abstractions, board and ARM/EFM32 support, plus firmware-update and boot support. A variant can change its board or transport without changing the core collection-to-journal-to-sender model.

Release verification

Evidence required before calling a gateway deployment ready

StageRequired checkPassing evidence
Target hardwareConfirm board configuration, power, installed firmware, and selected GSM/Internet transport.Target-board build and a powered device with the expected configuration.
BLE discoveryScan supported sensors and resolve their family before collection begins.Advertisement and correct device-path selection; not merely a generic BLE connection.
Current collectionRead a current value and persist it in the local journal.Collected sample plus successful durable local record.
Offline retentionInterrupt Internet after collection, then inspect pending work.The sample remains in the pending journal and is eligible for later retry.
Cloud recoveryRestore connectivity and send the pending batch.Server acknowledgement or commit response, not only a modem/TLS success message.
History replayVerify the first cursor, next-page timestamp, and completion condition.Replay starts at zero when no cursor exists, advances after retained samples, and ends on an end marker or short final page.
Verification scope: source code demonstrates intended architecture and module boundaries. It does not replace proof on the real board, with a real sensor, an offline period, restored network connectivity, and server-side persistence.