ThingIO¶

Plane: Data Plane

Purpose¶

IoT data processing, analytics, and dashboard API. ThingIO is the platform service that gives developers and operators the tools to query device telemetry, manage data processing pipelines, and register consumer applications — all scoped to the devices enrolled in their account.

ThingIO is built on Apache StreamPipes for pipeline execution. It provides account-scoped REST APIs that wrap StreamPipes with FlexGalaxy access control, and a Data API that consumer apps call to query telemetry and pipeline results.

Service Identity¶

Aspect	Value
Namespace	`thingio` (own namespace on TrustMint EKS cluster)
Backend	Python / FastAPI, PostgreSQL
Frontend	React SPA (Vite + Carbon Design)
Port	Backend: 8000, Frontend: 80
ECR	`trustmint/thingio-backend`, `trustmint/thingio`

Responsibilities¶

Provide REST APIs for scopes, telemetry queries, pipeline management, and consumer app registration
Manage three tiers of data processing pipelines (managed, app, platform)
Accept telemetry ingest from ThingsBoard and analytics results from StreamPipes via Integration API
Auto-provision managed pipelines when devices enroll
Sync scope membership when devices are enrolled/removed in TrustMint

How ThingIO Relates to Other Services¶

                    ┌─────────────┐
                    │ Applications │
                    │ (WES, CJ)   │
                    └──────┬──────┘
                           │ call ThingIO Data API
                           │ render own dashboards
                           ▼
┌──────────────┐    ┌─────────────┐    ┌──────────────┐
│  TrustMint    │───►│   ThingIO   │◄───│  StreamPipes  │
│  (device      │    │             │    │  (pipeline    │
│   enrollment) │    │ Data API    │    │   execution)  │
└──────────────┘    │ Pipeline API│    └──────────────┘
                    └──────┬──────┘
                           │ reads telemetry
                           ▼
                    ┌─────────────┐
                    │ ThingsBoard  │
                    │ (via Kafka)  │
                    └─────────────┘

TrustMint handles device enrollment → notifies ThingIO for scope sync and managed pipeline provisioning
ThingIO handles data processing APIs and telemetry queries
StreamPipes executes the actual pipelines; ThingIO wraps it with access control
Consumer apps (WES, ClearJanitor) call ThingIO APIs and render their own dashboards — ThingIO does NOT provide dashboard UI for business apps

API Architecture — Two Tiers¶

Management API (via KrakenD at `/thingio/v1/*`)¶

Exposed to consumer apps, the ThingIO frontend, and external developers. Auth: OAuth2 JWT from Keycloak or X-Account-Id header (set by KrakenD).

Resource	Endpoints	Description
Scopes	`GET/POST/PATCH/DELETE /scopes`	Named device filters
Things	`GET /scopes/{id}/things`, `GET /things/{id}`	Devices within a scope (proxied from TrustMint)
Telemetry	`GET /telemetry`, `GET /things/{id}/telemetry`	Time-series queries with scope/key/time filters
Pipelines	`GET/POST/PATCH/DELETE /pipelines`	Pipeline CRUD, templates, start/stop
Pipeline Templates	`GET /pipelines/templates`	List available managed + configurable templates
Consumer Apps	`GET/POST/PATCH/DELETE /apps`	App registration, permissions, ownership

Integration API (ClusterIP only at `/internal/v1/*`)¶

Never exposed via KrakenD. Auth: internal service token.

Endpoint	Consumer	Purpose
`POST /telemetry/ingest`	ThingsBoard, StreamPipes	Batch telemetry ingest
`POST /scopes/sync`	TrustMint	Device enrollment changes → update scope counts
`POST /pipelines/managed/provision`	TrustMint	Auto-provision Tier 1 pipelines on enrollment

Pipeline Tiers¶

Tier 1: Managed Pipelines (ThingIO provides)¶

Pre-defined, zero configuration. Exist the moment a device enrolls.

Template	Description	Input	Output
`device_health_rollup`	Battery trend, connectivity uptime, last-seen	`thingsboard.telemetry`	`sp.analytics.health`
`telemetry_aggregation`	1min/5min/1hr rollups per device type	`thingsboard.telemetry`	`sp.analytics.aggregation`
`anomaly_baseline`	Automatic anomaly scoring on numeric keys	`thingsboard.telemetry`	`sp.analytics.anomaly`

Tier 2: Configurable Pipelines (App self-service)¶

Consumer apps create from templates via POST /pipelines with parameters.

Template	Required Params	Description
`threshold_alert`	`key`, `op`, `value`	Alert when telemetry crosses threshold
`geo_fence`	`zone_id`, `device_types`	Alert on zone enter/leave
`rolling_average`	`key`, `window_minutes`	Rolling average over time window

Tier 3: Custom Pipelines (Operator-built)¶

Platform operators build in StreamPipes UI. ThingIO exposes results via the same query API. For complex analytics: ML inference, multi-source correlation, domain-specific pattern detection.

Uniform Consumption¶

All three tiers are queried through the same API. Consumer apps don’t care how a pipeline was created:

GET /thingio/v1/pipelines?tier=managed     → Tier 1 results
GET /thingio/v1/pipelines?tier=app         → Tier 2 results
GET /thingio/v1/pipelines?tier=platform    → Tier 3 results
GET /thingio/v1/pipelines/{id}             → Any tier by ID

Account Scoping¶

All ThingIO resources are scoped to the account that creates them.

Resource	Scope	Visibility
Scope	Account	Only filters account’s devices
Pipeline	Account	Only processes account’s telemetry
Consumer App	Account	Registered per account
Telemetry	Account	Only account’s device data

Frontend (ThingIO UI)¶

ThingIO provides a platform-level control plane UI — not a business dashboard. Accessed via Console gateway at /thingio/.

Navigation structure (AdminCenter pattern):

ThingIO
├── Scopes          (header tab)
│   └── Things      (within selected scope)
├── Telemetry       (header tab)
├── Pipelines       (header tab)
│   ├── Managed     (sidebar, Tier 1)
│   ├── App         (sidebar, Tier 2)
│   └── Platform    (sidebar, Tier 3)
└── Consumer Apps   (header tab)

Business apps (WES, ClearJanitor) build their own dashboards by calling ThingIO Data API. ThingIO UI is for platform operators to monitor scopes, pipelines, telemetry records, and consumer app registrations.

Consumer App Integration Pattern¶

Consumer apps call ThingIO APIs to get data and render their own widgets:

WES app
  │
  ├── reads Tier 1 (managed)
  │     GET /thingio/v1/things/{id}/telemetry
  │
  ├── creates Tier 2 (configurable)
  │     POST /thingio/v1/pipelines {template: "threshold_alert", ...}
  │     GET  /thingio/v1/pipelines/{id}
  │
  └── consumes Tier 3 (custom, operator-built)
        GET /thingio/v1/pipelines/{id}

Implementation¶

Aspect	Technology
Backend	Python 3.12, FastAPI, async SQLAlchemy + asyncpg
Database	PostgreSQL (in-cluster for dev, RDS for prod)
Pipeline engine	Apache StreamPipes (ThingIO delegates execution)
Message bus	Kafka (Strimzi) — telemetry ingest, analytics results
Frontend	React 18, Vite, IBM Carbon Design, `@fg/topbar`
Auth	Keycloak JWT / X-Account-Id header

Deployment¶

Component	Namespace	Replicas	Image
thingio-backend	`thingio`	2 (HPA)	`trustmint/thingio-backend`
thingio-frontend	`thingio`	1-2	`trustmint/thingio`
thingio-postgres	`thingio`	1	`postgres:15-alpine`

Network Access (NetworkPolicy)¶

Source	Access	Purpose
`api-gateway` ns	Port 8000	Management API via KrakenD
`trustmint` ns	Port 8000	Integration API (scope sync, pipeline provisioning)
`thingsboard` ns	Port 8000	Integration API (telemetry ingest)
`streampipes` ns	Port 8000	Integration API (analytics results)
`kafka` ns	Port 8000	Kafka consumer connections

Kafka ACLs (`thingio-consumer`)¶

Topic Pattern	Access	Purpose
`thingsboard.*`	Read	Telemetry data
`sp.analytics.*`	Read	StreamPipes analytics results
`trustmint.device.*`	Read	Device enrollment events

Dependencies¶

Service	Relationship
TrustMint	Device registry, enrollment events, scope sync
ThingsBoard	Telemetry data source (via Kafka)
StreamPipes	Pipeline execution engine
DotID	Authenticates API calls, account scoping
Kafka	Event bus for telemetry and analytics

Consumed By¶

Consumer	Usage
WES	Fleet telemetry, order throughput, zone utilization (via API)
ClearJanitor	Cleaning coverage, robot status (via API)
StarGate	Admin views for platform-wide telemetry monitoring
ThingIO UI	Platform operator control plane
Any consumer app	Register, create scopes/pipelines, query telemetry