Project 13 ยท Home Lab

A connected-fleet backend with a hot path that reacts in seconds and a cold path with cheap, queryable history.

Built withAWS CDK (TypeScript)
LevelIntermediate
Time to build1 day
Running cost~$1โ€“5 / day while simulating

The problem

IoT backends have two jobs that pull in opposite directions: react to each reading right now, and keep years of history affordably. Splitting the stream into a hot path and a cold path, with least-privilege device identity, handles both without either one slowing the other down.

What you’ll build

  • An AWS IoT Core topic rule fanning out fleet/+/+/telemetry to two paths
  • Hot path: Kinesis on-demand โ†’ Lambda updating “latest state per device” in DynamoDB, plus overheat/low-battery alerts by email and MQTT
  • Cold path: Firehose โ†’ S3 (gzip, partitioned by day) with an Athena table using partition projection
  • A least-privilege device policy: each thing can publish only its own topic
  • A live CloudWatch dashboard from Embedded Metric Format logs, plus a fleet simulator

Architecture

Trucks (MQTT + X.509) โ”€โ–ถ AWS IoT Core โ”€โ–ถ topic rule  SELECT *, topic(3) AS device_id ... 'fleet/+/+/telemetry'
                                          โ”œโ”€ Kinesis (on-demand) โ”€โ–ถ Lambda โ”€โ”ฌโ”€ DynamoDB  "latest state per device"
                                          โ”‚                                 โ”œโ”€ SNS + MQTT alerts/<device>  (overheat, low battery)
                                          โ”‚                                 โ””โ”€ EMF metrics โ”€โ–ถ CloudWatch dashboard "iot-fleet-live"
                                          โ””โ”€ Firehose โ”€โ–ถ S3 (gzip, dt=YYYY-MM-DD) โ”€โ–ถ Athena  iot_telemetry.readings

AWS services used

AWS IoT CoreKinesis Data StreamsData FirehoseLambdaDynamoDBS3AthenaCloudWatch

How it works

  1. ConnectDevices authenticate with X.509 certificates. Their identity is their authorization.
  2. Fan outOne SQL rule sends every reading to Kinesis (real time) and Firehose (archive).
  3. ReactLambda batches records, keeps the newest per device and alerts on anomalies.
  4. AnalyzeAthena queries the archive by day with no crawlers.

Key design decisions

  • Timestream for LiveAnalytics no longer accepts new customers, so DynamoDB + S3/Athena covers “now” and “history” more cheaply.
  • Partial batch failures mean one bad record doesn’t replay the whole batch.
  • EMF metrics give live dashboards with zero PutMetricData API calls.

Deploy it

npm install
npx cdk deploy -c alertEmail=you@example.com -c tempMaxC=85

Full prerequisites, parameters and clean-up steps are in the project README on the code page.

See it work

Run the simulator with 25 trucks driving around Austin. Watch readings flow on the dashboard, get an email when a truck overheats, and query the day’s peak temperatures in Athena.

Skills this demonstrates

Streaming architectureIoT securityHot/cold data pathsServerless at scale

Get the code

Every file needed to build this project, written as a deployable AWS CDK app and validated with the CloudFormation schema.


Leave a Reply

Your email address will not be published. Required fields are marked *