Skip to main content

Your side: the reference architecture

For solution architects and engineers. What a production integration needs in your environment. Any stack will do: what matters is the responsibility of each block and the contract it meets.

The partner-side reference architecture: collector, bar builder, ingest workload, key vault, series registry, segment store, event receiver, queue and workers, your app, observability

The building blocks​

#BlockMustExample realisations
1CollectorRead your sources in time order per series, and survive restarts without skipping or repeatingHistorian client, feed handler, file watcher
2Bar builderTurn readings into bars: ordered, non-overlapping, values above zero (offset if needed), gaps left as gapsYour code; examples/python/bars.py
3Ingest workloadCall Patternode from your backend only: key header, body hash, idempotency key, retries with backoff and retry-after, the stored edge with each new bar, one writer per seriesA service or function; examples/*/client
4Key vaultHold the live key (or OAuth signing key) and the webhook secret encrypted; readable only by blocks 3 and 7; versioned for rotationAWS Secrets Manager, Azure Key Vault, Google Secret Manager, HashiCorp Vault
5Series registryMap each opaque series name to your client and asset; hold each forming series' edge with a version, in the same write as its segmentsA table in your database; the segment store
6Segment storeKeep segments by client prefix, series, level and id; replace provisional segments, keep final ones; delete a client by deleting its prefixObject storage, a time-series or relational database
7Event receiverVerify the signature on the raw body, reject stale timestamps, drop seen event ids, answer 2xx within 10 seconds, queue the workA function behind HTTPS; or your event bus
8Queue and workersDo the work after the answer; retry; park failures in a dead-letter queueSQS, Pub/Sub, Service Bus, Kafka
9Your applicationRead segments from your store, never from Patternode; draw each chart at the level it can readYour product
10ObservabilityMeasure success rate, p99 latency, 429 rate, edge and out-of-order errors, event lag and dead-letter depth; propagate traceparentYour monitoring and tracing stack

AWS with Terraform builds blocks 4, 6, 7 and 8 and the policy for 3.

What Patternode assumes about your environment​

#AssumptionIf it does not hold
A1Your backend can make outbound HTTPS calls (and gRPC, if you use it) to your jurisdiction's endpointUse private connectivity, or an egress proxy that allows *.api.patternode.com
A2You have a secret store, and workloads read credentials from it at startIntroduce one before go-live; keys in code or images fail the production checklist
A3You can identify each series stably, and keep the mapping from opaque name to client and asset yourselfPatternode cannot recover it: it never learns who or what a series is
A4You can deliver each series' bars in time order, with one writer per series, keeping its edgeFunnel each series through one writer (a partition key, a lock, an actor); a bar before the edge is refused. Lose an edge and you can rebuild it from the series' bars
A5You can store segments: plan for up to one segment record per bar you send, across all levelsStore only the levels you draw, and re-fetch history as a finished series when needed
A6Your bar times come from your sources; Patternode never changes themNormalise time zones and clock offsets before building bars
A7An HTTPS endpoint can receive webhooks, or you use an event bus (Mission Critical)Poll jobs instead of receiving webhooks; slower, but complete
A11For jobs, storage you can presign links in (S3, Azure Blob, Google Cloud Storage)Send series inline in pieces of up to 5 MB instead
A8You can monitor and alert on your side of the integrationBuild it before go-live; Patternode's status page covers only Patternode's side
A9For Mission Critical: your callers run in two regions of the jurisdiction's cloud, or connect privatelyA single region of yours caps your availability below the service's
A10Two named technical contacts, and an on-call rota for Mission CriticalRequired by the order form

Sizing​

InputHow to estimate
Bars a second at peakSeries × (1 / bar width) × peak factor. Peaks: a storm (every point reporting), a market open or close
Edges to keepOne per forming series, a few kilobytes each
Segment store growthUp to one segment record per bar sent, a few hundred bytes each, plus an edge per forming series
Events a secondJobs finishing plus segments closing at the levels you subscribe to

Compare the peaks with your plan. Next: Deployment and resilience.