Unified multimodal neural codec transport protocol. 12 modalities. Per-modality encryption with independent forward-secrecy ratchets. AI-native discrete token streams. Sub-100ms end-to-end latency.
6 provisional patent applications filed at IP Australia, March 2026.
Join 200+ researchers and developers tracking the SSP specification
Real-time multimodal transport with bounded latency guarantees across all modality types.
Perceptual impact scoring determines which modalities degrade first under bandwidth pressure.
Register custom modality IDs (0x10+) without protocol changes. Future-proof by design.
Independent encryption keys per modality. Share audio without exposing biometric data.
Discrete token payloads designed for direct consumption by transformer architectures.
Dynamic bitrate allocation across modalities based on perceptual importance and available bandwidth.
Modality slots declare prediction dependencies enabling cross-modal compression gains.
HMAC-SHA256 integrity verification on every frame. Truncated to 64 bits for wire efficiency.
Designed for compatibility with ISO/IEC 23090-31:2025 Multimodal Information Handling System.
SSP fills capabilities absent from existing transport protocols. No existing standard supports 7+ modality framing, per-modality E2E encryption, or AI-native token output.
| Capability | RTP | WebRTC | MIHS | MoQT | SSP |
|---|---|---|---|---|---|
| 7+ modality framing | — | — | — | — | Yes |
| Per-modality E2E encryption | — | — | — | — | Yes |
| AI-native token output | — | — | — | — | Yes |
| Cross-modal prediction | — | — | — | — | Yes |
| Perceptual bitrate allocation | — | — | — | — | Yes |
| Latent-space watermarking | — | — | — | — | Yes |
| Selective disclosure | — | — | — | — | Yes |
| Audio/video streaming | Yes | Yes | Yes | Yes | Yes |
Each modality slot carries an independent encryption_key_id referencing a per-modality key established during SSP_KEY_EXCHANGE. Each modality MAY use an independent forward-secrecy ratchet chain.
12 registered modality IDs. Custom modalities from 0x10. Each modality slot carries independent codec, encryption, and synchronization configuration.
0x10+ reserved for custom modality registration
SSP frames carry AI-native discrete token payloads. Each modality produces tokens at its own rate. Cross-modal synchronization anchors align streams in time.
6 provisional patent applications filed at IP Australia, March 2026. PCT international phase deadline: March 2027. All assessed at novelty 8-10/10. Freedom-to-operate: GREEN across all 6.
SSP provides the multimodal sensing infrastructure layer for organizations that cannot trust third-party processing of biometric, spatial, or classified audio/video streams.
Per-modality encryption with independent forward-secrecy ratchets enables granular access control. Share audio transcription without video access. Share motion tracking without biometric data. Isolate thermal/spatial from emotional analysis. Each modality's key chain ratchets independently per SSP-ENC-001.
PROV-038 (Latent-Space Codec Watermarking) embeds imperceptible provenance markers in the latent space of neural codec tokens. Survives re-encoding, transcoding, and adversarial extraction attempts. Addresses Article 50 watermarking requirements for AI-generated audio/video content.
Reference implementation in Rust. Full frame serialization, wire format roundtrip, HMAC verification, modality slot encoding, and token stream parsing.
Each SSP frame carries 1-255 modality slots. Each slot has a 12-byte fixed header followed by variable payload bytes.
| Offset | Size | Field | Description |
|---|---|---|---|
| 0 | 1 | Modality ID | Registered modality identifier (0x01-0x0C, 0x10+) |
| 1 | 1 | Codec ID | Codec used for this modality's payload |
| 2 | 2 | Payload Length | Length of variable payload in bytes |
| 4 | 4 | Sync Anchor | Microsecond timestamp for cross-modal sync |
| 8 | 4 bits | QoS / LOD | Quality-of-service level for bandwidth adaptation |
| 8.5 | 4 bits | Prediction Deps | Cross-modal prediction dependency bitmap |
| 9 | 2 | Token Count | Number of discrete tokens in payload |
| 11 | 1 | Encryption Key ID | Per-modality key reference (0 = unencrypted) |
The Saj Sense Protocol specification is under active development. Join the waitlist to receive updates on new versions, reference implementations, and early access to the SDK.
No spam. Unsubscribe anytime.