Unified multimodal neural codec transport protocol. 12 modalities. Per-modality encryption with independent forward-secrecy ratchets. AI-native discrete token streams. Sub-100ms end-to-end latency.
Multiple provisional patent applications filed. Patent pending.
Join 200+ researchers and developers tracking the SSP specification
Real-time multimodal transport with bounded latency guarantees across all modality types.
Perceptual impact scoring determines which modalities degrade first under bandwidth pressure.
Register custom modality IDs (0x10+) without protocol changes. Future-proof by design.
Independent encryption keys per modality. Share audio without exposing biometric data.
Discrete token payloads designed for direct consumption by transformer architectures.
Dynamic bitrate allocation across modalities based on perceptual importance and available bandwidth.
Modality slots declare prediction dependencies enabling cross-modal compression gains.
Cryptographic integrity verification on every frame. Truncated MAC for wire efficiency.
Designed for compatibility with ISO/IEC 23090-31:2025 Multimodal Information Handling System.
SSP fills capabilities absent from existing transport protocols. No existing standard supports 7+ modality framing, per-modality E2E encryption, or AI-native token output.
| Capability | RTP | WebRTC | MIHS | MoQT | SSP |
|---|---|---|---|---|---|
| 7+ modality framing | — | — | — | — | Yes |
| Per-modality E2E encryption | — | — | — | — | Yes |
| AI-native token output | — | — | — | — | Yes |
| Cross-modal prediction | — | — | — | — | Yes |
| Perceptual bitrate allocation | — | — | — | — | Yes |
| Latent-space watermarking | — | — | — | — | Yes |
| Selective disclosure | — | — | — | — | Yes |
| Audio/video streaming | Yes | Yes | Yes | Yes | Yes |
Each modality slot carries an independent encryption_key_id referencing a per-modality key established during SSP_KEY_EXCHANGE. Each modality MAY use an independent forward-secrecy ratchet chain.
12 registered modality IDs. Custom modalities from 0x10. Each modality slot carries independent codec, encryption, and synchronization configuration.
0x10+ reserved for custom modality registration
SSP frames carry AI-native discrete token payloads. Each modality produces tokens at its own rate. Cross-modal synchronization anchors align streams in time.
Comprehensive patent portfolio covering the core SSP innovations. Multiple provisional patent applications filed with international protection pathways in progress.
SSP provides the multimodal sensing infrastructure layer for organizations that cannot trust third-party processing of biometric, spatial, or classified audio/video streams.
Per-modality encryption with independent forward-secrecy ratchets enables granular access control. Share audio transcription without video access. Share motion tracking without biometric data. Isolate thermal/spatial from emotional analysis. Each modality's key chain ratchets independently.
SSP's patented latent-space watermarking embeds imperceptible provenance markers in the latent space of neural codec tokens. Survives re-encoding, transcoding, and adversarial extraction attempts. Addresses Article 50 watermarking requirements for AI-generated audio/video content.
Reference implementation in Rust. Full frame serialization, wire format roundtrip, HMAC verification, modality slot encoding, and token stream parsing.
Each SSP frame carries 1-255 modality slots. Each slot has a compact fixed header followed by variable payload bytes.
Registered modality identifier with extensible custom range
Independent codec selection per modality with variable-length payload
Microsecond sync anchor, QoS level, and cross-modal prediction dependencies
Per-modality key reference for independent encryption and selective disclosure
SSP addresses multimodal transport requirements across industries where existing protocols fall short on encryption granularity, modality coverage, or AI-native processing.
Per-modality encryption for video calls enables selective disclosure for compliance. Share audio transcription with legal review without exposing video feeds. Grant AI assistants access to speech tokens while biometric data stays encrypted.
Sovereign multimodal processing with air-gapped deployment compatibility. On-device inference eliminates data exfiltration risk. Per-modality key management enables compartmentalized information handling across classification levels.
Patient data segregation by modality enables HIPAA-aligned selective access. Share physiological telemetry with monitoring systems while keeping video and audio encrypted. Biometric modality isolation prevents accidental cross-contamination.
Multi-sensor fusion with crypto-separated modalities. SSP's unified framing carries LiDAR, camera, radar, IMU, and thermal data in a single synchronized stream with cross-modal prediction for compression. Bandwidth-adaptive under satellite link constraints.
Patented latent-space watermarking embeds imperceptible provenance markers in neural codec tokens. Survives re-encoding, transcoding, and adversarial extraction. Detects AI-generated audio and video content at the codec level, not as a post-processing filter.
Existing protocols were designed for audio and video. SSP was designed from scratch for the multimodal, AI-native era.
Audio, video, haptic, spatial, biometric, motion, thermal, emotion -- all in a single synchronized frame. Not separate protocols stitched together with middleware. Not RTP for audio plus a custom channel for everything else.
Per-modality encryption is not a transport-layer wrapper. It is a first-class frame field. Each modality slot declares its own encryption key, enabling selective disclosure without protocol extensions, middleware layers, or application-level workarounds.
SSP uses perceptual impact scoring to determine degradation order. When bandwidth drops, lower-priority modalities gracefully reduce quality while critical streams maintain fidelity. No binary on/off. No codec renegotiation. Smooth transitions.
Start building with the SSP reference implementation. Rust-first, with Python bindings available.
The Saj Sense Protocol specification is under active development. Join the waitlist to receive updates on new versions, reference implementations, and early access to the SDK.
No spam. Unsubscribe anytime.