Encifher
CoProcessor Crates

Core

Core crate responsibilities, design, interfaces, and workflows

Overview & Design

System Overview

The encifher-core crate provides computation engine, core data types, cryptographic interfaces, and Sled-based storage utilities for Encifher coprocessor. Smart contracts operate on 16-byte symbolic references (handles) instead of raw ciphertexts. Core implements the decrypt → compute → encrypt pipeline and deterministic handle derivation used by other coprocessor services.

Core is a library crate which does not run any services.

Capabilities

  1. Computation engine

    Core compute operations :

    • Arithmetic: Add, Subtract, Multiply, Divide, Reminder
    • Relational: Equal, LessEqual, LessThan, NotEqual, GreaterEqual, GreaterThan
    • Bitwise: BitAnd, BitOr, BitXor, ShiftLeft, ShiftRight, RotLeft, RotRight
    • Control flow: IfThenElse
    • Unary: Negate, Not
    • Min/Max: Minimum, Maximum

    Additional Operation variants :

    • Special: TrivialEncrypt, VerifyCiphertext, Cast
    • Access control: SetAccess, IsAllowed
    • Decryption: Decrypt

  2. Cryptographic interface

    • Cipher trait with local Threshold-ElGamal implementation
    • Single and batch decryption helpers with fallback mechanisms

  3. Core data types

    • Handle (u128 type alias)
    • CiphertextWithCts: Container for encrypted data with CTS metadata
    • RequestParams: Request with handles and operation metadata
    • BatchData: Collection of groups for batch processing
    • Group: Computation group with inputs, expression, and outputs
    • BatchTree: Merkle tree over request-ciphertext pairs
    • RequestAndCiphertext: Combined request and encrypted result

  4. Storage utilities

    • KvDB: Key-value database wrapper on Sled with explicit flush

  5. Event ingestion

    • EventListener for Helius Laserstream integration
    • Kafka publishing capabilities via events module

  6. Clients

    • IndexerClient: For retreiving ciphertexts and updating computed ciphertexts.
    • KafkaClient: Generic producer/consumer for managing onchain computation requests.

Module Organization

  • Cryptography: accumulator, kms, verifier
  • Compute/operations: compute, ops
  • Storage/data: db, types, codec, constants
  • Integration: client, events, tee
  • Utilities: utils (handle derivation, encoding helpers, parsing)
  • Request handling: request (request type definitions)

Architecture & Workflows

Computation Flow

  1. Inputs: Callers pass serialized operands and per-operand flags that indicate scalar (plaintext) vs ciphertext
  2. Decrypt: Non-scalar operands are decrypted via Cipher trait (single or batched)
  3. Compute: Plaintext operations execute in process (arithmetic, comparisons, bitwise, control flow)
  4. Output: Callers may encrypt results (encrypt_result) and derive typed handles. Core does not persist results to external services.

Decryption helpers use scalar_byte to distinguish plaintext from encrypted inputs :

  • Value 1: treat as plaintext, skip decryption
  • Value 0: encrypted data, decrypt via Cipher trait

Data Model and Cryptographic Flow

Ciphertext Container

CiphertextWithCts encodes encrypted data with CTS metadata :

  • Encoding format: [4-byte LE cts_len] || cts || ciphertext_bytes
  • Hashing uses Keccak256 over cts || ciphertext_bytes
  • Strict deserialization with length validation (panics on invalid input)

Batch and Expression Model

BatchData contains one or more computation groups with validation:

  • Required non-zero batch_hash identifier
  • Validation enforces :
    • At least one group present
    • Expression syntax via regex ^[a-z0-9AB_+\-*/><!&|^=?:() ]+$
    • Balanced parentheses and correct operator/operand arity
    • Merkle proof verification against root hash
    • ECDSA signature verification over root_hash

Group represents single computation unit :

  • List of inputs with handles and optional ciphertexts
  • Expression string defining computation
  • Signer pubkey for authorization
  • Output handle and intermediate handles
  • Associated request parameters
  • Tokenizer for expression parsing with special handling for operations like decrypt, set_access

GroupInput provides per-input metadata :

  • Handle identifier
  • Optional ciphertext data
  • Merkle proof components (leaf hash, inclusion proof, root hash)
  • DER-encoded ECDSA signature for batch validation
  • Timestamp

BatchTree aggregates results :

  • Merkle tree (using Keccak256) over request-ciphertext pairs
  • Uses SimpleMerkleTree from chain_client crate

RequestParams encapsulates operation details :

  • Request ID and signer pubkey
  • LHS and RHS handles
  • Operation type and scalar flags
  • Optional fields: middle, anon_transfer_handles, decrypt_handles, access_handles, balance_conversion_handles

Signature Verification

Batch Validation :

  • Signatures are DER-encoded ECDSA over root_hash (32 bytes)
  • Verification uses secp256k1 public key from TEE_PUBLIC_KEY environment variable
  • Public key format: 65-byte uncompressed (0x04 prefix)
  • Signature::from_der(&signature) with message = input.root_hash.to_vec()

IndexerClient Fetches :

  • When merkle root signature is present: compact 64-byte ECDSA signature
  • Verified against TEE_PUBLIC_KEY before accepting response
  • Signature is over returned merkle_root (32 bytes)
  • Signature::from_compact(&signature)

Public Interfaces

Cipher Trait

The Cipher trait defines encryption/decryption interface :

pub trait Cipher {
    fn encrypt(&self, data: &[u8]) -> Result<CiphertextWithCts, Box<dyn std::error::Error>>;
    async fn decrypt(&mut self, cipher: CiphertextWithCts) -> Result<Vec<u8>, Box<dyn std::error::Error>>;
    async fn decrypt_batch(&mut self, cts: Vec<CiphertextWithCts>) -> Result<Vec<Vec<u8>>, Box<dyn std::error::Error>>;
}

IndexerClient

Client for retrieving ciphertexts from indexer service :

  • Configured with indexer base URL
  • Methods: indexer_fetch_ciphertext (single) and indexer_fetch_multiple_ciphertexts (batch)

Behavior :

  • Scalar mode (scalar_byte == 1): Returns 16-byte big-endian handle as plaintext
  • Encrypted mode (scalar_byte == 0):
    • Calls POST /v1/get-ciphertext endpoint
    • Verifies compact 64-byte ECDSA signature over merkle root (if present)
    • Returns serialized CiphertextWithCts or vec![0; 32] sentinel if not found
  • Read-only client; writes handled by processor/indexer services

KafkaClient

Generic Kafka client for producing/consuming JSON messages :

  • Type parameter T must be JSON serializable/deserializable
  • Can be configured as consumer OR producer (not both, mutually exclusive)
  • Consumer: with_consumer(broker, group_id, topic) streams messages
  • Producer: with_producer(broker) sends JSON messages to topics
  • Auto-serializes/deserializes messages

BN254 Accumulator

Cryptographic accumulator for membership proofs :

  • Manages G1/G2 group elements and accumulator state
  • Methods: new, add_member, membership_witness, verify_membership
  • hash_to_scalar: Converts arbitrary data to BN254 field elements via Keccak256
  • verify_membership_solana: Uses Solana's alt_bn128_pairing syscall for on-chain verification
  • Accumulator value is product of all added member scalars

TEE Helper

Provides access to TEE public key :

  • get_public_key(): Returns static reference to TEE public key bytes
  • Reads from TEE_PUBLIC_KEY environment variable (hex-encoded)
  • Expected format: 65-byte uncompressed secp256k1 (0x04 prefix)
  • Cached in OnceLock for efficiency
pub fn get_public_key() -> &'static Vec<u8> {
    TEE_PUBLIC_KEY.get_or_init(|| {
        let public_key = std::env::var("TEE_PUBLIC_KEY").expect("Failed to get public key");
        hex::decode(public_key).expect("Failed to decode public key")
    })
}

Submitter Integration

BatchTreeSubmission structure for chain submission :

  • Accumulator root (0x-prefixed hex, 32 bytes)
  • ECDSA signature over root with secp256k1 recovery ID
  • List of batch trees
  • Optional DA submission ID (set after publishing to data availability layer)
  • Note: Current on-chain transactions only include root, signature, and recovery_id

Utility Functions

Key helper functions in utils module :

  • append_type: Adds type bits to handle
  • uint128_to_bytes16 / bytes16_to_uint128: Handle conversion utilities
  • ascii_to_decimal: Number decoding (hex-aware)
  • decimal_to_ascii: Number encoding
  • normalize_to_32_bytes: Pads or truncates to 32-byte arrays

Storage Strategy

KvDB (Sled-based Key-Value Store)

Operations :

  • insert: Insert or update key-value pair (auto-flushes)
  • get: Retrieve value by key
  • remove: Delete key-value pair (auto-flushes)
  • contains_key: Check key existence
  • iter: Iterate over all entries
  • flush: Explicit flush to disk
  • shutdown: Graceful shutdown with flush

Storage Patterns:

  • ACL permissions use hashv(signer || handle) as key with boolean value

QueueDB (Sled-based FIFO Queue)

Operations :

  • push: Add item to tail (auto-flushes)
  • pop: Remove and return item from head (auto-flushes)
  • peek: View head item without removal
  • flush: Explicit flush to disk
  • shutdown: Graceful shutdown with flush

Implementation :

  • FIFO queue with __head__ and __tail__ metadata keys (static byte slices)
  • Entry keys are big-endian u64 indices
  • Every write operation explicitly flushes for durability

Error Handling & Resilience

Error Handling Patterns

  • Standard Result<T, E> returns throughout
  • Logging via log crate (info, warn, error levels)
  • No crate-level typed error enums; uses Box<dyn std::error::Error> for flexibility

Resilience Mechanisms

  • CiphertextWithCts parsing: Strict length validation; panics on invalid input to prevent unsafe partial decoding
  • Decryption fallbacks:
    • Single decrypt: Returns 32-byte vector [0,0,...,0,2] on failure
    • Batch decrypt: Treats vec![0; 32] as "missing ciphertext" sentinel; preserves sentinel in output
  • IndexerClient: Returns 32-byte zero vector sentinel when ciphertext not found
  • Threshold decryption: Returns 32-byte fallback vector on aggregation failure
  • Division by zero: Returns error instead of panic

Configuration

Environment Variables

  • TEE_PUBLIC_KEY (required): Hex-encoded 65-byte uncompressed secp256k1 public key for signature verification

Feature Flags

  • No feature flags defined in this crate

Runtime Configuration

  • Consumers pass configuration via function parameters
  • Database paths configured at open-time for KvDB and QueueDB
  • Shared configuration management available via config_loader crate (used by consumer services)

Tests

Unit Tests

Accumulator :

  • Membership proof verification for multiple members
  • Non-member rejection (no witness for non-members)
  • Solana pairing syscall verification
  • Byte serialization validation
  • Edge cases: zero points and large scalars

Compute :

  • Arithmetic: add, subtract, multiply, divide
  • Comparison: less_than, less_equal, equal
  • Bitwise: bit_and
  • Control flow: if_then_else
  • Encrypt/decrypt round-trips

KMS :

  • Local threshold encrypt/decrypt flows
  • Batch decryption
  • Partial decryption and aggregation using indices 0 and 2

Ops/Utils :

  • Handle derivation helpers
  • ACL permission management
  • Operand parsing and encoding

Integration Tests

Integration tests and service-level behaviors (indexer round-trips, production TEE/attestation, end-to-end batch processing) are implemented in consumer crates and services (processor, indexer, submitter).

CoProcessor Crates

Crates and their description

Processor

Processor service responsibilities, endpoints, data formats, and integrations