diff --git a/A2ML-ALIB-INTEGRATION-PLAN-2026-01-30.adoc b/A2ML-ALIB-INTEGRATION-PLAN-2026-01-30.adoc new file mode 100644 index 00000000..e40b2bab --- /dev/null +++ b/A2ML-ALIB-INTEGRATION-PLAN-2026-01-30.adoc @@ -0,0 +1,272 @@ +== a2ml + aLib Integration Plan for proven + +*Date:* 2026-01-30 *Status:* Task #3 - In Progress *Related Repos:* +proven, a2ml, aggregate-library + +''''' + +=== Overview + +*Integration Goal:* Establish proven as the reference implementation for +aggregate-library (aLib) operations, using a2ml for specification +markup. + +*Three Components:* + +[arabic] +. *proven* - Idris2 library with formally verified operations +. *aggregate-library (aLib)* - Methods repository specifying overlap +operations +. *a2ml* - Attested Markup Language for specs/docs + +''''' + +=== What is aLib? + +*Purpose:* A stress-test and methods lab demonstrating how to specify a +minimal overlap library across wildly different systems. + +*NOT:* - NOT a standard library replacement - NOT a proposal that all +languages share one stdlib - NOT a mandatory dependency + +*IS:* - A demonstration of specification methods - A conformance testing +framework - A stress-test for diversity + +*Spec Categories:* + +.... +aggregate-library/specs/ +├── arithmetic/ (add, subtract, multiply, divide, modulo) +├── collection/ (list operations) +├── comparison/ (equality, ordering) +├── conditional/ (if-then-else patterns) +├── logical/ (and, or, not) +└── string/ (concat, length, substring) +.... + +''''' + +=== What is a2ml? + +*Purpose:* A lightweight, Djot-like markup that compiles into a typed, +attested core. + +*Features:* - Progressive strictness (lax → checked → attested) - Strong +guarantees (required sections, resolved references, unique IDs) - Opaque +payloads preserved byte-for-byte - Renderer portability +(HTML/Markdown/PDF) + +*Idris2 Core:* + +.... +a2ml/src/A2ML/ +├── TypedCore.idr (typed core stub) +├── Translator.idr (translation stub) +└── Surface.idr (surface AST) +.... + +''''' + +=== Current Integration Status + +==== proven’s Role (per META.scm ADR-006) + +*Decision:* "`proven implements aLib core operations with formal +verification proofs`" + +*What proven provides:* - SafeMath implements aLib arithmetic: add, +subtract, multiply, divide, modulo - SafeString implements aLib string: +concat, length, substring - All implementations have termination proofs +(totality) - proven serves as gold standard for aLib correctness - +89-language FFI enables aLib compliance testing across platforms + +*Status:* ✓ *ALREADY IMPLEMENTED* + +proven’s SafeMath and SafeString already implement the aLib operations! +The integration exists in code but may need: 1. Documentation mapping +(proven operation → aLib spec) 2. Conformance test compliance 3. +a2ml-based specification documents + +==== What’s Missing + +[arabic] +. *Explicit aLib conformance testing* +* aLib specs have conformance test vectors +* proven should run these vectors +* Document compliance results +. *Specification in a2ml format* +* proven’s docs could use a2ml markup +* Provides attested, verifiable documentation +* Stronger structural guarantees than plain markdown +. *ECOSYSTEM.scm linkage* +* proven’s ECOSYSTEM.scm should explicitly list aLib relationship +* a2ml should be listed as documentation tool + +''''' + +=== Integration Tasks + +==== Task 3.1: Update proven’s ECOSYSTEM.scm ✓ + +Add explicit relationships: + +[source,scheme] +---- +(related-projects + ((project . "aggregate-library") + (type . "specification-consumer") + (relationship . "proven implements aLib core operations as reference implementation") + (url . "https://github.com/hyperpolymath/aggregate-library")) + + ((project . "a2ml") + (type . "documentation-tool") + (relationship . "a2ml provides attested markup for proven specifications") + (url . "https://github.com/hyperpolymath/standards/tree/main/a2ml"))) +---- + +==== Task 3.2: Create aLib Conformance Matrix + +*File:* `+docs/ALIB-CONFORMANCE.adoc+` (or `+.a2ml+` when a2ml is +stable) + +*Content:* + +.... +| aLib Spec | proven Module | Function | Status | Notes | +|-----------|---------------|----------|--------|-------| +| arithmetic/add | SafeMath | add_checked | ✓ PASS | Overflow detection | +| arithmetic/subtract | SafeMath | sub_checked | ✓ PASS | Underflow detection | +| arithmetic/multiply | SafeMath | mul_checked | ✓ PASS | Overflow detection | +| arithmetic/divide | SafeMath | div | ✓ PASS | Division by zero handling | +| arithmetic/modulo | SafeMath | mod | ✓ PASS | Division by zero handling | +| string/concat | SafeString | concat | ✓ PASS | No length overflow | +| string/length | SafeString | length | ✓ PASS | Returns natural number | +| string/substring | SafeString | substring | ✓ PASS | Bounds checking | +.... + +==== Task 3.3: Run aLib Conformance Tests + +*Action:* Execute aLib test vectors against proven + +*Location:* `+aggregate-library/tests/+` (if they exist) + +*Steps:* 1. Check if aLib has test vectors 2. Create test runner that +calls proven FFI 3. Document results in `+docs/ALIB-CONFORMANCE.adoc+` + +==== Task 3.4: Document a2ml Usage (Future) + +*When:* After a2ml reaches v1.0 + +*Action:* Convert proven specs to a2ml format + +*Example:* + +[source,a2ml] +---- +# SafeMath Specification + +@abstract: +SafeMath provides formally verified arithmetic operations with +overflow/underflow detection and division-by-zero prevention. +@end + +@requires: +- All operations must be total (no crashes) +- Overflow detection via dependent types +- Conformance with aggregate-library arithmetic specs +@end + +## Operations + +@spec(id="safe-add"): +**Function:** `add_checked : Int -> Int -> Result Int` + +**Semantics:** Addition with overflow detection + +**Proof Obligations:** +1. Totality: ∀ a b. add_checked a b returns (OK | Error) +2. Correctness: ∀ a b. no overflow → add_checked a b = OK (a + b) +3. Safety: ∀ a b. overflow → add_checked a b = Error Overflow +@end +---- + +*Benefits:* - Structural verification of docs - References must resolve +- Sections cannot be accidentally omitted - Attested markup provides +stronger guarantees + +''''' + +=== Integration Complete Checklist + +For Task #3 completion: + +* [ ] Update proven’s ECOSYSTEM.scm with aLib + a2ml relationships +* [ ] Create `+docs/ALIB-CONFORMANCE.adoc+` +* [ ] Check if aLib has test vectors +* [ ] Document which proven operations map to which aLib specs +* [ ] Note a2ml integration as future enhancement (v1.1+) +* [ ] Update proven’s README.adoc to mention aLib conformance + +''''' + +=== Immediate Actions (for v1.0) + +Since proven ALREADY implements aLib operations, the integration is +mostly about documentation: + +[arabic] +. *Document the relationship* (ECOSYSTEM.scm update) +. *Create conformance matrix* (which operations match which specs) +. *Verify compliance* (if aLib has test vectors, run them) + +This is NOT blocking v1.0 release - it’s documentation work that +clarifies existing compliance. + +''''' + +=== Future Enhancements (v1.1+) + +[arabic] +. *a2ml-based specs* - Convert proven docs to a2ml when stable +. *Automated conformance* - CI job that runs aLib test vectors +. *Cross-language validation* - Use proven’s 89 bindings to test aLib +across ecosystems +. *Contribute back to aLib* - Provide feedback on specs based on +proven’s formal verification insights + +''''' + +=== Questions to Resolve + +[arabic] +. *Does aggregate-library have test vectors?* +* Check `+aggregate-library/tests/+` or `+aggregate-library/test/+` +* If yes: Run them against proven +* If no: Create them based on aLib specs +. *Which aLib specs does proven already satisfy?* +* Arithmetic: ✓ (add, sub, mul, div, mod) +* String: ✓ (concat, length, substring) +* Collection: ? (check if SafeBuffer/SafeQueue satisfy) +* Comparison: ? (check if proven has these) +* Conditional: N/A (language feature, not library) +* Logical: ? (check if proven has these) +. *Should proven docs migrate to a2ml format?* +* Current: AsciiDoc (.adoc) +* Future: a2ml (.a2ml) +* Timeline: After a2ml v1.0 (per a2ml ROADMAP.adoc) + +''''' + +=== Next Steps + +[arabic] +. Update proven’s ECOSYSTEM.scm ← DO THIS NOW +. Create aLib conformance matrix ← DO THIS NOW +. Check for aLib test vectors ← INVESTIGATE +. Document in proven’s README ← DO THIS NOW +. Mark Task #3 complete +. Move to Task #4 (testing documentation) + +''''' + +_Plan created: 2026-01-30_ _Next: Execute integration tasks_ diff --git a/A2ML-ALIB-INTEGRATION-PLAN-2026-01-30.md b/A2ML-ALIB-INTEGRATION-PLAN-2026-01-30.md deleted file mode 100644 index dc3c0811..00000000 --- a/A2ML-ALIB-INTEGRATION-PLAN-2026-01-30.md +++ /dev/null @@ -1,263 +0,0 @@ -# a2ml + aLib Integration Plan for proven -**Date:** 2026-01-30 -**Status:** Task #3 - In Progress -**Related Repos:** proven, a2ml, aggregate-library - ---- - -## Overview - -**Integration Goal:** Establish proven as the reference implementation for aggregate-library (aLib) operations, using a2ml for specification markup. - -**Three Components:** - -1. **proven** - Idris2 library with formally verified operations -2. **aggregate-library (aLib)** - Methods repository specifying overlap operations -3. **a2ml** - Attested Markup Language for specs/docs - ---- - -## What is aLib? - -**Purpose:** A stress-test and methods lab demonstrating how to specify a minimal overlap library across wildly different systems. - -**NOT:** -- NOT a standard library replacement -- NOT a proposal that all languages share one stdlib -- NOT a mandatory dependency - -**IS:** -- A demonstration of specification methods -- A conformance testing framework -- A stress-test for diversity - -**Spec Categories:** -``` -aggregate-library/specs/ -├── arithmetic/ (add, subtract, multiply, divide, modulo) -├── collection/ (list operations) -├── comparison/ (equality, ordering) -├── conditional/ (if-then-else patterns) -├── logical/ (and, or, not) -└── string/ (concat, length, substring) -``` - ---- - -## What is a2ml? - -**Purpose:** A lightweight, Djot-like markup that compiles into a typed, attested core. - -**Features:** -- Progressive strictness (lax → checked → attested) -- Strong guarantees (required sections, resolved references, unique IDs) -- Opaque payloads preserved byte-for-byte -- Renderer portability (HTML/Markdown/PDF) - -**Idris2 Core:** -``` -a2ml/src/A2ML/ -├── TypedCore.idr (typed core stub) -├── Translator.idr (translation stub) -└── Surface.idr (surface AST) -``` - ---- - -## Current Integration Status - -### proven's Role (per META.scm ADR-006) - -**Decision:** "proven implements aLib core operations with formal verification proofs" - -**What proven provides:** -- SafeMath implements aLib arithmetic: add, subtract, multiply, divide, modulo -- SafeString implements aLib string: concat, length, substring -- All implementations have termination proofs (totality) -- proven serves as gold standard for aLib correctness -- 89-language FFI enables aLib compliance testing across platforms - -**Status:** ✓ **ALREADY IMPLEMENTED** - -proven's SafeMath and SafeString already implement the aLib operations! The integration exists in code but may need: -1. Documentation mapping (proven operation → aLib spec) -2. Conformance test compliance -3. a2ml-based specification documents - -### What's Missing - -1. **Explicit aLib conformance testing** - - aLib specs have conformance test vectors - - proven should run these vectors - - Document compliance results - -2. **Specification in a2ml format** - - proven's docs could use a2ml markup - - Provides attested, verifiable documentation - - Stronger structural guarantees than plain markdown - -3. **ECOSYSTEM.scm linkage** - - proven's ECOSYSTEM.scm should explicitly list aLib relationship - - a2ml should be listed as documentation tool - ---- - -## Integration Tasks - -### Task 3.1: Update proven's ECOSYSTEM.scm ✓ - -Add explicit relationships: -```scheme -(related-projects - ((project . "aggregate-library") - (type . "specification-consumer") - (relationship . "proven implements aLib core operations as reference implementation") - (url . "https://github.com/hyperpolymath/aggregate-library")) - - ((project . "a2ml") - (type . "documentation-tool") - (relationship . "a2ml provides attested markup for proven specifications") - (url . "https://github.com/hyperpolymath/standards/tree/main/a2ml"))) -``` - -### Task 3.2: Create aLib Conformance Matrix - -**File:** `docs/ALIB-CONFORMANCE.adoc` (or `.a2ml` when a2ml is stable) - -**Content:** -``` -| aLib Spec | proven Module | Function | Status | Notes | -|-----------|---------------|----------|--------|-------| -| arithmetic/add | SafeMath | add_checked | ✓ PASS | Overflow detection | -| arithmetic/subtract | SafeMath | sub_checked | ✓ PASS | Underflow detection | -| arithmetic/multiply | SafeMath | mul_checked | ✓ PASS | Overflow detection | -| arithmetic/divide | SafeMath | div | ✓ PASS | Division by zero handling | -| arithmetic/modulo | SafeMath | mod | ✓ PASS | Division by zero handling | -| string/concat | SafeString | concat | ✓ PASS | No length overflow | -| string/length | SafeString | length | ✓ PASS | Returns natural number | -| string/substring | SafeString | substring | ✓ PASS | Bounds checking | -``` - -### Task 3.3: Run aLib Conformance Tests - -**Action:** Execute aLib test vectors against proven - -**Location:** `aggregate-library/tests/` (if they exist) - -**Steps:** -1. Check if aLib has test vectors -2. Create test runner that calls proven FFI -3. Document results in `docs/ALIB-CONFORMANCE.adoc` - -### Task 3.4: Document a2ml Usage (Future) - -**When:** After a2ml reaches v1.0 - -**Action:** Convert proven specs to a2ml format - -**Example:** -```a2ml -# SafeMath Specification - -@abstract: -SafeMath provides formally verified arithmetic operations with -overflow/underflow detection and division-by-zero prevention. -@end - -@requires: -- All operations must be total (no crashes) -- Overflow detection via dependent types -- Conformance with aggregate-library arithmetic specs -@end - -## Operations - -@spec(id="safe-add"): -**Function:** `add_checked : Int -> Int -> Result Int` - -**Semantics:** Addition with overflow detection - -**Proof Obligations:** -1. Totality: ∀ a b. add_checked a b returns (OK | Error) -2. Correctness: ∀ a b. no overflow → add_checked a b = OK (a + b) -3. Safety: ∀ a b. overflow → add_checked a b = Error Overflow -@end -``` - -**Benefits:** -- Structural verification of docs -- References must resolve -- Sections cannot be accidentally omitted -- Attested markup provides stronger guarantees - ---- - -## Integration Complete Checklist - -For Task #3 completion: - -- [ ] Update proven's ECOSYSTEM.scm with aLib + a2ml relationships -- [ ] Create `docs/ALIB-CONFORMANCE.adoc` -- [ ] Check if aLib has test vectors -- [ ] Document which proven operations map to which aLib specs -- [ ] Note a2ml integration as future enhancement (v1.1+) -- [ ] Update proven's README.adoc to mention aLib conformance - ---- - -## Immediate Actions (for v1.0) - -Since proven ALREADY implements aLib operations, the integration is mostly about documentation: - -1. **Document the relationship** (ECOSYSTEM.scm update) -2. **Create conformance matrix** (which operations match which specs) -3. **Verify compliance** (if aLib has test vectors, run them) - -This is NOT blocking v1.0 release - it's documentation work that clarifies existing compliance. - ---- - -## Future Enhancements (v1.1+) - -1. **a2ml-based specs** - Convert proven docs to a2ml when stable -2. **Automated conformance** - CI job that runs aLib test vectors -3. **Cross-language validation** - Use proven's 89 bindings to test aLib across ecosystems -4. **Contribute back to aLib** - Provide feedback on specs based on proven's formal verification insights - ---- - -## Questions to Resolve - -1. **Does aggregate-library have test vectors?** - - Check `aggregate-library/tests/` or `aggregate-library/test/` - - If yes: Run them against proven - - If no: Create them based on aLib specs - -2. **Which aLib specs does proven already satisfy?** - - Arithmetic: ✓ (add, sub, mul, div, mod) - - String: ✓ (concat, length, substring) - - Collection: ? (check if SafeBuffer/SafeQueue satisfy) - - Comparison: ? (check if proven has these) - - Conditional: N/A (language feature, not library) - - Logical: ? (check if proven has these) - -3. **Should proven docs migrate to a2ml format?** - - Current: AsciiDoc (.adoc) - - Future: a2ml (.a2ml) - - Timeline: After a2ml v1.0 (per a2ml ROADMAP.adoc) - ---- - -## Next Steps - -1. Update proven's ECOSYSTEM.scm ← DO THIS NOW -2. Create aLib conformance matrix ← DO THIS NOW -3. Check for aLib test vectors ← INVESTIGATE -4. Document in proven's README ← DO THIS NOW -5. Mark Task #3 complete -6. Move to Task #4 (testing documentation) - ---- - -_Plan created: 2026-01-30_ -_Next: Execute integration tasks_ diff --git a/ARCHITECTURE-CLEANUP-2026-01-25.adoc b/ARCHITECTURE-CLEANUP-2026-01-25.adoc new file mode 100644 index 00000000..0f616098 --- /dev/null +++ b/ARCHITECTURE-CLEANUP-2026-01-25.adoc @@ -0,0 +1,292 @@ +== Architecture Cleanup - 2026-01-25 + +=== Problem Statement + +proven v0.9.0 was rejected from opam-repository due to *fundamental +architecture violations*: + +* *130 security issues* (5 CRITICAL, 124 HIGH) in language bindings +* *Rust bindings* contained full reimplementations instead of FFI +wrappers +* *ReScript bindings* used `+getExn+` that crashes on malformed input +* Code claimed "`formally verified`" but bypassed Idris2 verification +entirely + +*Root Cause:* LLMs kept adding native implementations in binding +languages, defeating the purpose of formal verification. + +=== Actions Taken + +==== 1. Deleted All Rust Reimplementations ✅ + +[source,bash] +---- +# Backed up to .UNSAFE-RUST-DELETED-20260125/ +# Then deleted: +rm bindings/rust/src/safe_*.rs # 38 files +---- + +*Rationale:* These were NOT calling Idris2. They were pure Rust code +with unsafe patterns. + +==== 2. Fixed ReScript Critical Issues ✅ + +Fixed 5 CRITICAL `+getExn+` calls in `+Proven_SafeCron.res+`: + +[source,rescript] +---- +// BEFORE (crashes on invalid input): +let fields = [ + Belt.Array.getExn(parts, 0), // CRITICAL + Belt.Array.getExn(parts, 1), // CRITICAL + // ... +] + +// AFTER (safe pattern matching): +switch parts { +| [field1, field2, field3, field4, field5] => // Handle +| _ => Error(InvalidFieldCount) +} +---- + +==== 3. Updated All Documentation ✅ + +* `+README.adoc+` - Emphasizes Idris2 first, architecture diagram +* `+bindings/rust/README.md+` - Explains FFI-only architecture +* `+.claude/CLAUDE.md+` - AI instructions to prevent recurrence +* `+META.scm+` - Added ADR-008 (FFI-only bindings), ADR-009 +(pre-submission validation) + +==== 4. Created CI/CD Enforcement ✅ + +`+.github/workflows/architecture-enforcement.yml+`: - Blocks +`+safe_*.rs+` files in Rust bindings - Blocks `+unwrap()+`, `+getExn+`, +`+Obj.magic+` patterns - Requires Idris2 build to pass - Runs hypatia +security scan (must show 0 critical/high) + +==== 5. Configured echidnabot Auto-Cleanup ✅ + +`+.echidnabot.toml+`: - `+enforcement_level = "strict"+` - +`+auto_cleanup.enabled = true+` - Auto-deletes forbidden patterns in +bindings/ - Creates PR with cleanup changes + +=== Architecture Now Enforced + +.... +Application (89 languages) + ↓ + Language Binding (FFI wrapper ONLY) ← No logic allowed + ↓ + Zig FFI Bridge (C ABI) + ↓ + Idris2 RefC Compiled Code + ↓ + Idris2 Source (src/Proven/*.idr) ← ALL LOGIC + PROOFS HERE +.... + +==== What Belongs Where + +[width="100%",cols="29%,43%,28%",options="header",] +|=== +|Component |Allowed Content |Forbidden +|`+src/Proven/*.idr+` |Logic, algorithms, proofs |N/A (Idris2 only) + +|`+ffi/zig/+` |C ABI bridge, FFI wrappers |Logic, algorithms + +|`+bindings/*/+` |Type conversions, error handling |`+safe_*.rs+`, +logic, `+unwrap()+`, `+getExn+` +|=== + +=== Comparison: Before vs After + +==== Before (WRONG - What We Deleted) + +[source,rust] +---- +// bindings/rust/src/safe_math.rs - DELETED +pub fn safe_add(a: i32, b: i32) -> Result { + a.checked_add(b).ok_or(Error::Overflow) // NOT VERIFIED +} +---- + +*Issues:* - No formal proof - Bypasses Idris2 verification - Had bugs +(unwrap calls that could panic) + +==== After (CORRECT - What We Need To Build) + +[source,rust] +---- +// bindings/rust/src/math.rs - FFI wrapper only +use crate::ffi; + +pub fn safe_add(a: i32, b: i32) -> Result { + unsafe { + let result = ffi::proven_safe_math_add(a, b); + if result.is_error { + Err(Error::from_c(result.error_code)) + } else { + Ok(result.value) + } + } +} +---- + +*Benefits:* - Calls Idris2 code with *mathematical proof* - FFI wrapper +is simple (hard to get wrong) - Same guarantee for all 89 language +bindings + +=== Why Idris2? + +*Idris2 provides:* - *Dependent types* - Types can depend on values - +*Totality checking* - Compiler proves functions terminate - *No runtime +exceptions* - All errors explicit in types - *Mathematical proofs* - +Code correctness verified at compile time + +*Example: Proven Safe Division* + +[source,idris] +---- +-- Idris2 code with dependent type proof +data NonZero : Nat -> Type where + IsNonZero : {n : Nat} -> (n /= 0) -> NonZero n + +safeDiv : (a : Int) -> (b : Int) -> {auto prf : NonZero b} -> Int +safeDiv a b = a `div` b -- Cannot call with b=0 (compiler prevents it) +---- + +This is *impossible* in Rust, ReScript, or any other binding language +without dependent types. + +=== Lessons Learned + +==== What Went Wrong + +[arabic] +. *LLMs added native code* instead of FFI wrappers +. *No CI enforcement* to catch violations +. *Unclear documentation* about architecture +. *No auto-cleanup* when violations occurred + +==== What We Fixed + +[arabic] +. ✅ Deleted all reimplementations +. ✅ CI blocks unsafe patterns +. ✅ Documentation emphasizes Idris2 first +. ✅ echidnabot auto-cleans violations + +==== Prevention Going Forward + +* *echidnabot scans daily* - Removes non-Idris code automatically +* *CI blocks merges* - Cannot merge if unsafe patterns present +* *Documentation prominent* - README, META.scm, .claude/CLAUDE.md all +emphasize architecture +* *Pre-submission validation* - Run hypatia scan before publishing to +registries + +=== Package Registry Impact + +==== Current Status + +*DO NOT publish yet* - Repo is in transitional state: - ✅ Unsafe code +deleted - ❌ FFI wrappers not yet implemented - Status: *Broken* +(bindings don’t work) + +==== Re-Publication Plan + +[arabic] +. *Yank v0.9.x* from registries with deprecation notice +. *Build FFI layer* - Implement Zig bridge properly +. *Create FFI wrappers* - Thin wrappers in each language +. *Test thoroughly* - Verify bindings call Idris2 +. *Publish v1.0.0* - Breaking change, FFI-based architecture + +=== Files Changed + +==== Deleted + +* `+bindings/rust/src/safe_*.rs+` (38 files) - Backup in +`+.UNSAFE-RUST-DELETED-20260125/+` + +==== Created + +* `+.github/workflows/architecture-enforcement.yml+` +* `+.claude/CLAUDE.md+` +* `+bindings/rust/README.md+` (rewritten) +* `+ARCHITECTURE-CLEANUP-2026-01-25.md+` (this file) + +==== Modified + +* `+README.adoc+` - Idris2-first messaging +* `+META.scm+` - ADR-008, ADR-009 +* `+.echidnabot.toml+` - Auto-cleanup configuration +* `+bindings/rescript/src/Proven_SafeCron.res+` - Fixed 5 critical +`+getExn+` calls + +=== Statistics + +==== Security Issues Fixed + +* *Before:* 130 findings (5 CRITICAL, 124 HIGH, 1 MEDIUM) +* *After:* 0 critical in ReScript, Rust bindings deleted + +==== Code Removed + +* 38 Rust files (~15,000 lines) +* All contained `+unwrap()+` and other unsafe patterns + +==== Architecture Violations + +* *Before:* 34 modules reimplemented in Rust, 5 with no Idris backing +* *After:* 0 reimplementations allowed + +=== Next Steps + +==== Immediate (This Week) + +[arabic] +. ✅ Document cleanup (DONE - this file) +. ⏳ Build Zig FFI bridge +. ⏳ Create Rust FFI wrapper template +. ⏳ Test Rust binding calls Idris2 + +==== Short-Term (This Month) + +[arabic] +. Replicate Rust FFI pattern for other languages +. Add fuzzing for FFI boundary +. Complete FFI wrappers for top 10 languages + +==== Long-Term (This Quarter) + +[arabic] +. Publish v1.0.0 to all package registries +. Auto-generate FFI wrappers from Idris2 types +. Add more Idris2 modules with proofs + +=== Validation Checklist + +Before re-publishing to any registry: + +* [ ] Zig FFI bridge implemented +* [ ] Language bindings call FFI (no logic in bindings) +* [ ] hypatia scan shows 0 critical/high issues +* [ ] All Idris2 modules build with totality checking +* [ ] Tests pass for FFI boundary +* [ ] Documentation updated +* [ ] CI passes all architecture enforcement checks + +=== References + +* Architecture decisions: `+META.scm+` (ADR-008, ADR-009) +* AI instructions: `+.claude/CLAUDE.md+` +* Binding template: `+bindings/rust/README.md+` +* CI enforcement: `+.github/workflows/architecture-enforcement.yml+` + +''''' + +*Summary:* proven is now correctly architected as an *Idris2 formal +verification library with FFI bindings*, not a collection of native +implementations. This cleanup prevents the architecture violations that +led to the opam rejection and ensures all code claiming "`proven`" is +actually mathematically verified. diff --git a/ARCHITECTURE-CLEANUP-2026-01-25.md b/ARCHITECTURE-CLEANUP-2026-01-25.md deleted file mode 100644 index 74c64932..00000000 --- a/ARCHITECTURE-CLEANUP-2026-01-25.md +++ /dev/null @@ -1,266 +0,0 @@ -# Architecture Cleanup - 2026-01-25 - -## Problem Statement - -proven v0.9.0 was rejected from opam-repository due to **fundamental architecture violations**: - -- **130 security issues** (5 CRITICAL, 124 HIGH) in language bindings -- **Rust bindings** contained full reimplementations instead of FFI wrappers -- **ReScript bindings** used `getExn` that crashes on malformed input -- Code claimed "formally verified" but bypassed Idris2 verification entirely - -**Root Cause:** LLMs kept adding native implementations in binding languages, defeating the purpose of formal verification. - -## Actions Taken - -### 1. Deleted All Rust Reimplementations ✅ - -```bash -# Backed up to .UNSAFE-RUST-DELETED-20260125/ -# Then deleted: -rm bindings/rust/src/safe_*.rs # 38 files -``` - -**Rationale:** These were NOT calling Idris2. They were pure Rust code with unsafe patterns. - -### 2. Fixed ReScript Critical Issues ✅ - -Fixed 5 CRITICAL `getExn` calls in `Proven_SafeCron.res`: - -```rescript -// BEFORE (crashes on invalid input): -let fields = [ - Belt.Array.getExn(parts, 0), // CRITICAL - Belt.Array.getExn(parts, 1), // CRITICAL - // ... -] - -// AFTER (safe pattern matching): -switch parts { -| [field1, field2, field3, field4, field5] => // Handle -| _ => Error(InvalidFieldCount) -} -``` - -### 3. Updated All Documentation ✅ - -- `README.adoc` - Emphasizes Idris2 first, architecture diagram -- `bindings/rust/README.md` - Explains FFI-only architecture -- `.claude/CLAUDE.md` - AI instructions to prevent recurrence -- `META.scm` - Added ADR-008 (FFI-only bindings), ADR-009 (pre-submission validation) - -### 4. Created CI/CD Enforcement ✅ - -`.github/workflows/architecture-enforcement.yml`: -- Blocks `safe_*.rs` files in Rust bindings -- Blocks `unwrap()`, `getExn`, `Obj.magic` patterns -- Requires Idris2 build to pass -- Runs hypatia security scan (must show 0 critical/high) - -### 5. Configured echidnabot Auto-Cleanup ✅ - -`.echidnabot.toml`: -- `enforcement_level = "strict"` -- `auto_cleanup.enabled = true` -- Auto-deletes forbidden patterns in bindings/ -- Creates PR with cleanup changes - -## Architecture Now Enforced - -``` -Application (89 languages) - ↓ - Language Binding (FFI wrapper ONLY) ← No logic allowed - ↓ - Zig FFI Bridge (C ABI) - ↓ - Idris2 RefC Compiled Code - ↓ - Idris2 Source (src/Proven/*.idr) ← ALL LOGIC + PROOFS HERE -``` - -### What Belongs Where - -| Component | Allowed Content | Forbidden | -|-----------|-----------------|-----------| -| `src/Proven/*.idr` | Logic, algorithms, proofs | N/A (Idris2 only) | -| `ffi/zig/` | C ABI bridge, FFI wrappers | Logic, algorithms | -| `bindings/*/` | Type conversions, error handling | `safe_*.rs`, logic, `unwrap()`, `getExn` | - -## Comparison: Before vs After - -### Before (WRONG - What We Deleted) - -```rust -// bindings/rust/src/safe_math.rs - DELETED -pub fn safe_add(a: i32, b: i32) -> Result { - a.checked_add(b).ok_or(Error::Overflow) // NOT VERIFIED -} -``` - -**Issues:** -- No formal proof -- Bypasses Idris2 verification -- Had bugs (unwrap calls that could panic) - -### After (CORRECT - What We Need To Build) - -```rust -// bindings/rust/src/math.rs - FFI wrapper only -use crate::ffi; - -pub fn safe_add(a: i32, b: i32) -> Result { - unsafe { - let result = ffi::proven_safe_math_add(a, b); - if result.is_error { - Err(Error::from_c(result.error_code)) - } else { - Ok(result.value) - } - } -} -``` - -**Benefits:** -- Calls Idris2 code with **mathematical proof** -- FFI wrapper is simple (hard to get wrong) -- Same guarantee for all 89 language bindings - -## Why Idris2? - -**Idris2 provides:** -- **Dependent types** - Types can depend on values -- **Totality checking** - Compiler proves functions terminate -- **No runtime exceptions** - All errors explicit in types -- **Mathematical proofs** - Code correctness verified at compile time - -**Example: Proven Safe Division** - -```idris --- Idris2 code with dependent type proof -data NonZero : Nat -> Type where - IsNonZero : {n : Nat} -> (n /= 0) -> NonZero n - -safeDiv : (a : Int) -> (b : Int) -> {auto prf : NonZero b} -> Int -safeDiv a b = a `div` b -- Cannot call with b=0 (compiler prevents it) -``` - -This is **impossible** in Rust, ReScript, or any other binding language without dependent types. - -## Lessons Learned - -### What Went Wrong - -1. **LLMs added native code** instead of FFI wrappers -2. **No CI enforcement** to catch violations -3. **Unclear documentation** about architecture -4. **No auto-cleanup** when violations occurred - -### What We Fixed - -1. ✅ Deleted all reimplementations -2. ✅ CI blocks unsafe patterns -3. ✅ Documentation emphasizes Idris2 first -4. ✅ echidnabot auto-cleans violations - -### Prevention Going Forward - -- **echidnabot scans daily** - Removes non-Idris code automatically -- **CI blocks merges** - Cannot merge if unsafe patterns present -- **Documentation prominent** - README, META.scm, .claude/CLAUDE.md all emphasize architecture -- **Pre-submission validation** - Run hypatia scan before publishing to registries - -## Package Registry Impact - -### Current Status - -**DO NOT publish yet** - Repo is in transitional state: -- ✅ Unsafe code deleted -- ❌ FFI wrappers not yet implemented -- Status: **Broken** (bindings don't work) - -### Re-Publication Plan - -1. **Yank v0.9.x** from registries with deprecation notice -2. **Build FFI layer** - Implement Zig bridge properly -3. **Create FFI wrappers** - Thin wrappers in each language -4. **Test thoroughly** - Verify bindings call Idris2 -5. **Publish v1.0.0** - Breaking change, FFI-based architecture - -## Files Changed - -### Deleted -- `bindings/rust/src/safe_*.rs` (38 files) - Backup in `.UNSAFE-RUST-DELETED-20260125/` - -### Created -- `.github/workflows/architecture-enforcement.yml` -- `.claude/CLAUDE.md` -- `bindings/rust/README.md` (rewritten) -- `ARCHITECTURE-CLEANUP-2026-01-25.md` (this file) - -### Modified -- `README.adoc` - Idris2-first messaging -- `META.scm` - ADR-008, ADR-009 -- `.echidnabot.toml` - Auto-cleanup configuration -- `bindings/rescript/src/Proven_SafeCron.res` - Fixed 5 critical `getExn` calls - -## Statistics - -### Security Issues Fixed - -- **Before:** 130 findings (5 CRITICAL, 124 HIGH, 1 MEDIUM) -- **After:** 0 critical in ReScript, Rust bindings deleted - -### Code Removed - -- 38 Rust files (~15,000 lines) -- All contained `unwrap()` and other unsafe patterns - -### Architecture Violations - -- **Before:** 34 modules reimplemented in Rust, 5 with no Idris backing -- **After:** 0 reimplementations allowed - -## Next Steps - -### Immediate (This Week) - -1. ✅ Document cleanup (DONE - this file) -2. ⏳ Build Zig FFI bridge -3. ⏳ Create Rust FFI wrapper template -4. ⏳ Test Rust binding calls Idris2 - -### Short-Term (This Month) - -1. Replicate Rust FFI pattern for other languages -2. Add fuzzing for FFI boundary -3. Complete FFI wrappers for top 10 languages - -### Long-Term (This Quarter) - -1. Publish v1.0.0 to all package registries -2. Auto-generate FFI wrappers from Idris2 types -3. Add more Idris2 modules with proofs - -## Validation Checklist - -Before re-publishing to any registry: - -- [ ] Zig FFI bridge implemented -- [ ] Language bindings call FFI (no logic in bindings) -- [ ] hypatia scan shows 0 critical/high issues -- [ ] All Idris2 modules build with totality checking -- [ ] Tests pass for FFI boundary -- [ ] Documentation updated -- [ ] CI passes all architecture enforcement checks - -## References - -- Architecture decisions: `META.scm` (ADR-008, ADR-009) -- AI instructions: `.claude/CLAUDE.md` -- Binding template: `bindings/rust/README.md` -- CI enforcement: `.github/workflows/architecture-enforcement.yml` - ---- - -**Summary:** proven is now correctly architected as an **Idris2 formal verification library with FFI bindings**, not a collection of native implementations. This cleanup prevents the architecture violations that led to the opam rejection and ensures all code claiming "proven" is actually mathematically verified. diff --git a/ARCHITECTURE-ROOT-CAUSE-ANALYSIS-2026-01-30.adoc b/ARCHITECTURE-ROOT-CAUSE-ANALYSIS-2026-01-30.adoc new file mode 100644 index 00000000..db51682a --- /dev/null +++ b/ARCHITECTURE-ROOT-CAUSE-ANALYSIS-2026-01-30.adoc @@ -0,0 +1,478 @@ +== proven Architecture - Root Cause Analysis + +*Date:* 2026-01-30 *Critical Discovery:* The architecture is +DISCONNECTED + +''''' + +=== 🚨 CRITICAL FINDING: Idris2 and Zig Are NOT Connected + +==== The Devastating Discovery + +*What we found:* + +[source,bash] +---- +# Idris2 FFI exports to C +$ grep -r "^export" src/Proven/*.idr +0 results + +# Idris2 foreign function declarations +$ grep -r "%foreign" src/Proven/*.idr +0 results +---- + +*What this means:* - ✅ Idris2 code exists with proofs - ❌ Idris2 code +is NOT exported to C/Zig - ❌ Zig does NOT call Idris2 - ⚠️ Zig +reimplemented everything natively (WITHOUT PROOFS!) + +*The architecture is BROKEN.* + +''''' + +=== Current Reality vs. Intended Architecture + +==== What We THOUGHT Was Happening: + +.... +User Code + ↓ +Language Binding (Python, Rust, etc.) + ↓ +Zig FFI (C ABI bridge) + ↓ +Idris2 Compiled Code ← PROOFS HERE ✓ +.... + +==== What’s ACTUALLY Happening: + +.... +User Code + ↓ +Language Binding (Python, Rust, etc.) + ↓ +Zig FFI (Native implementations - NO PROOFS!) ← PROBLEM! + +[Idris2 Code with Proofs] ← UNUSED, DISCONNECTED! +.... + +*The Idris2 proofs exist but are completely disconnected from the FFI!* + +''''' + +=== Evidence + +==== 1. Idris2 SafeMath.idr + +*Has proofs:* + +[source,idris] +---- +-- File: src/Proven/SafeMath.idr +module Proven.SafeMath + +%default total -- ✓ Totality checking enabled + +public export -- ✓ Exported for Idris2 use +div : Integer -> Integer -> Maybe Integer +div _ 0 = Nothing +div n d = Just (n `div` d) + +-- NO export for C FFI +-- NO %foreign declaration +---- + +*Functions are proven total but NOT exported to C!* + +==== 2. Zig FFI main.zig + +*Native reimplementation:* + +[source,zig] +---- +// File: ffi/zig/src/main.zig +export fn proven_math_div(numerator: i64, denominator: i64) IntResult { + if (denominator == 0) { + return .{ .status = .err_division_by_zero, .value = 0 }; + } + return .{ .status = .ok, .value = @divTrunc(numerator, denominator) }; +} + +// This is ZIG CODE, not calling Idris2! +// No proofs! +---- + +==== 3. Python Binding + +*Calls Zig (not Idris2):* + +[source,python] +---- +# File: bindings/python/proven/safe_math.py +lib = get_lib() +result = lib.proven_math_div(numerator, denominator) +# ↑ Calls Zig native code, NOT Idris2! +---- + +*The chain is broken!* + +''''' + +=== Root Cause: Missing FFI Export Layer + +==== What’s Missing + +*Idris2 modules need FFI export declarations:* + +[source,idris] +---- +-- What SafeMath.idr SHOULD have: +module Proven.SafeMath + +-- Export to C ABI +export +proven_safe_div : Int -> Int -> Int +proven_safe_div n d = + case div (cast n) (cast d) of + Just result => cast result + Nothing => 0 -- Error sentinel +---- + +*Then Zig wraps the Idris2 C export:* + +[source,zig] +---- +// Zig calls Idris2-generated C function +extern fn proven_safe_div(i64, i64) i64; + +export fn proven_math_div(numerator: i64, denominator: i64) IntResult { + const result = proven_safe_div(numerator, denominator); + // Handle error sentinel, wrap in Result +} +---- + +*Status:* ❌ *NOT IMPLEMENTED* + +''''' + +=== Why This Happened + +==== Theory: Development Evolution + +*Phase 1:* Created Idris2 proofs - Wrote proven modules in Idris2 - +Proved totality, safety properties - ✓ This part is complete + +*Phase 2:* Wanted to use from other languages - Needed FFI layer - +Idris2 FFI is complex (RefC backend, C codegen) - *Shortcut taken:* Zig +reimplemented functions natively + +*Phase 3:* Bindings call Zig - Python, Rust, etc. call Zig - ✓ This +works - ❌ But Zig doesn’t call Idris2! + +*Result:* The proven guarantees don’t reach the bindings. + +''''' + +=== The Honest Truth About Current Status + +==== What IS Formally Verified + +*Idris2 modules (79 files):* - ✓ Proven total (cannot crash/hang) - ✓ +Type-level invariants enforced - ✓ Dependent type proofs - ❌ *BUT: Not +used by bindings!* + +==== What Users Actually Get + +*When calling from Python/Rust/etc:* - They call Zig native code - 14 +functions call Idris2 (path, json, url, network) - 141 functions are +pure Zig (safe but unproven) + +*Verification status:* - 9% formally proven (the 14 that call Idris2) - +91% Zig-safe (builtin overflow detection, but NO formal proofs) + +''''' + +=== Architecture Options Analysis + +==== Option 1: Idris2 ABI + Zig FFI (INTENDED - FIX IT) + +*How it should work:* + +.... +Idris2 (proven) + → compiles to C via RefC backend + → Zig imports C functions + → Zig exports C ABI to languages + → Language bindings call Zig + → Zig calls Idris2-generated C +.... + +*Pros:* - ✓ Formal proofs reach the user - ✓ Single source of truth +(Idris2) - ✓ Zig is thin wrapper (no logic) + +*Cons:* - ⚠ FFI overhead (crossing boundary) - ⚠ Complex build (Idris2 + +Zig + bindings) - ⚠ Requires careful type marshalling + +*Recommendation:* ✅ *FIX THIS - IT’S THE RIGHT ARCHITECTURE* + +*What needs to happen:* 1. Add `+export+` declarations to all Idris2 +modules 2. Update Zig to call Idris2-generated C functions 3. Remove Zig +native implementations (keep only wrappers) + +*Estimated effort:* 2-4 weeks full-time + +''''' + +==== Option 2: Idris2 for Both ABI and FFI + +*How it would work:* + +.... +Idris2 (proven) + → directly generates language bindings + → No Zig layer +.... + +*Pros:* - ✓ No FFI overhead - ✓ Simpler build (no Zig) + +*Cons:* - ❌ Idris2 doesn’t support this - ❌ Would need custom codegen +for 89 languages - ❌ Massive engineering effort + +*Recommendation:* ❌ *NOT FEASIBLE* + +''''' + +==== Option 3: Zig for Both ABI and FFI + +*How it would work:* + +.... +Zig (reimplemented logic) + → exports to all languages + → No Idris2 in production +.... + +*Pros:* - ✓ Simpler (single language) - ✓ Better performance (no FFI +crossing) - ✓ Easier debugging + +*Cons:* - ❌ Loses all formal verification - ❌ Zig has no dependent +types - ❌ No totality checking - ❌ *Defeats the entire purpose of +proven!* + +*Recommendation:* ❌ *ABSOLUTELY NOT* + +''''' + +==== Option 4: Hybrid (Current Accidental State) + +*What’s happening now:* + +.... +14 functions: Zig → Idris2 (proven) ✓ +141 functions: Pure Zig (safe but unproven) ⚠ +.... + +*Pros:* - ✓ Works for users (functionally) - ✓ Zig overflow detection is +safe-ish - ✓ Critical operations (path, json) ARE proven + +*Cons:* - ❌ Misleading (claims proven, mostly isn’t) - ❌ Inconsistent +(some proven, some not) - ❌ Maintenance burden (two implementations) + +*Recommendation:* ⚠ *ACCEPTABLE FOR v1.0 WITH HONEST DISCLOSURE* +*Long-term:* ❌ *MUST FIX FOR v1.1* + +''''' + +=== Recommended Path Forward + +==== Immediate (v1.0 - Honest Disclosure) + +*Accept current state with transparency:* + +[arabic] +. *Update README.adoc:* + +[source,adoc] +---- +== Verification Status + +IMPORTANT: proven uses a hybrid architecture: + +* **Fully Verified (9%):** Path traversal, JSON parsing, URL parsing, + Network validation - These call Idris2 with formal proofs + +* **Zig-Safe (91%):** Arithmetic, checksums, string operations, etc. - + These use Zig builtin safety (overflow detection) but lack formal proofs + +See link:FFI-ARCHITECTURE-AUDIT-2026-01-30.md[Architecture Audit] for details. +---- + +[arabic, start=2] +. *Update docs/VERIFICATION-STATUS.md:* + +[source,markdown] +---- +| Module | Implementation | Status | +|--------|----------------|--------| +| SafePath | Idris2 | ✓ Formally Proven | +| SafeJson | Idris2 | ✓ Formally Proven | +| SafeUrl | Idris2 | ✓ Formally Proven | +| SafeNetwork | Idris2 | ✓ Formally Proven | +| SafeMath | Zig | ⚠ Safe (overflow detection) | +| SafeGeo | Zig | ⚠ Safe (bounds checks) | +| ... | ... | ... | +---- + +[arabic, start=3] +. *Release v1.0* with honest claims + +==== Short-term (v1.1 - Connect Architecture) + +*Fix the disconnection:* + +*Week 1-2: Add Idris2 FFI exports* + +[source,idris] +---- +-- For every module in src/Proven/SafeMath.idr +module Proven.SafeMath + +-- Existing proven functions +public export +div : Integer -> Integer -> Maybe Integer +div _ 0 = Nothing +div n d = Just (n `div` d) + +-- NEW: FFI export wrapper +export +ffi_safe_div : Ptr Word64 -> Ptr Word64 -> Ptr Word64 -> PrimIO () +ffi_safe_div ptrNum ptrDen ptrResult = toPrim $ do + num <- readWord64 ptrNum + den <- readWord64 ptrDen + case div (cast num) (cast den) of + Just res => writeWord64 ptrResult (cast res) + Nothing => writeWord64 ptrResult 0 -- Sentinel +---- + +*Week 3: Update Zig to call Idris2* + +[source,zig] +---- +// Import Idris2-generated C function +extern fn proven_Proven_SafeMath_ffi_safe_div( + num: *u64, + den: *u64, + result: *u64 +) callconv(.C) void; + +// Zig wrapper (no logic!) +export fn proven_math_div(numerator: i64, denominator: i64) IntResult { + var num: u64 = @bitCast(numerator); + var den: u64 = @bitCast(denominator); + var result: u64 = 0; + + proven_Proven_SafeMath_ffi_safe_div(&num, &den, &result); + + if (result == 0 and denominator != 0) { + return .{ .status = .err_division_by_zero, .value = 0 }; + } + return .{ .status = .ok, .value = @bitCast(result) }; +} +---- + +*Week 4: Test and validate* - All tests still pass - Benchmarks +(document FFI overhead) - Update documentation + +*Estimated effort:* 4 weeks, 1 person + +''''' + +==== Medium-term (v1.2 - Complete Bindings) + +*Week 5-8: Complete all bindings* - Python: 37 → 79 modules (add 42) - +Rust: TBD → 79 modules - Deno: TBD → 79 modules - Use binding generator +to automate + +''''' + +==== Long-term (v2.0 - Bidirectional FFI) + +*Months 3-12: Callback support* - Function pointers in Zig - Idris2 can +call back to language - Event handlers, plugins, async + +''''' + +=== Decision Matrix + +[width="100%",cols="20%,15%,25%,15%,25%",options="header",] +|=== +|Approach |Proofs |Performance |Effort |Recommended +|*Fix Option 1* (Idris2 ABI + Zig FFI) |✓ Full |⚠ FFI overhead |Medium +|✅ *YES* + +|Option 2 (Idris2 only) |✓ Full |✓ Best |Very High |❌ No + +|Option 3 (Zig only) |❌ None |✓ Best |Low |❌ *Never* + +|Option 4 (Keep hybrid) |⚠ Partial |✓ Good |None |⚠ v1.0 only +|=== + +''''' + +=== Honest Answer to "`Should we use Idris for both, Zig for both, or split?`" + +==== ✅ *ANSWER: ABIs in Idris, FFI in Zig (Option 1) - FIX IT* + +*Why:* 1. *Idris2 MUST be the ABI* - That’s where the proofs are 2. *Zig +SHOULD be FFI* - C ABI bridge to all languages 3. *Zig MUST NOT have +logic* - Pure passthrough only + +*The problem is NOT the architecture choice.* *The problem is the +IMPLEMENTATION - Idris2 isn’t exporting to Zig!* + +*Fix:* Connect Idris2 → Zig properly, remove Zig native code. + +''''' + +=== Verification + +==== To confirm this analysis: + +[source,bash] +---- +# Check if Idris2 generates C files when built +cd ~/Documents/hyperpolymath-repos/proven +idris2 --build proven.ipkg +find build -name "*.c" -o -name "*.h" + +# Expected: C files generated from Idris2 +# These should be callable from Zig +---- + +If NO C files exist → Idris2 isn’t compiling to C at all! If C files +exist → Need to import them in Zig. + +''''' + +=== Summary + +*Root Cause:* Idris2 modules are NOT exported to C/Zig FFI + +*Impact:* - Bindings call Zig native code (not proven) - Only 14/155 +functions actually use Idris2 - Users don’t get the formal verification +they expect + +*Solution:* - Fix Option 1 architecture (Idris2 ABI + Zig FFI) - Add +`+export+` declarations to all Idris2 modules - Update Zig to import and +call Idris2-generated C - Remove Zig native implementations + +*Timeline:* - v1.0: Release with honest disclosure (hybrid state) - +v1.1: Fix architecture (4 weeks) - v1.2: Complete bindings (4 weeks) - +v2.0: Bidirectional FFI (months) + +*Recommendation:* Delay v1.0 until v1.1 is complete (8 weeks total) + +''''' + +_Analysis completed: 2026-01-30_ _Critical finding: Architecture is +correct but disconnected_ _Estimated fix time: 4-8 weeks_ diff --git a/ARCHITECTURE-ROOT-CAUSE-ANALYSIS-2026-01-30.md b/ARCHITECTURE-ROOT-CAUSE-ANALYSIS-2026-01-30.md deleted file mode 100644 index ae1a88dd..00000000 --- a/ARCHITECTURE-ROOT-CAUSE-ANALYSIS-2026-01-30.md +++ /dev/null @@ -1,486 +0,0 @@ -# proven Architecture - Root Cause Analysis -**Date:** 2026-01-30 -**Critical Discovery:** The architecture is DISCONNECTED - ---- - -## 🚨 CRITICAL FINDING: Idris2 and Zig Are NOT Connected - -### The Devastating Discovery - -**What we found:** -```bash -# Idris2 FFI exports to C -$ grep -r "^export" src/Proven/*.idr -0 results - -# Idris2 foreign function declarations -$ grep -r "%foreign" src/Proven/*.idr -0 results -``` - -**What this means:** -- ✅ Idris2 code exists with proofs -- ❌ Idris2 code is NOT exported to C/Zig -- ❌ Zig does NOT call Idris2 -- ⚠️ Zig reimplemented everything natively (WITHOUT PROOFS!) - -**The architecture is BROKEN.** - ---- - -## Current Reality vs. Intended Architecture - -### What We THOUGHT Was Happening: - -``` -User Code - ↓ -Language Binding (Python, Rust, etc.) - ↓ -Zig FFI (C ABI bridge) - ↓ -Idris2 Compiled Code ← PROOFS HERE ✓ -``` - -### What's ACTUALLY Happening: - -``` -User Code - ↓ -Language Binding (Python, Rust, etc.) - ↓ -Zig FFI (Native implementations - NO PROOFS!) ← PROBLEM! - -[Idris2 Code with Proofs] ← UNUSED, DISCONNECTED! -``` - -**The Idris2 proofs exist but are completely disconnected from the FFI!** - ---- - -## Evidence - -### 1. Idris2 SafeMath.idr - -**Has proofs:** -```idris --- File: src/Proven/SafeMath.idr -module Proven.SafeMath - -%default total -- ✓ Totality checking enabled - -public export -- ✓ Exported for Idris2 use -div : Integer -> Integer -> Maybe Integer -div _ 0 = Nothing -div n d = Just (n `div` d) - --- NO export for C FFI --- NO %foreign declaration -``` - -**Functions are proven total but NOT exported to C!** - -### 2. Zig FFI main.zig - -**Native reimplementation:** -```zig -// File: ffi/zig/src/main.zig -export fn proven_math_div(numerator: i64, denominator: i64) IntResult { - if (denominator == 0) { - return .{ .status = .err_division_by_zero, .value = 0 }; - } - return .{ .status = .ok, .value = @divTrunc(numerator, denominator) }; -} - -// This is ZIG CODE, not calling Idris2! -// No proofs! -``` - -### 3. Python Binding - -**Calls Zig (not Idris2):** -```python -# File: bindings/python/proven/safe_math.py -lib = get_lib() -result = lib.proven_math_div(numerator, denominator) -# ↑ Calls Zig native code, NOT Idris2! -``` - -**The chain is broken!** - ---- - -## Root Cause: Missing FFI Export Layer - -### What's Missing - -**Idris2 modules need FFI export declarations:** - -```idris --- What SafeMath.idr SHOULD have: -module Proven.SafeMath - --- Export to C ABI -export -proven_safe_div : Int -> Int -> Int -proven_safe_div n d = - case div (cast n) (cast d) of - Just result => cast result - Nothing => 0 -- Error sentinel -``` - -**Then Zig wraps the Idris2 C export:** - -```zig -// Zig calls Idris2-generated C function -extern fn proven_safe_div(i64, i64) i64; - -export fn proven_math_div(numerator: i64, denominator: i64) IntResult { - const result = proven_safe_div(numerator, denominator); - // Handle error sentinel, wrap in Result -} -``` - -**Status:** ❌ **NOT IMPLEMENTED** - ---- - -## Why This Happened - -### Theory: Development Evolution - -**Phase 1:** Created Idris2 proofs -- Wrote proven modules in Idris2 -- Proved totality, safety properties -- ✓ This part is complete - -**Phase 2:** Wanted to use from other languages -- Needed FFI layer -- Idris2 FFI is complex (RefC backend, C codegen) -- **Shortcut taken:** Zig reimplemented functions natively - -**Phase 3:** Bindings call Zig -- Python, Rust, etc. call Zig -- ✓ This works -- ❌ But Zig doesn't call Idris2! - -**Result:** The proven guarantees don't reach the bindings. - ---- - -## The Honest Truth About Current Status - -### What IS Formally Verified - -**Idris2 modules (79 files):** -- ✓ Proven total (cannot crash/hang) -- ✓ Type-level invariants enforced -- ✓ Dependent type proofs -- ❌ **BUT: Not used by bindings!** - -### What Users Actually Get - -**When calling from Python/Rust/etc:** -- They call Zig native code -- 14 functions call Idris2 (path, json, url, network) -- 141 functions are pure Zig (safe but unproven) - -**Verification status:** -- 9% formally proven (the 14 that call Idris2) -- 91% Zig-safe (builtin overflow detection, but NO formal proofs) - ---- - -## Architecture Options Analysis - -### Option 1: Idris2 ABI + Zig FFI (INTENDED - FIX IT) - -**How it should work:** -``` -Idris2 (proven) - → compiles to C via RefC backend - → Zig imports C functions - → Zig exports C ABI to languages - → Language bindings call Zig - → Zig calls Idris2-generated C -``` - -**Pros:** -- ✓ Formal proofs reach the user -- ✓ Single source of truth (Idris2) -- ✓ Zig is thin wrapper (no logic) - -**Cons:** -- ⚠ FFI overhead (crossing boundary) -- ⚠ Complex build (Idris2 + Zig + bindings) -- ⚠ Requires careful type marshalling - -**Recommendation:** ✅ **FIX THIS - IT'S THE RIGHT ARCHITECTURE** - -**What needs to happen:** -1. Add `export` declarations to all Idris2 modules -2. Update Zig to call Idris2-generated C functions -3. Remove Zig native implementations (keep only wrappers) - -**Estimated effort:** 2-4 weeks full-time - ---- - -### Option 2: Idris2 for Both ABI and FFI - -**How it would work:** -``` -Idris2 (proven) - → directly generates language bindings - → No Zig layer -``` - -**Pros:** -- ✓ No FFI overhead -- ✓ Simpler build (no Zig) - -**Cons:** -- ❌ Idris2 doesn't support this -- ❌ Would need custom codegen for 89 languages -- ❌ Massive engineering effort - -**Recommendation:** ❌ **NOT FEASIBLE** - ---- - -### Option 3: Zig for Both ABI and FFI - -**How it would work:** -``` -Zig (reimplemented logic) - → exports to all languages - → No Idris2 in production -``` - -**Pros:** -- ✓ Simpler (single language) -- ✓ Better performance (no FFI crossing) -- ✓ Easier debugging - -**Cons:** -- ❌ Loses all formal verification -- ❌ Zig has no dependent types -- ❌ No totality checking -- ❌ **Defeats the entire purpose of proven!** - -**Recommendation:** ❌ **ABSOLUTELY NOT** - ---- - -### Option 4: Hybrid (Current Accidental State) - -**What's happening now:** -``` -14 functions: Zig → Idris2 (proven) ✓ -141 functions: Pure Zig (safe but unproven) ⚠ -``` - -**Pros:** -- ✓ Works for users (functionally) -- ✓ Zig overflow detection is safe-ish -- ✓ Critical operations (path, json) ARE proven - -**Cons:** -- ❌ Misleading (claims proven, mostly isn't) -- ❌ Inconsistent (some proven, some not) -- ❌ Maintenance burden (two implementations) - -**Recommendation:** ⚠ **ACCEPTABLE FOR v1.0 WITH HONEST DISCLOSURE** -**Long-term:** ❌ **MUST FIX FOR v1.1** - ---- - -## Recommended Path Forward - -### Immediate (v1.0 - Honest Disclosure) - -**Accept current state with transparency:** - -1. **Update README.adoc:** -```adoc -== Verification Status - -IMPORTANT: proven uses a hybrid architecture: - -* **Fully Verified (9%):** Path traversal, JSON parsing, URL parsing, - Network validation - These call Idris2 with formal proofs - -* **Zig-Safe (91%):** Arithmetic, checksums, string operations, etc. - - These use Zig builtin safety (overflow detection) but lack formal proofs - -See link:FFI-ARCHITECTURE-AUDIT-2026-01-30.md[Architecture Audit] for details. -``` - -2. **Update docs/VERIFICATION-STATUS.md:** -```markdown -| Module | Implementation | Status | -|--------|----------------|--------| -| SafePath | Idris2 | ✓ Formally Proven | -| SafeJson | Idris2 | ✓ Formally Proven | -| SafeUrl | Idris2 | ✓ Formally Proven | -| SafeNetwork | Idris2 | ✓ Formally Proven | -| SafeMath | Zig | ⚠ Safe (overflow detection) | -| SafeGeo | Zig | ⚠ Safe (bounds checks) | -| ... | ... | ... | -``` - -3. **Release v1.0** with honest claims - -### Short-term (v1.1 - Connect Architecture) - -**Fix the disconnection:** - -**Week 1-2: Add Idris2 FFI exports** -```idris --- For every module in src/Proven/SafeMath.idr -module Proven.SafeMath - --- Existing proven functions -public export -div : Integer -> Integer -> Maybe Integer -div _ 0 = Nothing -div n d = Just (n `div` d) - --- NEW: FFI export wrapper -export -ffi_safe_div : Ptr Word64 -> Ptr Word64 -> Ptr Word64 -> PrimIO () -ffi_safe_div ptrNum ptrDen ptrResult = toPrim $ do - num <- readWord64 ptrNum - den <- readWord64 ptrDen - case div (cast num) (cast den) of - Just res => writeWord64 ptrResult (cast res) - Nothing => writeWord64 ptrResult 0 -- Sentinel -``` - -**Week 3: Update Zig to call Idris2** -```zig -// Import Idris2-generated C function -extern fn proven_Proven_SafeMath_ffi_safe_div( - num: *u64, - den: *u64, - result: *u64 -) callconv(.C) void; - -// Zig wrapper (no logic!) -export fn proven_math_div(numerator: i64, denominator: i64) IntResult { - var num: u64 = @bitCast(numerator); - var den: u64 = @bitCast(denominator); - var result: u64 = 0; - - proven_Proven_SafeMath_ffi_safe_div(&num, &den, &result); - - if (result == 0 and denominator != 0) { - return .{ .status = .err_division_by_zero, .value = 0 }; - } - return .{ .status = .ok, .value = @bitCast(result) }; -} -``` - -**Week 4: Test and validate** -- All tests still pass -- Benchmarks (document FFI overhead) -- Update documentation - -**Estimated effort:** 4 weeks, 1 person - ---- - -### Medium-term (v1.2 - Complete Bindings) - -**Week 5-8: Complete all bindings** -- Python: 37 → 79 modules (add 42) -- Rust: TBD → 79 modules -- Deno: TBD → 79 modules -- Use binding generator to automate - ---- - -### Long-term (v2.0 - Bidirectional FFI) - -**Months 3-12: Callback support** -- Function pointers in Zig -- Idris2 can call back to language -- Event handlers, plugins, async - ---- - -## Decision Matrix - -| Approach | Proofs | Performance | Effort | Recommended | -|----------|--------|-------------|--------|-------------| -| **Fix Option 1** (Idris2 ABI + Zig FFI) | ✓ Full | ⚠ FFI overhead | Medium | ✅ **YES** | -| Option 2 (Idris2 only) | ✓ Full | ✓ Best | Very High | ❌ No | -| Option 3 (Zig only) | ❌ None | ✓ Best | Low | ❌ **Never** | -| Option 4 (Keep hybrid) | ⚠ Partial | ✓ Good | None | ⚠ v1.0 only | - ---- - -## Honest Answer to "Should we use Idris for both, Zig for both, or split?" - -### ✅ **ANSWER: ABIs in Idris, FFI in Zig (Option 1) - FIX IT** - -**Why:** -1. **Idris2 MUST be the ABI** - That's where the proofs are -2. **Zig SHOULD be FFI** - C ABI bridge to all languages -3. **Zig MUST NOT have logic** - Pure passthrough only - -**The problem is NOT the architecture choice.** -**The problem is the IMPLEMENTATION - Idris2 isn't exporting to Zig!** - -**Fix:** Connect Idris2 → Zig properly, remove Zig native code. - ---- - -## Verification - -### To confirm this analysis: - -```bash -# Check if Idris2 generates C files when built -cd ~/Documents/hyperpolymath-repos/proven -idris2 --build proven.ipkg -find build -name "*.c" -o -name "*.h" - -# Expected: C files generated from Idris2 -# These should be callable from Zig -``` - -If NO C files exist → Idris2 isn't compiling to C at all! -If C files exist → Need to import them in Zig. - ---- - -## Summary - -**Root Cause:** Idris2 modules are NOT exported to C/Zig FFI - -**Impact:** -- Bindings call Zig native code (not proven) -- Only 14/155 functions actually use Idris2 -- Users don't get the formal verification they expect - -**Solution:** -- Fix Option 1 architecture (Idris2 ABI + Zig FFI) -- Add `export` declarations to all Idris2 modules -- Update Zig to import and call Idris2-generated C -- Remove Zig native implementations - -**Timeline:** -- v1.0: Release with honest disclosure (hybrid state) -- v1.1: Fix architecture (4 weeks) -- v1.2: Complete bindings (4 weeks) -- v2.0: Bidirectional FFI (months) - -**Recommendation:** Delay v1.0 until v1.1 is complete (8 weeks total) - ---- - -_Analysis completed: 2026-01-30_ -_Critical finding: Architecture is correct but disconnected_ -_Estimated fix time: 4-8 weeks_ diff --git a/ARCHITECTURE.adoc b/ARCHITECTURE.adoc new file mode 100644 index 00000000..1c0a7a69 --- /dev/null +++ b/ARCHITECTURE.adoc @@ -0,0 +1,48 @@ +== Architecture + +=== Overview + +This repository follows a modular, maintainable architecture designed +for clarity, scalability, and long-term sustainability. + +=== Directory Structure + +.... +. +├── src/ # Source code +├── tests/ # Test suites +├── docs/ # Documentation +├── scripts/ # Utility scripts +├── config/ # Configuration files +├── LICENSE # License file +├── LICENSES/ # Full license texts +└── README.adoc # Project documentation +.... + +=== Design Principles + +* *Separation of Concerns*: Each module has a single responsibility +* *Testability*: Code is written to be easily testable +* *Documentation*: All public APIs are documented +* *Configuration*: Environment-specific settings are externalized + +=== Dependencies + +* External dependencies are minimized and clearly declared +* Version pinning is used for reproducibility + +=== Security Considerations + +* Sensitive data is never committed to the repository +* Secrets are managed through environment variables or secure vaults +* Regular dependency audits are performed + +=== Maintainability + +* Code follows consistent style guidelines +* Pull requests require review and CI checks +* Issues and discussions are tracked transparently + +''''' + +_Last updated: 2026-07-18_ diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md deleted file mode 100644 index 607e3d8c..00000000 --- a/ARCHITECTURE.md +++ /dev/null @@ -1,47 +0,0 @@ -# Architecture - -## Overview - -This repository follows a modular, maintainable architecture designed for clarity, scalability, and long-term sustainability. - -## Directory Structure - -``` -. -├── src/ # Source code -├── tests/ # Test suites -├── docs/ # Documentation -├── scripts/ # Utility scripts -├── config/ # Configuration files -├── LICENSE # License file -├── LICENSES/ # Full license texts -└── README.adoc # Project documentation -``` - -## Design Principles - -- **Separation of Concerns**: Each module has a single responsibility -- **Testability**: Code is written to be easily testable -- **Documentation**: All public APIs are documented -- **Configuration**: Environment-specific settings are externalized - -## Dependencies - -- External dependencies are minimized and clearly declared -- Version pinning is used for reproducibility - -## Security Considerations - -- Sensitive data is never committed to the repository -- Secrets are managed through environment variables or secure vaults -- Regular dependency audits are performed - -## Maintainability - -- Code follows consistent style guidelines -- Pull requests require review and CI checks -- Issues and discussions are tracked transparently - ---- - -*Last updated: 2026-07-18* diff --git a/BINDING-COMPLETENESS-AUDIT-2026-01-30.adoc b/BINDING-COMPLETENESS-AUDIT-2026-01-30.adoc new file mode 100644 index 00000000..76eadd04 --- /dev/null +++ b/BINDING-COMPLETENESS-AUDIT-2026-01-30.adoc @@ -0,0 +1,464 @@ +== proven Language Binding Completeness Audit + +*Date:* 2026-01-30 *Critical Finding:* Most bindings are INCOMPLETE + +''''' + +=== Executive Summary + +*CRITICAL ISSUES FOUND:* + +[arabic] +. ❌ *Bindings are NOT bidirectional* - Only unidirectional (Language → +Zig → Idris2) +. ❌ *Bindings are INCOMPLETE* - Most have ~47% module coverage +. ❌ *Missing important languages* - Several ecosystem-critical +languages absent + +''''' + +=== Module Coverage Analysis + +==== Idris2 Source Truth + +*Total Idris2 modules:* 79 Safe* modules in `+src/Proven/+` + +*Full module list:* + +.... +src/Proven/Safe*.idr (79 modules): +SafeAngle, SafeArgs, SafeBase64, SafeBitset, SafeBloom, SafeBuffer, +SafeCalculator, SafeCapability, SafeChecksum, SafeCircuitBreaker, +SafeColor, SafeCommand, SafeComplex, SafeConsensus, SafeContentType, +SafeCookie, SafeCron, SafeCrypto, SafeCSV, SafeCurrency, SafeDateTime, +SafeDecimal, SafeDigest, SafeEmail, SafeEnv, SafeFile, SafeFiniteField, +SafeFloat, SafeGeo, SafeGraph, SafeHeader, SafeHeap, SafeHex, SafeHtml, +SafeInterval, SafeJson, SafeJWT, SafeLog, SafeLRU, SafeMarkdown, +SafeMath, SafeMatrix, SafeMonotonic, SafeNetwork, SafeOrdering, +SafePassword, SafePath, SafePipe, SafePolicy, SafeProbability, +SafeProcess, SafeProvenance, SafeQueue, SafeRateLimiter, SafeRational, +SafeRegex, SafeRegistry, SafeResource, SafeRetry, SafeSchema, +SafeSemaphore, SafeSet, SafeShell, SafeSignal, SafeSQL, SafeStateMachine, +SafeString, SafeTensor, SafeTerminal, SafeTOML, SafeTransaction, +SafeTree, SafeUnionFind, SafeUnit, SafeUrl, SafeUUID, SafeVersion, +SafeXML, SafeYAML ++ Core.idr, FFI.idr, and others = 81 total modules +.... + +==== Zig FFI Coverage + +*Exported FFI functions:* 155 functions in `+ffi/zig/src/main.zig+` + +*Coverage:* 155 functions / 79 modules = ~2 functions per module +(average) + +*Gap:* Not all Idris2 modules have FFI exports yet! + +*Analysis:* - 14 functions call Idris2 (proven safe) - 141 functions are +Zig-native (safe but unproven) - *Missing:* Many Idris2 modules don’t +have ANY FFI export + +''''' + +=== Binding Coverage by Language + +==== Python Binding + +*Files:* 37 out of 79 modules (47% coverage) + +*Missing modules (42):* + +.... +SafeAngle ✓ (present) +SafeArgs ❌ (missing) +SafeBase64 ❌ +SafeBitset ❌ +SafeBloom ✓ +SafeBuffer ✓ +SafeCalculator ✓ +SafeCapability ❌ +SafeChecksum ✓ +SafeCircuitBreaker ✓ +SafeColor ✓ +SafeCommand ❌ +SafeComplex ❌ +SafeConsensus ❌ +... (pattern continues) +.... + +*Architecture:* ✓ *CORRECT* - All Python modules call Zig FFI, no native +logic + +*Status:* ⚠ *INCOMPLETE* - Only 47% of modules bound + +==== Rust Binding + +*Files:* 1 file (`+src/lib.rs+`) + +*Expected:* ~79 Safe* modules *Actual:* Unknown (need to check lib.rs +exports) + +*Status:* ⚠ *LIKELY INCOMPLETE* + +==== Deno/JavaScript Binding + +*Files:* Minimal structure + +*Status:* ⚠ *LIKELY INCOMPLETE* + +==== ReScript Binding + +*Files:* 3 files + +*Status:* ⚠ *LIKELY INCOMPLETE* + +==== Gleam Binding + +*Files:* 13 files (16% coverage if each = 1 module) + +*Status:* ⚠ *LIKELY INCOMPLETE* + +==== Other Bindings (58 languages) + +*Status:* ❌ *UNKNOWN* - Need systematic audit + +''''' + +=== Bidirectional FFI Analysis + +==== Current Status: ❌ *UNIDIRECTIONAL ONLY* + +*Direction 1: Language → Zig → Idris2* ✓ *WORKS* + +.... +Python code + → calls Python binding (safe_math.py) + → calls ctypes FFI (proven_math_div) + → calls Zig FFI (main.zig:proven_math_div) + → (sometimes) calls Idris2 (proven_idris_*) +.... + +*Direction 2: Idris2 → Zig → Language* ❌ *NOT IMPLEMENTED* + +.... +Idris2 code + → wants to call callback + → NO MECHANISM EXISTS + → Cannot call back into Python/Rust/etc. +.... + +==== What Bidirectional Would Enable + +*Use Cases:* 1. *Async callbacks* - Idris2 calls language code when +operation completes 2. *Plugin systems* - User provides functions, +Idris2 validates and calls them 3. *Event handlers* - Idris2 core +triggers language-side handlers 4. *Dependency injection* - Language +provides implementations of interfaces + +*Example (NOT CURRENTLY POSSIBLE):* + +[source,python] +---- +# Python provides a validator function +def my_validator(data: str) -> bool: + return len(data) < 100 + +# Idris2 wants to call this from within proven +proven.validate_with_callback(input_data, my_validator) # ❌ NOT POSSIBLE +---- + +==== Architecture for Bidirectional FFI + +*Required Changes:* + +[arabic] +. *Idris2 Layer:* + +[source,idris] +---- +-- Accept function pointer from FFI +validateWith : (validator : String -> Bool) -> String -> Bool +validateWith validator input = + if validator input -- Call foreign function + then True + else False + +-- FFI declaration +%foreign "C:proven_callback_validate, zig:main" +provenCallbackValidate : GCAnyPtr -> String -> Bool +---- + +[arabic, start=2] +. *Zig Layer:* + +[source,zig] +---- +// Function pointer type +pub const ValidatorFn = *const fn([*:0]const u8) callconv(.C) bool; + +// Accept function pointer from language bindings +export fn proven_set_validator(validator: ValidatorFn) void { + stored_validator = validator; +} + +// Call it from Idris2 +export fn proven_callback_validate(data: [*:0]const u8) bool { + return stored_validator(data); +} +---- + +[arabic, start=3] +. *Language Binding (Python):* + +[source,python] +---- +from ctypes import CFUNCTYPE, c_char_p, c_bool + +# Create callback type +VALIDATOR_CALLBACK = CFUNCTYPE(c_bool, c_char_p) + +# User's Python function +def my_validator(data: bytes) -> bool: + return len(data) < 100 + +# Register callback with proven +callback = VALIDATOR_CALLBACK(my_validator) +lib.proven_set_validator(callback) +---- + +*Status:* ❌ *NOT IMPLEMENTED* - Roadmap for v2.0 + +''''' + +=== Missing Critical Languages + +==== Currently Have (64 bindings): + +*General:* Ada, C, C++, Crystal, D, Dart, Deno, Elixir, Erlang, F#, +Gleam, Go, Haskell, Java, JavaScript, Julia, Kotlin, Lua, Nim, OCaml, +Perl, PHP, Python, R, Racket, ReScript, Ruby, Rust, Scala, Swift, +TypeScript, V, Zig + +*Functional:* Clojure, Common Lisp, Elm, Guile, PureScript + +*Shell:* Bash, Fish, PowerShell, Zsh + +*Config:* CUE, Dhall, HCL, Jsonnet, Nickel, Starlark + +*Domain:* Arduino, Cairo, GDScript, Ink, MicroPython, Move, Solidity, +Unity C#, Vyper + +*Legacy:* COBOL, Forth, Fortran + +*Research:* Alloy, CEL, GraphQL, Janus, Neuromorphic, OpenQASM, PromQL, +Q#, Rego, SPICE, Ternary, TLA+ + +*Low-Level:* AssemblyScript, Grain, Malbolge, VHDL, WAT + +==== MISSING Important Languages + +===== Tier 1 (Critical - Used in Production) + +❌ *C#* (non-Unity) - Major enterprise language - .NET ecosystem - +Azure, ASP.NET, Blazor - *Priority: HIGH* + +❌ *Objective-C* - iOS/macOS (legacy but still critical) - Older iOS +apps - macOS system integration - *Priority: MEDIUM* + +❌ *Zig* - Wait, is Zig binding present? - Check: We have +`+bindings/zig/+` but it might just be the FFI layer - *Priority: CHECK* + +===== Tier 2 (Emerging - Growing Ecosystems) + +❌ *Roc* - Functional language gaining traction - Fast, safe, functional +- Similar philosophy to proven - *Priority: MEDIUM* + +❌ *Koka* - Effect system research language - Microsoft Research - +Effect handlers - *Priority: LOW* (research) + +❌ *Lean 4* - Theorem prover that can compile to C - Similar to Idris2 - +Could validate proven’s proofs - *Priority: MEDIUM* + +❌ *Agda* - Dependently typed language - Proof assistant - Academic use +- *Priority: LOW* + +❌ *Coq* - Theorem prover - OCaml extraction - Could verify proven - +*Priority: LOW* + +===== Tier 3 (Specialized) + +❌ *Pony* - Actor-based language - Reference capabilities - +Concurrency-focused - *Priority: LOW* + +❌ *Virgil* - Lightweight systems language - Fast compilation - Embedded +systems - *Priority: LOW* + +❌ *Jai* - Jonathan Blow’s language - Game development - In beta - +*Priority: WAIT* (not public yet) + +''''' + +=== Binding Quality Tiers + +Based on completeness: + +==== Tier 1: Complete (>90% modules) + +*Status:* ❌ *NONE* + +==== Tier 2: Substantial (50-90% modules) + +*Candidates:* - Python (47%) - Close, needs 30 more modules + +==== Tier 3: Partial (20-50% modules) + +*Candidates:* - Gleam (16-20%?) - Need verification - Most others likely +here + +==== Tier 4: Stub (<20% modules) + +*Likely:* - Rust (1 file observed) - Deno (1 file observed) - ReScript +(3 files) - Most exotic languages + +''''' + +=== Recommendations + +==== Immediate (v1.0) + +[arabic] +. *Document incompleteness honestly* +* README should state "`Bindings are in progress, coverage varies`" +* List which languages have which modules +* Don’t claim "`89 complete bindings`" +. *Create binding coverage matrix* ++ +.... +Language | Modules | Coverage | Status +---------|---------|----------|------- +Python | 37/79 | 47% | Active +Rust | ?/79 | ?% | Unknown +Deno | ?/79 | ?% | Unknown +.... +. *Update STATE.scm* +* Change binding status from "`complete`" to "`partial`" +* Document actual coverage percentages + +==== Short-term (v1.1) + +[arabic, start=4] +. *Complete Tier 1 languages first* +* Python → 100% (add 42 modules) +* Rust → 100% (add most modules) +* Deno → 100% (JavaScript ecosystem critical) +* ReScript → 100% (approved by RSR) +. *Add missing critical languages* +* C# (non-Unity) +* Lean 4 (proof validation) +* Roc (emerging functional) +. *Create binding generator* +* Script to auto-generate bindings from Zig FFI +* Reduces manual work +* Ensures consistency + +==== Medium-term (v2.0) + +[arabic, start=7] +. *Implement bidirectional FFI* +* Function pointer support in Zig +* Callback registration API +* Type-safe marshalling +. *Complete all 89 bindings to 100%* +* Systematic completion +* Automated testing +* CI validation +. *Add Tier 2 missing languages* +* Complete the ecosystem + +''''' + +=== Binding Generator Proposal + +==== Auto-Generate from Zig FFI + +*Input:* `+ffi/zig/src/main.zig+` + +*Output:* Bindings for all languages + +*Example:* + +[source,bash] +---- +# Generate Python binding for all FFI functions +./scripts/generate-binding.sh python + +# Generates: +# bindings/python/proven/safe_angle.py +# bindings/python/proven/safe_args.py +# ... (all 79 modules) +---- + +*Template:* + +[source,python] +---- +# AUTO-GENERATED - DO NOT EDIT +# Generated from ffi/zig/src/main.zig + +class Safe{{ ModuleName }}: + @staticmethod + def {{ function_name }}({{ params }}) -> {{ return_type }}: + lib = get_lib() + result = lib.proven_{{ function_name }}({{ args }}) + return handle_result(result) +---- + +*Status:* ❌ *NOT CREATED* - Roadmap for v1.1 + +''''' + +=== Critical Gaps Summary + +[width="100%",cols="22%,28%,22%,28%",options="header",] +|=== +|Issue |Severity |Impact |Timeline +|Bindings incomplete (~47%) |HIGH |Users can’t access all modules |v1.1 +|Not bidirectional |MEDIUM |Limits use cases |v2.0 +|Missing C# binding |MEDIUM |Excludes .NET ecosystem |v1.1 +|No binding generator |LOW |Manual work, inconsistency |v1.1 +|Zig FFI incomplete |HIGH |Some Idris2 modules not exported |v1.0 +|=== + +''''' + +=== Action Items + +==== Before v1.0 Release + +* [ ] Update README to clarify binding completeness +* [ ] Create binding coverage matrix +* [ ] Update STATE.scm with accurate percentages +* [ ] Document known limitations + +==== v1.1 (Next 3 months) + +* [ ] Complete Python binding (100%) +* [ ] Complete Rust binding (100%) +* [ ] Complete Deno binding (100%) +* [ ] Complete ReScript binding (100%) +* [ ] Add C# binding +* [ ] Create binding generator +* [ ] Systematic binding audit (all 64 languages) + +==== v2.0 (Next 12 months) + +* [ ] Implement bidirectional FFI +* [ ] Complete all 89 bindings to 100% +* [ ] Add missing Tier 2 languages (Lean 4, Roc) +* [ ] Automated binding tests (all languages) + +''''' + +_Audit completed: 2026-01-30_ _Critical finding: Bindings are 47% +complete, not 100% as STATE.scm suggests_ _Recommendation: Honest +disclosure + systematic completion plan_ diff --git a/BINDING-COMPLETENESS-AUDIT-2026-01-30.md b/BINDING-COMPLETENESS-AUDIT-2026-01-30.md deleted file mode 100644 index 21174154..00000000 --- a/BINDING-COMPLETENESS-AUDIT-2026-01-30.md +++ /dev/null @@ -1,462 +0,0 @@ -# proven Language Binding Completeness Audit -**Date:** 2026-01-30 -**Critical Finding:** Most bindings are INCOMPLETE - ---- - -## Executive Summary - -**CRITICAL ISSUES FOUND:** - -1. ❌ **Bindings are NOT bidirectional** - Only unidirectional (Language → Zig → Idris2) -2. ❌ **Bindings are INCOMPLETE** - Most have ~47% module coverage -3. ❌ **Missing important languages** - Several ecosystem-critical languages absent - ---- - -## Module Coverage Analysis - -### Idris2 Source Truth - -**Total Idris2 modules:** 79 Safe* modules in `src/Proven/` - -**Full module list:** -``` -src/Proven/Safe*.idr (79 modules): -SafeAngle, SafeArgs, SafeBase64, SafeBitset, SafeBloom, SafeBuffer, -SafeCalculator, SafeCapability, SafeChecksum, SafeCircuitBreaker, -SafeColor, SafeCommand, SafeComplex, SafeConsensus, SafeContentType, -SafeCookie, SafeCron, SafeCrypto, SafeCSV, SafeCurrency, SafeDateTime, -SafeDecimal, SafeDigest, SafeEmail, SafeEnv, SafeFile, SafeFiniteField, -SafeFloat, SafeGeo, SafeGraph, SafeHeader, SafeHeap, SafeHex, SafeHtml, -SafeInterval, SafeJson, SafeJWT, SafeLog, SafeLRU, SafeMarkdown, -SafeMath, SafeMatrix, SafeMonotonic, SafeNetwork, SafeOrdering, -SafePassword, SafePath, SafePipe, SafePolicy, SafeProbability, -SafeProcess, SafeProvenance, SafeQueue, SafeRateLimiter, SafeRational, -SafeRegex, SafeRegistry, SafeResource, SafeRetry, SafeSchema, -SafeSemaphore, SafeSet, SafeShell, SafeSignal, SafeSQL, SafeStateMachine, -SafeString, SafeTensor, SafeTerminal, SafeTOML, SafeTransaction, -SafeTree, SafeUnionFind, SafeUnit, SafeUrl, SafeUUID, SafeVersion, -SafeXML, SafeYAML -+ Core.idr, FFI.idr, and others = 81 total modules -``` - -### Zig FFI Coverage - -**Exported FFI functions:** 155 functions in `ffi/zig/src/main.zig` - -**Coverage:** 155 functions / 79 modules = ~2 functions per module (average) - -**Gap:** Not all Idris2 modules have FFI exports yet! - -**Analysis:** -- 14 functions call Idris2 (proven safe) -- 141 functions are Zig-native (safe but unproven) -- **Missing:** Many Idris2 modules don't have ANY FFI export - ---- - -## Binding Coverage by Language - -### Python Binding - -**Files:** 37 out of 79 modules (47% coverage) - -**Missing modules (42):** -``` -SafeAngle ✓ (present) -SafeArgs ❌ (missing) -SafeBase64 ❌ -SafeBitset ❌ -SafeBloom ✓ -SafeBuffer ✓ -SafeCalculator ✓ -SafeCapability ❌ -SafeChecksum ✓ -SafeCircuitBreaker ✓ -SafeColor ✓ -SafeCommand ❌ -SafeComplex ❌ -SafeConsensus ❌ -... (pattern continues) -``` - -**Architecture:** ✓ **CORRECT** - All Python modules call Zig FFI, no native logic - -**Status:** ⚠ **INCOMPLETE** - Only 47% of modules bound - -### Rust Binding - -**Files:** 1 file (`src/lib.rs`) - -**Expected:** ~79 Safe* modules -**Actual:** Unknown (need to check lib.rs exports) - -**Status:** ⚠ **LIKELY INCOMPLETE** - -### Deno/JavaScript Binding - -**Files:** Minimal structure - -**Status:** ⚠ **LIKELY INCOMPLETE** - -### ReScript Binding - -**Files:** 3 files - -**Status:** ⚠ **LIKELY INCOMPLETE** - -### Gleam Binding - -**Files:** 13 files (16% coverage if each = 1 module) - -**Status:** ⚠ **LIKELY INCOMPLETE** - -### Other Bindings (58 languages) - -**Status:** ❌ **UNKNOWN** - Need systematic audit - ---- - -## Bidirectional FFI Analysis - -### Current Status: ❌ **UNIDIRECTIONAL ONLY** - -**Direction 1: Language → Zig → Idris2** ✓ **WORKS** -``` -Python code - → calls Python binding (safe_math.py) - → calls ctypes FFI (proven_math_div) - → calls Zig FFI (main.zig:proven_math_div) - → (sometimes) calls Idris2 (proven_idris_*) -``` - -**Direction 2: Idris2 → Zig → Language** ❌ **NOT IMPLEMENTED** -``` -Idris2 code - → wants to call callback - → NO MECHANISM EXISTS - → Cannot call back into Python/Rust/etc. -``` - -### What Bidirectional Would Enable - -**Use Cases:** -1. **Async callbacks** - Idris2 calls language code when operation completes -2. **Plugin systems** - User provides functions, Idris2 validates and calls them -3. **Event handlers** - Idris2 core triggers language-side handlers -4. **Dependency injection** - Language provides implementations of interfaces - -**Example (NOT CURRENTLY POSSIBLE):** -```python -# Python provides a validator function -def my_validator(data: str) -> bool: - return len(data) < 100 - -# Idris2 wants to call this from within proven -proven.validate_with_callback(input_data, my_validator) # ❌ NOT POSSIBLE -``` - -### Architecture for Bidirectional FFI - -**Required Changes:** - -1. **Idris2 Layer:** -```idris --- Accept function pointer from FFI -validateWith : (validator : String -> Bool) -> String -> Bool -validateWith validator input = - if validator input -- Call foreign function - then True - else False - --- FFI declaration -%foreign "C:proven_callback_validate, zig:main" -provenCallbackValidate : GCAnyPtr -> String -> Bool -``` - -2. **Zig Layer:** -```zig -// Function pointer type -pub const ValidatorFn = *const fn([*:0]const u8) callconv(.C) bool; - -// Accept function pointer from language bindings -export fn proven_set_validator(validator: ValidatorFn) void { - stored_validator = validator; -} - -// Call it from Idris2 -export fn proven_callback_validate(data: [*:0]const u8) bool { - return stored_validator(data); -} -``` - -3. **Language Binding (Python):** -```python -from ctypes import CFUNCTYPE, c_char_p, c_bool - -# Create callback type -VALIDATOR_CALLBACK = CFUNCTYPE(c_bool, c_char_p) - -# User's Python function -def my_validator(data: bytes) -> bool: - return len(data) < 100 - -# Register callback with proven -callback = VALIDATOR_CALLBACK(my_validator) -lib.proven_set_validator(callback) -``` - -**Status:** ❌ **NOT IMPLEMENTED** - Roadmap for v2.0 - ---- - -## Missing Critical Languages - -### Currently Have (64 bindings): - -**General:** Ada, C, C++, Crystal, D, Dart, Deno, Elixir, Erlang, F#, Gleam, Go, Haskell, Java, JavaScript, Julia, Kotlin, Lua, Nim, OCaml, Perl, PHP, Python, R, Racket, ReScript, Ruby, Rust, Scala, Swift, TypeScript, V, Zig - -**Functional:** Clojure, Common Lisp, Elm, Guile, PureScript - -**Shell:** Bash, Fish, PowerShell, Zsh - -**Config:** CUE, Dhall, HCL, Jsonnet, Nickel, Starlark - -**Domain:** Arduino, Cairo, GDScript, Ink, MicroPython, Move, Solidity, Unity C#, Vyper - -**Legacy:** COBOL, Forth, Fortran - -**Research:** Alloy, CEL, GraphQL, Janus, Neuromorphic, OpenQASM, PromQL, Q#, Rego, SPICE, Ternary, TLA+ - -**Low-Level:** AssemblyScript, Grain, Malbolge, VHDL, WAT - -### MISSING Important Languages - -#### Tier 1 (Critical - Used in Production) - -❌ **C#** (non-Unity) - Major enterprise language - - .NET ecosystem - - Azure, ASP.NET, Blazor - - **Priority: HIGH** - -❌ **Objective-C** - iOS/macOS (legacy but still critical) - - Older iOS apps - - macOS system integration - - **Priority: MEDIUM** - -❌ **Zig** - Wait, is Zig binding present? - - Check: We have `bindings/zig/` but it might just be the FFI layer - - **Priority: CHECK** - -#### Tier 2 (Emerging - Growing Ecosystems) - -❌ **Roc** - Functional language gaining traction - - Fast, safe, functional - - Similar philosophy to proven - - **Priority: MEDIUM** - -❌ **Koka** - Effect system research language - - Microsoft Research - - Effect handlers - - **Priority: LOW** (research) - -❌ **Lean 4** - Theorem prover that can compile to C - - Similar to Idris2 - - Could validate proven's proofs - - **Priority: MEDIUM** - -❌ **Agda** - Dependently typed language - - Proof assistant - - Academic use - - **Priority: LOW** - -❌ **Coq** - Theorem prover - - OCaml extraction - - Could verify proven - - **Priority: LOW** - -#### Tier 3 (Specialized) - -❌ **Pony** - Actor-based language - - Reference capabilities - - Concurrency-focused - - **Priority: LOW** - -❌ **Virgil** - Lightweight systems language - - Fast compilation - - Embedded systems - - **Priority: LOW** - -❌ **Jai** - Jonathan Blow's language - - Game development - - In beta - - **Priority: WAIT** (not public yet) - ---- - -## Binding Quality Tiers - -Based on completeness: - -### Tier 1: Complete (>90% modules) - -**Status:** ❌ **NONE** - -### Tier 2: Substantial (50-90% modules) - -**Candidates:** -- Python (47%) - Close, needs 30 more modules - -### Tier 3: Partial (20-50% modules) - -**Candidates:** -- Gleam (16-20%?) - Need verification -- Most others likely here - -### Tier 4: Stub (<20% modules) - -**Likely:** -- Rust (1 file observed) -- Deno (1 file observed) -- ReScript (3 files) -- Most exotic languages - ---- - -## Recommendations - -### Immediate (v1.0) - -1. **Document incompleteness honestly** - - README should state "Bindings are in progress, coverage varies" - - List which languages have which modules - - Don't claim "89 complete bindings" - -2. **Create binding coverage matrix** - ``` - Language | Modules | Coverage | Status - ---------|---------|----------|------- - Python | 37/79 | 47% | Active - Rust | ?/79 | ?% | Unknown - Deno | ?/79 | ?% | Unknown - ``` - -3. **Update STATE.scm** - - Change binding status from "complete" to "partial" - - Document actual coverage percentages - -### Short-term (v1.1) - -4. **Complete Tier 1 languages first** - - Python → 100% (add 42 modules) - - Rust → 100% (add most modules) - - Deno → 100% (JavaScript ecosystem critical) - - ReScript → 100% (approved by RSR) - -5. **Add missing critical languages** - - C# (non-Unity) - - Lean 4 (proof validation) - - Roc (emerging functional) - -6. **Create binding generator** - - Script to auto-generate bindings from Zig FFI - - Reduces manual work - - Ensures consistency - -### Medium-term (v2.0) - -7. **Implement bidirectional FFI** - - Function pointer support in Zig - - Callback registration API - - Type-safe marshalling - -8. **Complete all 89 bindings to 100%** - - Systematic completion - - Automated testing - - CI validation - -9. **Add Tier 2 missing languages** - - Complete the ecosystem - ---- - -## Binding Generator Proposal - -### Auto-Generate from Zig FFI - -**Input:** `ffi/zig/src/main.zig` - -**Output:** Bindings for all languages - -**Example:** -```bash -# Generate Python binding for all FFI functions -./scripts/generate-binding.sh python - -# Generates: -# bindings/python/proven/safe_angle.py -# bindings/python/proven/safe_args.py -# ... (all 79 modules) -``` - -**Template:** -```python -# AUTO-GENERATED - DO NOT EDIT -# Generated from ffi/zig/src/main.zig - -class Safe{{ ModuleName }}: - @staticmethod - def {{ function_name }}({{ params }}) -> {{ return_type }}: - lib = get_lib() - result = lib.proven_{{ function_name }}({{ args }}) - return handle_result(result) -``` - -**Status:** ❌ **NOT CREATED** - Roadmap for v1.1 - ---- - -## Critical Gaps Summary - -| Issue | Severity | Impact | Timeline | -|-------|----------|--------|----------| -| Bindings incomplete (~47%) | HIGH | Users can't access all modules | v1.1 | -| Not bidirectional | MEDIUM | Limits use cases | v2.0 | -| Missing C# binding | MEDIUM | Excludes .NET ecosystem | v1.1 | -| No binding generator | LOW | Manual work, inconsistency | v1.1 | -| Zig FFI incomplete | HIGH | Some Idris2 modules not exported | v1.0 | - ---- - -## Action Items - -### Before v1.0 Release - -- [ ] Update README to clarify binding completeness -- [ ] Create binding coverage matrix -- [ ] Update STATE.scm with accurate percentages -- [ ] Document known limitations - -### v1.1 (Next 3 months) - -- [ ] Complete Python binding (100%) -- [ ] Complete Rust binding (100%) -- [ ] Complete Deno binding (100%) -- [ ] Complete ReScript binding (100%) -- [ ] Add C# binding -- [ ] Create binding generator -- [ ] Systematic binding audit (all 64 languages) - -### v2.0 (Next 12 months) - -- [ ] Implement bidirectional FFI -- [ ] Complete all 89 bindings to 100% -- [ ] Add missing Tier 2 languages (Lean 4, Roc) -- [ ] Automated binding tests (all languages) - ---- - -_Audit completed: 2026-01-30_ -_Critical finding: Bindings are 47% complete, not 100% as STATE.scm suggests_ -_Recommendation: Honest disclosure + systematic completion plan_ diff --git a/CHANGELOG.adoc b/CHANGELOG.adoc new file mode 100644 index 00000000..db259d9f --- /dev/null +++ b/CHANGELOG.adoc @@ -0,0 +1,97 @@ +== Changelog + +All notable changes to *proven* are documented here. + +=== Unreleased + +* Creating remaining 12 server apps (proven-httpd through proven-wasm) +* GPU/VPU/TPU/Crypto hardware backend modules +* Framework convenience re-export modules (Web, Crypto, Network, Data, +System, Math, Hardware) +* Container hardening with stapeln, firewalld, svalinn-compose +* Comprehensive test suite expansion (all 104 modules) +* Cross-repo integration with hypatia and gitbot-fleet + +=== 1.2.0 — 2026-02-22 + +==== CRITICAL REMEDIATION + +This release addresses findings from an honest audit of the codebase. + +==== Fixed — Formal Verification + +* *believe_me eliminated*: 0 instances remaining (down from ~4,566 +across 38+ files) +* *assert_total eliminated*: All uses replaced with structurally total +implementations +* *All TODOs removed*: No remaining TODO/STUB/FIXME markers in source + +==== Fixed — Architecture Compliance + +* All language bindings now call Idris2 via Zig FFI (no reimplemented +logic) +* 31 missing bindings created and wired to FFI layer +* Rust binding license corrected from Apache-2.0 to MPL-2.0 +* Containerfile converted from Docker-style to OCI-compliant +Containerfile +* RefC compilation pipeline built (`+scripts/build-refc.sh+`) + +==== Added — RSR Compliance + +* `+0-AI-MANIFEST.a2ml+` — Canonical AI entry point (universal agent +protocol) +* `+.github/CODEOWNERS+` — Code ownership for PR review routing +* `+MAINTAINERS.adoc+` — Project maintainer documentation +* `+.well-known/security.txt+` — RFC 9116 security contact +(securitytxt.org) +* `+.editorconfig+` SPDX header fixed from MPL-2.0 to MPL-2.0 + +==== Changed — Honest Documentation + +* `+STATE.scm+` updated with accurate completion percentages (55%, not +100%) +* Honest accounting: core Idris2 ~95%, apps 7%, GPU/crypto 0%, +convenience modules 0% +* Module count clarified: 104 core Safe* modules, 258 total .idr files +(including FFI wrappers) +* Binding count updated: 120+ targets (18 complete, 102 scaffolded) + +=== 1.1.0 + +==== Added + +* 14 new Idris2 modules: SafeShell, SafePipe, SafeProcess, SafeSignal, +SafeTerminal, SafeSemaphore, SafeCalculator, SafeCSV, SafeDecimal, +SafeRational, SafeComplex, SafeSet, SafeHeap, SafeMatrix (104 total, up +from 74) +* 14 new FFI exports in Zig bridge layer for new modules +* 6 additional data structure modules: SafeBitset, SafeInterval, +SafeUnionFind (with proofs) + +==== Fixed + +* SafeCapability: proof hole for `+capSubsetRefl+` filled with correct +reflexivity proof +* ECHIDNA/Soundness: fixed soundness proof structure +* SafeFloat: corrected dependent type proofs +* SafeCert FFI: hostname wildcard matching aligned with source module +(no-dots-in-prefix check) +* SafeML FFI: replaced `+parsePositive+` with scalar building blocks +* ipkg: removed duplicate module entries + +==== Changed + +* SPDX license headers standardised to MPL-2.0 across all 245+ source +files +* Copyright lines updated to "`Jonathan D.A. Jewell (hyperpolymath)`" +format +* All TODO stubs in SafeDigest and SafeRegistry converted to +documentation notes +* pack.toml version, license, and author fields corrected + +=== 1.0.0 + +* Initial public release of Idris2 verified modules. +* 74 core modules with dependent type proofs. +* 89 binding targets via Zig FFI bridge. +* ECHIDNA integration for proof verification. diff --git a/CHANGELOG.md b/CHANGELOG.md deleted file mode 100644 index bf6a32aa..00000000 --- a/CHANGELOG.md +++ /dev/null @@ -1,71 +0,0 @@ -# Changelog - -All notable changes to **proven** are documented here. - -## Unreleased - -- Creating remaining 12 server apps (proven-httpd through proven-wasm) -- GPU/VPU/TPU/Crypto hardware backend modules -- Framework convenience re-export modules (Web, Crypto, Network, Data, System, Math, Hardware) -- Container hardening with stapeln, firewalld, svalinn-compose -- Comprehensive test suite expansion (all 104 modules) -- Cross-repo integration with hypatia and gitbot-fleet - -## 1.2.0 — 2026-02-22 - -### CRITICAL REMEDIATION - -This release addresses findings from an honest audit of the codebase. - -### Fixed — Formal Verification -- **believe_me eliminated**: 0 instances remaining (down from ~4,566 across 38+ files) -- **assert_total eliminated**: All uses replaced with structurally total implementations -- **All TODOs removed**: No remaining TODO/STUB/FIXME markers in source - -### Fixed — Architecture Compliance -- All language bindings now call Idris2 via Zig FFI (no reimplemented logic) -- 31 missing bindings created and wired to FFI layer -- Rust binding license corrected from Apache-2.0 to MPL-2.0 -- Containerfile converted from Docker-style to OCI-compliant Containerfile -- RefC compilation pipeline built (`scripts/build-refc.sh`) - -### Added — RSR Compliance -- `0-AI-MANIFEST.a2ml` — Canonical AI entry point (universal agent protocol) -- `.github/CODEOWNERS` — Code ownership for PR review routing -- `MAINTAINERS.adoc` — Project maintainer documentation -- `.well-known/security.txt` — RFC 9116 security contact (securitytxt.org) -- `.editorconfig` SPDX header fixed from MPL-2.0 to MPL-2.0 - -### Changed — Honest Documentation -- `STATE.scm` updated with accurate completion percentages (55%, not 100%) -- Honest accounting: core Idris2 ~95%, apps 7%, GPU/crypto 0%, convenience modules 0% -- Module count clarified: 104 core Safe* modules, 258 total .idr files (including FFI wrappers) -- Binding count updated: 120+ targets (18 complete, 102 scaffolded) - -## 1.1.0 - -### Added -- 14 new Idris2 modules: SafeShell, SafePipe, SafeProcess, SafeSignal, SafeTerminal, SafeSemaphore, SafeCalculator, SafeCSV, SafeDecimal, SafeRational, SafeComplex, SafeSet, SafeHeap, SafeMatrix (104 total, up from 74) -- 14 new FFI exports in Zig bridge layer for new modules -- 6 additional data structure modules: SafeBitset, SafeInterval, SafeUnionFind (with proofs) - -### Fixed -- SafeCapability: proof hole for `capSubsetRefl` filled with correct reflexivity proof -- ECHIDNA/Soundness: fixed soundness proof structure -- SafeFloat: corrected dependent type proofs -- SafeCert FFI: hostname wildcard matching aligned with source module (no-dots-in-prefix check) -- SafeML FFI: replaced `parsePositive` with scalar building blocks -- ipkg: removed duplicate module entries - -### Changed -- SPDX license headers standardised to MPL-2.0 across all 245+ source files -- Copyright lines updated to "Jonathan D.A. Jewell (hyperpolymath)" format -- All TODO stubs in SafeDigest and SafeRegistry converted to documentation notes -- pack.toml version, license, and author fields corrected - -## 1.0.0 - -- Initial public release of Idris2 verified modules. -- 74 core modules with dependent type proofs. -- 89 binding targets via Zig FFI bridge. -- ECHIDNA integration for proof verification. diff --git a/CODE_OF_CONDUCT.adoc b/CODE_OF_CONDUCT.adoc new file mode 100644 index 00000000..21fb60f9 --- /dev/null +++ b/CODE_OF_CONDUCT.adoc @@ -0,0 +1,83 @@ +== Barnet Code of Conduct + +Like the technical community as a whole, the Barnet team and community +is made up of a mixture of professionals and volunteers from all over +the world, working on every aspect of the mission - including +mentorship, teaching, and connecting people. + +Diversity is one of our huge strengths, but it can also lead to +communication issues and unhappiness. To that end, we have a few ground +rules that we ask people to adhere to. This code applies equally to +founders, mentors and those seeking help and guidance. + +This isn’t an exhaustive list of things that you can’t do. Rather, take +it in the spirit in which it’s intended - a guide to make it easier to +enrich all of us and the technical communities in which we participate. + +This code of conduct applies to all spaces managed by the Barnet project +or . This includes IRC, the mailing lists, the issue tracker, DSF +events, and any other forums created by the project team which the +community uses for communication. In addition, violations of this code +outside these spaces may affect a person’s ability to participate within +them. + +If you believe someone is violating the code of conduct, we ask that you +report it by emailing j.d.a.jewell@open.ac.uk. For more details please +see our 86 Brookside Road + +* *Be friendly and patient.* +* *Be welcoming.* We strive to be a community that welcomes and supports +people of all backgrounds and identities. This includes, but is not +limited to members of any race, ethnicity, culture, national origin, +colour, immigration status, social and economic class, educational +level, sex, sexual orientation, gender identity and expression, age, +size, family status, political belief, religion, and mental and physical +ability. +* *Be considerate.* Your work will be used by other people, and you in +turn will depend on the work of others. Any decision you take will +affect users and colleagues, and you should take those consequences into +account when making decisions. Remember that we’re a world-wide +community, so you might not be communicating in someone else’s primary +language. +* *Be respectful.* Not all of us will agree all the time, but +disagreement is no excuse for poor behavior and poor manners. We might +all experience some frustration now and then, but we cannot allow that +frustration to turn into a personal attack. It’s important to remember +that a community where people feel uncomfortable or threatened is not a +productive one. Members of the Barnet community should be respectful +when dealing with other members as well as with people outside the +Barnet community. +* *Be careful in the words that you choose.* We are a community of +professionals, and we conduct ourselves professionally. Be kind to +others. Do not insult or put down other participants. Harassment and +other exclusionary behavior aren’t acceptable. This includes, but is not +limited to: +* Violent threats or language directed against another person. +* Discriminatory jokes and language. +* Posting sexually explicit or violent material. +* Posting (or threatening to post) other people’s personally identifying +information ("`doxing`"). +* Personal insults, especially those using racist or sexist terms. +* Unwelcome sexual attention. +* Advocating for, or encouraging, any of the above behavior. +* Repeated harassment of others. In general, if someone asks you to +stop, then stop. +* *When we disagree, try to understand why.* Disagreements, both social +and technical, happen all the time and Barnet is no exception. It is +important that we resolve disagreements and differing views +constructively. Remember that we’re different. The strength of Barnet +comes from its varied community, people from a wide range of +backgrounds. Different people have different perspectives on issues. +Being unable to understand why someone holds a viewpoint doesn’t mean +that they’re wrong. Don’t forget that it is human to err and blaming +each other doesn’t get us anywhere. Instead, focus on helping to resolve +issues and learning from mistakes. + +Original text courtesy of the +http://web.archive.org/web/20141109123859/http://speakup.io/coc.html[Speak +Up! project]. + +=== Questions? + +If you have questions, please see . If that doesn’t answer your +questions, feel free to mailto:j.d.a.jewell@open.ac.uk[contact us]. diff --git a/CODE_OF_CONDUCT.md b/CODE_OF_CONDUCT.md deleted file mode 100644 index bb138073..00000000 --- a/CODE_OF_CONDUCT.md +++ /dev/null @@ -1,32 +0,0 @@ -# Barnet Code of Conduct - -Like the technical community as a whole, the Barnet team and community is made up of a mixture of professionals and volunteers from all over the world, working on every aspect of the mission - including mentorship, teaching, and connecting people. - -Diversity is one of our huge strengths, but it can also lead to communication issues and unhappiness. To that end, we have a few ground rules that we ask people to adhere to. This code applies equally to founders, mentors and those seeking help and guidance. - -This isn’t an exhaustive list of things that you can’t do. Rather, take it in the spirit in which it’s intended - a guide to make it easier to enrich all of us and the technical communities in which we participate. - -This code of conduct applies to all spaces managed by the Barnet project or . This includes IRC, the mailing lists, the issue tracker, DSF events, and any other forums created by the project team which the community uses for communication. In addition, violations of this code outside these spaces may affect a person's ability to participate within them. - -If you believe someone is violating the code of conduct, we ask that you report it by emailing [j.d.a.jewell@open.ac.uk](mailto:j.d.a.jewell@open.ac.uk). For more details please see our 86 Brookside Road - -- **Be friendly and patient.** -- **Be welcoming.** We strive to be a community that welcomes and supports people of all backgrounds and identities. This includes, but is not limited to members of any race, ethnicity, culture, national origin, colour, immigration status, social and economic class, educational level, sex, sexual orientation, gender identity and expression, age, size, family status, political belief, religion, and mental and physical ability. -- **Be considerate.** Your work will be used by other people, and you in turn will depend on the work of others. Any decision you take will affect users and colleagues, and you should take those consequences into account when making decisions. Remember that we're a world-wide community, so you might not be communicating in someone else's primary language. -- **Be respectful.** Not all of us will agree all the time, but disagreement is no excuse for poor behavior and poor manners. We might all experience some frustration now and then, but we cannot allow that frustration to turn into a personal attack. It’s important to remember that a community where people feel uncomfortable or threatened is not a productive one. Members of the Barnet community should be respectful when dealing with other members as well as with people outside the Barnet community. -- **Be careful in the words that you choose.** We are a community of professionals, and we conduct ourselves professionally. Be kind to others. Do not insult or put down other participants. Harassment and other exclusionary behavior aren't acceptable. This includes, but is not limited to: - - Violent threats or language directed against another person. - - Discriminatory jokes and language. - - Posting sexually explicit or violent material. - - Posting (or threatening to post) other people's personally identifying information ("doxing"). - - Personal insults, especially those using racist or sexist terms. - - Unwelcome sexual attention. - - Advocating for, or encouraging, any of the above behavior. - - Repeated harassment of others. In general, if someone asks you to stop, then stop. -- **When we disagree, try to understand why.** Disagreements, both social and technical, happen all the time and Barnet is no exception. It is important that we resolve disagreements and differing views constructively. Remember that we’re different. The strength of Barnet comes from its varied community, people from a wide range of backgrounds. Different people have different perspectives on issues. Being unable to understand why someone holds a viewpoint doesn’t mean that they’re wrong. Don’t forget that it is human to err and blaming each other doesn’t get us anywhere. Instead, focus on helping to resolve issues and learning from mistakes. - -Original text courtesy of the [Speak Up! project](http://web.archive.org/web/20141109123859/http://speakup.io/coc.html). - -## Questions? - -If you have questions, please see . If that doesn't answer your questions, feel free to [contact us](mailto:j.d.a.jewell@open.ac.uk). diff --git a/CONTRIBUTING.adoc b/CONTRIBUTING.adoc index 2521ad51..858f44fd 100644 --- a/CONTRIBUTING.adoc +++ b/CONTRIBUTING.adoc @@ -1,106 +1,71 @@ -// SPDX-License-Identifier: CC-BY-SA-4.0 -= Contributing to proven +== Contributing -Thank you for your interest in contributing! +Thank you for your interest in contributing! We follow a "`Dual-Track`" +architecture where human-readable documentation lives in the root and +machine-readable policies live in `+.machine_readable/+`. -== Code of Conduct +=== How to Contribute -This project follows the https://www.contributor-covenant.org/version/2/1/code_of_conduct/[Contributor Covenant v2.1]. +We welcome contributions in many forms: -== How to Contribute +* *Code:* Improving the core stack or extensions +* *Documentation:* Enhancing docs or AI manifests +* *Testing:* Adding property-based tests or formal proofs +* *Bug reports:* Filing clear, reproducible issues -=== Reporting Bugs - -1. Check existing issues first -2. Use the bug report template -3. Include reproduction steps -4. Specify your environment (OS, language versions) - -=== Suggesting Features - -1. Open a discussion first -2. Explain the use case -3. Consider backward compatibility - -=== Pull Requests - -1. Fork the repository -2. Create a feature branch (`git checkout -b feature/amazing-feature`) -3. Make your changes -4. Run tests (`idris2 --build proven.ipkg`) -5. Commit with conventional commits (`feat:`, `fix:`, `docs:`, etc.) -6. Push and open a PR - -== Development Setup +=== Getting Started -[source,bash] ----- -# Clone -git clone https://github.com/hyperpolymath/proven -cd proven +[arabic] +. *Read the AI Manifest:* Start with `+0-AI-MANIFEST.a2ml+` (if present) +to understand the repository structure. +. *Environment:* Use `+nix develop+` or `+direnv allow+` to set up your +tools. +. *Task Runner:* Use `+just+` to see available commands +(`+just --list+`). -# Install Idris 2 (via pack) -pack install-deps +=== Development Workflow -# Install Zig -# See: https://ziglang.org/download/ +==== Branch Naming -# Environment fallback (Guix) -guix time-machine -C guix/channels.scm -- \ - shell -m guix/manifest.scm +.... +docs/short-description # Documentation +test/what-added # Test additions +feat/short-description # New features +fix/issue-number-description # Bug fixes +refactor/what-changed # Code improvements +security/what-fixed # Security fixes +.... -# Environment fallback (Nix) -nix-shell nix/shard.nix +==== Commit Messages -# Pin cadence (Guix + Nix) -# Monthly on the first Saturday, plus CVE-driven updates as needed: -./scripts/update-env-pins.sh +We follow https://www.conventionalcommits.org/[Conventional Commits]: -# Build -idris2 --build proven.ipkg +.... +(): -# Test -idris2 --build proven.ipkg ----- +[optional body] -== Code Style +[optional footer] +.... -=== Idris 2 +Types: `+feat+`, `+fix+`, `+docs+`, `+test+`, `+refactor+`, `+ci+`, +`+chore+`, `+security+` -- 2-space indentation -- Explicit type signatures for all exports -- Document with `|||` doc comments -- Prove properties where practical - -=== Zig - -- 4-space indentation -- Use `zig fmt` -- Prefer `comptime` verification -- Document public APIs - -=== Python Bindings - -- Use Black formatter -- Type hints required -- Docstrings for all public functions - -== Commit Messages +=== Reporting Bugs -Follow https://www.conventionalcommits.org/[Conventional Commits]: +Before reporting: 1. Search existing issues 2. Check if it’s already +fixed in `+main+` -- `feat:` New feature -- `fix:` Bug fix -- `docs:` Documentation only -- `style:` Formatting, no code change -- `refactor:` Code restructuring -- `test:` Adding tests -- `chore:` Maintenance +When reporting, include: - Clear, descriptive title - Environment +details (OS, versions, toolchain) - Steps to reproduce - Expected vs +actual behaviour -== Contact +=== Code of Conduct -Questions? Reach out at j.d.a.jewell@open.ac.uk or open a GitHub issue. +All contributors are expected to adhere to our +link:CODE_OF_CONDUCT.md[Code of Conduct]. -== License +=== License -By contributing, you agree that your contributions will be licensed under MPL-2.0. +By contributing, you agree that your contributions will be licensed +under the same license as the project (see LICENSE). diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md deleted file mode 100644 index 80ecdac8..00000000 --- a/CONTRIBUTING.md +++ /dev/null @@ -1,66 +0,0 @@ - -# Contributing - -Thank you for your interest in contributing! We follow a "Dual-Track" architecture where human-readable documentation lives in the root and machine-readable policies live in `.machine_readable/`. - -## How to Contribute - -We welcome contributions in many forms: - -- **Code:** Improving the core stack or extensions -- **Documentation:** Enhancing docs or AI manifests -- **Testing:** Adding property-based tests or formal proofs -- **Bug reports:** Filing clear, reproducible issues - -## Getting Started - -1. **Read the AI Manifest:** Start with `0-AI-MANIFEST.a2ml` (if present) to understand the repository structure. -2. **Environment:** Use `nix develop` or `direnv allow` to set up your tools. -3. **Task Runner:** Use `just` to see available commands (`just --list`). - -## Development Workflow - -### Branch Naming - -``` -docs/short-description # Documentation -test/what-added # Test additions -feat/short-description # New features -fix/issue-number-description # Bug fixes -refactor/what-changed # Code improvements -security/what-fixed # Security fixes -``` - -### Commit Messages - -We follow [Conventional Commits](https://www.conventionalcommits.org/): - -``` -(): - -[optional body] - -[optional footer] -``` - -Types: `feat`, `fix`, `docs`, `test`, `refactor`, `ci`, `chore`, `security` - -## Reporting Bugs - -Before reporting: -1. Search existing issues -2. Check if it's already fixed in `main` - -When reporting, include: -- Clear, descriptive title -- Environment details (OS, versions, toolchain) -- Steps to reproduce -- Expected vs actual behaviour - -## Code of Conduct - -All contributors are expected to adhere to our [Code of Conduct](CODE_OF_CONDUCT.md). - -## License - -By contributing, you agree that your contributions will be licensed under the same license as the project (see [LICENSE](LICENSE)). diff --git a/FFI-ARCHITECTURE-AUDIT-2026-01-30.adoc b/FFI-ARCHITECTURE-AUDIT-2026-01-30.adoc new file mode 100644 index 00000000..34138345 --- /dev/null +++ b/FFI-ARCHITECTURE-AUDIT-2026-01-30.adoc @@ -0,0 +1,425 @@ +== proven FFI Architecture Audit + +*Date:* 2026-01-30 *Auditor:* Claude (via user request) *Scope:* Verify +Idris2 ABI + Zig FFI compliance across all bindings + +''''' + +=== Executive Summary + +*Finding:* proven uses a *HYBRID architecture* - not pure Idris2-only as +originally assumed. + +* *14 Idris2-backed functions* (complex parsing/validation) +* *~141 Zig-native functions* (simple operations) +* *Total: 155 exported FFI functions* + +*Status:* Architecture is *pragmatic but not fully proven*. Only +functions calling Idris2 have formal verification guarantees. + +''''' + +=== Architecture Analysis + +==== Current Architecture (Hybrid) + +.... +┌─────────────────────────────────────────────────┐ +│ Language Bindings (Rust, Python, etc) │ +└────────────────────┬────────────────────────────┘ + │ + ┌───────────▼──────────┐ + │ Zig FFI (155 fns) │ + │ │ + │ 14 → Idris2 ✓ │ (proven safe) + │ 141 → Zig native ⚠ │ (not proven) + └──────────────────────┘ + │ + ┌───────────▼──────────┐ + │ Idris2 Core │ + │ (14 functions) │ + │ PROVEN SAFE ✓ │ + └──────────────────────┘ +.... + +==== Design Rationale (Inferred) + +The split appears intentional: + +*Idris2-backed (complex, proven):* - Path traversal detection - JSON +parsing/validation - URL parsing (RFC 3986) - Network address parsing +(IPv4/IPv6) + +*Zig-native (simple, unproven but safe):* - Math operations +(`+@addWithOverflow+`, `+@subWithOverflow+`) - Checksums (CRC32 via +`+std.hash.Crc32+`) - Geo calculations (Haversine formula) - String +formatting + +*Why?* - Zig’s builtin safety features (`+@addWithOverflow+`, +`+@mulWithOverflow+`) provide overflow detection - Standard library +functions (`+std.hash.Crc32+`) are well-tested - Performance: Avoiding +FFI crossing for trivial operations - Pragmatism: Not everything needs +formal proofs + +''''' + +=== Function Categorization + +==== Category 1: Idris2-Backed (PROVEN SAFE) ✓ + +These 14 functions call Idris2 via FFI and have formal verification: + +[width="100%",cols="31%,30%,14%,25%",options="header",] +|=== +|Export Function |Idris2 Function |Module |Proof Status +|`+proven_path_has_traversal+` |`+proven_idris_path_has_traversal+` +|SafePath |Proven + +|`+proven_path_sanitize_filename+` +|`+proven_idris_path_sanitize_filename+` |SafePath |Proven + +|`+proven_json_is_valid+` |`+proven_idris_json_is_valid+` |SafeJson +|Proven + +|`+proven_json_get_type+` |`+proven_idris_json_get_type+` |SafeJson +|Proven + +|`+proven_network_parse_ipv4+` |`+proven_idris_network_parse_ipv4+` +|SafeNetwork |Proven + +|`+proven_network_ipv4_is_private+` +|`+proven_idris_network_ipv4_is_private+` |SafeNetwork |Proven + +|`+proven_network_ipv4_is_loopback+` +|`+proven_idris_network_ipv4_is_loopback+` |SafeNetwork |Proven + +|`+proven_url_is_valid+` |`+proven_idris_url_is_valid+` |SafeUrl |Proven + +|`+proven_url_scheme+` |`+proven_idris_url_scheme+` |SafeUrl |Proven + +|`+proven_url_host+` |`+proven_idris_url_host+` |SafeUrl |Proven + +|`+proven_url_port+` |`+proven_idris_url_port+` |SafeUrl |Proven + +|`+proven_url_path+` |`+proven_idris_url_path+` |SafeUrl |Proven + +|`+proven_url_query+` |`+proven_idris_url_query+` |SafeUrl |Proven + +|`+proven_url_fragment+` |`+proven_idris_url_fragment+` |SafeUrl |Proven +|=== + +*Verification:* These functions have: - Dependent type proofs in Idris2 +- Totality checking (cannot hang or crash) - Property-based tests - +Formal verification via echidnabot + +==== Category 2: Zig-Native (SAFE BUT UNPROVEN) ⚠ + +~141 functions implemented directly in Zig using: - Zig builtins +(`+@addWithOverflow+`, `+@subWithOverflow+`, `+@mulWithOverflow+`) - Zig +standard library (`+std.hash.Crc32+`, `+std.math+`) - Manual bounds +checking + +*Examples:* + +===== SafeMath (Zig builtins - overflow safe) + +[source,zig] +---- +export fn proven_math_add_checked(a: i64, b: i64) IntResult { + const result = @addWithOverflow(a, b); + if (result[1] != 0) { + return .{ .status = .err_overflow, .value = 0 }; + } + return .{ .status = .ok, .value = result[0] }; +} +---- + +*Safety:* Zig’s `+@addWithOverflow+` detects overflow *Proof:* None +(relies on Zig compiler correctness) + +===== SafeChecksum (Zig stdlib) + +[source,zig] +---- +export fn proven_checksum_crc32(ptr: ?[*]const u8, len: usize) IntResult { + if (ptr == null) { + return .{ .status = .err_null_pointer, .value = 0 }; + } + const data = ptr.?[0..len]; + const crc = std.hash.Crc32.hash(data); // ← Zig stdlib + return .{ .status = .ok, .value = crc }; +} +---- + +*Safety:* Null check + bounds-safe slice *Proof:* None (relies on Zig +stdlib correctness) + +===== SafeGeo (manual implementation) + +[source,zig] +---- +export fn proven_geo_distance(a: GeoCoordinate, b: GeoCoordinate) FloatResult { + const EARTH_RADIUS: f64 = 6371000; // meters + const lat1 = a.latitude * std.math.pi / 180; + const lat2 = b.latitude * std.math.pi / 180; + // ... Haversine formula ... + return .{ .status = .ok, .value = EARTH_RADIUS * c }; +} +---- + +*Safety:* Bounds-checked float math *Proof:* None (manual +implementation) + +''''' + +=== Binding Compliance Audit + +==== Language Bindings → Zig FFI + +All checked bindings correctly call Zig FFI (no native safety logic): + +[width="100%",cols="25%,29%,29%,17%",options="header",] +|=== +|Language |FFI Method |Compliance |Notes +|Python |`+ctypes+` |✓ PASS |Calls `+lib.proven_*+` via ctypes + +|Rust |`+extern "C"+` |✓ PASS |`+#![forbid(unsafe_code)]+` in binding +layer + +|Deno/JS |`+Deno.dlopen+` |✓ PASS |FFI via Deno native + +|Go |`+cgo+` |✓ PASS |C bindings to Zig + +|ReScript |(compiled JS) |✓ PASS |Via JavaScript binding + +|Gleam |BEAM FFI |✓ PASS |NIFs call Zig + +|Elixir |NIFs |✓ PASS |Native Implemented Functions + +|Julia |`+ccall+` |✓ PASS |Direct C FFI + +|Haskell |FFI |✓ PASS |Foreign imports + +|OCaml |Ctypes |✓ PASS |OCaml ctypes library +|=== + +*Result:* All bindings are thin wrappers - NO native safety logic +reimplementation. + +==== Zig FFI → Idris2 + +*Issue:* Only 14/155 functions (9%) call Idris2. + +*Impact:* - 91% of operations lack formal verification - But: Zig +operations use safe patterns (overflow detection, null checks, stdlib) + +''''' + +=== Bidirectional FFI Status + +==== Question: Is FFI bidirectional (Idris2 ↔ Zig ↔ Languages)? + +*Current State:* *Partially unidirectional* + +*Direction 1:* Languages → Zig → Idris2 ✓ - Bindings call Zig functions +- 14 Zig functions call Idris2 - Works correctly + +*Direction 2:* Idris2 → Zig → Languages ❌ - *No evidence of callbacks +found* - Idris2 cannot call back into host language code - No function +pointer passing mechanisms observed + +*Ephapax Note:* The `+DYADIC-FFI-DESIGN.md+` document discusses +affine/linear types but doesn’t implement bidirectional FFI - it’s about +how Ephapax types map to the same Zig functions. + +*Recommendation:* Bidirectional FFI (callbacks) requires: 1. Zig +accepting function pointers from languages 2. Idris2 calling those +function pointers 3. Type-safe marshalling both ways + +This is a *v2.0 feature* per ROADMAP (SafeConcurrency, advanced FFI). + +''''' + +=== Security Implications + +==== What IS Formally Verified? + +*Verified (14 functions):* - Path traversal detection ✓ - JSON structure +validation ✓ - URL parsing (RFC compliance) ✓ - IPv4/IPv6 address +parsing ✓ + +*Critical operations with proofs:* - No directory traversal +(`+../../../etc/passwd+`) - No malformed JSON crashes - No URL injection +vulnerabilities - No invalid network addresses + +==== What is NOT Formally Verified? + +*Unverified (141 functions):* - Math overflow detection (relies on Zig +builtins) - Checksums (relies on Zig stdlib) - Float operations +(NaN/Infinity checks manual) - Geographic calculations (manual formulas) +- String operations (manual bounds checks) - Most data structure +operations + +*Still Safe Because:* - Zig enforces bounds checking by default - +Overflow builtins are compiler-verified - Standard library functions are +well-tested - Manual checks follow safe patterns + +*Risk Level:* *LOW-MEDIUM* - Not cryptographically proven - But: +industry-standard safe practices - Better than C/C++ (memory safe) - +Worse than pure Idris2 (no proofs) + +''''' + +=== Recommendations + +==== For v1.0 Release (Immediate) + +[arabic] +. *Update Documentation* +* README.adoc should clarify hybrid architecture +* State which modules are proven vs safe-but-unproven +* Update marketing: "`Mathematically proven for critical operations, +Zig-safe for performance`" +. *Add Verification Badges* ++ +[source,adoc] +---- +| Module | Verification Level | +|--------|-------------------| +| SafePath | ✓ Formally Proven (Idris2) | +| SafeJson | ✓ Formally Proven (Idris2) | +| SafeUrl | ✓ Formally Proven (Idris2) | +| SafeNetwork | ✓ Formally Proven (Idris2) | +| SafeMath | ⚠ Zig-Safe (overflow builtins) | +| SafeGeo | ⚠ Zig-Safe (manual checks) | +| SafeChecksum | ⚠ Zig-Safe (stdlib) | +---- +. *Create `+docs/VERIFICATION-STATUS.md+`* +* List all 155 functions with verification status +* Explain Zig safety features used +* Roadmap for migrating to Idris2 + +==== For v1.1 (3 months) + +[arabic, start=4] +. *Migrate High-Risk Functions to Idris2* +* *Priority 1:* SafeMath (overflow-critical) +* *Priority 2:* SafeFloat (NaN/Inf handling) +* *Priority 3:* SafeGeo (correctness-critical) +. *Benchmark FFI Overhead* +* Measure performance cost of Idris2 calls +* Document which operations justify Zig-native +* Justify hybrid architecture with data +. *Expand Idris2 Coverage* +* Goal: 50% of functions Idris2-backed +* Focus on security-critical operations +* Keep performance-critical in Zig (with documentation) + +==== For v2.0 (12 months) + +[arabic, start=7] +. *Bidirectional FFI* +* Implement callback mechanism (Idris2 → Zig → Language) +* Enable async operations from Idris2 +* Support concurrency primitives +. *Full Verification* (aspirational) +* 100% Idris2-backed (if performance allows) +* OR: Formally verify Zig layer (separate effort) +* OR: Accept hybrid as permanent (document thoroughly) + +''''' + +=== Comparison to Other Verified Libraries + +[width="100%",cols="20%,20%,28%,32%",options="header",] +|=== +|Library |Language |FFI Strategy |Verification % +|*proven (current)* |Idris2 + Zig |Hybrid |~9% (14/155 fns) + +|CompCert |Coq |OCaml extraction |100% (compiler) + +|Lean 4 stdlib |Lean 4 |Direct C FFI |~70% (estimated) + +|F* stdlib |F* |Extraction to C |~50% (estimated) + +|*proven (v1.1 target)* |Idris2 + Zig |Hybrid |~50% (75/155 fns) + +|*proven (v2.0 target)* |Idris2 + Zig |Mostly verified |~90% (140/155 +fns) +|=== + +''''' + +=== Conclusion + +*Architecture Status:* ✓ *ACCEPTABLE FOR V1* + +*Justification:* 1. Hybrid approach is *pragmatic* (performance + +verification) 2. Critical operations (parsing, injection prevention) +*ARE proven* 3. Zig-native operations use *safe patterns* (overflow +detection, bounds checks) 4. All bindings correctly *call Zig FFI* (no +logic duplication) + +*Required Actions Before v1:* - [ ] Update documentation to reflect +hybrid architecture - [ ] Create verification status badges/table - [ ] +Write `+docs/VERIFICATION-STATUS.md+` - [ ] Add migration roadmap to +ROADMAP.adoc (already done ✓) + +*Not Blocking v1:* - Migrating Zig functions to Idris2 (v1.1+) - +Bidirectional FFI (v2.0+) - Full verification (v2.0+) + +*Risk Assessment:* *LOW* - No memory unsafety - Critical operations +proven - Zig provides strong safety guarantees - Better than 99% of +libraries in the wild + +''''' + +=== Appendix: Full Function List + +==== Idris2-Backed Functions (14) + +*SafePath (2):* 1. `+proven_path_has_traversal+` → +`+proven_idris_path_has_traversal+` 2. `+proven_path_sanitize_filename+` +→ `+proven_idris_path_sanitize_filename+` + +*SafeJson (2):* 3. `+proven_json_is_valid+` → +`+proven_idris_json_is_valid+` 4. `+proven_json_get_type+` → +`+proven_idris_json_get_type+` + +*SafeNetwork (3):* 5. `+proven_network_parse_ipv4+` → +`+proven_idris_network_parse_ipv4+` 6. +`+proven_network_ipv4_is_private+` → +`+proven_idris_network_ipv4_is_private+` 7. +`+proven_network_ipv4_is_loopback+` → +`+proven_idris_network_ipv4_is_loopback+` + +*SafeUrl (7):* 8. `+proven_url_is_valid+` → +`+proven_idris_url_is_valid+` 9. `+proven_url_scheme+` → +`+proven_idris_url_scheme+` 10. `+proven_url_host+` → +`+proven_idris_url_host+` 11. `+proven_url_port+` → +`+proven_idris_url_port+` 12. `+proven_url_path+` → +`+proven_idris_url_path+` 13. `+proven_url_query+` → +`+proven_idris_url_query+` 14. `+proven_url_fragment+` → +`+proven_idris_url_fragment+` + +==== Zig-Native Functions (~141) + +_Full list available via:_ + +[source,bash] +---- +cd ~/Documents/hyperpolymath-repos/proven/ffi/zig/src +grep "^export fn proven" main.zig +---- + +*Categories:* - SafeMath: 10+ functions (add, sub, mul, div, mod, abs, +etc.) - SafeFloat: 8+ functions (NaN checks, division, comparisons) - +SafeChecksum: 6+ functions (CRC32, Adler32, FNV, Luhn) - SafeGeo: 5+ +functions (distance, bounds, validation) - SafeString: 15+ functions +(escape, sanitize, validate) - SafeCrypto: 4+ functions (constant-time +compare, random) - SafeBuffer: 10+ functions (ring buffer, bounded +writes) - … (and 80+ more) + +''''' + +_Audit completed: 2026-01-30_ _Next: Task #3 (a2ml integration)_ diff --git a/FFI-ARCHITECTURE-AUDIT-2026-01-30.md b/FFI-ARCHITECTURE-AUDIT-2026-01-30.md deleted file mode 100644 index ffb01284..00000000 --- a/FFI-ARCHITECTURE-AUDIT-2026-01-30.md +++ /dev/null @@ -1,394 +0,0 @@ -# proven FFI Architecture Audit -**Date:** 2026-01-30 -**Auditor:** Claude (via user request) -**Scope:** Verify Idris2 ABI + Zig FFI compliance across all bindings - ---- - -## Executive Summary - -**Finding:** proven uses a **HYBRID architecture** - not pure Idris2-only as originally assumed. - -- **14 Idris2-backed functions** (complex parsing/validation) -- **~141 Zig-native functions** (simple operations) -- **Total: 155 exported FFI functions** - -**Status:** Architecture is **pragmatic but not fully proven**. Only functions calling Idris2 have formal verification guarantees. - ---- - -## Architecture Analysis - -### Current Architecture (Hybrid) - -``` -┌─────────────────────────────────────────────────┐ -│ Language Bindings (Rust, Python, etc) │ -└────────────────────┬────────────────────────────┘ - │ - ┌───────────▼──────────┐ - │ Zig FFI (155 fns) │ - │ │ - │ 14 → Idris2 ✓ │ (proven safe) - │ 141 → Zig native ⚠ │ (not proven) - └──────────────────────┘ - │ - ┌───────────▼──────────┐ - │ Idris2 Core │ - │ (14 functions) │ - │ PROVEN SAFE ✓ │ - └──────────────────────┘ -``` - -### Design Rationale (Inferred) - -The split appears intentional: - -**Idris2-backed (complex, proven):** -- Path traversal detection -- JSON parsing/validation -- URL parsing (RFC 3986) -- Network address parsing (IPv4/IPv6) - -**Zig-native (simple, unproven but safe):** -- Math operations (`@addWithOverflow`, `@subWithOverflow`) -- Checksums (CRC32 via `std.hash.Crc32`) -- Geo calculations (Haversine formula) -- String formatting - -**Why?** -- Zig's builtin safety features (`@addWithOverflow`, `@mulWithOverflow`) provide overflow detection -- Standard library functions (`std.hash.Crc32`) are well-tested -- Performance: Avoiding FFI crossing for trivial operations -- Pragmatism: Not everything needs formal proofs - ---- - -## Function Categorization - -### Category 1: Idris2-Backed (PROVEN SAFE) ✓ - -These 14 functions call Idris2 via FFI and have formal verification: - -| Export Function | Idris2 Function | Module | Proof Status | -|-----------------|-----------------|--------|--------------| -| `proven_path_has_traversal` | `proven_idris_path_has_traversal` | SafePath | Proven | -| `proven_path_sanitize_filename` | `proven_idris_path_sanitize_filename` | SafePath | Proven | -| `proven_json_is_valid` | `proven_idris_json_is_valid` | SafeJson | Proven | -| `proven_json_get_type` | `proven_idris_json_get_type` | SafeJson | Proven | -| `proven_network_parse_ipv4` | `proven_idris_network_parse_ipv4` | SafeNetwork | Proven | -| `proven_network_ipv4_is_private` | `proven_idris_network_ipv4_is_private` | SafeNetwork | Proven | -| `proven_network_ipv4_is_loopback` | `proven_idris_network_ipv4_is_loopback` | SafeNetwork | Proven | -| `proven_url_is_valid` | `proven_idris_url_is_valid` | SafeUrl | Proven | -| `proven_url_scheme` | `proven_idris_url_scheme` | SafeUrl | Proven | -| `proven_url_host` | `proven_idris_url_host` | SafeUrl | Proven | -| `proven_url_port` | `proven_idris_url_port` | SafeUrl | Proven | -| `proven_url_path` | `proven_idris_url_path` | SafeUrl | Proven | -| `proven_url_query` | `proven_idris_url_query` | SafeUrl | Proven | -| `proven_url_fragment` | `proven_idris_url_fragment` | SafeUrl | Proven | - -**Verification:** These functions have: -- Dependent type proofs in Idris2 -- Totality checking (cannot hang or crash) -- Property-based tests -- Formal verification via echidnabot - -### Category 2: Zig-Native (SAFE BUT UNPROVEN) ⚠ - -~141 functions implemented directly in Zig using: -- Zig builtins (`@addWithOverflow`, `@subWithOverflow`, `@mulWithOverflow`) -- Zig standard library (`std.hash.Crc32`, `std.math`) -- Manual bounds checking - -**Examples:** - -#### SafeMath (Zig builtins - overflow safe) -```zig -export fn proven_math_add_checked(a: i64, b: i64) IntResult { - const result = @addWithOverflow(a, b); - if (result[1] != 0) { - return .{ .status = .err_overflow, .value = 0 }; - } - return .{ .status = .ok, .value = result[0] }; -} -``` -**Safety:** Zig's `@addWithOverflow` detects overflow -**Proof:** None (relies on Zig compiler correctness) - -#### SafeChecksum (Zig stdlib) -```zig -export fn proven_checksum_crc32(ptr: ?[*]const u8, len: usize) IntResult { - if (ptr == null) { - return .{ .status = .err_null_pointer, .value = 0 }; - } - const data = ptr.?[0..len]; - const crc = std.hash.Crc32.hash(data); // ← Zig stdlib - return .{ .status = .ok, .value = crc }; -} -``` -**Safety:** Null check + bounds-safe slice -**Proof:** None (relies on Zig stdlib correctness) - -#### SafeGeo (manual implementation) -```zig -export fn proven_geo_distance(a: GeoCoordinate, b: GeoCoordinate) FloatResult { - const EARTH_RADIUS: f64 = 6371000; // meters - const lat1 = a.latitude * std.math.pi / 180; - const lat2 = b.latitude * std.math.pi / 180; - // ... Haversine formula ... - return .{ .status = .ok, .value = EARTH_RADIUS * c }; -} -``` -**Safety:** Bounds-checked float math -**Proof:** None (manual implementation) - ---- - -## Binding Compliance Audit - -### Language Bindings → Zig FFI - -All checked bindings correctly call Zig FFI (no native safety logic): - -| Language | FFI Method | Compliance | Notes | -|----------|------------|------------|-------| -| Python | `ctypes` | ✓ PASS | Calls `lib.proven_*` via ctypes | -| Rust | `extern "C"` | ✓ PASS | `#![forbid(unsafe_code)]` in binding layer | -| Deno/JS | `Deno.dlopen` | ✓ PASS | FFI via Deno native | -| Go | `cgo` | ✓ PASS | C bindings to Zig | -| ReScript | (compiled JS) | ✓ PASS | Via JavaScript binding | -| Gleam | BEAM FFI | ✓ PASS | NIFs call Zig | -| Elixir | NIFs | ✓ PASS | Native Implemented Functions | -| Julia | `ccall` | ✓ PASS | Direct C FFI | -| Haskell | FFI | ✓ PASS | Foreign imports | -| OCaml | Ctypes | ✓ PASS | OCaml ctypes library | - -**Result:** All bindings are thin wrappers - NO native safety logic reimplementation. - -### Zig FFI → Idris2 - -**Issue:** Only 14/155 functions (9%) call Idris2. - -**Impact:** -- 91% of operations lack formal verification -- But: Zig operations use safe patterns (overflow detection, null checks, stdlib) - ---- - -## Bidirectional FFI Status - -### Question: Is FFI bidirectional (Idris2 ↔ Zig ↔ Languages)? - -**Current State:** **Partially unidirectional** - -**Direction 1:** Languages → Zig → Idris2 ✓ -- Bindings call Zig functions -- 14 Zig functions call Idris2 -- Works correctly - -**Direction 2:** Idris2 → Zig → Languages ❌ -- **No evidence of callbacks found** -- Idris2 cannot call back into host language code -- No function pointer passing mechanisms observed - -**Ephapax Note:** -The `DYADIC-FFI-DESIGN.md` document discusses affine/linear types but doesn't implement bidirectional FFI - it's about how Ephapax types map to the same Zig functions. - -**Recommendation:** -Bidirectional FFI (callbacks) requires: -1. Zig accepting function pointers from languages -2. Idris2 calling those function pointers -3. Type-safe marshalling both ways - -This is a **v2.0 feature** per ROADMAP (SafeConcurrency, advanced FFI). - ---- - -## Security Implications - -### What IS Formally Verified? - -**Verified (14 functions):** -- Path traversal detection ✓ -- JSON structure validation ✓ -- URL parsing (RFC compliance) ✓ -- IPv4/IPv6 address parsing ✓ - -**Critical operations with proofs:** -- No directory traversal (`../../../etc/passwd`) -- No malformed JSON crashes -- No URL injection vulnerabilities -- No invalid network addresses - -### What is NOT Formally Verified? - -**Unverified (141 functions):** -- Math overflow detection (relies on Zig builtins) -- Checksums (relies on Zig stdlib) -- Float operations (NaN/Infinity checks manual) -- Geographic calculations (manual formulas) -- String operations (manual bounds checks) -- Most data structure operations - -**Still Safe Because:** -- Zig enforces bounds checking by default -- Overflow builtins are compiler-verified -- Standard library functions are well-tested -- Manual checks follow safe patterns - -**Risk Level:** **LOW-MEDIUM** -- Not cryptographically proven -- But: industry-standard safe practices -- Better than C/C++ (memory safe) -- Worse than pure Idris2 (no proofs) - ---- - -## Recommendations - -### For v1.0 Release (Immediate) - -1. **Update Documentation** - - README.adoc should clarify hybrid architecture - - State which modules are proven vs safe-but-unproven - - Update marketing: "Mathematically proven for critical operations, Zig-safe for performance" - -2. **Add Verification Badges** - ```adoc - | Module | Verification Level | - |--------|-------------------| - | SafePath | ✓ Formally Proven (Idris2) | - | SafeJson | ✓ Formally Proven (Idris2) | - | SafeUrl | ✓ Formally Proven (Idris2) | - | SafeNetwork | ✓ Formally Proven (Idris2) | - | SafeMath | ⚠ Zig-Safe (overflow builtins) | - | SafeGeo | ⚠ Zig-Safe (manual checks) | - | SafeChecksum | ⚠ Zig-Safe (stdlib) | - ``` - -3. **Create `docs/VERIFICATION-STATUS.md`** - - List all 155 functions with verification status - - Explain Zig safety features used - - Roadmap for migrating to Idris2 - -### For v1.1 (3 months) - -4. **Migrate High-Risk Functions to Idris2** - - **Priority 1:** SafeMath (overflow-critical) - - **Priority 2:** SafeFloat (NaN/Inf handling) - - **Priority 3:** SafeGeo (correctness-critical) - -5. **Benchmark FFI Overhead** - - Measure performance cost of Idris2 calls - - Document which operations justify Zig-native - - Justify hybrid architecture with data - -6. **Expand Idris2 Coverage** - - Goal: 50% of functions Idris2-backed - - Focus on security-critical operations - - Keep performance-critical in Zig (with documentation) - -### For v2.0 (12 months) - -7. **Bidirectional FFI** - - Implement callback mechanism (Idris2 → Zig → Language) - - Enable async operations from Idris2 - - Support concurrency primitives - -8. **Full Verification** (aspirational) - - 100% Idris2-backed (if performance allows) - - OR: Formally verify Zig layer (separate effort) - - OR: Accept hybrid as permanent (document thoroughly) - ---- - -## Comparison to Other Verified Libraries - -| Library | Language | FFI Strategy | Verification % | -|---------|----------|--------------|----------------| -| **proven (current)** | Idris2 + Zig | Hybrid | ~9% (14/155 fns) | -| CompCert | Coq | OCaml extraction | 100% (compiler) | -| Lean 4 stdlib | Lean 4 | Direct C FFI | ~70% (estimated) | -| F* stdlib | F* | Extraction to C | ~50% (estimated) | -| **proven (v1.1 target)** | Idris2 + Zig | Hybrid | ~50% (75/155 fns) | -| **proven (v2.0 target)** | Idris2 + Zig | Mostly verified | ~90% (140/155 fns) | - ---- - -## Conclusion - -**Architecture Status:** ✓ **ACCEPTABLE FOR V1** - -**Justification:** -1. Hybrid approach is **pragmatic** (performance + verification) -2. Critical operations (parsing, injection prevention) **ARE proven** -3. Zig-native operations use **safe patterns** (overflow detection, bounds checks) -4. All bindings correctly **call Zig FFI** (no logic duplication) - -**Required Actions Before v1:** -- [ ] Update documentation to reflect hybrid architecture -- [ ] Create verification status badges/table -- [ ] Write `docs/VERIFICATION-STATUS.md` -- [ ] Add migration roadmap to ROADMAP.adoc (already done ✓) - -**Not Blocking v1:** -- Migrating Zig functions to Idris2 (v1.1+) -- Bidirectional FFI (v2.0+) -- Full verification (v2.0+) - -**Risk Assessment:** **LOW** -- No memory unsafety -- Critical operations proven -- Zig provides strong safety guarantees -- Better than 99% of libraries in the wild - ---- - -## Appendix: Full Function List - -### Idris2-Backed Functions (14) - -**SafePath (2):** -1. `proven_path_has_traversal` → `proven_idris_path_has_traversal` -2. `proven_path_sanitize_filename` → `proven_idris_path_sanitize_filename` - -**SafeJson (2):** -3. `proven_json_is_valid` → `proven_idris_json_is_valid` -4. `proven_json_get_type` → `proven_idris_json_get_type` - -**SafeNetwork (3):** -5. `proven_network_parse_ipv4` → `proven_idris_network_parse_ipv4` -6. `proven_network_ipv4_is_private` → `proven_idris_network_ipv4_is_private` -7. `proven_network_ipv4_is_loopback` → `proven_idris_network_ipv4_is_loopback` - -**SafeUrl (7):** -8. `proven_url_is_valid` → `proven_idris_url_is_valid` -9. `proven_url_scheme` → `proven_idris_url_scheme` -10. `proven_url_host` → `proven_idris_url_host` -11. `proven_url_port` → `proven_idris_url_port` -12. `proven_url_path` → `proven_idris_url_path` -13. `proven_url_query` → `proven_idris_url_query` -14. `proven_url_fragment` → `proven_idris_url_fragment` - -### Zig-Native Functions (~141) - -*Full list available via:* -```bash -cd ~/Documents/hyperpolymath-repos/proven/ffi/zig/src -grep "^export fn proven" main.zig -``` - -**Categories:** -- SafeMath: 10+ functions (add, sub, mul, div, mod, abs, etc.) -- SafeFloat: 8+ functions (NaN checks, division, comparisons) -- SafeChecksum: 6+ functions (CRC32, Adler32, FNV, Luhn) -- SafeGeo: 5+ functions (distance, bounds, validation) -- SafeString: 15+ functions (escape, sanitize, validate) -- SafeCrypto: 4+ functions (constant-time compare, random) -- SafeBuffer: 10+ functions (ring buffer, bounded writes) -- ... (and 80+ more) - ---- - -_Audit completed: 2026-01-30_ -_Next: Task #3 (a2ml integration)_ diff --git a/GOVERNANCE.adoc b/GOVERNANCE.adoc new file mode 100644 index 00000000..9b836fb2 --- /dev/null +++ b/GOVERNANCE.adoc @@ -0,0 +1,60 @@ +== Governance + +=== Overview + +This project is governed by the following principles and structures to +ensure transparent, inclusive, and effective decision-making. + +=== Roles and Responsibilities + +==== Maintainers + +Maintainers are responsible for: - Reviewing and merging pull requests - +Managing releases and versioning - Ensuring code quality and standards - +Triaging issues and bug reports - Community engagement and support + +==== Contributors + +Contributors are expected to: - Follow the code of conduct - Submit +well-documented pull requests - Write tests for new functionality - +Maintain existing tests - Update documentation as needed + +=== Decision Making + +==== Minor Changes + +* Can be made by any maintainer +* Include bug fixes, documentation updates, dependency updates + +==== Major Changes + +* Require discussion in issues or pull requests +* Include new features, architectural changes, API changes +* Need approval from at least 2 maintainers + +==== Breaking Changes + +* Require RFC (Request for Comments) process +* Need approval from majority of maintainers +* Must include migration guide + +=== Code of Conduct + +All participants are expected to follow our Code of Conduct. Violations +can be reported to the maintainers. + +=== Communication + +* *Issues*: For bug reports and feature requests +* *Discussions*: For questions and general discussion +* *Pull Requests*: For code contributions + +=== Licensing + +All contributions are made under the terms of the repository’s LICENSE +file. By submitting a pull request, you agree to license your +contributions accordingly. + +''''' + +_Last updated: 2026-07-18_ diff --git a/GOVERNANCE.md b/GOVERNANCE.md deleted file mode 100644 index e27364c7..00000000 --- a/GOVERNANCE.md +++ /dev/null @@ -1,60 +0,0 @@ -# Governance - -## Overview - -This project is governed by the following principles and structures to ensure transparent, inclusive, and effective decision-making. - -## Roles and Responsibilities - -### Maintainers - -Maintainers are responsible for: -- Reviewing and merging pull requests -- Managing releases and versioning -- Ensuring code quality and standards -- Triaging issues and bug reports -- Community engagement and support - -### Contributors - -Contributors are expected to: -- Follow the code of conduct -- Submit well-documented pull requests -- Write tests for new functionality -- Maintain existing tests -- Update documentation as needed - -## Decision Making - -### Minor Changes -- Can be made by any maintainer -- Include bug fixes, documentation updates, dependency updates - -### Major Changes -- Require discussion in issues or pull requests -- Include new features, architectural changes, API changes -- Need approval from at least 2 maintainers - -### Breaking Changes -- Require RFC (Request for Comments) process -- Need approval from majority of maintainers -- Must include migration guide - -## Code of Conduct - -All participants are expected to follow our Code of Conduct. Violations can be reported to the maintainers. - -## Communication - -- **Issues**: For bug reports and feature requests -- **Discussions**: For questions and general discussion -- **Pull Requests**: For code contributions - -## Licensing - -All contributions are made under the terms of the repository's LICENSE file. -By submitting a pull request, you agree to license your contributions accordingly. - ---- - -*Last updated: 2026-07-18* diff --git a/PROJECT-ISSUES.adoc b/PROJECT-ISSUES.adoc new file mode 100644 index 00000000..2d99cb7c --- /dev/null +++ b/PROJECT-ISSUES.adoc @@ -0,0 +1,61 @@ +== Project Issues Kanban Board + +*Repository:* hyperpolymath/proven + +*Last Updated:* 2026-06-03 + +*Purpose:* Track GitHub issues through kanban workflow columns + +=== Kanban Columns + +[width="99%",cols="14%,8%,10%,12%,12%,10%,14%,12%,8%",options="header",] +|=== +|Issue |Title |Status |Priority |Assignee |Labels |Blocked By |Due Date +|Notes +|#80 |SafePassword.Strength windows/detectPatterns/analyzeStrength +covering→total refactor |In Progress |High |- |proof-debt, Phase-3 |- |- +|Discharges 3 OWED sites: strengthScoreBounded, higherImpliesLower, +veryStrongSatisfiesAll + +|#83 |- |Backlog |High |- |proof-debt, Phase-3 |- |- |Phase 3 unblocking +|=== + +=== Column Definitions + +* *Backlog* - Issues not yet started, awaiting triage or resources +* *In Progress* - Actively being worked on +* *Blocked* - Waiting on dependencies or external factors + +* *Review* - Ready for review, testing, or approval +* *Done* - Completed and verified + +=== Status Transitions + +.... +Backlog → In Progress → Review → Done + ↓ + Blocked (when dependencies arise) +.... + +=== Issue #80 Details + +*Related Files:* - `+src/Proven/SafePassword/Proofs.idr+` (lines +257-258, 454-458, 471-474) - `+src/Proven/SafePassword.idr+` (strength +analysis functions) + +*Tasks:* 1. Refactor `+windows+` function from covering to total 2. +Refactor `+detectPatterns+` function from covering to total + +3. Refactor `+analyzeStrength+` function from covering to total 4. +Discharge `+strengthScoreBounded+` theorem 5. Discharge +`+higherImpliesLower+` theorem 6. Discharge `+veryStrongSatisfiesAll+` +theorem + +*Estimated Effort:* ~2-3 hours + +*Dependencies:* None identified + +=== Issue #83 Details + +*Status:* Backlog - needs investigation to determine specific scope + +''''' + +_Auto-generated from GitHub issues - sync with +hyperpolymath/proven/issues_ diff --git a/PROJECT-ISSUES.md b/PROJECT-ISSUES.md deleted file mode 100644 index eb7b59d3..00000000 --- a/PROJECT-ISSUES.md +++ /dev/null @@ -1,58 +0,0 @@ - -# Project Issues Kanban Board - -**Repository:** hyperpolymath/proven -**Last Updated:** 2026-06-03 -**Purpose:** Track GitHub issues through kanban workflow columns - -## Kanban Columns - -| Issue | Title | Status | Priority | Assignee | Labels | Blocked By | Due Date | Notes | -|-------|-------|--------|----------|----------|--------|-----------|----------|-------| -| #80 | SafePassword.Strength windows/detectPatterns/analyzeStrength covering→total refactor | In Progress | High | - | proof-debt, Phase-3 | - | - | Discharges 3 OWED sites: strengthScoreBounded, higherImpliesLower, veryStrongSatisfiesAll | -| #83 | - | Backlog | High | - | proof-debt, Phase-3 | - | - | Phase 3 unblocking | - -## Column Definitions - -- **Backlog** - Issues not yet started, awaiting triage or resources -- **In Progress** - Actively being worked on -- **Blocked** - Waiting on dependencies or external factors -- **Review** - Ready for review, testing, or approval -- **Done** - Completed and verified - -## Status Transitions - -``` -Backlog → In Progress → Review → Done - ↓ - Blocked (when dependencies arise) -``` - -## Issue #80 Details - -**Related Files:** -- `src/Proven/SafePassword/Proofs.idr` (lines 257-258, 454-458, 471-474) -- `src/Proven/SafePassword.idr` (strength analysis functions) - -**Tasks:** -1. Refactor `windows` function from covering to total -2. Refactor `detectPatterns` function from covering to total -3. Refactor `analyzeStrength` function from covering to total -4. Discharge `strengthScoreBounded` theorem -5. Discharge `higherImpliesLower` theorem -6. Discharge `veryStrongSatisfiesAll` theorem - -**Estimated Effort:** ~2-3 hours - -**Dependencies:** None identified - -## Issue #83 Details - -**Status:** Backlog - needs investigation to determine specific scope - ---- - -*Auto-generated from GitHub issues - sync with hyperpolymath/proven/issues* diff --git a/PROOF-NEEDS.adoc b/PROOF-NEEDS.adoc new file mode 100644 index 00000000..facb0094 --- /dev/null +++ b/PROOF-NEEDS.adoc @@ -0,0 +1,368 @@ +== PROOF-NEEDS.md — proven + +____ +*Re-audit update 2026-05-20.* The 39 "`carried forward, pending +Proofs.idr re-audit`" directories have now been re-audited +(hyperpolymath/standards#158 Fork A). Net finding: of those 39, *6 are +clean* (no bodyless decls; SafeChecksum, SafeBuffer, SafeBloom, +SafeCryptoAccel, SafeHKDF, SafeFPGA — these set the OWED convention), +*28 carried bodyless decls* which have now been annotated as +`+||| OWED:+` + `+0+` erased-multiplicity per the convention (PRs +hyperpolymath/proven#37-64, all DRAFT pending estate CI clear; 8 locally +idris2-0.8.0-verified). Total: 250 bodyless decls surfaced and made +discoverable. See the new "`OWED-with-justification convention`" section +below and `+.machine_readable/6a2/META.a2ml+` ADR-001. + +*Honesty refresh 2026-05-18.* `+proven+` is the estate trust root, so +this ledger must not overclaim. The prior edition asserted "`the most +complete Idris2 proof infrastructure in the ecosystem`" and "`~30 +modules lack proof files`". Adversarial re-audit shows that framing was +*dishonest by a wide margin*: the real gap is ~3x larger, and dozens of +modules ship verbatim security claims in their doc headers +(`+Prevents:+`, `+formally verified+`, `+guaranteed+`, `+cannot crash+`) +with *zero discharged theorem*. This is the same silent-overclaim class +the sibling security audit found in SafeMCP / SafeOAuth / SafeWebAuthn / +SafeWebhook / SafeCapability / SafeAttestation / SafeJWK / +SafeSecretShare / SafeWebSocket. This edition enumerates the debt +honestly. Refs hyperpolymath/standards#124. +____ + +=== Definitions (what counts) + +* *Proven* = a discharged theorem exists: either +`+src/Proven//Proofs.idr+` states and discharges ≥1 theorem (`+Refl+` +/ `+with+`-block / `+impossible+` / `+absurd+` / decidable `+Dec+`), +*or* the single-file `+src/Proven/.idr+` contains such a discharged +theorem inline. +* *Witness-only (OVERCLAIM, marked ✗)* = the module ships a +`+Prevents:+` / `+Guarantees:+` / `+formally verified+` / +`+cannot crash+` / `+no false negative+` doc claim (or `+data ...Proof+` +witness types / proof-carrying record _fields_) but *no discharged +theorem*. A `+data+` witness type is an _obligation a constructor must +satisfy_, not a discharged proof. A record field of proof type is an +obligation pushed onto the caller, not discharged here. A signature with +`+@ Assumed:+` in its doc is an explicit postulate. +* *Claim-free* = the module makes no safety/security promise; absence of +proof is honest. +* ABI/FFI wrappers (`+src/Proven/FFI/Safe*.idr+`, `+bindings/...+`) +inherit, never independently establish, proofs. + +=== Honest counts + +[width="100%",cols="54%,>46%",options="header",] +|=== +|Bucket |Count +|Safe* directories with a companion `+Proofs.idr+` |41 + +|— of which are *stubs* (header only, no theorem) → treat as UNPROVEN +|*0* (`+SafeCommand+` discharged by proven#21; `+SafeDateTime+` +discharged on branch `+proof-debt/standards-132-safedatetime-stub+`) + +|Single-file `+src/Proven/Safe*.idr+` modules (no `+Proofs.idr+` dir) +|133 + +|— single-file modules shipping a safety/security *doc claim* |*76* +|=== + +=== Security-critical decidable proofs landed (2026-05-18) + +The following security-critical modules previously shipped only inline +witness _types_ (`+StrongKey+`, `+StrongCert+`, …) with a "`Prevents: +…`" module-doc claim but *no discharged theorem* that the predicate +rejects the bad case. Real verified `+Proofs.idr+` files now exist +(`+idris2 --check+` exit 0), modelled on `+SafeCommand/Proofs.idr+`: + +* *SafeSSH/Proofs.idr* — exhaustive per-constructor `+Refl+`: DSA +provably rejected by `+isWeakAlgorithm+`/`+validateKey+`; `+StrongKey+` +uninhabitable for DSA. Pure enum, *no bridge axiom*. +* *SafeCert/Proofs.idr* — SHA-1 provably weak, RSA-2048 provably not +strong; `+StrongCert+` uninhabitable for SHA-1 / RSA-2048; temporal +expiry guard. Pure enum, *no bridge axiom*. +* *SafePromptInjection/Proofs.idr* — per-char delimiter-escape soundness ++ canonical attack-vector detection + `+RejectUnsafe+` enforcement; one +explicit named erased string bridge axiom only. + +Still owed (single-file, witness-type-only, "`Prevents`" doc claim +undischarged — honest ledger): SafeMCP, SafeOAuth, SafeWebAuthn, +SafeWebSocket, SafeWebhook, SafeCapability, SafeJWK, SafeSecretShare. + +`+SafeAttestation+` was previously listed here but its companion +`+src/Proven/SafeAttestation/Proofs.idr+` (189 ln, `+idris2 --check+` +exit 0, "`Zero `+believe_me+` / `+idris_crash+`, zero OWED`") discharges +enum self-equality, parser rejection of unknown / weak algorithms, and +record-projection anchors. The remaining hash-recomputation claim is +OWED at the FFI seam (the actual hash computation happens in +`+Proven.SafeCrypto+`); the module header was softened in proven#76 to +reflect this. It therefore belongs in the "`Security-critical decidable +proofs landed`" section above, not in this still-owed list. + +Companion `+Proofs.idr+` content was _not_ re-verified +theorem-by-theorem in this pass; the 39 non-stub directories are carried +forward as claimed-proven pending a separate Proofs.idr discharge audit. +*The single-file population is the exposure this ledger now owns +honestly.* + +==== Adversarial sample of the single-file non-security population + +32 modules sampled across the alphabet (SafeAngle…SafeUnit), excluding +the 12 modules the sibling security audit handled and +SafeCommand/SafeDateTime: + +[cols=",>,>",options="header",] +|=== +|Outcome |Count |Fraction +|Genuinely proven (≥1 inline discharged theorem) |*1* |~3% +|Witness-only / doc-claim *OVERCLAIM (✗)* |*13* |~41% +|Claim-free (honest silence) |*18* |~56% +|=== + +* Only *SafeOrdering* carries a real discharged theorem +(`+seqTotalOrder+`, `+with+`-block, `+Refl+`, `+%default total+`). Its +`+PartialOrder+`/`+TotalOrder+` records additionally push proof _fields_ +onto callers (obligations, not discharged here). +* *Extrapolation:* with 76/133 single-file modules carrying a +safety/security doc claim and the sampled proven-rate ≈3%, on the order +of *~70 single-file modules are silent overclaims* — the prior ledger +acknowledged ~30 _gaps_ total and never used the word "`overclaim`". +Ledger accuracy verdict: the origin/main edition was *dishonest — it +understated the unproven gap by roughly 3x and entirely omitted the +doc-claim-without-theorem class.* + +=== Owed — single-file modules shipping a security claim with NO theorem (✗) + +Each line: module — verbatim doc claim — why it is owed. + +==== Security-relevant overclaims (highest concern) + +* *SafeDigest* ✗ — `+src/Proven/SafeDigest.idr:5+` _"`formally verified +digest parsing, validation, and constant-time comparison to prevent +timing attacks.`"_ — WORST OFFENDER: claims "`formally verified`" yet +`+constantTimeReflexive+`/`+constantTimeSymmetric+`/`+verifyTransitive+` +(`+:225–248+`) are *explicitly undischarged* (`+@ Assumed:+` in their +own docstrings). Crypto material + actively false "`verified`" word. +* *SafeArchive* ✗ — `+src/Proven/SafeArchive.idr:5+` _"`Provides +type-safe archive member validation that prevents: Zip Slip / Symlink +attacks / Zip bombs / Path injection via null bytes.`"_ — +`+hasPathTraversal+` is a `+Bool+` predicate with no theorem that a +non-flagged entry is traversal-free. Supply- chain / unpack surface. +* *SafeCBOR* ✗ — `+src/Proven/SafeCBOR.idr:7+` _"`Prevents: integer +overflow, indefinite-length bombs, tag confusion.`"_ — used in +COSE/WebAuthn/FIDO2 per its own header; no overflow/bomb/tag theorem. +Authn-adjacent. +* *SafeConsensus* ✗ — `+src/Proven/SafeConsensus.idr:9+` _"`Log +replication with consistency guarantees`"_ — no safety/agreement +theorem. Distributed-state surface. +* *SafeLRU* ✗ — `+src/Proven/SafeLRU.idr:6+` _"`that cannot overflow or +corrupt cache state.`"_ — no invariant theorem; cache-poisoning +adjacency. + +==== Lower-blast-radius overclaims (still owed) + +* *SafeBloom* ✗ — `+:6+` _"`with guaranteed no false negatives.`"_ (the +entire correctness contract of a Bloom filter; undischarged) +* *SafeMatrix* ✗ — `+:3+` _"`matrix operations that cannot crash`"_ +* *SafeSet* ✗ — `+:3+` _"`set operations that cannot crash`"_ +* *SafeRational* ✗ — `+:6+` _"`safe operations that cannot crash.`"_ +* *SafeGraph* ✗ — `+:197+` _"`Bounded by fuel … to guarantee +termination`"_ (asserted, not proven) +* *SafeProbability* ✗ — `+:20+` _"`A probability value guaranteed to be +in [0, 1]`"_ (no smart-constructor refinement proof; `+impossible+` is a +_value_ named impossible, not a proof) +* *SafeTree* ✗ — `+:6+` traversal/manipulation safety, no theorem +* *SafeDecimal* ✗ — `+:54+` "`Structurally decreasing on scale ensures +totality`" (relies on `+%default total+` only; no stated lemma) + +==== Long-tail enumeration (Phase 2 Days 19-21, partial — 2026-05-27) + +Adversarial-grep + per-module inspection of the long tail surfaced *6 +additional confirmed single-file overclaims* beyond the 13 enumerated +above. Headers softened in proven#76 (follow-on commit; see +`+docs/proof-debt-triage.md+` §11): + +* *SafeSupplyChain* ✗ HIGH — `+src/Proven/SafeSupplyChain.idr:7+` +_"`Prevents: tampered builds, unattested artifacts, provenance +forgery.`"_ — SLSA attestation layer; tamper-prevention theorems OWED. +Real-world supply-chain blast radius. +* *SafePBKDF2* ✗ HIGH — `+src/Proven/SafePBKDF2.idr:6+` _"`Prevents: low +iteration counts, short salts, weak derived key lengths.`"_ — crypto +parameter validator; prevention theorems OWED. +* *SafeProvenance* ✗ MEDIUM — `+src/Proven/SafeProvenance.idr:4+` +_"`Formally verified change tracking and audit trails`"_ — provides +records + predicates only; integrity/lineage theorems OWED. +* *SafePolicy* ✗ MEDIUM — `+src/Proven/SafePolicy.idr:4+` _"`Formally +verified policy enforcement`"_ — AST-level predicates only; enforcement +soundness theorems OWED. +* *SafeRegistry* ✗ MEDIUM — `+src/Proven/SafeRegistry.idr:5-6+` +_"`formally verified parsing of OCI/Docker image references with +guarantees of correctness and termination`"_ — parsers via Bool +validators; correctness/termination theorems OWED. +* *SafeSchema* ✗ MEDIUM — `+src/Proven/SafeSchema.idr:4-10+` _"`Formally +verified schema evolution`"_ + _"`compatibility proofs`"_ + +_"`correctness guarantees`"_ — type definitions + migration scaffolding; +compatibility/correctness theorems OWED. + +*Confirmed actually-proven* (re-classification, NOT in the OWED list): + +* *SafeTrust* — `+src/Proven/SafeTrust.idr:296-303+` +`+satisfiesMonotone+` discharged via exhaustive pattern-match + `+Refl+` +per arm. The header’s "`Formally verified`" / "`proven monotone`" claim +IS valid. +* *SafeOrdering* — already noted in §"`Adversarial sample`" as the only +single-file module in that sample with a discharged theorem. + +==== Long-tail still extrapolated + +The Phase 2 Days 19-21 audit surfaced 6 new instances but did NOT +exhaustively read all 76 claim-bearing single-file modules (Bash +permission boundaries on the audit sub-agent prevented full +enumeration). On the order of *~54 single-file modules remain +extrapolated* as overclaim candidates per the original 3% +genuinely-proven sample rate. A full enumeration is mechanical: every +`+src/Proven/Safe*.idr+` matching +`+Prevents:|formally verified|cannot crash|guarantee|no false negative| traversal|injection|attack+` +without a discharged theorem (no `+Refl+` / `+Dec+` / `+impossible+` / +`+absurd+` / `+with+` block) is OWED. Phase 3 may batch-process these +via the three cross-cutting overclaim patterns identified in the Days +19-21 audit: + +[arabic] +. *"`Formally verified`" without evidence* — batch-fix template: +`+formally verified X+` → +`+X via Bool predicates; theorems OWED — see PROOF-NEEDS.md+`. +. *"`Prevents: X`" without prevention theorem* — batch-fix template: +`+Prevents: X+` → +`+Aims to prevent (via Bool predicates; soundness theorems OWED — see PROOF-NEEDS.md): X+`. +. *"`guarantees`" / "`correctness`" / "`termination`" prose claims* — +batch-fix template: remove adjectives + add OWED pointer. + +=== OWED-with-justification convention (adopted 2026-05-20) + +Bodyless type signatures in `+Proofs.idr+` files are _implicit +postulates_ — Idris2 parses them as axioms with no proof body. To make +the trust posture visible (to readers and to grep), each bodyless +declaration carries: + +[source,idris] +---- +||| OWED: +||| Held back by . Discharge once +||| . +0 declarationName : Type +---- + +Three parts: triple-pipe doc-comment with claim+blocker+discharge +condition; leading `+0+` (Idris2 quantitative-type-theory +erased-multiplicity marker — runtime-erased); original bare type +signature. + +*Do NOT use the `+postulate+` keyword.* Zero `+Proofs.idr+` files in +`+proven+` use it. The OWED+`+0+`+bare-sig pattern is chosen so that the +reason each obligation exists is discoverable, and erasure means proof +gaps cannot silently affect runtime behaviour. + +Canonical example: `+src/Proven/SafeChecksum/Proofs.idr+` (L24-100). +Also see SafeBuffer, SafeBloom, SafeCryptoAccel, SafeHKDF, SafeFPGA — +these landed 2026-05-20 and set the convention. + +==== Blocker families surfaced in the 2026-05-20 audit + +[width="100%",cols="21%,37%,42%",options="header",] +|=== +|Family |Typical shape |Discharge route +|String FFI opacity |claims about +`+unpack+`/`+pack+`/`+ord+`/`+toLower+`/`+prim__eq_String+` |Typed +String/Char primitive layer, or Class-J axiom set parallel to +gossamer/boj-server + +|Numeric-literal Refl gaps |`+Bits32+`/`+Bits64+`/large-`+Nat+` literal +equality |`+Data.Bits+` reflective tactic or `+Integer+`-backed bound + +|Covering-not-total reduction |e.g. `+gcd n 0 = n+` under +`+Data.Nat.gcd+` declared `+covering+` (cf. SafeMath PR#46) |Upstream +Idris2 stdlib promoting to `+total+`, or local total reimplementation + +|Foldl-predicate gaps |claims about `+Data.List.all+`/`+any+` on +abstract lists |Cons-distribution lemmas inline (cf. proof-of-work +PR#60) + +|Structurally OWED |e.g. SafeJWT `+validatedJWTFromValidation+` (record +type lacks provenance proof field) |Type-side widening, not just FFI +seam +|=== + +==== Fork A scope vs. Fork B scope + +Fork A (this campaign) = make every bodyless decl explicit with a +justified OWED note. Surfacing, not discharging. *Complete 2026-05-20* +via PRs hyperpolymath/proven#37-64. + +Fork B (per-module discharge triage) is the next layer — selectively +proving the dischargeable subset. Quick-win candidates surfaced in the +audit: SafeCrypto `+modernIsSecure+`/`+standardIsSecure+` (3-line +`+isSecure+` refactor), SafePath already-Refl-able pairs (already done +in PR#57). Other modules require deeper triage. + +=== Honestly proven (carried forward, Proofs.idr re-audited 2026-05-20) + +39 directories: SafeAPIKey, SafeArgs, SafeBase64, SafeCORS, SafeCSP, +SafeCSRF, SafeCSV, SafeContentType, SafeCookie, SafeCrypto, SafeEmail, +SafeEnv, SafeFile, SafeHSTS, SafeHTTP, SafeHeader, SafeHtml, SafeJWT, +SafeJson, SafeMath, SafeNetwork, SafeOTP, SafePassword, SafePath, +SafeRBAC, SafeRateLimiter, SafeRecord, SafeRedirect, SafeRegex, SafeSQL, +SafeSRI, SafeSSRF, SafeSemVer, SafeShell, SafeString, SafeTOML, SafeUrl, +SafeXML, SafeYAML — *plus* SafeOrdering (single-file, one discharged +theorem). Their `+Proofs.idr+` bodies were re-audited 2026-05-20 under +standards#158 Fork A: of the 39 directory-form modules, 6 are clean +(zero bodyless decls), 28 had bodyless decls now annotated as explicit +OWED, 5 had only fully-discharged proofs already. No silent postulates +remain. + +=== Stubs — proof absence disguised as presence (CRITICAL) — _now empty_ + +Both audited stubs have been discharged (no remaining "`header-only`" +`+Proofs.idr+` in the audited set): + +[width="100%",cols="46%,27%,27%",options="header",] +|=== +|Module |Was |Now +|SafeCommand |`+Proofs.idr+` 8 ln, header only — CRITICAL |proven#21 — +160 ln real injection-safety proofs, `+idris2 --check+` exit 0, no +escapes + +|SafeDateTime |`+Proofs.idr+` 5 ln, header only — LOW |real +`+daysInMonth+` band lemmas (28..31, non-zero) + `+makeDate+` +smart-constructor soundness +(`+makeDate ... = Just dt -> dateGuard dt.year dt.month dt.day = True+`), +`+idris2 --check+` exit 0, no escapes — landed in this PR +|=== + +=== What needs proving (priority) + +[arabic] +. *Stubs first* — done. Both audited stubs (`+SafeCommand+`, +`+SafeDateTime+`) now carry genuine machine-checked theorems (proven#21 ++ this PR); the misleading "`proof-absence-as-presence`" class is closed +for the audited set. +. *Strip or discharge the security overclaims* — for every ✗ module +either discharge the claimed theorem or downgrade the doc header to a +non-promising description. Lead with *SafeDigest* ("`formally verified`" +is actively false), then SafeArchive, SafeCBOR. +. Sibling-audited security modules (SafeMCP, SafeOAuth, SafeWebAuthn, +SafeWebSocket, SafeWebhook, SafeCapability, SafeAttestation, SafeJWK, +SafeSecretShare) — see that audit; same OWED class. +. Re-audit the 39 carried-forward `+Proofs.idr+` bodies (not done here). + +=== Recommended prover + +*Idris2* — this _is_ the Idris2 proof library. Use a non-stub directory +(`+src/Proven/SafeSQL/Proofs.idr+`, `+SafeHTTP/Proofs.idr+`) as the +structural template; mirror `+SafeOrdering.seqTotalOrder+` for inline +single-file proofs. + +=== Priority + +*CRITICAL* — proven is the estate-wide trust root. The dominant risk is +not "`modules lacking proofs`" but *modules asserting +safety/verification in prose that no theorem backs*. Until each ✗ is +discharged or its doc claim retracted, those headers must be treated as +unverified marketing, not guarantees. diff --git a/PROOF-NEEDS.md b/PROOF-NEEDS.md deleted file mode 100644 index 2ff8208c..00000000 --- a/PROOF-NEEDS.md +++ /dev/null @@ -1,312 +0,0 @@ -# PROOF-NEEDS.md — proven - -> **Re-audit update 2026-05-20.** The 39 "carried forward, pending Proofs.idr -> re-audit" directories have now been re-audited (hyperpolymath/standards#158 -> Fork A). Net finding: of those 39, **6 are clean** (no bodyless decls; -> SafeChecksum, SafeBuffer, SafeBloom, SafeCryptoAccel, SafeHKDF, SafeFPGA — -> these set the OWED convention), **28 carried bodyless decls** which have -> now been annotated as `||| OWED:` + `0 ` erased-multiplicity per the -> convention (PRs hyperpolymath/proven#37-64, all DRAFT pending estate CI -> clear; 8 locally idris2-0.8.0-verified). Total: 250 bodyless decls -> surfaced and made discoverable. See the new "OWED-with-justification -> convention" section below and `.machine_readable/6a2/META.a2ml` ADR-001. -> -> **Honesty refresh 2026-05-18.** `proven` is the estate trust root, so this -> ledger must not overclaim. The prior edition asserted "the most complete -> Idris2 proof infrastructure in the ecosystem" and "~30 modules lack proof -> files". Adversarial re-audit shows that framing was **dishonest by a wide -> margin**: the real gap is ~3x larger, and dozens of modules ship verbatim -> security claims in their doc headers (`Prevents:`, `formally verified`, -> `guaranteed`, `cannot crash`) with **zero discharged theorem**. This is the -> same silent-overclaim class the sibling security audit found in SafeMCP / -> SafeOAuth / SafeWebAuthn / SafeWebhook / SafeCapability / SafeAttestation / -> SafeJWK / SafeSecretShare / SafeWebSocket. This edition enumerates the debt -> honestly. Refs hyperpolymath/standards#124. - -## Definitions (what counts) - -- **Proven** = a discharged theorem exists: either `src/Proven//Proofs.idr` - states and discharges ≥1 theorem (`Refl` / `with`-block / `impossible` / - `absurd` / decidable `Dec`), **or** the single-file `src/Proven/.idr` - contains such a discharged theorem inline. -- **Witness-only (OVERCLAIM, marked ✗)** = the module ships a `Prevents:` / - `Guarantees:` / `formally verified` / `cannot crash` / `no false negative` - doc claim (or `data ...Proof` witness types / proof-carrying record - *fields*) but **no discharged theorem**. A `data` witness type is an - *obligation a constructor must satisfy*, not a discharged proof. A record - field of proof type is an obligation pushed onto the caller, not discharged - here. A signature with `@ Assumed:` in its doc is an explicit postulate. -- **Claim-free** = the module makes no safety/security promise; absence of - proof is honest. -- ABI/FFI wrappers (`src/Proven/FFI/Safe*.idr`, `bindings/...`) inherit, never - independently establish, proofs. - -## Honest counts - -| Bucket | Count | -|--------|------:| -| Safe\* directories with a companion `Proofs.idr` | 41 | -| — of which are **stubs** (header only, no theorem) → treat as UNPROVEN | **0** (`SafeCommand` discharged by proven#21; `SafeDateTime` discharged on branch `proof-debt/standards-132-safedatetime-stub`) | -| Single-file `src/Proven/Safe*.idr` modules (no `Proofs.idr` dir) | 133 | -| — single-file modules shipping a safety/security **doc claim** | **76** | - -## Security-critical decidable proofs landed (2026-05-18) - -The following security-critical modules previously shipped only inline -witness *types* (`StrongKey`, `StrongCert`, …) with a "Prevents: …" -module-doc claim but **no discharged theorem** that the predicate -rejects the bad case. Real verified `Proofs.idr` files now exist -(`idris2 --check` exit 0), modelled on `SafeCommand/Proofs.idr`: - -- **SafeSSH/Proofs.idr** — exhaustive per-constructor `Refl`: DSA - provably rejected by `isWeakAlgorithm`/`validateKey`; `StrongKey` - uninhabitable for DSA. Pure enum, **no bridge axiom**. -- **SafeCert/Proofs.idr** — SHA-1 provably weak, RSA-2048 provably not - strong; `StrongCert` uninhabitable for SHA-1 / RSA-2048; temporal - expiry guard. Pure enum, **no bridge axiom**. -- **SafePromptInjection/Proofs.idr** — per-char delimiter-escape - soundness + canonical attack-vector detection + `RejectUnsafe` - enforcement; one explicit named erased string bridge axiom only. - -Still owed (single-file, witness-type-only, "Prevents" doc claim -undischarged — honest ledger): SafeMCP, SafeOAuth, SafeWebAuthn, -SafeWebSocket, SafeWebhook, SafeCapability, SafeJWK, SafeSecretShare. - -`SafeAttestation` was previously listed here but its companion -`src/Proven/SafeAttestation/Proofs.idr` (189 ln, `idris2 --check` exit 0, -"Zero `believe_me` / `idris_crash`, zero OWED") discharges enum -self-equality, parser rejection of unknown / weak algorithms, and -record-projection anchors. The remaining hash-recomputation claim is -OWED at the FFI seam (the actual hash computation happens in -`Proven.SafeCrypto`); the module header was softened in proven#76 to -reflect this. It therefore belongs in the "Security-critical decidable -proofs landed" section above, not in this still-owed list. - -Companion `Proofs.idr` content was *not* re-verified theorem-by-theorem in this -pass; the 39 non-stub directories are carried forward as claimed-proven pending -a separate Proofs.idr discharge audit. **The single-file population is the -exposure this ledger now owns honestly.** - -### Adversarial sample of the single-file non-security population - -32 modules sampled across the alphabet (SafeAngle…SafeUnit), excluding the 12 -modules the sibling security audit handled and SafeCommand/SafeDateTime: - -| Outcome | Count | Fraction | -|---------|------:|---------:| -| Genuinely proven (≥1 inline discharged theorem) | **1** | ~3% | -| Witness-only / doc-claim **OVERCLAIM (✗)** | **13** | ~41% | -| Claim-free (honest silence) | **18** | ~56% | - -- Only **SafeOrdering** carries a real discharged theorem - (`seqTotalOrder`, `with`-block, `Refl`, `%default total`). Its - `PartialOrder`/`TotalOrder` records additionally push proof *fields* onto - callers (obligations, not discharged here). -- **Extrapolation:** with 76/133 single-file modules carrying a safety/security - doc claim and the sampled proven-rate ≈3%, on the order of **~70 single-file - modules are silent overclaims** — the prior ledger acknowledged ~30 *gaps* - total and never used the word "overclaim". Ledger accuracy verdict: the - origin/main edition was **dishonest — it understated the unproven gap by - roughly 3x and entirely omitted the doc-claim-without-theorem class.** - -## Owed — single-file modules shipping a security claim with NO theorem (✗) - -Each line: module — verbatim doc claim — why it is owed. - -### Security-relevant overclaims (highest concern) - -- **SafeDigest** ✗ — `src/Proven/SafeDigest.idr:5` *"formally verified digest - parsing, validation, and constant-time comparison to prevent timing - attacks."* — WORST OFFENDER: claims "formally verified" yet - `constantTimeReflexive`/`constantTimeSymmetric`/`verifyTransitive` - (`:225–248`) are **explicitly undischarged** (`@ Assumed:` in their own - docstrings). Crypto material + actively false "verified" word. -- **SafeArchive** ✗ — `src/Proven/SafeArchive.idr:5` *"Provides type-safe - archive member validation that prevents: Zip Slip / Symlink attacks / Zip - bombs / Path injection via null bytes."* — `hasPathTraversal` is a `Bool` - predicate with no theorem that a non-flagged entry is traversal-free. Supply- - chain / unpack surface. -- **SafeCBOR** ✗ — `src/Proven/SafeCBOR.idr:7` *"Prevents: integer overflow, - indefinite-length bombs, tag confusion."* — used in COSE/WebAuthn/FIDO2 per - its own header; no overflow/bomb/tag theorem. Authn-adjacent. -- **SafeConsensus** ✗ — `src/Proven/SafeConsensus.idr:9` *"Log replication with - consistency guarantees"* — no safety/agreement theorem. Distributed-state - surface. -- **SafeLRU** ✗ — `src/Proven/SafeLRU.idr:6` *"that cannot overflow or corrupt - cache state."* — no invariant theorem; cache-poisoning adjacency. - -### Lower-blast-radius overclaims (still owed) - -- **SafeBloom** ✗ — `:6` *"with guaranteed no false negatives."* (the entire - correctness contract of a Bloom filter; undischarged) -- **SafeMatrix** ✗ — `:3` *"matrix operations that cannot crash"* -- **SafeSet** ✗ — `:3` *"set operations that cannot crash"* -- **SafeRational** ✗ — `:6` *"safe operations that cannot crash."* -- **SafeGraph** ✗ — `:197` *"Bounded by fuel … to guarantee termination"* - (asserted, not proven) -- **SafeProbability** ✗ — `:20` *"A probability value guaranteed to be in - [0, 1]"* (no smart-constructor refinement proof; `impossible` is a *value* - named impossible, not a proof) -- **SafeTree** ✗ — `:6` traversal/manipulation safety, no theorem -- **SafeDecimal** ✗ — `:54` "Structurally decreasing on scale ensures - totality" (relies on `%default total` only; no stated lemma) - -### Long-tail enumeration (Phase 2 Days 19-21, partial — 2026-05-27) - -Adversarial-grep + per-module inspection of the long tail surfaced **6 -additional confirmed single-file overclaims** beyond the 13 enumerated -above. Headers softened in proven#76 (follow-on commit; see -`docs/proof-debt-triage.md` §11): - -- **SafeSupplyChain** ✗ HIGH — `src/Proven/SafeSupplyChain.idr:7` - *"Prevents: tampered builds, unattested artifacts, provenance forgery."* - — SLSA attestation layer; tamper-prevention theorems OWED. Real-world - supply-chain blast radius. -- **SafePBKDF2** ✗ HIGH — `src/Proven/SafePBKDF2.idr:6` *"Prevents: low - iteration counts, short salts, weak derived key lengths."* — crypto - parameter validator; prevention theorems OWED. -- **SafeProvenance** ✗ MEDIUM — `src/Proven/SafeProvenance.idr:4` - *"Formally verified change tracking and audit trails"* — provides - records + predicates only; integrity/lineage theorems OWED. -- **SafePolicy** ✗ MEDIUM — `src/Proven/SafePolicy.idr:4` *"Formally - verified policy enforcement"* — AST-level predicates only; enforcement - soundness theorems OWED. -- **SafeRegistry** ✗ MEDIUM — `src/Proven/SafeRegistry.idr:5-6` - *"formally verified parsing of OCI/Docker image references with - guarantees of correctness and termination"* — parsers via Bool - validators; correctness/termination theorems OWED. -- **SafeSchema** ✗ MEDIUM — `src/Proven/SafeSchema.idr:4-10` *"Formally - verified schema evolution"* + *"compatibility proofs"* + - *"correctness guarantees"* — type definitions + migration scaffolding; - compatibility/correctness theorems OWED. - -**Confirmed actually-proven** (re-classification, NOT in the OWED list): - -- **SafeTrust** — `src/Proven/SafeTrust.idr:296-303` `satisfiesMonotone` - discharged via exhaustive pattern-match + `Refl` per arm. The header's - "Formally verified" / "proven monotone" claim IS valid. -- **SafeOrdering** — already noted in §"Adversarial sample" as the only - single-file module in that sample with a discharged theorem. - -### Long-tail still extrapolated - -The Phase 2 Days 19-21 audit surfaced 6 new instances but did NOT -exhaustively read all 76 claim-bearing single-file modules (Bash -permission boundaries on the audit sub-agent prevented full enumeration). -On the order of **~54 single-file modules remain extrapolated** as -overclaim candidates per the original 3% genuinely-proven sample rate. -A full enumeration is mechanical: every `src/Proven/Safe*.idr` matching -`Prevents:|formally verified|cannot crash|guarantee|no false negative| -traversal|injection|attack` without a discharged theorem (no `Refl` / -`Dec` / `impossible` / `absurd` / `with` block) is OWED. Phase 3 may -batch-process these via the three cross-cutting overclaim patterns -identified in the Days 19-21 audit: - -1. **"Formally verified" without evidence** — batch-fix template: - `formally verified X` → `X via Bool predicates; theorems OWED — see - PROOF-NEEDS.md`. -2. **"Prevents: X" without prevention theorem** — batch-fix template: - `Prevents: X` → `Aims to prevent (via Bool predicates; soundness - theorems OWED — see PROOF-NEEDS.md): X`. -3. **"guarantees" / "correctness" / "termination" prose claims** — - batch-fix template: remove adjectives + add OWED pointer. - -## OWED-with-justification convention (adopted 2026-05-20) - -Bodyless type signatures in `Proofs.idr` files are *implicit postulates* — -Idris2 parses them as axioms with no proof body. To make the trust posture -visible (to readers and to grep), each bodyless declaration carries: - -```idris -||| OWED: -||| Held back by . Discharge once -||| . -0 declarationName : Type -``` - -Three parts: triple-pipe doc-comment with claim+blocker+discharge condition; -leading `0` (Idris2 quantitative-type-theory erased-multiplicity marker — -runtime-erased); original bare type signature. - -**Do NOT use the `postulate` keyword.** Zero `Proofs.idr` files in `proven` -use it. The OWED+`0`+bare-sig pattern is chosen so that the reason each -obligation exists is discoverable, and erasure means proof gaps cannot -silently affect runtime behaviour. - -Canonical example: `src/Proven/SafeChecksum/Proofs.idr` (L24-100). Also see -SafeBuffer, SafeBloom, SafeCryptoAccel, SafeHKDF, SafeFPGA — these landed -2026-05-20 and set the convention. - -### Blocker families surfaced in the 2026-05-20 audit - -| Family | Typical shape | Discharge route | -|--------|---------------|-----------------| -| String FFI opacity | claims about `unpack`/`pack`/`ord`/`toLower`/`prim__eq_String` | Typed String/Char primitive layer, or Class-J axiom set parallel to gossamer/boj-server | -| Numeric-literal Refl gaps | `Bits32`/`Bits64`/large-`Nat` literal equality | `Data.Bits` reflective tactic or `Integer`-backed bound | -| Covering-not-total reduction | e.g. `gcd n 0 = n` under `Data.Nat.gcd` declared `covering` (cf. SafeMath PR#46) | Upstream Idris2 stdlib promoting to `total`, or local total reimplementation | -| Foldl-predicate gaps | claims about `Data.List.all`/`any` on abstract lists | Cons-distribution lemmas inline (cf. proof-of-work PR#60) | -| Structurally OWED | e.g. SafeJWT `validatedJWTFromValidation` (record type lacks provenance proof field) | Type-side widening, not just FFI seam | - -### Fork A scope vs. Fork B scope - -Fork A (this campaign) = make every bodyless decl explicit with a justified -OWED note. Surfacing, not discharging. **Complete 2026-05-20** via PRs -hyperpolymath/proven#37-64. - -Fork B (per-module discharge triage) is the next layer — selectively -proving the dischargeable subset. Quick-win candidates surfaced in the -audit: SafeCrypto `modernIsSecure`/`standardIsSecure` (3-line `isSecure` -refactor), SafePath already-Refl-able pairs (already done in PR#57). Other -modules require deeper triage. - -## Honestly proven (carried forward, Proofs.idr re-audited 2026-05-20) - -39 directories: SafeAPIKey, SafeArgs, SafeBase64, SafeCORS, SafeCSP, SafeCSRF, -SafeCSV, SafeContentType, SafeCookie, SafeCrypto, SafeEmail, SafeEnv, SafeFile, -SafeHSTS, SafeHTTP, SafeHeader, SafeHtml, SafeJWT, SafeJson, SafeMath, -SafeNetwork, SafeOTP, SafePassword, SafePath, SafeRBAC, SafeRateLimiter, -SafeRecord, SafeRedirect, SafeRegex, SafeSQL, SafeSRI, SafeSSRF, SafeSemVer, -SafeShell, SafeString, SafeTOML, SafeUrl, SafeXML, SafeYAML — **plus** -SafeOrdering (single-file, one discharged theorem). Their `Proofs.idr` bodies -were re-audited 2026-05-20 under standards#158 Fork A: of the 39 directory-form -modules, 6 are clean (zero bodyless decls), 28 had bodyless decls now -annotated as explicit OWED, 5 had only fully-discharged proofs already. No -silent postulates remain. - -## Stubs — proof absence disguised as presence (CRITICAL) — *now empty* - -Both audited stubs have been discharged (no remaining "header-only" -`Proofs.idr` in the audited set): - -| Module | Was | Now | -|--------|-----|-----| -| SafeCommand | `Proofs.idr` 8 ln, header only — CRITICAL | proven#21 — 160 ln real injection-safety proofs, `idris2 --check` exit 0, no escapes | -| SafeDateTime | `Proofs.idr` 5 ln, header only — LOW | real `daysInMonth` band lemmas (28..31, non-zero) + `makeDate` smart-constructor soundness (`makeDate ... = Just dt -> dateGuard dt.year dt.month dt.day = True`), `idris2 --check` exit 0, no escapes — landed in this PR | - -## What needs proving (priority) - -1. **Stubs first** — done. Both audited stubs (`SafeCommand`, - `SafeDateTime`) now carry genuine machine-checked theorems - (proven#21 + this PR); the misleading "proof-absence-as-presence" - class is closed for the audited set. -2. **Strip or discharge the security overclaims** — for every ✗ module either - discharge the claimed theorem or downgrade the doc header to a non-promising - description. Lead with **SafeDigest** ("formally verified" is actively - false), then SafeArchive, SafeCBOR. -3. Sibling-audited security modules (SafeMCP, SafeOAuth, SafeWebAuthn, - SafeWebSocket, SafeWebhook, SafeCapability, SafeAttestation, SafeJWK, - SafeSecretShare) — see that audit; same OWED class. -4. Re-audit the 39 carried-forward `Proofs.idr` bodies (not done here). - -## Recommended prover - -**Idris2** — this *is* the Idris2 proof library. Use a non-stub directory -(`src/Proven/SafeSQL/Proofs.idr`, `SafeHTTP/Proofs.idr`) as the structural -template; mirror `SafeOrdering.seqTotalOrder` for inline single-file proofs. - -## Priority - -**CRITICAL** — proven is the estate-wide trust root. The dominant risk is not -"modules lacking proofs" but **modules asserting safety/verification in prose -that no theorem backs**. Until each ✗ is discharged or its doc claim retracted, -those headers must be treated as unverified marketing, not guarantees. diff --git a/PROVEN-COMPREHENSIVE-ANALYSIS-2026-01-30.adoc b/PROVEN-COMPREHENSIVE-ANALYSIS-2026-01-30.adoc new file mode 100644 index 00000000..0103163e --- /dev/null +++ b/PROVEN-COMPREHENSIVE-ANALYSIS-2026-01-30.adoc @@ -0,0 +1,669 @@ +== PROVEN Library - Comprehensive Analysis & V1 Release Readiness + +*Date:* 2026-01-30 *Status:* Pre-V1 Release Audit *Repository:* +github.com/hyperpolymath/proven + +=== Executive Summary + +*Core Architecture:* Idris2 library (45 source files, 90+ modules) with +FFI bindings to 89 languages *Current Version:* 1.0.0-production-release +(per STATE.scm) *Primary Issues:* Language policy violations, missing +ROADMAP, incomplete testing documentation + +''''' + +=== ASCII Architecture Diagram (Anatomy) + +.... +┌─────────────────────────────────────────────────────────────────┐ +│ PROVEN LIBRARY ECOSYSTEM │ +└─────────────────────────────────────────────────────────────────┘ + + ┌──────────────────┐ + │ User's App Code │ + │ (Any Language) │ + └────────┬─────────┘ + │ + ┌────────────────────────┼────────────────────────┐ + │ │ │ + ▼ ▼ ▼ +┌──────────────┐ ┌──────────────┐ ┌──────────────┐ +│ Rust FFI │ │ Deno/JS FFI │ │ Gleam FFI │ +│ Binding │ ... │ Binding │ ... │ Binding │ +└──────┬───────┘ └──────┬───────┘ └──────┬───────┘ + │ │ │ + └────────────────────────┼────────────────────────┘ + │ + ┌───────────▼──────────┐ + │ ZIG FFI BRIDGE │ + │ (Pure ABI layer) │ + │ • No safety logic │ + │ • C ABI compatible │ + │ • Cross-platform │ + └───────────┬──────────┘ + │ + ┌───────────▼──────────────────────────────┐ + │ IDRIS2 CORE LIBRARY │ + │ │ + │ ┌────────────┐ ┌──────────────────┐ │ + │ │ Safe Types │ │ Dependent Types │ │ + │ │ • Result │ │ • Proofs │ │ + │ │ • Maybe │ │ • Totality check │ │ + │ └────────────┘ └──────────────────┘ │ + │ │ + │ ┌─────────────────────────────────┐ │ + │ │ 90+ Safety Modules │ │ + │ │ • SafeMath • SafeString │ │ + │ │ • SafeJson • SafeUrl │ │ + │ │ • SafePath • SafeCrypto │ │ + │ │ • SafeSQL • SafeNetwork │ │ + │ │ • ... (86 more modules) │ │ + │ └─────────────────────────────────┘ │ + └──────────────────────────────────────────┘ + +┌─────────────────────────────────────────────────────────────────┐ +│ VERIFICATION LAYERS │ +├─────────────────────────────────────────────────────────────────┤ +│ Compile Time: Type checker + Totality checker │ +│ Test Time: Property tests (29 files) + Unit tests (20) │ +│ Runtime: ClusterFuzzLite fuzzing │ +│ Formal: Echidnabot smart contract verification │ +└─────────────────────────────────────────────────────────────────┘ +.... + +''''' + +=== ASCII Process/Flow Diagram (Physiology) + +.... +USER CODE EXECUTION FLOW +═══════════════════════════════════════════════════════════════ + +1. API Call + ┌─────────────────────────────────────┐ + │ user_code.rs │ + │ │ + │ let safe_num = safe_add(5, 3)?; │ + └─────────────┬───────────────────────┘ + │ + ▼ +2. Rust Binding Layer (src/lib.rs) + ┌─────────────────────────────────────┐ + │ pub fn safe_add(a: i64, b: i64) │ + │ -> Result { │ + │ unsafe { │ + │ zig_ffi_safe_add(a, b) │ + │ } │ + │ } │ + └─────────────┬───────────────────────┘ + │ FFI call + ▼ +3. Zig FFI Bridge (ffi/zig/src/safe_math.zig) + ┌─────────────────────────────────────┐ + │ export fn zig_ffi_safe_add( │ + │ a: i64, b: i64 │ + │ ) callconv(.C) Result_i64 { │ + │ return idris2_safe_add(a, b); │ + │ } │ + └─────────────┬───────────────────────┘ + │ C ABI call + ▼ +4. Idris2 Core (src/Proven/SafeMath.idr) + ┌─────────────────────────────────────────────────┐ + │ export │ + │ idris2_safe_add : Int -> Int -> Result Int │ + │ idris2_safe_add a b = │ + │ if willOverflow a b │ + │ then Error Overflow │ + │ else OK (a + b) │ + │ │ + │ -- Totality checked: ✓ (no infinite loops) │ + │ -- Proofs: ∀a,b. result ∈ [INT_MIN, INT_MAX] │ + └─────────────┬───────────────────────────────────┘ + │ + ▼ +5. Result Propagation (back up the stack) + ┌─────────────────────────────────────┐ + │ Idris2 → Zig → Rust → User Code │ + │ │ + │ OK 8 (success case) │ + │ Error e (overflow/error case) │ + └─────────────────────────────────────┘ + +VERIFICATION AT EACH LAYER +═══════════════════════════════════════════════════════════════ + +Idris2 Layer: + ├─ Compile-time totality check (no crashes) + ├─ Type-level proofs (bounds, invariants) + └─ No exceptions (all fallible ops return Result) + +Zig Layer: + ├─ Comptime verification where possible + ├─ No safety logic (pure pass-through) + └─ C ABI compatibility enforced + +Binding Layer: + ├─ Type conversions only + ├─ Error propagation to native Result types + └─ No business logic + +Testing: + ├─ Property tests: ∀ inputs, check invariants hold + ├─ Unit tests: Known input/output pairs + ├─ Fuzz tests: Random inputs via ClusterFuzzLite + └─ Formal verification: Echidnabot for contracts +.... + +''''' + +=== Current Status (from STATE.scm) + +==== Core Implementation + +* *Idris2 Modules:* 90+ modules (100% complete per STATE.scm) +* *Test Files:* 29 property tests + 20 unit tests ✓ +* *Fuzzing:* ClusterFuzzLite integration ✓ +* *CI/CD:* GitHub Actions workflows ✓ +* *Documentation:* README.adoc ✓, CONTRIBUTING.adoc ✓, SECURITY.md ✓ + +==== Language Bindings (89 targets) + +STATE.scm marks all bindings as "`complete`" but quality varies: + +*Approved Languages (per RSR):* - ✓ Rust (bindings/rust/) - ✓ Deno +(bindings/deno/) - ✓ Gleam (bindings/gleam/) - ✓ Elixir +(bindings/elixir/) - ✓ Haskell (bindings/haskell/) - ✓ OCaml +(bindings/ocaml/) - ✓ Ada (bindings/ada/) - ✓ Julia (bindings/julia/) - +✓ Bash (bindings/bash/) - ✓ Nickel (bindings/nickel/) - ✓ Guile Scheme +(bindings/guile/) + +*BANNED Languages Found:* - ❌ Go (bindings/go/) - VIOLATION: Go is +banned per RSR - ❌ Python (bindings/python/) - VIOLATION: Python is +banned per RSR - ❌ TypeScript (bindings/typescript/) - VIOLATION: +TypeScript is banned per RSR - ❌ JavaScript with npm +(bindings/javascript/package.json) - VIOLATION: npm/Node.js banned - ❌ +ReScript with node_modules (bindings/rescript/node_modules) - VIOLATION: +Should use Deno + +''''' + +=== CRITICAL ISSUES BLOCKING V1 RELEASE + +==== 1. Language Policy Violations (RSR Non-Compliance) + +*Issue:* Repository contains banned languages contrary to Rhodium +Standard Repositories. + +*Violations:* + +.... +bindings/go/ ← BANNED (use Rust instead) +bindings/python/ ← BANNED (use Julia/Rust/ReScript) +bindings/typescript/ ← BANNED (use ReScript) +bindings/javascript/package.json ← BANNED (use Deno with deno.json) +bindings/rescript/node_modules/ ← BANNED (migrate to Deno runtime) +bindings/malbolge/compiler.py ← BANNED (Python file) +bindings/malbolge/safe_malbolge.py ← BANNED (Python file) +.... + +*Action Required:* 1. *DELETE* bindings/go/, bindings/python/, +bindings/typescript/ 2. *MIGRATE* bindings/javascript/ to use Deno +(delete package.json, create deno.json) 3. *MIGRATE* bindings/rescript/ +to use Deno (delete node_modules, package.json, package-lock.json) 4. +*REWRITE* bindings/malbolge/*.py in Julia or Rust + +*Timeline:* IMMEDIATE (blocks v1 release) + +==== 2. Missing ROADMAP File + +*Issue:* No ROADMAP.md or ROADMAP.adoc exists in repository root. + +*Current State:* - STATE.scm contains milestones (v0.1.0 through v1.0.0) +- No forward-looking roadmap for v1.1, v2.0, v3.0 + +*Action Required:* Create `+ROADMAP.adoc+` with: - Completed milestones +(v0.1.0 - v1.0.0) - v1.1.0 plans - v2.0.0 vision - v3.0.0 long-term +goals + +==== 3. Testing Infrastructure Gaps + +*Current Testing:* - ✓ Property tests (29 files) - ✓ Unit tests (20 +files) - ✓ ClusterFuzzLite fuzzing - ✓ Echidnabot (.echidnabot.toml +exists) + +*Missing/Undocumented:* - ⚠ *Formal Verification Status:* Echidnabot +config exists but no results documented - ⚠ *Stress Testing:* No stress +test suite found - ⚠ *Compilation Testing:* Not documented (though CI +likely does this) - ⚠ *Attack Surface Analysis:* Not documented - ⚠ +*Benchmarking:* `+benchmarks/+` directory exists but no results in docs + +*Action Required:* 1. Run and document Echidnabot results (formal +verification) 2. Create stress test suite (concurrent operations, +resource limits) 3. Document compilation test matrix (platforms, Idris +versions) 4. Perform attack surface analysis (FFI boundaries, unsafe +blocks) 5. Run benchmarks and document performance characteristics + +==== 4. Missing Contractiles and a2ml/k9-svc + +*Issue:* No contractiles or a2ml/k9-svc integration found in repository. + +*Search Results:* + +[source,bash] +---- +$ find . -name "*contractile*" -o -name "*a2ml*" -o -name "*k9-svc*" +(no results) +---- + +*Action Required:* - Clarify: What are contractiles in the context of +proven? - Clarify: Is a2ml/k9-svc integration required for v1? - If +required: Add integration and documentation + +==== 5. Cleanup Required + +*Stray Backup Directories:* + +.... +.UNSAFE-ALL-BINDINGS-DELETED-20260125-095646/ +.UNSAFE-ALL-BINDINGS-DELETED-20260125-095657/ +.UNSAFE-ALL-BINDINGS-DELETED-20260125-095712/ +.UNSAFE-ALL-BINDINGS-DELETED-FINAL-20260125/ +.... + +*Action:* Delete these backup directories (old binding cleanup +artifacts) + +''''' + +=== Route to V2 and V3 + +==== v1.0.0 → v1.1.0 (Stabilization & Performance) + +*Timeline:* 1-2 months *Focus:* Hardening, optimization, ecosystem +growth + +*Planned Features:* 1. *Performance Optimization* - Reduce FFI crossing +overhead (batching APIs) - WASM compilation target optimization - +Benchmark-driven tuning + +[arabic, start=2] +. *Extended Bindings* +* Fill gaps in config languages (CUE, Starlark, HCL) +* Quantum computing bindings (Q#, OpenQASM) maturation +* Neuromorphic computing binding enhancements +. *Tooling Improvements* +* VS Code extension for proven types +* LSP integration for supported languages +* Better error messages at FFI boundary +. *Documentation* +* Video tutorials +* Interactive playground +* Migration guides from unsafe libraries + +==== v2.0.0 (Advanced Verification & Concurrency) + +*Timeline:* 6-12 months *Focus:* Concurrency proofs, advanced +verification + +*Major Features:* 1. *SafeConcurrency Module* - Type-safe concurrency +primitives - Data race prevention via types - Deadlock-free guarantees - +Actor model implementation + +[arabic, start=2] +. *Enhanced Verification* +* SMT solver integration (Z3, CVC5) +* Automated proof search +* Verification condition generation +* Runtime assertion synthesis +. *Advanced Safety Modules* +* SafeMemory (region-based memory safety) +* SafeProtocol (verified protocol implementations) +* SafeSmartContract (extended contract verification) +* SafeML (safe machine learning pipelines) +. *Ecosystem Integration* +* Package manager integration (cargo, npm alternatives) +* Build system plugins (Bazel, Buck2) +* IDE deep integration + +==== v3.0.0 (Distributed Systems & Formal Methods) + +*Timeline:* 18-24 months *Focus:* Distributed correctness, advanced +formal methods + +*Major Features:* 1. *SafeDistributed Module* - Consensus algorithm +verification (Raft, Paxos) - Network partition safety - Byzantine fault +tolerance - Distributed transaction proofs + +[arabic, start=2] +. *Advanced Type Features* +* Refinement types for all modules +* Session types for protocols +* Linear types for resource management +* Higher-order verification +. *Proof Automation* +* Tactic language for custom proofs +* Automated invariant discovery +* Proof repair and suggestions +* Interactive proof assistants +. *Research Integration* +* Academic paper implementations +* Benchmark suite for verification research +* Collaboration with PL research groups +* Formal methods education platform + +''''' + +=== Testing Requirements (Pre-V1 Checklist) + +==== ✓ Completed Tests + +[arabic] +. *Unit Tests* (20 files) +* Location: `+tests/+` +* Status: COMPLETE +. *Property Tests* (29 files) +* Location: `+tests/+` +* Status: COMPLETE +. *Fuzzing* (ClusterFuzzLite) +* Config: `+.clusterfuzzlite/+` +* Status: CONFIGURED + +==== ⚠ Incomplete/Undocumented Tests + +[arabic, start=4] +. *Execution Testing* +* Required: Test actual execution in all supported language bindings +* Status: LIKELY DONE (CI runs tests) but NOT DOCUMENTED +* Action: Create `+docs/TESTING.md+` documenting execution test results +. *Formal Verification* (Echidnabot) +* Config: `+.echidnabot.toml+` exists +* Status: CONFIGURED but NO RESULTS DOCUMENTED +* Action: Run Echidnabot, capture results, add to docs +. *Stress Testing* +* Required: High load, concurrent operations, resource exhaustion +* Status: NOT FOUND +* Action: Create `+tests/stress/+` with stress test suite +. *Compilation Testing* +* Required: Test compilation on all platforms (Linux, macOS, Windows, +BSD) +* Required: Test multiple Idris2 versions +* Status: LIKELY DONE (CI) but NOT DOCUMENTED +* Action: Document compilation matrix in `+docs/TESTING.md+` +. *Attack Surface Analysis* +* Required: Analyze FFI boundaries, unsafe blocks, trust assumptions +* Status: NOT FOUND +* Action: Create `+docs/ATTACK-SURFACE.md+` with threat model +. *Benchmarking* +* Directory: `+benchmarks/+` exists +* Status: CODE EXISTS but NO RESULTS DOCUMENTED +* Action: Run benchmarks, create `+docs/BENCHMARKS.md+` with results + +==== Testing Checklist for V1 + +* [ ] Run all unit tests, document results +* [ ] Run all property tests, document results +* [ ] Execute ClusterFuzzLite, document findings +* [ ] Run Echidnabot formal verification, document proofs +* [ ] Create and run stress test suite +* [ ] Document compilation test matrix +* [ ] Perform attack surface analysis +* [ ] Run benchmarks, document performance +* [ ] Test all 89 language bindings (at least smoke tests) +* [ ] Create comprehensive `+docs/TESTING.md+` + +''''' + +=== File Organization Audit + +==== ✓ Present and Correct + +* STATE.scm (comprehensive, up-to-date) +* META.scm (ADRs, design rationale) +* ECOSYSTEM.scm (project relationships) +* README.adoc (excellent, detailed) +* CONTRIBUTING.adoc ✓ +* CODE_OF_CONDUCT.md ✓ +* SECURITY.md ✓ +* CHANGELOG.md ✓ +* .editorconfig ✓ +* LICENSE files (MPL-2.0-or-later) ✓ + +==== ⚠ Missing or Incomplete + +* *ROADMAP.adoc* - MISSING (required for RSR compliance) +* *docs/TESTING.md* - MISSING (should document all test results) +* *docs/BENCHMARKS.md* - MISSING (benchmark results) +* *docs/ATTACK-SURFACE.md* - MISSING (security analysis) +* *.well-known/* directory - NOT CHECKED (RSR requirement) + +==== 🗑 To Delete + +* `+.UNSAFE-ALL-BINDINGS-DELETED-*+` directories (4 backup dirs) +* `+bindings/go/+` (language violation) +* `+bindings/python/+` (language violation) +* `+bindings/typescript/+` (language violation) +* `+bindings/javascript/package.json+` (migrate to Deno) +* `+bindings/rescript/node_modules/+` (migrate to Deno) +* `+bindings/malbolge/*.py+` (rewrite in Rust/Julia) + +''''' + +=== Code Optimization Opportunities + +==== 1. Idris Code Incorporation + +*Current State:* - Core library is 100% Idris2 - 45 .idr source files in +`+src/Proven/+` + +*Optimization:* - Review proven-malbolge-toolchain for reusable Idris +patterns - Incorporate proven-concat streaming optimizations - Integrate +ProvenCrypto.jl Julia code for numeric algorithms (convert to Idris) + +*Action:* Audit related repos for Idris code to merge: + +[source,bash] +---- +~/Documents/hyperpolymath-repos/ephapax-proven/ +~/Documents/hyperpolymath-repos/ephapax-proven-ffi/ +~/Documents/hyperpolymath-repos/ephapax-proven-sys/ +~/Documents/hyperpolymath-repos/proven-concat/ +~/Documents/hyperpolymath-repos/ProvenCrypto.jl/ +~/Documents/hyperpolymath-repos/proven-http/ +~/Documents/hyperpolymath-repos/proven-malbolge-toolchain/ +~/Documents/hyperpolymath-repos/proven-tui/ +---- + +==== 2. FFI Overhead Reduction + +*Current Issue:* Each operation crosses FFI boundary (Rust → Zig → +Idris2) + +*Optimization Strategies:* 1. *Batching API:* Accept arrays of +operations, process in Idris2, return array 2. *Caching:* Memoize pure +functions at Zig layer 3. *WASM Optimization:* Compile Idris2 directly +to WASM (bypass Zig for web) + +==== 3. Compiler Optimization Flags + +*Check:* Are optimal Idris2 compiler flags used? - `+--cg chez+` vs +`+--cg refc+` (C backend) benchmark - Inlining thresholds - Dead code +elimination + +''''' + +=== Annotation Status + +*Requirement:* All files must be annotated. + +*Current State:* - Idris2 files: Likely have doc comments (need +verification) - Binding files: Varies by language - Documentation files: +Well-written + +*Action Required:* 1. Audit all .idr files for documentation comments 2. +Ensure all public functions have doc comments 3. Add module-level +documentation to all files 4. Verify SPDX headers on all source files + +''''' + +=== RSR Compliance Checklist + +Based on Rhodium Standard Repositories (RSR): + +* [x] STATE.scm present and current +* [x] META.scm present with ADRs +* [x] ECOSYSTEM.scm present +* [x] README.adoc comprehensive +* [x] CONTRIBUTING.adoc +* [x] CODE_OF_CONDUCT.md +* [x] SECURITY.md +* [x] LICENSE files (MPL-2.0-or-later) +* [ ] *ROADMAP.adoc* - MISSING +* [x] .editorconfig +* [ ] *No banned languages* - VIOLATED (Go, Python, TypeScript, npm) +* [ ] .well-known/ directory - NOT VERIFIED +* [ ] All files annotated - NOT VERIFIED +* [x] GitHub Actions workflows with SPDX headers +* [x] OpenSSF Scorecard (assumed passing based on org standards) + +''''' + +=== Package Re-Release Readiness + +==== Blockers (MUST FIX) + +[arabic] +. ❌ *Remove banned languages* (Go, Python, TypeScript, npm) +. ❌ *Create ROADMAP.adoc* +. ❌ *Document testing results* (formal verification, stress, +benchmarks) +. ❌ *Clarify contractiles/a2ml/k9-svc requirement* + +==== Recommended (SHOULD FIX) + +[arabic, start=5] +. ⚠ Clean up .UNSAFE-* backup directories +. ⚠ Verify all files have SPDX headers +. ⚠ Verify all public APIs have doc comments +. ⚠ Run and document benchmarks +. ⚠ Perform attack surface analysis + +==== Nice to Have (MAY FIX) + +[arabic, start=10] +. ○ Incorporate Idris code from related repos +. ○ Optimize FFI overhead +. ○ Add tooling (VS Code extension, LSP) + +''''' + +=== Recommended Action Plan + +==== Phase 1: Critical Blockers (Week 1) + +[arabic] +. *Delete banned language bindings* ++ +[source,bash] +---- +cd ~/Documents/hyperpolymath-repos/proven +rm -rf bindings/go bindings/python bindings/typescript +rm bindings/javascript/package.json +rm -rf bindings/rescript/node_modules bindings/rescript/package*.json +rm bindings/malbolge/*.py +git commit -m "Remove banned languages per RSR" +---- +. *Create ROADMAP.adoc* +* Extract milestones from STATE.scm +* Add v1.1, v2.0, v3.0 vision (see above) +. *Run and document testing* +* Execute Echidnabot: `+echidnabot run+` +* Create stress tests in `+tests/stress/+` +* Run benchmarks, capture results +* Create `+docs/TESTING.md+` + +==== Phase 2: Quality Assurance (Week 2) + +[arabic, start=4] +. *Annotation audit* +* Check all .idr files for doc comments +* Add SPDX headers where missing +* Verify module-level docs +. *Attack surface analysis* +* Document FFI trust boundaries +* List all `+unsafe+` blocks +* Create threat model +* Write `+docs/ATTACK-SURFACE.md+` +. *Benchmarking* +* Run all benchmarks +* Create `+docs/BENCHMARKS.md+` +* Compare to unsafe alternatives + +==== Phase 3: Optimization (Week 3-4) + +[arabic, start=7] +. *Code optimization* +* Audit related repos for Idris code to merge +* Implement batching APIs +* Optimize compiler flags +. *Documentation polish* +* Video walkthrough +* Interactive examples +* Migration guides + +==== Phase 4: Release (Week 5) + +[arabic, start=9] +. *Final checklist* +* All tests passing +* Documentation complete +* No RSR violations +* Version tagged +. *Package publication* +* GitHub Packages (Container) +* Language-specific registries (crates.io, JSR, etc.) +* Announce on hyperpolymath channels + +''''' + +=== Questions Requiring Clarification + +[arabic] +. *Contractiles:* What are contractiles in the proven context? Are they +required for v1? +. *a2ml/k9-svc:* What is a2ml/k9-svc? Is integration required for v1 +release? +. *Malbolge bindings:* The bindings/malbolge/ directory has Python +files. Should Malbolge bindings be: +* Rewritten in Rust/Julia? +* Removed entirely (Malbolge is esoteric/impractical)? +* Kept as educational examples? +. *ReScript migration:* Should bindings/rescript/ use: +* Deno runtime (deno.json)? +* Bun is banned, so not an option +* Direct compilation to JS (ES6 modules) without runtime? +. *JavaScript bindings:* Should bindings/javascript/ be: +* Deleted (TypeScript/JavaScript banned)? +* Migrated to Deno (bindings/deno/ already exists)? +* Kept as pure ES6 modules without npm? + +''''' + +=== Summary + +*proven* is architecturally sound with excellent Idris2 core code and +comprehensive documentation. The main issues are: + +[arabic] +. *Language policy violations* (banned languages present) +. *Missing ROADMAP* +. *Testing documentation gaps* +. *Unclear contractiles/a2ml requirement* + +With 1-2 weeks of cleanup and documentation work, proven will be ready +for v1.0.0 release. The architecture is solid, the core implementation +is complete, and the verification foundations are in place. + +The route to v2 and v3 focuses on concurrency, advanced verification, +and distributed systems—all natural extensions of the current +architecture. diff --git a/PROVEN-COMPREHENSIVE-ANALYSIS-2026-01-30.md b/PROVEN-COMPREHENSIVE-ANALYSIS-2026-01-30.md deleted file mode 100644 index 5bc82ae1..00000000 --- a/PROVEN-COMPREHENSIVE-ANALYSIS-2026-01-30.md +++ /dev/null @@ -1,682 +0,0 @@ -# PROVEN Library - Comprehensive Analysis & V1 Release Readiness -**Date:** 2026-01-30 -**Status:** Pre-V1 Release Audit -**Repository:** github.com/hyperpolymath/proven - -## Executive Summary - -**Core Architecture:** Idris2 library (45 source files, 90+ modules) with FFI bindings to 89 languages -**Current Version:** 1.0.0-production-release (per STATE.scm) -**Primary Issues:** Language policy violations, missing ROADMAP, incomplete testing documentation - ---- - -## ASCII Architecture Diagram (Anatomy) - -``` -┌─────────────────────────────────────────────────────────────────┐ -│ PROVEN LIBRARY ECOSYSTEM │ -└─────────────────────────────────────────────────────────────────┘ - - ┌──────────────────┐ - │ User's App Code │ - │ (Any Language) │ - └────────┬─────────┘ - │ - ┌────────────────────────┼────────────────────────┐ - │ │ │ - ▼ ▼ ▼ -┌──────────────┐ ┌──────────────┐ ┌──────────────┐ -│ Rust FFI │ │ Deno/JS FFI │ │ Gleam FFI │ -│ Binding │ ... │ Binding │ ... │ Binding │ -└──────┬───────┘ └──────┬───────┘ └──────┬───────┘ - │ │ │ - └────────────────────────┼────────────────────────┘ - │ - ┌───────────▼──────────┐ - │ ZIG FFI BRIDGE │ - │ (Pure ABI layer) │ - │ • No safety logic │ - │ • C ABI compatible │ - │ • Cross-platform │ - └───────────┬──────────┘ - │ - ┌───────────▼──────────────────────────────┐ - │ IDRIS2 CORE LIBRARY │ - │ │ - │ ┌────────────┐ ┌──────────────────┐ │ - │ │ Safe Types │ │ Dependent Types │ │ - │ │ • Result │ │ • Proofs │ │ - │ │ • Maybe │ │ • Totality check │ │ - │ └────────────┘ └──────────────────┘ │ - │ │ - │ ┌─────────────────────────────────┐ │ - │ │ 90+ Safety Modules │ │ - │ │ • SafeMath • SafeString │ │ - │ │ • SafeJson • SafeUrl │ │ - │ │ • SafePath • SafeCrypto │ │ - │ │ • SafeSQL • SafeNetwork │ │ - │ │ • ... (86 more modules) │ │ - │ └─────────────────────────────────┘ │ - └──────────────────────────────────────────┘ - -┌─────────────────────────────────────────────────────────────────┐ -│ VERIFICATION LAYERS │ -├─────────────────────────────────────────────────────────────────┤ -│ Compile Time: Type checker + Totality checker │ -│ Test Time: Property tests (29 files) + Unit tests (20) │ -│ Runtime: ClusterFuzzLite fuzzing │ -│ Formal: Echidnabot smart contract verification │ -└─────────────────────────────────────────────────────────────────┘ -``` - ---- - -## ASCII Process/Flow Diagram (Physiology) - -``` -USER CODE EXECUTION FLOW -═══════════════════════════════════════════════════════════════ - -1. API Call - ┌─────────────────────────────────────┐ - │ user_code.rs │ - │ │ - │ let safe_num = safe_add(5, 3)?; │ - └─────────────┬───────────────────────┘ - │ - ▼ -2. Rust Binding Layer (src/lib.rs) - ┌─────────────────────────────────────┐ - │ pub fn safe_add(a: i64, b: i64) │ - │ -> Result { │ - │ unsafe { │ - │ zig_ffi_safe_add(a, b) │ - │ } │ - │ } │ - └─────────────┬───────────────────────┘ - │ FFI call - ▼ -3. Zig FFI Bridge (ffi/zig/src/safe_math.zig) - ┌─────────────────────────────────────┐ - │ export fn zig_ffi_safe_add( │ - │ a: i64, b: i64 │ - │ ) callconv(.C) Result_i64 { │ - │ return idris2_safe_add(a, b); │ - │ } │ - └─────────────┬───────────────────────┘ - │ C ABI call - ▼ -4. Idris2 Core (src/Proven/SafeMath.idr) - ┌─────────────────────────────────────────────────┐ - │ export │ - │ idris2_safe_add : Int -> Int -> Result Int │ - │ idris2_safe_add a b = │ - │ if willOverflow a b │ - │ then Error Overflow │ - │ else OK (a + b) │ - │ │ - │ -- Totality checked: ✓ (no infinite loops) │ - │ -- Proofs: ∀a,b. result ∈ [INT_MIN, INT_MAX] │ - └─────────────┬───────────────────────────────────┘ - │ - ▼ -5. Result Propagation (back up the stack) - ┌─────────────────────────────────────┐ - │ Idris2 → Zig → Rust → User Code │ - │ │ - │ OK 8 (success case) │ - │ Error e (overflow/error case) │ - └─────────────────────────────────────┘ - -VERIFICATION AT EACH LAYER -═══════════════════════════════════════════════════════════════ - -Idris2 Layer: - ├─ Compile-time totality check (no crashes) - ├─ Type-level proofs (bounds, invariants) - └─ No exceptions (all fallible ops return Result) - -Zig Layer: - ├─ Comptime verification where possible - ├─ No safety logic (pure pass-through) - └─ C ABI compatibility enforced - -Binding Layer: - ├─ Type conversions only - ├─ Error propagation to native Result types - └─ No business logic - -Testing: - ├─ Property tests: ∀ inputs, check invariants hold - ├─ Unit tests: Known input/output pairs - ├─ Fuzz tests: Random inputs via ClusterFuzzLite - └─ Formal verification: Echidnabot for contracts -``` - ---- - -## Current Status (from STATE.scm) - -### Core Implementation -- **Idris2 Modules:** 90+ modules (100% complete per STATE.scm) -- **Test Files:** 29 property tests + 20 unit tests ✓ -- **Fuzzing:** ClusterFuzzLite integration ✓ -- **CI/CD:** GitHub Actions workflows ✓ -- **Documentation:** README.adoc ✓, CONTRIBUTING.adoc ✓, SECURITY.md ✓ - -### Language Bindings (89 targets) -STATE.scm marks all bindings as "complete" but quality varies: - -**Approved Languages (per RSR):** -- ✓ Rust (bindings/rust/) -- ✓ Deno (bindings/deno/) -- ✓ Gleam (bindings/gleam/) -- ✓ Elixir (bindings/elixir/) -- ✓ Haskell (bindings/haskell/) -- ✓ OCaml (bindings/ocaml/) -- ✓ Ada (bindings/ada/) -- ✓ Julia (bindings/julia/) -- ✓ Bash (bindings/bash/) -- ✓ Nickel (bindings/nickel/) -- ✓ Guile Scheme (bindings/guile/) - -**BANNED Languages Found:** -- ❌ Go (bindings/go/) - VIOLATION: Go is banned per RSR -- ❌ Python (bindings/python/) - VIOLATION: Python is banned per RSR -- ❌ TypeScript (bindings/typescript/) - VIOLATION: TypeScript is banned per RSR -- ❌ JavaScript with npm (bindings/javascript/package.json) - VIOLATION: npm/Node.js banned -- ❌ ReScript with node_modules (bindings/rescript/node_modules) - VIOLATION: Should use Deno - ---- - -## CRITICAL ISSUES BLOCKING V1 RELEASE - -### 1. Language Policy Violations (RSR Non-Compliance) - -**Issue:** Repository contains banned languages contrary to Rhodium Standard Repositories. - -**Violations:** -``` -bindings/go/ ← BANNED (use Rust instead) -bindings/python/ ← BANNED (use Julia/Rust/ReScript) -bindings/typescript/ ← BANNED (use ReScript) -bindings/javascript/package.json ← BANNED (use Deno with deno.json) -bindings/rescript/node_modules/ ← BANNED (migrate to Deno runtime) -bindings/malbolge/compiler.py ← BANNED (Python file) -bindings/malbolge/safe_malbolge.py ← BANNED (Python file) -``` - -**Action Required:** -1. **DELETE** bindings/go/, bindings/python/, bindings/typescript/ -2. **MIGRATE** bindings/javascript/ to use Deno (delete package.json, create deno.json) -3. **MIGRATE** bindings/rescript/ to use Deno (delete node_modules, package.json, package-lock.json) -4. **REWRITE** bindings/malbolge/*.py in Julia or Rust - -**Timeline:** IMMEDIATE (blocks v1 release) - -### 2. Missing ROADMAP File - -**Issue:** No ROADMAP.md or ROADMAP.adoc exists in repository root. - -**Current State:** -- STATE.scm contains milestones (v0.1.0 through v1.0.0) -- No forward-looking roadmap for v1.1, v2.0, v3.0 - -**Action Required:** -Create `ROADMAP.adoc` with: -- Completed milestones (v0.1.0 - v1.0.0) -- v1.1.0 plans -- v2.0.0 vision -- v3.0.0 long-term goals - -### 3. Testing Infrastructure Gaps - -**Current Testing:** -- ✓ Property tests (29 files) -- ✓ Unit tests (20 files) -- ✓ ClusterFuzzLite fuzzing -- ✓ Echidnabot (.echidnabot.toml exists) - -**Missing/Undocumented:** -- ⚠ **Formal Verification Status:** Echidnabot config exists but no results documented -- ⚠ **Stress Testing:** No stress test suite found -- ⚠ **Compilation Testing:** Not documented (though CI likely does this) -- ⚠ **Attack Surface Analysis:** Not documented -- ⚠ **Benchmarking:** `benchmarks/` directory exists but no results in docs - -**Action Required:** -1. Run and document Echidnabot results (formal verification) -2. Create stress test suite (concurrent operations, resource limits) -3. Document compilation test matrix (platforms, Idris versions) -4. Perform attack surface analysis (FFI boundaries, unsafe blocks) -5. Run benchmarks and document performance characteristics - -### 4. Missing Contractiles and a2ml/k9-svc - -**Issue:** No contractiles or a2ml/k9-svc integration found in repository. - -**Search Results:** -```bash -$ find . -name "*contractile*" -o -name "*a2ml*" -o -name "*k9-svc*" -(no results) -``` - -**Action Required:** -- Clarify: What are contractiles in the context of proven? -- Clarify: Is a2ml/k9-svc integration required for v1? -- If required: Add integration and documentation - -### 5. Cleanup Required - -**Stray Backup Directories:** -``` -.UNSAFE-ALL-BINDINGS-DELETED-20260125-095646/ -.UNSAFE-ALL-BINDINGS-DELETED-20260125-095657/ -.UNSAFE-ALL-BINDINGS-DELETED-20260125-095712/ -.UNSAFE-ALL-BINDINGS-DELETED-FINAL-20260125/ -``` - -**Action:** Delete these backup directories (old binding cleanup artifacts) - ---- - -## Route to V2 and V3 - -### v1.0.0 → v1.1.0 (Stabilization & Performance) - -**Timeline:** 1-2 months -**Focus:** Hardening, optimization, ecosystem growth - -**Planned Features:** -1. **Performance Optimization** - - Reduce FFI crossing overhead (batching APIs) - - WASM compilation target optimization - - Benchmark-driven tuning - -2. **Extended Bindings** - - Fill gaps in config languages (CUE, Starlark, HCL) - - Quantum computing bindings (Q#, OpenQASM) maturation - - Neuromorphic computing binding enhancements - -3. **Tooling Improvements** - - VS Code extension for proven types - - LSP integration for supported languages - - Better error messages at FFI boundary - -4. **Documentation** - - Video tutorials - - Interactive playground - - Migration guides from unsafe libraries - -### v2.0.0 (Advanced Verification & Concurrency) - -**Timeline:** 6-12 months -**Focus:** Concurrency proofs, advanced verification - -**Major Features:** -1. **SafeConcurrency Module** - - Type-safe concurrency primitives - - Data race prevention via types - - Deadlock-free guarantees - - Actor model implementation - -2. **Enhanced Verification** - - SMT solver integration (Z3, CVC5) - - Automated proof search - - Verification condition generation - - Runtime assertion synthesis - -3. **Advanced Safety Modules** - - SafeMemory (region-based memory safety) - - SafeProtocol (verified protocol implementations) - - SafeSmartContract (extended contract verification) - - SafeML (safe machine learning pipelines) - -4. **Ecosystem Integration** - - Package manager integration (cargo, npm alternatives) - - Build system plugins (Bazel, Buck2) - - IDE deep integration - -### v3.0.0 (Distributed Systems & Formal Methods) - -**Timeline:** 18-24 months -**Focus:** Distributed correctness, advanced formal methods - -**Major Features:** -1. **SafeDistributed Module** - - Consensus algorithm verification (Raft, Paxos) - - Network partition safety - - Byzantine fault tolerance - - Distributed transaction proofs - -2. **Advanced Type Features** - - Refinement types for all modules - - Session types for protocols - - Linear types for resource management - - Higher-order verification - -3. **Proof Automation** - - Tactic language for custom proofs - - Automated invariant discovery - - Proof repair and suggestions - - Interactive proof assistants - -4. **Research Integration** - - Academic paper implementations - - Benchmark suite for verification research - - Collaboration with PL research groups - - Formal methods education platform - ---- - -## Testing Requirements (Pre-V1 Checklist) - -### ✓ Completed Tests - -1. **Unit Tests** (20 files) - - Location: `tests/` - - Status: COMPLETE - -2. **Property Tests** (29 files) - - Location: `tests/` - - Status: COMPLETE - -3. **Fuzzing** (ClusterFuzzLite) - - Config: `.clusterfuzzlite/` - - Status: CONFIGURED - -### ⚠ Incomplete/Undocumented Tests - -4. **Execution Testing** - - Required: Test actual execution in all supported language bindings - - Status: LIKELY DONE (CI runs tests) but NOT DOCUMENTED - - Action: Create `docs/TESTING.md` documenting execution test results - -5. **Formal Verification** (Echidnabot) - - Config: `.echidnabot.toml` exists - - Status: CONFIGURED but NO RESULTS DOCUMENTED - - Action: Run Echidnabot, capture results, add to docs - -6. **Stress Testing** - - Required: High load, concurrent operations, resource exhaustion - - Status: NOT FOUND - - Action: Create `tests/stress/` with stress test suite - -7. **Compilation Testing** - - Required: Test compilation on all platforms (Linux, macOS, Windows, BSD) - - Required: Test multiple Idris2 versions - - Status: LIKELY DONE (CI) but NOT DOCUMENTED - - Action: Document compilation matrix in `docs/TESTING.md` - -8. **Attack Surface Analysis** - - Required: Analyze FFI boundaries, unsafe blocks, trust assumptions - - Status: NOT FOUND - - Action: Create `docs/ATTACK-SURFACE.md` with threat model - -9. **Benchmarking** - - Directory: `benchmarks/` exists - - Status: CODE EXISTS but NO RESULTS DOCUMENTED - - Action: Run benchmarks, create `docs/BENCHMARKS.md` with results - -### Testing Checklist for V1 - -- [ ] Run all unit tests, document results -- [ ] Run all property tests, document results -- [ ] Execute ClusterFuzzLite, document findings -- [ ] Run Echidnabot formal verification, document proofs -- [ ] Create and run stress test suite -- [ ] Document compilation test matrix -- [ ] Perform attack surface analysis -- [ ] Run benchmarks, document performance -- [ ] Test all 89 language bindings (at least smoke tests) -- [ ] Create comprehensive `docs/TESTING.md` - ---- - -## File Organization Audit - -### ✓ Present and Correct - -- STATE.scm (comprehensive, up-to-date) -- META.scm (ADRs, design rationale) -- ECOSYSTEM.scm (project relationships) -- README.adoc (excellent, detailed) -- CONTRIBUTING.adoc ✓ -- CODE_OF_CONDUCT.md ✓ -- SECURITY.md ✓ -- CHANGELOG.md ✓ -- .editorconfig ✓ -- LICENSE files (MPL-2.0-or-later) ✓ - -### ⚠ Missing or Incomplete - -- **ROADMAP.adoc** - MISSING (required for RSR compliance) -- **docs/TESTING.md** - MISSING (should document all test results) -- **docs/BENCHMARKS.md** - MISSING (benchmark results) -- **docs/ATTACK-SURFACE.md** - MISSING (security analysis) -- **.well-known/** directory - NOT CHECKED (RSR requirement) - -### 🗑 To Delete - -- `.UNSAFE-ALL-BINDINGS-DELETED-*` directories (4 backup dirs) -- `bindings/go/` (language violation) -- `bindings/python/` (language violation) -- `bindings/typescript/` (language violation) -- `bindings/javascript/package.json` (migrate to Deno) -- `bindings/rescript/node_modules/` (migrate to Deno) -- `bindings/malbolge/*.py` (rewrite in Rust/Julia) - ---- - -## Code Optimization Opportunities - -### 1. Idris Code Incorporation - -**Current State:** -- Core library is 100% Idris2 -- 45 .idr source files in `src/Proven/` - -**Optimization:** -- Review proven-malbolge-toolchain for reusable Idris patterns -- Incorporate proven-concat streaming optimizations -- Integrate ProvenCrypto.jl Julia code for numeric algorithms (convert to Idris) - -**Action:** Audit related repos for Idris code to merge: -```bash -~/Documents/hyperpolymath-repos/ephapax-proven/ -~/Documents/hyperpolymath-repos/ephapax-proven-ffi/ -~/Documents/hyperpolymath-repos/ephapax-proven-sys/ -~/Documents/hyperpolymath-repos/proven-concat/ -~/Documents/hyperpolymath-repos/ProvenCrypto.jl/ -~/Documents/hyperpolymath-repos/proven-http/ -~/Documents/hyperpolymath-repos/proven-malbolge-toolchain/ -~/Documents/hyperpolymath-repos/proven-tui/ -``` - -### 2. FFI Overhead Reduction - -**Current Issue:** Each operation crosses FFI boundary (Rust → Zig → Idris2) - -**Optimization Strategies:** -1. **Batching API:** Accept arrays of operations, process in Idris2, return array -2. **Caching:** Memoize pure functions at Zig layer -3. **WASM Optimization:** Compile Idris2 directly to WASM (bypass Zig for web) - -### 3. Compiler Optimization Flags - -**Check:** Are optimal Idris2 compiler flags used? -- `--cg chez` vs `--cg refc` (C backend) benchmark -- Inlining thresholds -- Dead code elimination - ---- - -## Annotation Status - -**Requirement:** All files must be annotated. - -**Current State:** -- Idris2 files: Likely have doc comments (need verification) -- Binding files: Varies by language -- Documentation files: Well-written - -**Action Required:** -1. Audit all .idr files for documentation comments -2. Ensure all public functions have doc comments -3. Add module-level documentation to all files -4. Verify SPDX headers on all source files - ---- - -## RSR Compliance Checklist - -Based on Rhodium Standard Repositories (RSR): - -- [x] STATE.scm present and current -- [x] META.scm present with ADRs -- [x] ECOSYSTEM.scm present -- [x] README.adoc comprehensive -- [x] CONTRIBUTING.adoc -- [x] CODE_OF_CONDUCT.md -- [x] SECURITY.md -- [x] LICENSE files (MPL-2.0-or-later) -- [ ] **ROADMAP.adoc** - MISSING -- [x] .editorconfig -- [ ] **No banned languages** - VIOLATED (Go, Python, TypeScript, npm) -- [ ] .well-known/ directory - NOT VERIFIED -- [ ] All files annotated - NOT VERIFIED -- [x] GitHub Actions workflows with SPDX headers -- [x] OpenSSF Scorecard (assumed passing based on org standards) - ---- - -## Package Re-Release Readiness - -### Blockers (MUST FIX) - -1. ❌ **Remove banned languages** (Go, Python, TypeScript, npm) -2. ❌ **Create ROADMAP.adoc** -3. ❌ **Document testing results** (formal verification, stress, benchmarks) -4. ❌ **Clarify contractiles/a2ml/k9-svc requirement** - -### Recommended (SHOULD FIX) - -5. ⚠ Clean up .UNSAFE-* backup directories -6. ⚠ Verify all files have SPDX headers -7. ⚠ Verify all public APIs have doc comments -8. ⚠ Run and document benchmarks -9. ⚠ Perform attack surface analysis - -### Nice to Have (MAY FIX) - -10. ○ Incorporate Idris code from related repos -11. ○ Optimize FFI overhead -12. ○ Add tooling (VS Code extension, LSP) - ---- - -## Recommended Action Plan - -### Phase 1: Critical Blockers (Week 1) - -1. **Delete banned language bindings** - ```bash - cd ~/Documents/hyperpolymath-repos/proven - rm -rf bindings/go bindings/python bindings/typescript - rm bindings/javascript/package.json - rm -rf bindings/rescript/node_modules bindings/rescript/package*.json - rm bindings/malbolge/*.py - git commit -m "Remove banned languages per RSR" - ``` - -2. **Create ROADMAP.adoc** - - Extract milestones from STATE.scm - - Add v1.1, v2.0, v3.0 vision (see above) - -3. **Run and document testing** - - Execute Echidnabot: `echidnabot run` - - Create stress tests in `tests/stress/` - - Run benchmarks, capture results - - Create `docs/TESTING.md` - -### Phase 2: Quality Assurance (Week 2) - -4. **Annotation audit** - - Check all .idr files for doc comments - - Add SPDX headers where missing - - Verify module-level docs - -5. **Attack surface analysis** - - Document FFI trust boundaries - - List all `unsafe` blocks - - Create threat model - - Write `docs/ATTACK-SURFACE.md` - -6. **Benchmarking** - - Run all benchmarks - - Create `docs/BENCHMARKS.md` - - Compare to unsafe alternatives - -### Phase 3: Optimization (Week 3-4) - -7. **Code optimization** - - Audit related repos for Idris code to merge - - Implement batching APIs - - Optimize compiler flags - -8. **Documentation polish** - - Video walkthrough - - Interactive examples - - Migration guides - -### Phase 4: Release (Week 5) - -9. **Final checklist** - - All tests passing - - Documentation complete - - No RSR violations - - Version tagged - -10. **Package publication** - - GitHub Packages (Container) - - Language-specific registries (crates.io, JSR, etc.) - - Announce on hyperpolymath channels - ---- - -## Questions Requiring Clarification - -1. **Contractiles:** What are contractiles in the proven context? Are they required for v1? - -2. **a2ml/k9-svc:** What is a2ml/k9-svc? Is integration required for v1 release? - -3. **Malbolge bindings:** The bindings/malbolge/ directory has Python files. Should Malbolge bindings be: - - Rewritten in Rust/Julia? - - Removed entirely (Malbolge is esoteric/impractical)? - - Kept as educational examples? - -4. **ReScript migration:** Should bindings/rescript/ use: - - Deno runtime (deno.json)? - - Bun is banned, so not an option - - Direct compilation to JS (ES6 modules) without runtime? - -5. **JavaScript bindings:** Should bindings/javascript/ be: - - Deleted (TypeScript/JavaScript banned)? - - Migrated to Deno (bindings/deno/ already exists)? - - Kept as pure ES6 modules without npm? - ---- - -## Summary - -**proven** is architecturally sound with excellent Idris2 core code and comprehensive documentation. The main issues are: - -1. **Language policy violations** (banned languages present) -2. **Missing ROADMAP** -3. **Testing documentation gaps** -4. **Unclear contractiles/a2ml requirement** - -With 1-2 weeks of cleanup and documentation work, proven will be ready for v1.0.0 release. The architecture is solid, the core implementation is complete, and the verification foundations are in place. - -The route to v2 and v3 focuses on concurrency, advanced verification, and distributed systems—all natural extensions of the current architecture. diff --git a/SECURITY-CRITICAL-FIXES-PLAN.adoc b/SECURITY-CRITICAL-FIXES-PLAN.adoc new file mode 100644 index 00000000..bb42f485 --- /dev/null +++ b/SECURITY-CRITICAL-FIXES-PLAN.adoc @@ -0,0 +1,226 @@ +== Security Critical Fixes for proven Repo + +=== Analysis Date: 2026-01-25 + +____ +*Status:* Package rejected from opam-repository due to poor code quality +*Scanner Results:* 130 findings (5 CRITICAL, 124 HIGH, 1 MEDIUM) *Root +Cause:* Claims "`formally verified safety`" but uses unsafe patterns +throughout +____ + +''''' + +=== CRITICAL Issues (Must Fix Before Re-Submission) + +==== 1. Proven_SafeCron.res (5 CRITICAL) + +*Location:* `+bindings/rescript/src/Proven_SafeCron.res+` lines 309-313 + +*Problem:* + +[source,rescript] +---- +// UNSAFE: getExn can crash if array doesn't have 5 elements +let minuteField = Belt.Array.getExn(fields, 0) +let hourField = Belt.Array.getExn(fields, 1) +let dayField = Belt.Array.getExn(fields, 2) +let monthField = Belt.Array.getExn(fields, 3) +let dowField = Belt.Array.getExn(fields, 4) +---- + +*Fix:* + +[source,rescript] +---- +// SAFE: Pattern match on exactly 5 fields +switch fields { +| [minuteField, hourField, dayField, monthField, dowField] => + // Process fields... +| _ => Error(InvalidFieldCount) +} +---- + +*Impact:* HIGH - This is the parse() function for cron expressions. +Malformed input WILL crash the application. + +*Estimated Fix Time:* 10 minutes + +''''' + +=== HIGH Severity Issues (124 unwrap() calls) + +==== Priority Files (Top 5 - 63 unwraps total) + +[cols=",,",options="header",] +|=== +|File |Unwraps |Description +|`+safe_ml.rs+` |15 |Machine learning validation +|`+safe_tensor.rs+` |14 |Tensor operations +|`+safe_float.rs+` |14 |Floating point validation +|`+safe_version.rs+` |10 |Semantic version parsing +|`+safe_math.rs+` |10 |Mathematical operations +|=== + +*Pattern:* All files named `+safe_*+` but using unsafe `+unwrap()+` +calls. + +*Fix Strategy:* 1. Replace `+unwrap()+` with +`+expect("descriptive message")+` 2. Or use `+?+` operator for proper +error propagation 3. Or use pattern matching for explicit handling + +*Example Fix (safe_ml.rs):* + +[source,rust] +---- +// BEFORE: +let value = json["key"].as_str().unwrap(); + +// AFTER (Option 1 - expect): +let value = json["key"].as_str() + .expect("JSON key 'key' must be a string - validation bug"); + +// AFTER (Option 2 - ? operator): +let value = json["key"].as_str() + .ok_or(Error::InvalidJson)?; + +// AFTER (Option 3 - match): +let value = match json["key"].as_str() { + Some(v) => v, + None => return Err(Error::InvalidJson), +}; +---- + +*Estimated Fix Time:* 2-3 hours for all 124 unwraps + +''''' + +=== Additional Issues (From opam PR Review) + +==== 1. License Compliance ✅ FIXED + +* Changed from `+PMPL-1.0+` to `+MPL-2.0+` (corrected license +identifier) +* *Note:* Line 1 of SafeCron.res still says `+PMPL-1.0+` - needs update + +==== 2. RFC 5321 Compliance Claims ❌ NOT IMPLEMENTED + +* Reviewer noted: "`claims RFC 5321 compliance but doesn’t implement it +correctly`" +* Need to either: (a) fix implementation, or (b) remove claims from +documentation + +==== 3. dune-project Structure ✅ FIXED + +* Fixed invalid first line (SPDX header before language declaration) + +''''' + +=== Fix Priority + +==== Phase 1: CRITICAL (Must Do Before Re-Submission) ⏱️ 15 mins + +[arabic] +. Fix 5 `+getExn+` calls in Proven_SafeCron.res +. Update license header in Proven_SafeCron.res (`+PMPL-1.0+` → +`+MPL-2.0+`) +. Test that cron parsing doesn’t crash on malformed input + +==== Phase 2: HIGH PRIORITY (Should Do) ⏱️ 2-3 hours + +[arabic] +. Fix top 5 files (63 unwraps) +* safe_ml.rs (15) +* safe_tensor.rs (14) +* safe_float.rs (14) +* safe_version.rs (10) +* safe_math.rs (10) +. Add fuzzing tests to catch panics +. Verify RFC 5321 claims or remove them + +==== Phase 3: MEDIUM PRIORITY (Good To Have) ⏱️ 3-4 hours + +[arabic] +. Fix remaining 61 unwraps in other files +. Add comprehensive error handling tests +. Update documentation to accurately reflect safety guarantees + +''''' + +=== Testing Plan + +==== 1. Critical Path Testing + +[source,bash] +---- +# Test cron parsing with malformed input +rescript build # Ensure SafeCron compiles +dune test # Run existing tests + +# Fuzzing (if ClusterFuzzLite configured) +cargo fuzz run fuzz_cron -- -max_total_time=60 +---- + +==== 2. Regression Testing + +[source,bash] +---- +# After each fix, verify: +1. Code compiles +2. Tests pass +3. No new unwrap/getExn calls added +---- + +==== 3. Dogfooding + +[source,bash] +---- +# Scan proven repo with hypatia after fixes +./hypatia-v2 ../proven 2>/dev/null | jq '.scan_info' +# Expected: 0 critical, <50 high (down from 124) +---- + +''''' + +=== Risk Assessment + +==== If NOT Fixed: + +* ❌ Package will be rejected from opam-repository again +* ❌ Production use WILL cause crashes on malformed input +* ❌ Reputation damage ("`formally verified`" library that crashes) +* ❌ Cannot be trusted for safety-critical applications + +==== If Fixed: + +* ✅ Safe to use in production +* ✅ Can be re-submitted to opam-repository +* ✅ Builds trust with OCaml community +* ✅ Validates hypatia scanner effectiveness + +''''' + +=== Automation Opportunities + +==== Auto-Fix Candidates (Low Risk) + +* Unwrap → expect conversion (mechanical transformation) +* License header updates (text replacement) + +==== Manual Review Required (High Risk) + +* getExn → pattern matching (logic changes) +* RFC 5321 compliance verification (domain knowledge) + +''''' + +=== Next Steps + +[arabic] +. *Immediate:* Fix 5 critical getExn calls in SafeCron +. *This Session:* Fix top 5 unwrap files if time permits +. *Follow-up:* Complete remaining unwraps + comprehensive testing +. *Re-submission:* After all critical + high priority fixes verified + +*Estimated Total Time:* 3-4 hours for complete fix *Critical Path Time:* +15 minutes for re-submission eligibility diff --git a/SECURITY-CRITICAL-FIXES-PLAN.md b/SECURITY-CRITICAL-FIXES-PLAN.md deleted file mode 100644 index b2609618..00000000 --- a/SECURITY-CRITICAL-FIXES-PLAN.md +++ /dev/null @@ -1,189 +0,0 @@ -# Security Critical Fixes for proven Repo -## Analysis Date: 2026-01-25 - -> **Status:** Package rejected from opam-repository due to poor code quality -> **Scanner Results:** 130 findings (5 CRITICAL, 124 HIGH, 1 MEDIUM) -> **Root Cause:** Claims "formally verified safety" but uses unsafe patterns throughout - ---- - -## CRITICAL Issues (Must Fix Before Re-Submission) - -### 1. Proven_SafeCron.res (5 CRITICAL) - -**Location:** `bindings/rescript/src/Proven_SafeCron.res` lines 309-313 - -**Problem:** -```rescript -// UNSAFE: getExn can crash if array doesn't have 5 elements -let minuteField = Belt.Array.getExn(fields, 0) -let hourField = Belt.Array.getExn(fields, 1) -let dayField = Belt.Array.getExn(fields, 2) -let monthField = Belt.Array.getExn(fields, 3) -let dowField = Belt.Array.getExn(fields, 4) -``` - -**Fix:** -```rescript -// SAFE: Pattern match on exactly 5 fields -switch fields { -| [minuteField, hourField, dayField, monthField, dowField] => - // Process fields... -| _ => Error(InvalidFieldCount) -} -``` - -**Impact:** HIGH - This is the parse() function for cron expressions. Malformed input WILL crash the application. - -**Estimated Fix Time:** 10 minutes - ---- - -## HIGH Severity Issues (124 unwrap() calls) - -### Priority Files (Top 5 - 63 unwraps total) - -| File | Unwraps | Description | -|------|---------|-------------| -| `safe_ml.rs` | 15 | Machine learning validation | -| `safe_tensor.rs` | 14 | Tensor operations | -| `safe_float.rs` | 14 | Floating point validation | -| `safe_version.rs` | 10 | Semantic version parsing | -| `safe_math.rs` | 10 | Mathematical operations | - -**Pattern:** All files named `safe_*` but using unsafe `unwrap()` calls. - -**Fix Strategy:** -1. Replace `unwrap()` with `expect("descriptive message")` -2. Or use `?` operator for proper error propagation -3. Or use pattern matching for explicit handling - -**Example Fix (safe_ml.rs):** -```rust -// BEFORE: -let value = json["key"].as_str().unwrap(); - -// AFTER (Option 1 - expect): -let value = json["key"].as_str() - .expect("JSON key 'key' must be a string - validation bug"); - -// AFTER (Option 2 - ? operator): -let value = json["key"].as_str() - .ok_or(Error::InvalidJson)?; - -// AFTER (Option 3 - match): -let value = match json["key"].as_str() { - Some(v) => v, - None => return Err(Error::InvalidJson), -}; -``` - -**Estimated Fix Time:** 2-3 hours for all 124 unwraps - ---- - -## Additional Issues (From opam PR Review) - -### 1. License Compliance ✅ FIXED -- Changed from `PMPL-1.0` to `MPL-2.0` (corrected license identifier) -- **Note:** Line 1 of SafeCron.res still says `PMPL-1.0` - needs update - -### 2. RFC 5321 Compliance Claims ❌ NOT IMPLEMENTED -- Reviewer noted: "claims RFC 5321 compliance but doesn't implement it correctly" -- Need to either: (a) fix implementation, or (b) remove claims from documentation - -### 3. dune-project Structure ✅ FIXED -- Fixed invalid first line (SPDX header before language declaration) - ---- - -## Fix Priority - -### Phase 1: CRITICAL (Must Do Before Re-Submission) ⏱️ 15 mins -1. Fix 5 `getExn` calls in Proven_SafeCron.res -2. Update license header in Proven_SafeCron.res (`PMPL-1.0` → `MPL-2.0`) -3. Test that cron parsing doesn't crash on malformed input - -### Phase 2: HIGH PRIORITY (Should Do) ⏱️ 2-3 hours -1. Fix top 5 files (63 unwraps) - - safe_ml.rs (15) - - safe_tensor.rs (14) - - safe_float.rs (14) - - safe_version.rs (10) - - safe_math.rs (10) -2. Add fuzzing tests to catch panics -3. Verify RFC 5321 claims or remove them - -### Phase 3: MEDIUM PRIORITY (Good To Have) ⏱️ 3-4 hours -1. Fix remaining 61 unwraps in other files -2. Add comprehensive error handling tests -3. Update documentation to accurately reflect safety guarantees - ---- - -## Testing Plan - -### 1. Critical Path Testing -```bash -# Test cron parsing with malformed input -rescript build # Ensure SafeCron compiles -dune test # Run existing tests - -# Fuzzing (if ClusterFuzzLite configured) -cargo fuzz run fuzz_cron -- -max_total_time=60 -``` - -### 2. Regression Testing -```bash -# After each fix, verify: -1. Code compiles -2. Tests pass -3. No new unwrap/getExn calls added -``` - -### 3. Dogfooding -```bash -# Scan proven repo with hypatia after fixes -./hypatia-v2 ../proven 2>/dev/null | jq '.scan_info' -# Expected: 0 critical, <50 high (down from 124) -``` - ---- - -## Risk Assessment - -### If NOT Fixed: -- ❌ Package will be rejected from opam-repository again -- ❌ Production use WILL cause crashes on malformed input -- ❌ Reputation damage ("formally verified" library that crashes) -- ❌ Cannot be trusted for safety-critical applications - -### If Fixed: -- ✅ Safe to use in production -- ✅ Can be re-submitted to opam-repository -- ✅ Builds trust with OCaml community -- ✅ Validates hypatia scanner effectiveness - ---- - -## Automation Opportunities - -### Auto-Fix Candidates (Low Risk) -- Unwrap → expect conversion (mechanical transformation) -- License header updates (text replacement) - -### Manual Review Required (High Risk) -- getExn → pattern matching (logic changes) -- RFC 5321 compliance verification (domain knowledge) - ---- - -## Next Steps - -1. **Immediate:** Fix 5 critical getExn calls in SafeCron -2. **This Session:** Fix top 5 unwrap files if time permits -3. **Follow-up:** Complete remaining unwraps + comprehensive testing -4. **Re-submission:** After all critical + high priority fixes verified - -**Estimated Total Time:** 3-4 hours for complete fix -**Critical Path Time:** 15 minutes for re-submission eligibility diff --git a/SECURITY.adoc b/SECURITY.adoc new file mode 100644 index 00000000..56eebc0a --- /dev/null +++ b/SECURITY.adoc @@ -0,0 +1,67 @@ +== Security Policy + +=== Supported Versions + +[cols=",",options="header",] +|=== +|Version |Supported +|0.x.x |:white_check_mark: +|=== + +=== Reporting a Vulnerability + +*Do not report security vulnerabilities through public GitHub issues.* + +Instead, please report them via: + +[arabic] +. *Email*: security@hyperpolymath.org (preferred) +. *GitHub Security Advisories*: +https://github.com/hyperpolymath/bulletproof-core/security/advisories/new[Create +a private advisory] + +==== What to include + +* Type of vulnerability (buffer overflow, injection, etc.) +* Full path to affected source file(s) +* Step-by-step instructions to reproduce +* Proof-of-concept or exploit code (if available) +* Impact assessment + +==== Response Timeline + +* *Acknowledgment*: Within 48 hours +* *Initial assessment*: Within 7 days +* *Resolution target*: Within 90 days (may vary based on severity) + +==== Safe Harbor + +We consider security research conducted in accordance with this policy +to be: - Authorized - Lawful - Helpful + +We will not pursue legal action against researchers who follow this +policy. + +=== Security Measures + +This project implements: + +* [x] Dependabot alerts enabled +* [x] CodeQL static analysis +* [x] OpenSSF Scorecard compliance +* [x] Signed commits required +* [x] Branch protection enabled +* [ ] Formal verification (Idris 2 dependent types) +* [ ] Security audit (planned for 1.0) + +=== Known Limitations + +The core Idris 2 code is formally verified. However: + +[arabic] +. *Language bindings* (Python, Rust, JS) are human-written and may +contain bugs +. *FFI boundaries* are potential attack surfaces +. *Build toolchain* (Zig, C compiler) is not verified + +Report issues in any layer — we take all security concerns seriously. diff --git a/SECURITY.md b/SECURITY.md deleted file mode 100644 index f98940fe..00000000 --- a/SECURITY.md +++ /dev/null @@ -1,61 +0,0 @@ -# Security Policy - -## Supported Versions - -| Version | Supported | -| ------- | ------------------ | -| 0.x.x | :white_check_mark: | - -## Reporting a Vulnerability - -**Do not report security vulnerabilities through public GitHub issues.** - -Instead, please report them via: - -1. **Email**: security@hyperpolymath.org (preferred) -2. **GitHub Security Advisories**: [Create a private advisory](https://github.com/hyperpolymath/bulletproof-core/security/advisories/new) - -### What to include - -- Type of vulnerability (buffer overflow, injection, etc.) -- Full path to affected source file(s) -- Step-by-step instructions to reproduce -- Proof-of-concept or exploit code (if available) -- Impact assessment - -### Response Timeline - -- **Acknowledgment**: Within 48 hours -- **Initial assessment**: Within 7 days -- **Resolution target**: Within 90 days (may vary based on severity) - -### Safe Harbor - -We consider security research conducted in accordance with this policy to be: -- Authorized -- Lawful -- Helpful - -We will not pursue legal action against researchers who follow this policy. - -## Security Measures - -This project implements: - -- [x] Dependabot alerts enabled -- [x] CodeQL static analysis -- [x] OpenSSF Scorecard compliance -- [x] Signed commits required -- [x] Branch protection enabled -- [ ] Formal verification (Idris 2 dependent types) -- [ ] Security audit (planned for 1.0) - -## Known Limitations - -The core Idris 2 code is formally verified. However: - -1. **Language bindings** (Python, Rust, JS) are human-written and may contain bugs -2. **FFI boundaries** are potential attack surfaces -3. **Build toolchain** (Zig, C compiler) is not verified - -Report issues in any layer — we take all security concerns seriously. diff --git a/TEST-NEEDS.adoc b/TEST-NEEDS.adoc new file mode 100644 index 00000000..0f645c81 --- /dev/null +++ b/TEST-NEEDS.adoc @@ -0,0 +1,155 @@ +== TEST-NEEDS.md — proven + +____ +Generated 2026-03-29 by punishing audit. Updated 2026-04-04: CRG Grade C +blitz — P2P + E2E + aspect tests added. +____ + +=== CRG Grade: C — ACHIEVED 2026-04-04 + +*Test count after blitz:* 54 tests pass in Rust binding (up from 8) + +[width="100%",cols="20%,56%,10%,14%",options="header",] +|=== +|Category |Module |Count |Status +|Unit |`+core::tests+` |8 |PASS +|P2P |`+tests::p2p+` (proptest) |16 |PASS +|E2E |`+tests::e2e+` (structural/reflexive) |10 |PASS +|Aspect |`+tests::aspect+` (security/boundary) |20 |PASS +|*Total* | |*54* |*PASS* +|=== + +==== P2P Tests Added (`+bindings/rust/src/tests/p2p.rs+`) + +Property-based tests using `+proptest+`: - +`+prop_bounded_percentage_in_range+` — any i64 in [0,100] accepted by +Bounded<0,100> - `+prop_bounded_percentage_out_of_range+` — any i64 +outside [0,100] produces OutOfBounds - `+prop_bounded_port_valid+` — +Bounded<0,65535> accepts full port range - `+prop_bounded_byte_valid+` — +Bounded<0,255> accepts full byte range - +`+prop_non_empty_from_nonempty_vec+` — NonEmpty preserves length +invariant - `+prop_non_empty_from_empty_vec_is_none+` — NonEmpty rejects +empty input - `+prop_non_empty_roundtrip+` — from_vec → to_vec preserves +all elements - `+prop_status_ok_is_ok+` — STATUS_OK always maps to Ok - +`+prop_nonzero_status_is_err+` — any non-zero status produces Err - +`+prop_int_result_ok_preserves_value+` — OK IntResult forwards value +unchanged - `+prop_int_result_overflow_is_err+` — STATUS_ERR_OVERFLOW +always maps to Error::Overflow - `+prop_int_result_underflow_is_err+` — +STATUS_ERR_UNDERFLOW always maps to Error::Underflow - +`+prop_int_result_div_zero_is_err+` — STATUS_ERR_DIVISION_BY_ZERO always +maps correctly - `+prop_error_display_never_empty+` — Display never +produces empty string for any variant - +`+prop_error_clone_equals_original+` — Clone/PartialEq consistency - +`+prop_status_to_result_is_deterministic+` — same code always yields +same variant + +==== E2E Tests Added (`+bindings/rust/src/tests/e2e.rs+`) + +Structural/reflexive end-to-end tests (no libproven.so required): - +`+e2e_ffi_status_constants_are_defined+` — all 12 status constants have +correct values - `+e2e_full_status_to_error_chain+` — every error code +maps to the correct Error variant - `+e2e_int_result_chain_ok+` — +IntResult OK chain: status=0, value=42 → Ok(42) - +`+e2e_int_result_chain_errors+` — IntResult error chain: +overflow/underflow/div-zero - `+e2e_ffi_struct_sizes_match_c_abi+` — +IntResult=16B, BoolResult=5–8B, FloatResult=16B - +`+e2e_type_aliases_exported+` — Bounded/NonEmpty accessible from crate +root - `+e2e_error_implements_std_error+` — Error is boxable as dyn +std::error::Error - `+e2e_error_display_messages+` — all variants have +meaningful display strings - `+e2e_bounded_const_bounds+` — MIN/MAX +const accessors are correct - `+e2e_int_result_alignment+` — IntResult +has 8-byte alignment (C ABI match) + +==== Aspect Tests Added (`+bindings/rust/src/tests/aspect.rs+`) + +Security, boundary, and concurrency aspects: - +`+aspect_bounded_i64_max_is_rejected+` — i64::MAX rejected by Percentage +- `+aspect_bounded_i64_min_is_rejected+` — i64::MIN rejected by any +non-negative type - `+aspect_bounded_minus_one_is_rejected+` — -1 +rejected by Port [0,65535] - `+aspect_bounded_65536_exceeds_port_range+` +— 65536 rejected by Port - `+aspect_bounded_zero_is_valid+` — 0 is valid +lower boundary - `+aspect_bounded_100_is_valid+` — 100 is valid upper +boundary for Percentage - `+aspect_bounded_101_exceeds_percentage+` — +101 is rejected - `+aspect_ffi_null_pointer_is_error+` — +STATUS_ERR_NULL_POINTER never succeeds - +`+aspect_ffi_int_result_null_pointer_discards_value+` — error value +field is ignored - `+aspect_overflow_and_underflow_are_distinct+` — two +distinct error variants - `+aspect_div_zero_is_distinct_from_overflow+` +— div-zero not confused with overflow - +`+aspect_ffi_allocation_failed_is_error+` — allocation failures always +produce Err - `+aspect_not_implemented_status_maps_correctly+` — -99 +maps to NotImplemented - `+aspect_unknown_status_codes_produce_error+` — +gaps in code space → Error::Unknown - +`+aspect_int_result_error_value_is_discarded+` — value field ignored for +all error codes - `+aspect_error_variants_are_distinct+` — no two +variants accidentally equal - `+aspect_error_is_send_sync+` — Error: +Send + Sync - `+aspect_result_is_send_sync+` — Result: Send + Sync - +`+aspect_bounded_get_always_in_range+` — get() always within [MIN,MAX] - +`+aspect_only_zero_is_success+` — STATUS_OK=0 is the unique success code + +''''' + +=== Original State (Pre-Blitz) + +[width="100%",cols="50%,25%,25%",options="header",] +|=== +|Category |Count |Notes +|Unit tests |~15 |Zig FFI integration_test, Go proven_test, Gleam +proven_test, Elixir proven_test, Lua proven_spec, Ruby proven_spec, Nim +test_proven, OCaml test_proven, Ada test_proven, Ephapax tests +(test_all.zig, test_simple.c), Python test_benchmarks + +|Integration |1 |Zig FFI integration test + +|E2E |0 |None + +|Benchmarks |5 |Rust benches (benchmarks.rs, safe_math.rs), JavaScript +benchmark.mjs, Python test_benchmarks.py, benchmarks.ipkg (Idris2) +|=== + +*Source modules:* ~1005 across Idris2 core (312 .idr files), Zig FFI (91 +files), Rust (76 files), and 100+ language bindings. Bindings span ~120 +languages from Ada to Zsh. + +=== Remaining Gaps (Post-Blitz) + +==== Runtime P2P Tests (require libproven.so) + +* [ ] Safe math overflow: `+SafeMath::add(i64::MAX, 1)+` → +`+Err(Overflow)+` +* [ ] Safe math underflow: `+SafeMath::sub(i64::MIN, 1)+` → +`+Err(Underflow)+` +* [ ] Safe div-zero: `+SafeMath::div(1, 0)+` → `+Err(DivisionByZero)+` +* [ ] Safe abs of MIN: `+SafeMath::abs(i64::MIN)+` → `+Err(Overflow)+` + +==== Full Proof-Chain E2E (require Idris2 build + libproven.so) + +* [ ] Idris2 spec → Zig FFI → Rust binding → verification roundtrip +* [ ] Multi-language consistency: Rust + Gleam + Elixir agree on all +operations + +==== Cross-Binding Coverage + +* [ ] 89% of bindings (109/120) still untested +* [ ] Cross-binding consistency: same operation in 120 bindings produces +identical results + +==== Benchmarks Needed + +* [ ] Zig FFI call overhead +* [ ] Cross-binding comparison (all 120 languages, same operations) +* [ ] Compilation time benchmarks + +==== Self-Tests + +* [ ] All 312 Idris2 proofs type-check (CI gate) +* [ ] FFI exports match Idris2 specifications +* [ ] Binding API surface matches core API + +=== Priority + +*CRG Grade C ACHIEVED* for the Rust binding via structural and +property-based tests. The next step toward Grade B+ is runtime +integration tests (requiring `+libproven.so+` from a full Zig+Idris2 +build) and expanding coverage to other language bindings (currently 9.2% +coverage across 120 languages). diff --git a/TEST-NEEDS.md b/TEST-NEEDS.md deleted file mode 100644 index cd9447f7..00000000 --- a/TEST-NEEDS.md +++ /dev/null @@ -1,117 +0,0 @@ -# TEST-NEEDS.md — proven - -> Generated 2026-03-29 by punishing audit. -> Updated 2026-04-04: CRG Grade C blitz — P2P + E2E + aspect tests added. - -## CRG Grade: C — ACHIEVED 2026-04-04 - -**Test count after blitz:** 54 tests pass in Rust binding (up from 8) - -| Category | Module | Count | Status | -|-------------|---------------------------------------|-------|----------| -| Unit | `core::tests` | 8 | PASS | -| P2P | `tests::p2p` (proptest) | 16 | PASS | -| E2E | `tests::e2e` (structural/reflexive) | 10 | PASS | -| Aspect | `tests::aspect` (security/boundary) | 20 | PASS | -| **Total** | | **54**| **PASS** | - -### P2P Tests Added (`bindings/rust/src/tests/p2p.rs`) - -Property-based tests using `proptest`: -- `prop_bounded_percentage_in_range` — any i64 in [0,100] accepted by Bounded<0,100> -- `prop_bounded_percentage_out_of_range` — any i64 outside [0,100] produces OutOfBounds -- `prop_bounded_port_valid` — Bounded<0,65535> accepts full port range -- `prop_bounded_byte_valid` — Bounded<0,255> accepts full byte range -- `prop_non_empty_from_nonempty_vec` — NonEmpty preserves length invariant -- `prop_non_empty_from_empty_vec_is_none` — NonEmpty rejects empty input -- `prop_non_empty_roundtrip` — from_vec → to_vec preserves all elements -- `prop_status_ok_is_ok` — STATUS_OK always maps to Ok -- `prop_nonzero_status_is_err` — any non-zero status produces Err -- `prop_int_result_ok_preserves_value` — OK IntResult forwards value unchanged -- `prop_int_result_overflow_is_err` — STATUS_ERR_OVERFLOW always maps to Error::Overflow -- `prop_int_result_underflow_is_err` — STATUS_ERR_UNDERFLOW always maps to Error::Underflow -- `prop_int_result_div_zero_is_err` — STATUS_ERR_DIVISION_BY_ZERO always maps correctly -- `prop_error_display_never_empty` — Display never produces empty string for any variant -- `prop_error_clone_equals_original` — Clone/PartialEq consistency -- `prop_status_to_result_is_deterministic` — same code always yields same variant - -### E2E Tests Added (`bindings/rust/src/tests/e2e.rs`) - -Structural/reflexive end-to-end tests (no libproven.so required): -- `e2e_ffi_status_constants_are_defined` — all 12 status constants have correct values -- `e2e_full_status_to_error_chain` — every error code maps to the correct Error variant -- `e2e_int_result_chain_ok` — IntResult OK chain: status=0, value=42 → Ok(42) -- `e2e_int_result_chain_errors` — IntResult error chain: overflow/underflow/div-zero -- `e2e_ffi_struct_sizes_match_c_abi` — IntResult=16B, BoolResult=5–8B, FloatResult=16B -- `e2e_type_aliases_exported` — Bounded/NonEmpty accessible from crate root -- `e2e_error_implements_std_error` — Error is boxable as dyn std::error::Error -- `e2e_error_display_messages` — all variants have meaningful display strings -- `e2e_bounded_const_bounds` — MIN/MAX const accessors are correct -- `e2e_int_result_alignment` — IntResult has 8-byte alignment (C ABI match) - -### Aspect Tests Added (`bindings/rust/src/tests/aspect.rs`) - -Security, boundary, and concurrency aspects: -- `aspect_bounded_i64_max_is_rejected` — i64::MAX rejected by Percentage -- `aspect_bounded_i64_min_is_rejected` — i64::MIN rejected by any non-negative type -- `aspect_bounded_minus_one_is_rejected` — -1 rejected by Port [0,65535] -- `aspect_bounded_65536_exceeds_port_range` — 65536 rejected by Port -- `aspect_bounded_zero_is_valid` — 0 is valid lower boundary -- `aspect_bounded_100_is_valid` — 100 is valid upper boundary for Percentage -- `aspect_bounded_101_exceeds_percentage` — 101 is rejected -- `aspect_ffi_null_pointer_is_error` — STATUS_ERR_NULL_POINTER never succeeds -- `aspect_ffi_int_result_null_pointer_discards_value` — error value field is ignored -- `aspect_overflow_and_underflow_are_distinct` — two distinct error variants -- `aspect_div_zero_is_distinct_from_overflow` — div-zero not confused with overflow -- `aspect_ffi_allocation_failed_is_error` — allocation failures always produce Err -- `aspect_not_implemented_status_maps_correctly` — -99 maps to NotImplemented -- `aspect_unknown_status_codes_produce_error` — gaps in code space → Error::Unknown -- `aspect_int_result_error_value_is_discarded` — value field ignored for all error codes -- `aspect_error_variants_are_distinct` — no two variants accidentally equal -- `aspect_error_is_send_sync` — Error: Send + Sync -- `aspect_result_is_send_sync` — Result: Send + Sync -- `aspect_bounded_get_always_in_range` — get() always within [MIN,MAX] -- `aspect_only_zero_is_success` — STATUS_OK=0 is the unique success code - ---- - -## Original State (Pre-Blitz) - -| Category | Count | Notes | -|-------------|-------|-------| -| Unit tests | ~15 | Zig FFI integration_test, Go proven_test, Gleam proven_test, Elixir proven_test, Lua proven_spec, Ruby proven_spec, Nim test_proven, OCaml test_proven, Ada test_proven, Ephapax tests (test_all.zig, test_simple.c), Python test_benchmarks | -| Integration | 1 | Zig FFI integration test | -| E2E | 0 | None | -| Benchmarks | 5 | Rust benches (benchmarks.rs, safe_math.rs), JavaScript benchmark.mjs, Python test_benchmarks.py, benchmarks.ipkg (Idris2) | - -**Source modules:** ~1005 across Idris2 core (312 .idr files), Zig FFI (91 files), Rust (76 files), and 100+ language bindings. Bindings span ~120 languages from Ada to Zsh. - -## Remaining Gaps (Post-Blitz) - -### Runtime P2P Tests (require libproven.so) -- [ ] Safe math overflow: `SafeMath::add(i64::MAX, 1)` → `Err(Overflow)` -- [ ] Safe math underflow: `SafeMath::sub(i64::MIN, 1)` → `Err(Underflow)` -- [ ] Safe div-zero: `SafeMath::div(1, 0)` → `Err(DivisionByZero)` -- [ ] Safe abs of MIN: `SafeMath::abs(i64::MIN)` → `Err(Overflow)` - -### Full Proof-Chain E2E (require Idris2 build + libproven.so) -- [ ] Idris2 spec → Zig FFI → Rust binding → verification roundtrip -- [ ] Multi-language consistency: Rust + Gleam + Elixir agree on all operations - -### Cross-Binding Coverage -- [ ] 89% of bindings (109/120) still untested -- [ ] Cross-binding consistency: same operation in 120 bindings produces identical results - -### Benchmarks Needed -- [ ] Zig FFI call overhead -- [ ] Cross-binding comparison (all 120 languages, same operations) -- [ ] Compilation time benchmarks - -### Self-Tests -- [ ] All 312 Idris2 proofs type-check (CI gate) -- [ ] FFI exports match Idris2 specifications -- [ ] Binding API surface matches core API - -## Priority - -**CRG Grade C ACHIEVED** for the Rust binding via structural and property-based tests. The next step toward Grade B+ is runtime integration tests (requiring `libproven.so` from a full Zig+Idris2 build) and expanding coverage to other language bindings (currently 9.2% coverage across 120 languages). diff --git a/TOPOLOGY.md b/TOPOLOGY.adoc similarity index 89% rename from TOPOLOGY.md rename to TOPOLOGY.adoc index 3a76b9bc..eb7a8a46 100644 --- a/TOPOLOGY.md +++ b/TOPOLOGY.adoc @@ -1,13 +1,8 @@ - - - - +== proven — System Architecture & Completion Dashboard -# proven — System Architecture & Completion Dashboard +=== System Architecture -## System Architecture - -``` +.... ┌─────────────────────────────────────────────┐ │ APPLICATION LAYER (7%) │ │ │ @@ -89,11 +84,11 @@ │ SafeCryptoHW ░░ SafeVulkanCompute ░░ │ │ SafeOpenCL ░░ SafeMetal ░░ SafeCUDA ░░ │ └─────────────────────────────────────────────────────────────┘ -``` +.... -## Completion Dashboard +=== Completion Dashboard -``` +.... COMPONENT STATUS NOTES ───────────────────────────────── ────────────────── ───────────────────────────────── CORE MODULES (IDRIS2) @@ -153,23 +148,25 @@ RSR COMPLIANCE ───────────────────────────────────────────────────────────────────────────── OVERALL: ██████░░░░ 55% Core solid, ecosystem incomplete -``` +.... -## Key Dependencies +=== Key Dependencies -| Dependency | Version | Purpose | Status | -|----------------|----------|-----------------------------------|--------------| -| Idris2 | >= 0.6.0 | Core language + totality checker | Required | -| Zig | 0.13+ | FFI bridge compilation | Required | -| pack | latest | Idris2 package manager | NOT INSTALLED| -| Chainguard | latest | Container base images | Planned | -| stapeln | latest | Container layer management | Planned | -| hypatia | v2 | Security scanning | Integrated | -| echidnabot | latest | Proof verification bot | Integrated | +[width="100%",cols="23%,13%,46%,18%",options="header",] +|=== +|Dependency |Version |Purpose |Status +|Idris2 |>= 0.6.0 |Core language + totality checker |Required +|Zig |0.13+ |FFI bridge compilation |Required +|pack |latest |Idris2 package manager |NOT INSTALLED +|Chainguard |latest |Container base images |Planned +|stapeln |latest |Container layer management |Planned +|hypatia |v2 |Security scanning |Integrated +|echidnabot |latest |Proof verification bot |Integrated +|=== -## Build Pipeline +=== Build Pipeline -``` +.... idris2 --codegen refc proven-ffi.ipkg │ ▼ @@ -183,16 +180,17 @@ idris2 --codegen refc proven-ffi.ipkg │ ▼ Language bindings link against libproven -``` +.... -## Update Protocol +=== Update Protocol This file is maintained by both humans and AI agents. When updating: -1. **After completing a component**: Change its bar and percentage -2. **After adding a component**: Add a new row in the appropriate section -3. **After architectural changes**: Update the ASCII diagram -4. **Date**: Update the `Last updated` comment at the top of this file +[arabic] +. *After completing a component*: Change its bar and percentage +. *After adding a component*: Add a new row in the appropriate section +. *After architectural changes*: Update the ASCII diagram +. *Date*: Update the `+Last updated+` comment at the top of this file -Progress bars use: `█` (filled) and `░` (empty), 10 characters wide. -Percentages: 0%, 10%, 20%, ... 100% (in 10% increments). +Progress bars use: `+█+` (filled) and `+░+` (empty), 10 characters wide. +Percentages: 0%, 10%, 20%, … 100% (in 10% increments). diff --git a/audits/audit-ffi-bindings-2026-05-26.adoc b/audits/audit-ffi-bindings-2026-05-26.adoc new file mode 100644 index 00000000..e0882fd7 --- /dev/null +++ b/audits/audit-ffi-bindings-2026-05-26.adoc @@ -0,0 +1,133 @@ +== Audit: FFI binding `+unsafe+` blocks across `+bindings/+` and `+ffi/+` + +*Auditor*: Jonathan D.A. Jewell *Date*: 2026-05-26 *Scope*: all +`+panic-attack assail+` Critical/High `+UnsafeCode+` + `+UnsafeFFI+` +findings located under `+bindings/*/src/+`, `+ffi/*/src/+`, and +`+domain-specific/*/ffi/*/src/+`. *Cross-reference*: campaign tracker +https://github.com/hyperpolymath/panic-attack/issues/32[hyperpolymath/panic-attack#32]. +*Registry*: `+audits/assail-classifications.a2ml+`. + +=== Context + +`+proven+` is the Idris2-implemented library whose logic is *formally +verified* (dependent types + totality checking). Each binding-language +directory (`+bindings//src/+`) is a _thin wrapper_ over the C ABI +exposed by `+libproven+` via the Zig FFI bridge (`+ffi/zig/+`). Wrappers +do exactly three things: + +[arabic] +. Declare `+extern "C"+` FFI signatures matching `+libproven.h+`. +. Provide safe-language wrappers around the unsafe FFI calls (allocating +buffers, mapping `+ProvenStatus+` to language-native errors). +. Free allocated strings via `+proven_free_string+` to satisfy the ABI’s +ownership contract. + +*No logic is reimplemented in any binding.* Every function calls through +to the formally-verified Idris2 implementation. The `+unsafe+` blocks +(or equivalent in each language) exist solely at the C-ABI boundary, +where they are required by the language to call across. + +=== Per-binding rationale + +The classifications cover the following binding/ffi roots: + +* `+bindings/ada/src/+` — Ada `+pragma Interface (C)+` callouts. The +`+.adb+` carries the FFI shim only. +* `+bindings/c/+` — direct C consumer of `+libproven.h+`; not unsafe per +Rust semantics but flagged by the same detector. +* `+bindings/elixir/+` — Erlang NIF + Port wrappers. +* `+bindings/ephapax-affine/src/+` — AffineScript binding compiled to +Rust; unsafe blocks at the `+extern "C"+` boundary. +* `+bindings/ephapax-linear/src/+` — Rust linear-types binding to +`+libproven+`; same FFI pattern as `+bindings/rust/+`. +* `+bindings/go/+` — cgo wrappers. +* `+bindings/haskell/+` — `+foreign import ccall+` FFI. +* `+bindings/rust/src/+` — canonical Rust binding; every `+unsafe+` +block has an inline `+// SAFETY:+` comment and calls a `+proven_*+` C +function. +* `+bindings/sml/src/+` — SML/NJ FFI to the `+libproven+` shared object. +* `+bindings/vcl/+` — Varnish-VCL binding (HTTP edge use case). +* `+bindings/zig/src/+` — Zig direct binding (also acts as the canonical +reference for the FFI shape). +* `+ffi/beam/src/proven_nif.zig+` — BEAM NIF in Zig; bridges Erlang to +`+libproven+`. +* `+domain-specific/http/ffi/zig/src/http.zig+` — http-specific Zig FFI +bridge. + +=== Anti-gameability + +The registry is `+audits/assail-classifications.a2ml+` — a separate file +from any binding source under scan. Adding a new `+unsafe+` block inside +`+bindings//src/+` cannot self-suppress; the registry edit + an +update to a section in this audit doc is required, both of which are +reviewable. + +The classification is *scoped to FFI-boundary files*. Any `+unsafe+` +block outside `+bindings/*/src/+`, `+ffi/*/src/+`, or +`+domain-specific/*/ffi/*/src/+` remains visible to assail and will be +flagged unsuppressed. + +=== Verification + +Locally on this branch: `+panic-attack assail . --headless+` reports +`+UnsafeCode: 0 active, 88 suppressed+` and +`+UnsafeFFI: 0 active, 41 suppressed+` after the registry is loaded. The +non-FFI findings (CommandInjection, HardcodedSecret, +DynamicCodeExecution, SupplyChain, etc.) remain visible and will be +handled in separate follow-up PRs. + +=== Out of scope this audit + +* `+CommandInjection+` (15 findings) — separate triage; mostly +tooling/build scripts. +* `+HardcodedSecret+` (6 findings) — separate triage; mostly test +fixtures. +* `+DynamicCodeExecution+` (5 findings) — separate triage. +* `+SupplyChain+`, `+UnsafeTypeCoercion+`, `+UncheckedAllocation+`, +`+PanicPath+` — separate triage. + +Refs hyperpolymath/panic-attack#32. + +=== §HardcodedSecret — Idris2 protocol type identifiers (6 entries) + +panic-attack PA009 also fires on *identifier names* that match the +secret-detection pattern set. Six Idris2 modules trip this detector +because they declare types or string-match patterns for protocols that +_talk about_ passwords / keys / tokens: + +* `+src/Proven/SafePassword/Strength.idr+` — password-strength +validation module. +* `+apps/proven-socks/src/SOCKS/Types.idr+` — SOCKS protocol type +definitions. +* `+apps/proven-ssh-bastion/src/SSH/Auth.idr+` — +`+authMethodFromString "password" = Password+`: the string +`+"password"+` is the SSH protocol’s `+auth-method+` name, not a +credential. +* `+apps/proven-kms/src/KMS/Types.idr+` — KMS protocol type definitions. +* `+apps/proven-radius/src/RADIUS/Types.idr+` — RADIUS attribute names +(password attribute is part of the protocol). +* `+apps/proven-authserver/src/Authserver/Types.idr+` — authserver +protocol type definitions. + +None of these files contain credential literals — they declare or +reference protocol-defined names. Classification: +`+protocol-type-identifier+`. + +=== §CommandInjection — binding-wrapper naming patterns (15 entries) + +PA003 fires on identifiers/keywords whose names overlap with +shell-execution primitives: + +* `+bindings/bash/proven.sh+` calls +`+_proven_call calculator eval "$1"+` — `+eval+` here is the *libproven +calculator’s method name*, not bash `+eval+`. The actual call goes +through `+_proven_call+`, which sanitises via libproven. +* `+bindings/guile/proven/safe-*.scm+` (13 files) — every Guile +safe-wrapper declares `+#:use-module (system foreign)+`. `+system+` is a +*Guile module path component* (the foreign-function module lives under +`+system+`), not a `+system()+` call. +* `+bindings/prolog/safe_string.pl+` — exports an `+escape_shell/2+` +*sanitisation predicate*. The name describes its purpose (escape for +shell consumption); it does not execute anything. + +Classification: `+binding-wrapper-naming+`. diff --git a/audits/audit-ffi-bindings-2026-05-26.md b/audits/audit-ffi-bindings-2026-05-26.md deleted file mode 100644 index 873d6cef..00000000 --- a/audits/audit-ffi-bindings-2026-05-26.md +++ /dev/null @@ -1,82 +0,0 @@ - - -# Audit: FFI binding `unsafe` blocks across `bindings/` and `ffi/` - -**Auditor**: Jonathan D.A. Jewell -**Date**: 2026-05-26 -**Scope**: all `panic-attack assail` Critical/High `UnsafeCode` + `UnsafeFFI` findings located under `bindings/*/src/`, `ffi/*/src/`, and `domain-specific/*/ffi/*/src/`. -**Cross-reference**: campaign tracker [hyperpolymath/panic-attack#32](https://github.com/hyperpolymath/panic-attack/issues/32). -**Registry**: `audits/assail-classifications.a2ml`. - -## Context - -`proven` is the Idris2-implemented library whose logic is **formally verified** (dependent types + totality checking). Each binding-language directory (`bindings//src/`) is a *thin wrapper* over the C ABI exposed by `libproven` via the Zig FFI bridge (`ffi/zig/`). Wrappers do exactly three things: - -1. Declare `extern "C"` FFI signatures matching `libproven.h`. -2. Provide safe-language wrappers around the unsafe FFI calls (allocating buffers, mapping `ProvenStatus` to language-native errors). -3. Free allocated strings via `proven_free_string` to satisfy the ABI's ownership contract. - -**No logic is reimplemented in any binding.** Every function calls through to the formally-verified Idris2 implementation. The `unsafe` blocks (or equivalent in each language) exist solely at the C-ABI boundary, where they are required by the language to call across. - -## Per-binding rationale - -The classifications cover the following binding/ffi roots: - -- `bindings/ada/src/` — Ada `pragma Interface (C)` callouts. The `.adb` carries the FFI shim only. -- `bindings/c/` — direct C consumer of `libproven.h`; not unsafe per Rust semantics but flagged by the same detector. -- `bindings/elixir/` — Erlang NIF + Port wrappers. -- `bindings/ephapax-affine/src/` — AffineScript binding compiled to Rust; unsafe blocks at the `extern "C"` boundary. -- `bindings/ephapax-linear/src/` — Rust linear-types binding to `libproven`; same FFI pattern as `bindings/rust/`. -- `bindings/go/` — cgo wrappers. -- `bindings/haskell/` — `foreign import ccall` FFI. -- `bindings/rust/src/` — canonical Rust binding; every `unsafe` block has an inline `// SAFETY:` comment and calls a `proven_*` C function. -- `bindings/sml/src/` — SML/NJ FFI to the `libproven` shared object. -- `bindings/vcl/` — Varnish-VCL binding (HTTP edge use case). -- `bindings/zig/src/` — Zig direct binding (also acts as the canonical reference for the FFI shape). -- `ffi/beam/src/proven_nif.zig` — BEAM NIF in Zig; bridges Erlang to `libproven`. -- `domain-specific/http/ffi/zig/src/http.zig` — http-specific Zig FFI bridge. - -## Anti-gameability - -The registry is `audits/assail-classifications.a2ml` — a separate file from any binding source under scan. Adding a new `unsafe` block inside `bindings//src/` cannot self-suppress; the registry edit + an update to a section in this audit doc is required, both of which are reviewable. - -The classification is **scoped to FFI-boundary files**. Any `unsafe` block outside `bindings/*/src/`, `ffi/*/src/`, or `domain-specific/*/ffi/*/src/` remains visible to assail and will be flagged unsuppressed. - -## Verification - -Locally on this branch: `panic-attack assail . --headless` reports `UnsafeCode: 0 active, 88 suppressed` and `UnsafeFFI: 0 active, 41 suppressed` after the registry is loaded. The non-FFI findings (CommandInjection, HardcodedSecret, DynamicCodeExecution, SupplyChain, etc.) remain visible and will be handled in separate follow-up PRs. - -## Out of scope this audit - -- `CommandInjection` (15 findings) — separate triage; mostly tooling/build scripts. -- `HardcodedSecret` (6 findings) — separate triage; mostly test fixtures. -- `DynamicCodeExecution` (5 findings) — separate triage. -- `SupplyChain`, `UnsafeTypeCoercion`, `UncheckedAllocation`, `PanicPath` — separate triage. - -Refs hyperpolymath/panic-attack#32. - -## §HardcodedSecret — Idris2 protocol type identifiers (6 entries) - -panic-attack PA009 also fires on **identifier names** that match the secret-detection pattern set. Six Idris2 modules trip this detector because they declare types or string-match patterns for protocols that *talk about* passwords / keys / tokens: - -- `src/Proven/SafePassword/Strength.idr` — password-strength validation module. -- `apps/proven-socks/src/SOCKS/Types.idr` — SOCKS protocol type definitions. -- `apps/proven-ssh-bastion/src/SSH/Auth.idr` — `authMethodFromString "password" = Password`: the string `"password"` is the SSH protocol's `auth-method` name, not a credential. -- `apps/proven-kms/src/KMS/Types.idr` — KMS protocol type definitions. -- `apps/proven-radius/src/RADIUS/Types.idr` — RADIUS attribute names (password attribute is part of the protocol). -- `apps/proven-authserver/src/Authserver/Types.idr` — authserver protocol type definitions. - -None of these files contain credential literals — they declare or reference protocol-defined names. Classification: `protocol-type-identifier`. - -## §CommandInjection — binding-wrapper naming patterns (15 entries) - -PA003 fires on identifiers/keywords whose names overlap with shell-execution primitives: - -- `bindings/bash/proven.sh` calls `_proven_call calculator eval "$1"` — `eval` here is the **libproven calculator's method name**, not bash `eval`. The actual call goes through `_proven_call`, which sanitises via libproven. -- `bindings/guile/proven/safe-*.scm` (13 files) — every Guile safe-wrapper declares `#:use-module (system foreign)`. `system` is a **Guile module path component** (the foreign-function module lives under `system`), not a `system()` call. -- `bindings/prolog/safe_string.pl` — exports an `escape_shell/2` **sanitisation predicate**. The name describes its purpose (escape for shell consumption); it does not execute anything. - -Classification: `binding-wrapper-naming`. diff --git a/audits/audit-track-c-2026-06-23.adoc b/audits/audit-track-c-2026-06-23.adoc new file mode 100644 index 00000000..14845a48 --- /dev/null +++ b/audits/audit-track-c-2026-06-23.adoc @@ -0,0 +1,231 @@ +== Audit: Track C — non-FFI Critical/High `+panic-attack assail+` findings + +*Auditor*: Jonathan D.A. Jewell *Date*: 2026-06-23 *Scope*: the 33 +Critical/High `+panic-attack assail+` findings aggregated in +https://github.com/hyperpolymath/proven/issues/68[hyperpolymath/proven#68] +("`Track C`"), i.e. every Critical/High finding *except* the +`+UnsafeCode+` / `+UnsafeFFI+` findings handled by Track A +(audit-ffi-bindings-2026-05-26.md, PR #67). *Cross-reference*: campaign +tracker +https://github.com/hyperpolymath/panic-attack/issues/32[hyperpolymath/panic-attack#32]. +*Registry*: `+audits/assail-classifications.a2ml+`. + +=== Context + +`+proven+` is the Idris2-implemented library whose logic is *formally +verified* (dependent types + totality checking). Each +`+bindings//+` directory is a _thin wrapper_ over the C ABI +exposed by `+libproven+` via the Zig FFI bridge; no logic is +reimplemented in any binding (repo `+.claude/CLAUDE.md+`, ADR-008). + +The `+panic-attack assail+` detectors used here are substring/keyword +pattern matchers. They fire on the _names_ of methods, modules, FFI +symbols, and on words appearing in comments — without semantic +understanding of whether the matched token is an executable call. The +triage below reads each flagged site and records, per file, whether it +is a genuine defect (→ fixed at source) or a false positive (→ +classified in the registry, which is a separate file that a new unsafe +block cannot self-suppress). + +=== Disposition summary + +[width="100%",cols="34%,33%,33%",options="header",] +|=== +|Category |Count |Disposition +|CommandInjection |15 |false positive — `+binding-wrapper-naming+` +(registered earlier; verified again here) + +|HardcodedSecret |6 |false positive — `+protocol-type-identifier+` +(registered earlier; verified again here) + +|DynamicCodeExecution |5 |false positive — `+binding-wrapper-naming+` / +`+generated-code+` + +|UnsafeTypeCoercion |2 |false positive — `+legitimate-ffi+` (Nim) / +`+documentation-reference+` (Haskell) + +|PanicPath |1 |false positive — `+documentation-reference+` (Haskell) + +|UncheckedAllocation |2 |*1 fixed at source* (`+stubs.c+`) + 1 false +positive (`+proven_udf.c+`, already NULL-checked) + +|SupplyChain |2 |resolved — both `+flake.nix+` files removed in the +Nix→Guix migration + +|*Total* |*33* | +|=== + +=== The one genuine defect — fixed at source + +==== `+bindings/ephapax/src/stubs.c+` — UncheckedAllocation (Critical) + +Four allocations dereferenced their result without a NULL check: + +* `+idris_proven_lru_new+` — `+malloc(sizeof(LRUCache))+` then +`+cache->capacity = …+` +* `+idris_proven_buffer_new+` — `+malloc(sizeof(Buffer))+` then +`+buf->capacity = …+`, and a second `+malloc(capacity)+` for +`+buf->data+` +* `+idris_proven_resource_new_handle+` — +`+malloc(sizeof(ResourceHandle))+` then `+handle->resource_id = …+` + +Under allocation failure (OOM, or an attacker-influenced large +`+capacity+`) each would dereference NULL and crash. The file is an +explicitly temporary test stub +(`+// TODO: Replace with actual Idris2-compiled proven library+`), but +the unchecked dereference is a real defect, so it is fixed rather than +suppressed. + +*Fix*: each pointer-returning constructor now returns `+NULL+` on +allocation failure, matching the file’s existing convention +(pointer-returning functions return a pointer; the partial `+Buffer+` +case frees `+buf+` before returning `+NULL+` to avoid a leak). Callers +across the FFI boundary already treat a `+NULL+` handle as a failed +constructor. + +=== False positives — classified + +==== §DynamicCodeExecution — `+eval+` is a libproven method name (5) + +Every flagged site routes the expression to libproven’s verified +calculator via the `+proven_calculator_eval+` C ABI export; no +language-level `+eval()+` of attacker input occurs. + +* `+bindings/deno/src/safe_calculator.ts+`, +`+bindings/javascript/src/safe_calculator.js+` — a method _named_ +`+eval()+` whose body calls `+symbols.proven_calculator_eval(…)+`. +* `+bindings/assemblyscript/src/ffi.ts+` — an `+@external+` FFI +_declaration_ of `+proven_calculator_eval+`; AssemblyScript→Wasm has no +`+eval()+` primitive. +* `+bindings/elm/interop/proven-elm-ports.js+` — calls +`+lib.proven_calculator_eval(…)+` over an Elm port; JS `+eval()+` is +never referenced. +* `+build-simple/exec/test_ffi_exports_app/test_ffi_exports.ss+` — +Idris2 Chez backend generated test harness; the `+eval+` is +`+blodwen-eval-scheme+` in the Idris runtime-support prelude, never +invoked with runtime input. Classification: `+generated-code+`. + +____ +Naming note (non-blocking): the public `+eval()+` method on the JS/TS +calculator bindings is a readability hazard precisely because it tripped +this detector. A future major-version rename to `+evaluate()+` would +remove the ambiguity. This is an API-cosmetic change, not a security +issue, and is out of scope for this PR. +____ + +==== §UnsafeTypeCoercion (2) + +* `+bindings/nim/src/proven/safe_hex.nim+` — +`+cast[pointer](unsafeAddr a[0])+` marshals a Nim string buffer to C +`+void*+` at the libproven FFI boundary; each cast is length-guarded +(`+a.len > 0+`) with a dummy sentinel for the empty case. Idiomatic +safe-Nim FFI. Classification: `+legitimate-ffi+`. +* `+bindings/haskell/src/Proven.hs+` — the token `+unsafeCoerce+` +appears only in the module Haddock docstring asserting its _absence_. No +such call exists. Classification: `+documentation-reference+`. + +==== §PanicPath (1) + +* `+bindings/haskell/src/Proven.hs+` — `+error+`/`+undefined+` appear +only in the same docstring and in error-constructor type names +(e.g. `+ErrEncodingError+`); no `+error+`/`+undefined+` call exists. +Classification: `+documentation-reference+`. + +==== §UncheckedAllocation — false positive (1) + +* `+bindings/sql/mysql/proven_udf.c+` — every `+malloc+` (the +`+initid->ptr+` allocations in the `+*_init+` UDF entry points) is +immediately followed by an `+if (ptr == NULL)+` guard that frees any +owned string, sets `+*is_null = 1+` and returns, per the MySQL UDF +convention. No unchecked dereference. Classification: +`+already-null-checked+`. + +==== §SupplyChain — resolved by file removal (2) + +Both flagged files (`+flake.nix+`, `+bindings/nix/flake.nix+`) were +removed during the estate Nix→Guix migration (commit `+ee62f7e+`). +Build-time dependency pinning is now provided by `+guix.scm+` + +`+guix/channels.scm+`. The surviving `+bindings/nix/default.nix+` is a +consumer-facing expression and was not flagged for unpinned flake +inputs. Classification: `+resolved-file-removed+`. (Per estate policy +Nix is not used for `+proven+`’s own build; the `+bindings/nix/+` +directory remains only as a binding offered _to_ Nix consumers of +`+libproven+`.) + +==== §CommandInjection (15) and §HardcodedSecret (6) + +Re-verified during this pass; rationale unchanged from +audit-ffi-bindings-2026-05-26.md (§CommandInjection, §HardcodedSecret) +and already present in the registry. +`+eval+`/`+system+`/`+escape_shell+` are method names, the Guile module +path `+(system foreign)+`, and a sanitiser predicate name respectively; +the `+HardcodedSecret+` sites are protocol/type identifiers and a +common-password _dictionary_ (used by the strength checker), not +credential literals. + +=== Verification & scanner-mechanism note (panic-attack 2.5.5, 2026-06-23) + +This triage was verified by building `+panic-attack 2.5.5+` from source +and running `+panic-attack assail . --headless --output-format json+` +against this branch. Filtering on the scanner’s own `+suppressed+` +field, the result is: + +* *The only genuinely-active (`+suppressed != true+`) Critical/High +finding in the entire repository is `+bindings/ephapax/src/stubs.c+` +(UncheckedAllocation).* Every other Track-C finding is auto-suppressed +by the scanner’s 2.5.5 context-aware engine (`+kanren+` rules + +`+test_context+` classification), and `+SupplyChain+` is gone entirely +(the `+flake.nix+` files no longer exist). + +Two mechanism facts discovered while verifying, which callers of this +audit should know: + +[arabic] +. *The registry file is not consumed by panic-attack 2.5.5.* Suppression +in 2.5.5 is driven by (a) the context-aware `+kanren+` engine that sets +`+WeakPoint.suppressed+`, and (b) inline `+// panic-attack: accepted+` +comment markers. The `+audits/assail-classifications.a2ml+` registry is +*not read by the scanner* (nor by the `+panicbot+` fleet wrapper, which +reads `+.machine_readable/bot_directives/panicbot.a2ml+` and then +honours the scanner’s `+suppressed+` flag). The registry therefore +stands as the *reviewable audit trail* for these dispositions; it is +not, under 2.5.5, an active suppression input. It is retained because it +is the estate-canonical record and because it predates (and may again +post-date) this scanner version. +. *`+stubs.c+`’s finding has no available suppression path in 2.5.5, and +that is acceptable.* The `+UncheckedAllocation+` detector is +pattern-based — it fires on the presence of `+malloc(+` regardless of a +following NULL check — and it emits a _file-level_ finding with no line +number. Inline `+// panic-attack: accepted+` markers are line-gated +(`+assail/mod.rs+` only consults a marker when `+WeakPoint.line+` is +`+Some+`), so they cannot suppress a file-level finding; and the +`+kanren+` "`null-check guarding`" rule (`+suppress_unchecked_alloc+`) +is defined but its premise fact `+context(File, "null_checked")+` is +emitted nowhere, so it never fires. The defect is nonetheless *genuinely +fixed in source* (the NULL checks are real and correct); the residual +scanner report is a known false-positive-after-fix with no 2.5.5 +suppression hook. Both gaps — file-level findings being unsuppressable, +and the unwired `+null_checked+` rule — are scanner issues, noted +upstream at hyperpolymath/panic-attack#32. + +=== Anti-gameability + +Identical to the Track A audit: the registry +(`+audits/assail-classifications.a2ml+`) is a separate file from any +scanned source. Adding a new `+eval+`/`+malloc+`/`+cast+` site inside a +binding cannot self-suppress; a reviewable registry edit plus an entry +in this audit doc is required. The one genuine defect was fixed in +source, not suppressed, so it will re-fire if regressed. + +=== Upstream note (panic-attack) + +The recurring root cause of this noise is that the +`+DynamicCodeExecution+`, `+CommandInjection+`, `+PanicPath+` and +`+UnsafeTypeCoercion+` detectors match the _substring_ (`+eval+`, +`+system+`, `+error+`, `+unsafeCoerce+`) rather than an AST call node, +so they fire on method names, FFI symbol names, module paths and +comments. A foundational reduction in false positives belongs in the +scanner — tracked at hyperpolymath/panic-attack#32 — and is out of scope +for this repository’s PR. + +Refs hyperpolymath/proven#68, hyperpolymath/panic-attack#32. diff --git a/audits/audit-track-c-2026-06-23.md b/audits/audit-track-c-2026-06-23.md deleted file mode 100644 index 2d522190..00000000 --- a/audits/audit-track-c-2026-06-23.md +++ /dev/null @@ -1,196 +0,0 @@ - - -# Audit: Track C — non-FFI Critical/High `panic-attack assail` findings - -**Auditor**: Jonathan D.A. Jewell -**Date**: 2026-06-23 -**Scope**: the 33 Critical/High `panic-attack assail` findings aggregated in -[hyperpolymath/proven#68](https://github.com/hyperpolymath/proven/issues/68) -("Track C"), i.e. every Critical/High finding **except** the `UnsafeCode` / -`UnsafeFFI` findings handled by Track A -([audit-ffi-bindings-2026-05-26.md](audit-ffi-bindings-2026-05-26.md), PR #67). -**Cross-reference**: campaign tracker -[hyperpolymath/panic-attack#32](https://github.com/hyperpolymath/panic-attack/issues/32). -**Registry**: `audits/assail-classifications.a2ml`. - -## Context - -`proven` is the Idris2-implemented library whose logic is **formally verified** -(dependent types + totality checking). Each `bindings//` directory is a -*thin wrapper* over the C ABI exposed by `libproven` via the Zig FFI bridge; no -logic is reimplemented in any binding (repo `.claude/CLAUDE.md`, ADR-008). - -The `panic-attack assail` detectors used here are substring/keyword pattern -matchers. They fire on the *names* of methods, modules, FFI symbols, and on -words appearing in comments — without semantic understanding of whether the -matched token is an executable call. The triage below reads each flagged site -and records, per file, whether it is a genuine defect (→ fixed at source) or a -false positive (→ classified in the registry, which is a separate file that a -new unsafe block cannot self-suppress). - -## Disposition summary - -| Category | Count | Disposition | -|---|---|---| -| CommandInjection | 15 | false positive — `binding-wrapper-naming` (registered earlier; verified again here) | -| HardcodedSecret | 6 | false positive — `protocol-type-identifier` (registered earlier; verified again here) | -| DynamicCodeExecution | 5 | false positive — `binding-wrapper-naming` / `generated-code` | -| UnsafeTypeCoercion | 2 | false positive — `legitimate-ffi` (Nim) / `documentation-reference` (Haskell) | -| PanicPath | 1 | false positive — `documentation-reference` (Haskell) | -| UncheckedAllocation | 2 | **1 fixed at source** (`stubs.c`) + 1 false positive (`proven_udf.c`, already NULL-checked) | -| SupplyChain | 2 | resolved — both `flake.nix` files removed in the Nix→Guix migration | -| **Total** | **33** | | - -## The one genuine defect — fixed at source - -### `bindings/ephapax/src/stubs.c` — UncheckedAllocation (Critical) - -Four allocations dereferenced their result without a NULL check: - -- `idris_proven_lru_new` — `malloc(sizeof(LRUCache))` then `cache->capacity = …` -- `idris_proven_buffer_new` — `malloc(sizeof(Buffer))` then `buf->capacity = …`, - and a second `malloc(capacity)` for `buf->data` -- `idris_proven_resource_new_handle` — `malloc(sizeof(ResourceHandle))` then - `handle->resource_id = …` - -Under allocation failure (OOM, or an attacker-influenced large `capacity`) each -would dereference NULL and crash. The file is an explicitly temporary test stub -(`// TODO: Replace with actual Idris2-compiled proven library`), but the -unchecked dereference is a real defect, so it is fixed rather than suppressed. - -**Fix**: each pointer-returning constructor now returns `NULL` on allocation -failure, matching the file's existing convention (pointer-returning functions -return a pointer; the partial `Buffer` case frees `buf` before returning `NULL` -to avoid a leak). Callers across the FFI boundary already treat a `NULL` handle -as a failed constructor. - -## False positives — classified - -### §DynamicCodeExecution — `eval` is a libproven method name (5) - -Every flagged site routes the expression to libproven's verified calculator via -the `proven_calculator_eval` C ABI export; no language-level `eval()` of -attacker input occurs. - -- `bindings/deno/src/safe_calculator.ts`, `bindings/javascript/src/safe_calculator.js` - — a method *named* `eval()` whose body calls `symbols.proven_calculator_eval(…)`. -- `bindings/assemblyscript/src/ffi.ts` — an `@external` FFI *declaration* of - `proven_calculator_eval`; AssemblyScript→Wasm has no `eval()` primitive. -- `bindings/elm/interop/proven-elm-ports.js` — calls `lib.proven_calculator_eval(…)` - over an Elm port; JS `eval()` is never referenced. -- `build-simple/exec/test_ffi_exports_app/test_ffi_exports.ss` — Idris2 Chez - backend generated test harness; the `eval` is `blodwen-eval-scheme` in the - Idris runtime-support prelude, never invoked with runtime input. - Classification: `generated-code`. - -> Naming note (non-blocking): the public `eval()` method on the JS/TS calculator -> bindings is a readability hazard precisely because it tripped this detector. A -> future major-version rename to `evaluate()` would remove the ambiguity. This is -> an API-cosmetic change, not a security issue, and is out of scope for this PR. - -### §UnsafeTypeCoercion (2) - -- `bindings/nim/src/proven/safe_hex.nim` — `cast[pointer](unsafeAddr a[0])` - marshals a Nim string buffer to C `void*` at the libproven FFI boundary; each - cast is length-guarded (`a.len > 0`) with a dummy sentinel for the empty case. - Idiomatic safe-Nim FFI. Classification: `legitimate-ffi`. -- `bindings/haskell/src/Proven.hs` — the token `unsafeCoerce` appears only in - the module Haddock docstring asserting its *absence*. No such call exists. - Classification: `documentation-reference`. - -### §PanicPath (1) - -- `bindings/haskell/src/Proven.hs` — `error`/`undefined` appear only in the - same docstring and in error-constructor type names (e.g. `ErrEncodingError`); - no `error`/`undefined` call exists. Classification: `documentation-reference`. - -### §UncheckedAllocation — false positive (1) - -- `bindings/sql/mysql/proven_udf.c` — every `malloc` (the `initid->ptr` - allocations in the `*_init` UDF entry points) is immediately followed by an - `if (ptr == NULL)` guard that frees any owned string, sets `*is_null = 1` and - returns, per the MySQL UDF convention. No unchecked dereference. - Classification: `already-null-checked`. - -### §SupplyChain — resolved by file removal (2) - -Both flagged files (`flake.nix`, `bindings/nix/flake.nix`) were removed during -the estate Nix→Guix migration (commit `ee62f7e`). Build-time dependency pinning -is now provided by `guix.scm` + `guix/channels.scm`. The surviving -`bindings/nix/default.nix` is a consumer-facing expression and was not flagged -for unpinned flake inputs. Classification: `resolved-file-removed`. (Per estate -policy Nix is not used for `proven`'s own build; the `bindings/nix/` directory -remains only as a binding offered *to* Nix consumers of `libproven`.) - -### §CommandInjection (15) and §HardcodedSecret (6) - -Re-verified during this pass; rationale unchanged from -[audit-ffi-bindings-2026-05-26.md](audit-ffi-bindings-2026-05-26.md) -(§CommandInjection, §HardcodedSecret) and already present in the registry. -`eval`/`system`/`escape_shell` are method names, the Guile module path -`(system foreign)`, and a sanitiser predicate name respectively; the -`HardcodedSecret` sites are protocol/type identifiers and a common-password -*dictionary* (used by the strength checker), not credential literals. - -## Verification & scanner-mechanism note (panic-attack 2.5.5, 2026-06-23) - -This triage was verified by building `panic-attack 2.5.5` from source and running -`panic-attack assail . --headless --output-format json` against this branch. -Filtering on the scanner's own `suppressed` field, the result is: - -- **The only genuinely-active (`suppressed != true`) Critical/High finding in the - entire repository is `bindings/ephapax/src/stubs.c` (UncheckedAllocation).** - Every other Track-C finding is auto-suppressed by the scanner's 2.5.5 - context-aware engine (`kanren` rules + `test_context` classification), and - `SupplyChain` is gone entirely (the `flake.nix` files no longer exist). - -Two mechanism facts discovered while verifying, which callers of this audit -should know: - -1. **The registry file is not consumed by panic-attack 2.5.5.** Suppression in - 2.5.5 is driven by (a) the context-aware `kanren` engine that sets - `WeakPoint.suppressed`, and (b) inline `// panic-attack: accepted` comment - markers. The `audits/assail-classifications.a2ml` registry is **not read by - the scanner** (nor by the `panicbot` fleet wrapper, which reads - `.machine_readable/bot_directives/panicbot.a2ml` and then honours the - scanner's `suppressed` flag). The registry therefore stands as the **reviewable - audit trail** for these dispositions; it is not, under 2.5.5, an active - suppression input. It is retained because it is the estate-canonical record - and because it predates (and may again post-date) this scanner version. - -2. **`stubs.c`'s finding has no available suppression path in 2.5.5, and that is - acceptable.** The `UncheckedAllocation` detector is pattern-based — it fires on - the presence of `malloc(` regardless of a following NULL check — and it emits a - *file-level* finding with no line number. Inline `// panic-attack: accepted` - markers are line-gated (`assail/mod.rs` only consults a marker when - `WeakPoint.line` is `Some`), so they cannot suppress a file-level finding; and - the `kanren` "null-check guarding" rule (`suppress_unchecked_alloc`) is defined - but its premise fact `context(File, "null_checked")` is emitted nowhere, so it - never fires. The defect is nonetheless **genuinely fixed in source** (the NULL - checks are real and correct); the residual scanner report is a known - false-positive-after-fix with no 2.5.5 suppression hook. Both gaps — - file-level findings being unsuppressable, and the unwired `null_checked` rule — - are scanner issues, noted upstream at hyperpolymath/panic-attack#32. - -## Anti-gameability - -Identical to the Track A audit: the registry -(`audits/assail-classifications.a2ml`) is a separate file from any scanned -source. Adding a new `eval`/`malloc`/`cast` site inside a binding cannot -self-suppress; a reviewable registry edit plus an entry in this audit doc is -required. The one genuine defect was fixed in source, not suppressed, so it will -re-fire if regressed. - -## Upstream note (panic-attack) - -The recurring root cause of this noise is that the `DynamicCodeExecution`, -`CommandInjection`, `PanicPath` and `UnsafeTypeCoercion` detectors match the -*substring* (`eval`, `system`, `error`, `unsafeCoerce`) rather than an AST call -node, so they fire on method names, FFI symbol names, module paths and comments. -A foundational reduction in false positives belongs in the scanner — tracked at -hyperpolymath/panic-attack#32 — and is out of scope for this repository's PR. - -Refs hyperpolymath/proven#68, hyperpolymath/panic-attack#32. diff --git a/audits/chapel-symbol-audit-2026-05-30.adoc b/audits/chapel-symbol-audit-2026-05-30.adoc new file mode 100644 index 00000000..6eb60b8b --- /dev/null +++ b/audits/chapel-symbol-audit-2026-05-30.adoc @@ -0,0 +1,143 @@ +== Chapel binding — symbol-audit snapshot (2026-05-30) + +This is the static audit table produced during the Chapel binding +standup (`+docs/adr/0002-chapel-binding-standup.adoc+`). It maps every +`+extern proc+` in `+bindings/chapel/src/LibProven.chpl+` to its +declaration in `+bindings/c/include/proven.h+` and to the actual export +in `+ffi/zig/src/main.zig+`. + +The dynamic equivalent is the `+chapel-symbol-audit+` CI job in +`+.github/workflows/chapel-ci.yml+`, which runs `+nm -D libproven.so+` +against `+bindings/chapel/symbol-manifest.txt+` on every PR. + +=== WIRED (present in `+libproven.so+`; binding wrapper links + runs) + +[width="100%",cols="34%,10%,12%,11%,33%",options="header",] +|=== +|C symbol |proven.h |Zig export |ABI match |Chapel wrapper +|`+proven_path_has_traversal+` |✓ |✓ |✓ |`+SafePath.hasTraversal+` + +|`+proven_header_has_crlf+` |✓ |✓ |✓ |`+SafeHeader.hasCrlf+` (new) + +|`+proven_free_string+` |✓ |✓ |✓ |`+LibProven.provenFreeString+` + +|`+proven_version+` |(not in .h as string accessor) |✓ (returns +`+[*:0]const u8+`) |n/a (stub) |`+LibProven.libraryVersion+` + +|`+proven_build_info+` |(not in .h) |✓ (returns `+[*:0]const u8+`) |n/a +(stub) |`+LibProven.libraryBuildInfo+` +|=== + +Notes: + +* `+proven_version+` and `+proven_build_info+` are stub helpers in +`+ffi/zig/src/main.zig+` (no Idris2 backing); the C ABI in `+proven.h+` +declares per-component accessors (`+proven_version_major+` / `+_minor+` +/ `+_patch+`) but those have no Zig export today. The Chapel binding +exposes the stub accessors as `+libraryVersion()+` / +`+libraryBuildInfo()+` for the smoke test’s use; they are not the +documented long-term API. + +=== GATED (declared per proven.h; NOT exported from libproven.so) + +These extern declarations are kept in the binding as the documented ABI +contract. Calling any of them from a Chapel program produces a linker +error today. Each will move to WIRED as the corresponding Zig export +lands under proven#88. + +==== Lifecycle (ABI in flux — proven#88 sign-off pending) + +[width="100%",cols="30%,23%,27%,20%",options="header",] +|=== +|C symbol |proven.h ABI |Zig main.zig ABI |Resolution +|`+proven_init+` |`+int32_t (void)+` |`+?*Handle ()+` |Pick one +(proven#88) + +|`+proven_deinit+` |`+void (void)+` |(not exported; +`+proven_free(?*Handle)+` exists) |Pick one (proven#88) + +|`+proven_is_initialized+` |`+bool (void)+` |`+u32 (?*Handle)+` |Pick +one (proven#88) +|=== + +==== SafeMath (8 symbols — all GATED on proven#88) + +`+proven_math_add_checked+`, `+proven_math_sub_checked+`, +`+proven_math_mul_checked+`, `+proven_math_div+`, `+proven_math_mod+`, +`+proven_math_abs_safe+`, `+proven_math_clamp+`, +`+proven_math_pow_checked+`. + +==== SafeString (4 symbols) + +`+proven_string_is_valid_utf8+`, `+proven_string_escape_sql+`, +`+proven_string_escape_html+`, `+proven_string_escape_js+`. + +==== SafePath (1 remaining symbol — `+hasTraversal+` is WIRED above) + +`+proven_path_sanitize_filename+`. + +==== SafeEmail (1 symbol) + +`+proven_email_is_valid+`. + +==== SafeUrl (2 symbols) + +`+proven_http_url_encode+`, `+proven_http_url_decode+`. + +==== SafeNetwork (3 symbols) + +`+proven_network_parse_ipv4+`, `+proven_network_ipv4_is_private+`, +`+proven_network_ipv4_is_loopback+`. + +==== SafeCrypto (2 symbols) + +`+proven_crypto_constant_time_eq+`, `+proven_crypto_random_bytes+`. + +==== SafeJson (2 symbols) + +`+proven_json_is_valid+`, `+proven_json_get_type+`. + +==== SafeDateTime (4 symbols) + +`+proven_datetime_parse+`, `+proven_datetime_format_iso8601+`, +`+proven_datetime_is_leap_year+`, `+proven_datetime_days_in_month+`. + +==== SafeFloat (5 symbols) + +`+proven_float_div+`, `+proven_float_is_finite+`, +`+proven_float_is_nan+`, `+proven_float_sqrt+`, `+proven_float_ln+`. + +==== SafeHex (1 symbol) + +`+proven_hex_encode+`. + +==== SafeColor (1 symbol) + +`+proven_color_to_hex+`. + +==== SafeAngle (4 symbols) + +`+proven_angle_deg_to_rad+`, `+proven_angle_rad_to_deg+`, +`+proven_angle_normalize_degrees+`, `+proven_angle_normalize_radians+`. + +==== SafeCalculator (1 symbol) + +`+proven_calculator_eval+`. + +==== Version components (5 symbols) + +`+proven_ffi_abi_version+`, `+proven_version_major+`, +`+proven_version_minor+`, `+proven_version_patch+`, +`+proven_module_count+`. + +=== Summary + +* WIRED: *5 symbols* (out of ~50 declared in `+LibProven.chpl+`) +* GATED: ~45 symbols, all blocked on proven#88 +* Test coverage of WIRED: 4 test modules (`+TestSafePath+`, +`+TestSafeHeader+`, `+TestLibraryInfo+`, `+TestFfiContract+`) plus the +smoke target. + +The chapel-symbol-audit CI job enforces this snapshot: any drift in +which symbols `+libproven.so+` exports versus `+symbol-manifest.txt+` +fails the PR. diff --git a/audits/chapel-symbol-audit-2026-05-30.md b/audits/chapel-symbol-audit-2026-05-30.md deleted file mode 100644 index 43fa5d0b..00000000 --- a/audits/chapel-symbol-audit-2026-05-30.md +++ /dev/null @@ -1,131 +0,0 @@ - - -# Chapel binding — symbol-audit snapshot (2026-05-30) - -This is the static audit table produced during the Chapel binding -standup (`docs/adr/0002-chapel-binding-standup.adoc`). It maps every -`extern proc` in `bindings/chapel/src/LibProven.chpl` to its -declaration in `bindings/c/include/proven.h` and to the actual export -in `ffi/zig/src/main.zig`. - -The dynamic equivalent is the `chapel-symbol-audit` CI job in -`.github/workflows/chapel-ci.yml`, which runs `nm -D libproven.so` -against `bindings/chapel/symbol-manifest.txt` on every PR. - -## WIRED (present in `libproven.so`; binding wrapper links + runs) - -| C symbol | proven.h | Zig export | ABI match | Chapel wrapper | -|--------------------------------|----------|------------|-----------|---------------------------------| -| `proven_path_has_traversal` | ✓ | ✓ | ✓ | `SafePath.hasTraversal` | -| `proven_header_has_crlf` | ✓ | ✓ | ✓ | `SafeHeader.hasCrlf` (new) | -| `proven_free_string` | ✓ | ✓ | ✓ | `LibProven.provenFreeString` | -| `proven_version` | (not in .h as string accessor) | ✓ (returns `[*:0]const u8`) | n/a (stub) | `LibProven.libraryVersion` | -| `proven_build_info` | (not in .h) | ✓ (returns `[*:0]const u8`) | n/a (stub) | `LibProven.libraryBuildInfo` | - -Notes: - -- `proven_version` and `proven_build_info` are stub helpers in - `ffi/zig/src/main.zig` (no Idris2 backing); the C ABI in - `proven.h` declares per-component accessors (`proven_version_major` - / `_minor` / `_patch`) but those have no Zig export today. The - Chapel binding exposes the stub accessors as - `libraryVersion()` / `libraryBuildInfo()` for the smoke test's - use; they are not the documented long-term API. - -## GATED (declared per proven.h; NOT exported from libproven.so) - -These extern declarations are kept in the binding as the documented -ABI contract. Calling any of them from a Chapel program produces a -linker error today. Each will move to WIRED as the corresponding Zig -export lands under proven#88. - -### Lifecycle (ABI in flux — proven#88 sign-off pending) - -| C symbol | proven.h ABI | Zig main.zig ABI | Resolution | -|---------------------------|----------------------|--------------------------|-------------------| -| `proven_init` | `int32_t (void)` | `?*Handle ()` | Pick one (proven#88) | -| `proven_deinit` | `void (void)` | (not exported; `proven_free(?*Handle)` exists) | Pick one (proven#88) | -| `proven_is_initialized` | `bool (void)` | `u32 (?*Handle)` | Pick one (proven#88) | - -### SafeMath (8 symbols — all GATED on proven#88) - -`proven_math_add_checked`, `proven_math_sub_checked`, -`proven_math_mul_checked`, `proven_math_div`, `proven_math_mod`, -`proven_math_abs_safe`, `proven_math_clamp`, `proven_math_pow_checked`. - -### SafeString (4 symbols) - -`proven_string_is_valid_utf8`, `proven_string_escape_sql`, -`proven_string_escape_html`, `proven_string_escape_js`. - -### SafePath (1 remaining symbol — `hasTraversal` is WIRED above) - -`proven_path_sanitize_filename`. - -### SafeEmail (1 symbol) - -`proven_email_is_valid`. - -### SafeUrl (2 symbols) - -`proven_http_url_encode`, `proven_http_url_decode`. - -### SafeNetwork (3 symbols) - -`proven_network_parse_ipv4`, `proven_network_ipv4_is_private`, -`proven_network_ipv4_is_loopback`. - -### SafeCrypto (2 symbols) - -`proven_crypto_constant_time_eq`, `proven_crypto_random_bytes`. - -### SafeJson (2 symbols) - -`proven_json_is_valid`, `proven_json_get_type`. - -### SafeDateTime (4 symbols) - -`proven_datetime_parse`, `proven_datetime_format_iso8601`, -`proven_datetime_is_leap_year`, `proven_datetime_days_in_month`. - -### SafeFloat (5 symbols) - -`proven_float_div`, `proven_float_is_finite`, `proven_float_is_nan`, -`proven_float_sqrt`, `proven_float_ln`. - -### SafeHex (1 symbol) - -`proven_hex_encode`. - -### SafeColor (1 symbol) - -`proven_color_to_hex`. - -### SafeAngle (4 symbols) - -`proven_angle_deg_to_rad`, `proven_angle_rad_to_deg`, -`proven_angle_normalize_degrees`, `proven_angle_normalize_radians`. - -### SafeCalculator (1 symbol) - -`proven_calculator_eval`. - -### Version components (5 symbols) - -`proven_ffi_abi_version`, `proven_version_major`, `proven_version_minor`, -`proven_version_patch`, `proven_module_count`. - -## Summary - -- WIRED: **5 symbols** (out of ~50 declared in `LibProven.chpl`) -- GATED: ~45 symbols, all blocked on proven#88 -- Test coverage of WIRED: 4 test modules (`TestSafePath`, - `TestSafeHeader`, `TestLibraryInfo`, `TestFfiContract`) plus the - smoke target. - -The chapel-symbol-audit CI job enforces this snapshot: any drift in -which symbols `libproven.so` exports versus `symbol-manifest.txt` -fails the PR. diff --git a/bindings/dart/CHANGELOG.adoc b/bindings/dart/CHANGELOG.adoc new file mode 100644 index 00000000..0b900a75 --- /dev/null +++ b/bindings/dart/CHANGELOG.adoc @@ -0,0 +1,11 @@ +== Changelog + +=== 0.9.0 + +* Initial public release +* Safe arithmetic operations with overflow protection +* XSS prevention utilities +* Path traversal protection +* Email validation +* IP address classification +* Cryptographic constant-time comparison diff --git a/bindings/dart/CHANGELOG.md b/bindings/dart/CHANGELOG.md deleted file mode 100644 index 7cb22069..00000000 --- a/bindings/dart/CHANGELOG.md +++ /dev/null @@ -1,11 +0,0 @@ -# Changelog - -## 0.9.0 - -- Initial public release -- Safe arithmetic operations with overflow protection -- XSS prevention utilities -- Path traversal protection -- Email validation -- IP address classification -- Cryptographic constant-time comparison diff --git a/bindings/ephapax/BUILD-SUCCESS.md b/bindings/ephapax/BUILD-SUCCESS.adoc similarity index 55% rename from bindings/ephapax/BUILD-SUCCESS.md rename to bindings/ephapax/BUILD-SUCCESS.adoc index 29e72b4a..9cccb0d0 100644 --- a/bindings/ephapax/BUILD-SUCCESS.md +++ b/bindings/ephapax/BUILD-SUCCESS.adoc @@ -1,38 +1,37 @@ -# Build Success - Ephapax ↔ Proven FFI +== Build Success - Ephapax ↔ Proven FFI -**Date:** 2026-01-24 -**Status:** ✅ Steps 1 & 2 Complete +*Date:* 2026-01-24 *Status:* ✅ Steps 1 & 2 Complete ---- +''''' -## What We Built +=== What We Built -### Step 1: Build the FFI Adapter ✅ +==== Step 1: Build the FFI Adapter ✅ -**Created:** -- C stub implementations (`src/stubs.c`) - 220 lines -- Zig FFI adapters (3 files) - 200 lines -- Build system (Makefile) - works with current tools -- Shared library: `libephapax_proven.so` +*Created:* - C stub implementations (`+src/stubs.c+`) - 220 lines - Zig +FFI adapters (3 files) - 200 lines - Build system (Makefile) - works +with current tools - Shared library: `+libephapax_proven.so+` -**Components:** -1. **LRU Cache FFI** (`lru_adapter.zig`) - 87 lines -2. **Buffer FFI** (`buffer_adapter.zig`) - 63 lines -3. **Resource FFI** (`resource_adapter.zig`) - 56 lines +*Components:* 1. *LRU Cache FFI* (`+lru_adapter.zig+`) - 87 lines 2. +*Buffer FFI* (`+buffer_adapter.zig+`) - 63 lines 3. *Resource FFI* +(`+resource_adapter.zig+`) - 56 lines -**Build Output:** -```bash +*Build Output:* + +[source,bash] +---- $ make gcc -std=c11 -Wall -Wextra -fPIC -O2 -shared -o libephapax_proven.so src/stubs.c ✓ Library built successfully -``` +---- + +''''' ---- +==== Step 2: Test It ✅ -### Step 2: Test It ✅ +*Test Results:* -**Test Results:** -``` +.... === Ephapax ↔ Proven FFI Test === Test 1: LRU Cache @@ -52,17 +51,18 @@ Test 3: Resource Handle ✓ Resource Handle test passed === All FFI tests passed! === -``` +.... -**All tests passing!** 🎉 +*All tests passing!* 🎉 ---- +''''' -## Architecture Validated +=== Architecture Validated The FFI architecture works as designed: -```text +[source,text] +---- ┌───────────────────────────────────────────────────┐ │ C Test Program (test_simple.c) │ │ │ @@ -84,44 +84,52 @@ The FFI architecture works as designed: │ Currently: Simple test stubs │ │ Future: Idris2-compiled proven library │ └───────────────────────────────────────────────────┘ -``` +---- -**Key Insight:** The FFI layer works whether called directly from C or via Zig adapters. +*Key Insight:* The FFI layer works whether called directly from C or via +Zig adapters. ---- +''''' -## One FFI Handles Both Modes +=== One FFI Handles Both Modes -As designed, **ONE set of FFI functions** handles both affine and linear modes: +As designed, *ONE set of FFI functions* handles both affine and linear +modes: -```c +[source,c] +---- // Same C functions called in both modes: LRUCache* cache = idris_proven_lru_new(1024); cache = idris_proven_lru_put(cache, key, key_len, val, val_len); -``` +---- -**Affine mode (Ephapax):** -```ephapax +*Affine mode (Ephapax):* + +[source,ephapax] +---- let cache = AffineAPI.new(1024); // Calls: idris_proven_lru_new let cache = AffineAPI.put(cache, k, v); // Calls: idris_proven_lru_put // Optional cleanup -``` +---- + +*Linear mode (Ephapax):* -**Linear mode (Ephapax):** -```ephapax +[source,ephapax] +---- let! cache = LinearAPI.new(1024); // Calls: SAME idris_proven_lru_new let! cache = LinearAPI.put(cache, k, v); // Calls: SAME idris_proven_lru_put LinearAPI.free(cache); // REQUIRED cleanup -``` +---- -**Difference:** Ephapax's type checker enforcement, not the FFI. +*Difference:* Ephapax’s type checker enforcement, not the FFI. ---- +''''' -## Files Created +=== Files Created -### Core Implementation -``` +==== Core Implementation + +.... bindings/ephapax/ ├── src/ │ ├── stubs.c # C stub implementations (220 LOC) @@ -140,18 +148,20 @@ bindings/ephapax/ ├── README.md # Documentation ├── DYADIC-FFI-DESIGN.md # Design rationale └── BUILD-SUCCESS.md # This document -``` +.... ---- +''''' -## Next Steps +=== Next Steps -### Step 3: Integrate with Ephapax Compiler (Week 1-2) +==== Step 3: Integrate with Ephapax Compiler (Week 1-2) Now that the FFI works, we can integrate with Ephapax: -**Option A: Direct C Integration (Faster)** -```rust +*Option A: Direct C Integration (Faster)* + +[source,rust] +---- // In ephapax-cli/src/main.rs #[link(name = "ephapax_proven")] extern "C" { @@ -159,23 +169,26 @@ extern "C" { fn idris_proven_lru_put(...) -> *mut c_void; // ... } -``` +---- + +*Option B: Via Rust Bindings (Cleaner)* -**Option B: Via Rust Bindings (Cleaner)** -```rust +[source,rust] +---- // Create ephapax-proven-sys crate // Use bindgen to generate Rust bindings from C header -``` +---- -**Recommended:** Start with Option A for rapid iteration. +*Recommended:* Start with Option A for rapid iteration. ---- +''''' -### Step 4: Replace Stubs with Real Idris2 Code (Week 2-3) +==== Step 4: Replace Stubs with Real Idris2 Code (Week 2-3) Once integration works, replace stubs with actual proven library: -```bash +[source,bash] +---- # 1. Compile proven library to C with Idris2 cd /var/mnt/eclipse/repos/proven idris2 --codegen refc proven.ipkg @@ -185,86 +198,93 @@ gcc -shared -o libephapax_proven.so \ build/exec/idris_proven_lru.c \ build/exec/idris_proven_buffer.c \ ... -``` +---- ---- +''''' -## Performance Characteristics +=== Performance Characteristics -### Current (Stub Implementation) +==== Current (Stub Implementation) -| Operation | Time | Notes | -|-----------|------|-------| -| LRU new | 30ns | malloc only | -| LRU put | 40ns | Simple counter increment | -| LRU get | 10ns | Always returns NULL | -| Buffer write | 50ns | memcpy | -| Resource acquire | 20ns | State change | +[cols=",,",options="header",] +|=== +|Operation |Time |Notes +|LRU new |30ns |malloc only +|LRU put |40ns |Simple counter increment +|LRU get |10ns |Always returns NULL +|Buffer write |50ns |memcpy +|Resource acquire |20ns |State change +|=== -**Total overhead:** ~100-200ns per operation +*Total overhead:* ~100-200ns per operation ---- +''''' -### Expected (Proven Library) +==== Expected (Proven Library) -| Operation | Time | Notes | -|-----------|------|-------| -| LRU new | 50ns | Idris2 → C compiled code | -| LRU put | 80ns | Proven correct eviction | -| LRU get | 60ns | Hash lookup | -| Buffer write | 60ns | Bounds-checked write | -| Resource acquire | 30ns | State machine | +[cols=",,",options="header",] +|=== +|Operation |Time |Notes +|LRU new |50ns |Idris2 → C compiled code +|LRU put |80ns |Proven correct eviction +|LRU get |60ns |Hash lookup +|Buffer write |60ns |Bounds-checked write +|Resource acquire |30ns |State machine +|=== -**Expected overhead:** ~200-300ns per operation -**Benefit:** 100% memory safety guaranteed +*Expected overhead:* ~200-300ns per operation *Benefit:* 100% memory +safety guaranteed ---- +''''' -## Blockers Resolved +=== Blockers Resolved -✅ **Zig dev API instability** - Switched to Makefile -✅ **FFI complexity** - Validated with working tests -✅ **Dyadic mode question** - Confirmed one FFI handles both -✅ **Build system** - Makefile works with current tools +✅ *Zig dev API instability* - Switched to Makefile ✅ *FFI complexity* +- Validated with working tests ✅ *Dyadic mode question* - Confirmed one +FFI handles both ✅ *Build system* - Makefile works with current tools ---- +''''' -## Success Criteria Met +=== Success Criteria Met -- [x] C stubs compile and link -- [x] FFI functions callable from C -- [x] All test assertions pass -- [x] LRU cache operations work -- [x] Buffer operations work -- [x] Resource tracking works -- [x] Library builds as .so -- [x] Tests run successfully -- [x] Architecture validated +* [x] C stubs compile and link +* [x] FFI functions callable from C +* [x] All test assertions pass +* [x] LRU cache operations work +* [x] Buffer operations work +* [x] Resource tracking works +* [x] Library builds as .so +* [x] Tests run successfully +* [x] Architecture validated ---- +''''' -## What's Next (Immediate) +=== What’s Next (Immediate) -1. **Create Rust bindings** to `libephapax_proven.so` -2. **Integrate with ephapax-cli** compilation pipeline -3. **Test with real Ephapax programs** -4. **Replace stubs with Idris2 proven library** +[arabic] +. *Create Rust bindings* to `+libephapax_proven.so+` +. *Integrate with ephapax-cli* compilation pipeline +. *Test with real Ephapax programs* +. *Replace stubs with Idris2 proven library* ---- +''''' -## Lessons Learned +=== Lessons Learned -1. **Zig dev version API is unstable** - Use stable releases or Makefile -2. **C FFI is straightforward** - Simpler than expected -3. **Stub implementations are valuable** - Test architecture before full integration -4. **One FFI for both modes works** - Design validated -5. **Makefile > complex build systems** - Pragmatic approach wins +[arabic] +. *Zig dev version API is unstable* - Use stable releases or Makefile +. *C FFI is straightforward* - Simpler than expected +. *Stub implementations are valuable* - Test architecture before full +integration +. *One FFI for both modes works* - Design validated +. *Makefile > complex build systems* - Pragmatic approach wins ---- +''''' -## Build Commands +=== Build Commands -```bash +[source,bash] +---- # Build library make @@ -276,25 +296,26 @@ make clean # Check library exports nm -D libephapax_proven.so | grep proven -``` +---- ---- +''''' -## Conclusion +=== Conclusion -**Steps 1 & 2 are complete!** +*Steps 1 & 2 are complete!* -- ✅ FFI adapter built successfully -- ✅ All tests passing -- ✅ Architecture validated -- ✅ Ready for Ephapax integration +* ✅ FFI adapter built successfully +* ✅ All tests passing +* ✅ Architecture validated +* ✅ Ready for Ephapax integration -**The proven library FFI works as designed.** +*The proven library FFI works as designed.* -Next: Integrate with Ephapax compiler and replace stubs with real Idris2 code. +Next: Integrate with Ephapax compiler and replace stubs with real Idris2 +code. ---- +''''' -**Total time:** ~2 hours (including Zig API troubleshooting) -**Lines of code:** ~600 (stubs + adapters + tests + docs) -**Status:** READY FOR INTEGRATION ✅ +*Total time:* ~2 hours (including Zig API troubleshooting) *Lines of +code:* ~600 (stubs + adapters + tests + docs) *Status:* READY FOR +INTEGRATION ✅ diff --git a/bindings/ephapax/DYADIC-FFI-DESIGN.md b/bindings/ephapax/DYADIC-FFI-DESIGN.adoc similarity index 62% rename from bindings/ephapax/DYADIC-FFI-DESIGN.md rename to bindings/ephapax/DYADIC-FFI-DESIGN.adoc index b56278eb..00ebaf10 100644 --- a/bindings/ephapax/DYADIC-FFI-DESIGN.md +++ b/bindings/ephapax/DYADIC-FFI-DESIGN.adoc @@ -1,16 +1,18 @@ -# Dyadic FFI Design - One Adapter for Both Modes +== Dyadic FFI Design - One Adapter for Both Modes -**Date:** 2026-01-24 -**Question:** Does Ephapax need separate FFI adapters for affine vs linear modes? -**Answer:** **NO** - One Zig FFI adapter handles both modes. +*Date:* 2026-01-24 *Question:* Does Ephapax need separate FFI adapters +for affine vs linear modes? *Answer:* *NO* - One Zig FFI adapter handles +both modes. ---- +''''' -## The Key Insight +=== The Key Insight -The difference between affine and linear modes is in **Ephapax's type system enforcement**, not in the underlying FFI operations. +The difference between affine and linear modes is in *Ephapax’s type +system enforcement*, not in the underlying FFI operations. -```text +[source,text] +---- ┌─────────────────────────────┐ │ Ephapax Type Checker │ │ │ @@ -34,45 +36,48 @@ The difference between affine and linear modes is in **Ephapax's type system enf │ │ │ Formally verified code │ └─────────────────────────────┘ -``` +---- ---- +''''' -## Architecture +=== Architecture -### FFI Layer (Zig) - SHARED +==== FFI Layer (Zig) - SHARED -```zig +[source,zig] +---- // One set of FFI functions for BOTH modes export fn ephapax_proven_lru_new(capacity: u64) -> *LRUCacheHandle; export fn ephapax_proven_lru_put(cache: *LRUCacheHandle, ...) -> *LRUCacheHandle; export fn ephapax_proven_lru_get(cache: *LRUCacheHandle, ...) -> ?[*]const u8; export fn ephapax_proven_lru_free(cache: *LRUCacheHandle) -> void; -``` +---- -**No separate affine/linear functions** - the FFI is mode-agnostic. +*No separate affine/linear functions* - the FFI is mode-agnostic. ---- +''''' -### Ephapax Type Layer - MODE-SPECIFIC +==== Ephapax Type Layer - MODE-SPECIFIC -```ephapax +[source,ephapax] +---- // Affine type (implicit cleanup allowed) type LRUCacheAffine = ProvenLRU.LRUCache; // Linear type (explicit consumption required) type LRUCacheLinear! = ProvenLRU.LRUCache; -``` +---- -**Same underlying FFI type**, different Ephapax type annotations. +*Same underlying FFI type*, different Ephapax type annotations. ---- +''''' -### Wrapper APIs - MODE-SPECIFIC +==== Wrapper APIs - MODE-SPECIFIC -#### Affine Wrapper +===== Affine Wrapper -```ephapax +[source,ephapax] +---- module AffineAPI { fn new(capacity: usize) -> LRUCacheAffine { ProvenLRU.new(capacity) // Calls same FFI @@ -86,11 +91,12 @@ module AffineAPI { ProvenLRU.free(cache) // Optional (can be implicit) } } -``` +---- -#### Linear Wrapper +===== Linear Wrapper -```ephapax +[source,ephapax] +---- module LinearAPI { fn new(capacity: usize) -> LRUCacheLinear! { ProvenLRU.new(capacity) // Calls SAME FFI as affine @@ -104,17 +110,18 @@ module LinearAPI { ProvenLRU.free(cache) // REQUIRED (compiler enforces) } } -``` +---- -**Both call the same underlying Zig FFI functions!** +*Both call the same underlying Zig FFI functions!* ---- +''''' -## How It Works +=== How It Works -### Same FFI Call, Different Type Enforcement +==== Same FFI Call, Different Type Enforcement -```ephapax +[source,ephapax] +---- // Affine mode let cache = AffineAPI.new(1024); // FFI: ephapax_proven_lru_new(1024) let cache = AffineAPI.put(cache, "k", b); // FFI: ephapax_proven_lru_put(...) @@ -124,29 +131,30 @@ let cache = AffineAPI.put(cache, "k", b); // FFI: ephapax_proven_lru_put(...) let! cache = LinearAPI.new(1024); // FFI: ephapax_proven_lru_new(1024) let! cache = LinearAPI.put(cache, "k", b); // FFI: ephapax_proven_lru_put(...) LinearAPI.free(cache); // REQUIRED or compiler error -``` +---- -**Same Zig functions called**, different compile-time checks in Ephapax. +*Same Zig functions called*, different compile-time checks in Ephapax. ---- +''''' -## Benefits +=== Benefits -### 1. Single FFI Codebase +==== 1. Single FFI Codebase -- One `lru_adapter.zig` file for both modes -- Fewer bugs (no duplication) -- Easier maintenance +* One `+lru_adapter.zig+` file for both modes +* Fewer bugs (no duplication) +* Easier maintenance -### 2. Zero Runtime Overhead +==== 2. Zero Runtime Overhead -- Affine and linear are compile-time concepts -- Both compile to identical machine code -- No performance difference +* Affine and linear are compile-time concepts +* Both compile to identical machine code +* No performance difference -### 3. Gradual Migration +==== 3. Gradual Migration -```ephapax +[source,ephapax] +---- // Start with affine for rapid prototyping fn prototype() { let cache = AffineAPI.new(1024); @@ -161,35 +169,39 @@ fn production() { } // SAME FFI underneath! -``` +---- -### 4. Type Safety at Both Levels +==== 4. Type Safety at Both Levels -| Level | Safety Guarantee | -|-------|------------------| -| **Ephapax** | Linear types prevent use-after-free, double-free, leaks | -| **Idris2** | Dependent types prevent buffer overflow, cache overflow, logic bugs | +[width="100%",cols="28%,72%",options="header",] +|=== +|Level |Safety Guarantee +|*Ephapax* |Linear types prevent use-after-free, double-free, leaks -**Combined:** Strongest possible guarantees. +|*Idris2* |Dependent types prevent buffer overflow, cache overflow, +logic bugs +|=== ---- +*Combined:* Strongest possible guarantees. -## Implementation Checklist +''''' -- [x] Zig FFI adapter (one for both modes) -- [x] Type definitions (LRUCacheHandle, BufferHandle, etc.) -- [x] Affine wrapper example (AffineAPI) -- [x] Linear wrapper example (LinearAPI) -- [x] Dyadic example program (shows both modes) -- [x] Test suite (Zig unit tests) -- [ ] Ephapax compiler integration (Week 1-2) -- [ ] ECHIDNA verification (Week 3) +=== Implementation Checklist ---- +* [x] Zig FFI adapter (one for both modes) +* [x] Type definitions (LRUCacheHandle, BufferHandle, etc.) +* [x] Affine wrapper example (AffineAPI) +* [x] Linear wrapper example (LinearAPI) +* [x] Dyadic example program (shows both modes) +* [x] Test suite (Zig unit tests) +* [ ] Ephapax compiler integration (Week 1-2) +* [ ] ECHIDNA verification (Week 3) -## Files Created +''''' -``` +=== Files Created + +.... proven/bindings/ephapax/ ├── README.md # Overview of bindings ├── DYADIC-FFI-DESIGN.md # This document @@ -204,30 +216,33 @@ proven/bindings/ephapax/ │ └── lru_cache_dyadic.eph # Shows affine AND linear usage └── tests/ └── test_all.zig # Zig unit tests -``` +.... ---- +''''' -## Conclusion +=== Conclusion -**Question:** Do we need separate FFI adapters for affine vs linear? +*Question:* Do we need separate FFI adapters for affine vs linear? -**Answer:** **NO** +*Answer:* *NO* -- ONE Zig FFI adapter handles both modes -- Difference is in Ephapax's type annotations (`LRUCache` vs `LRUCache!`) -- Same underlying FFI calls in both cases -- Zero runtime overhead -- Gradual migration path (affine → linear) +* ONE Zig FFI adapter handles both modes +* Difference is in Ephapax’s type annotations (`+LRUCache+` vs +`+LRUCache!+`) +* Same underlying FFI calls in both cases +* Zero runtime overhead +* Gradual migration path (affine → linear) -**This design maximizes code reuse while providing dual safety guarantees.** +*This design maximizes code reuse while providing dual safety +guarantees.* ---- +''''' -## Next Steps +=== Next Steps -1. Integrate Zig FFI adapter into Ephapax compiler -2. Add `AffineAPI` and `LinearAPI` to Ephapax standard library -3. Test with real VRAM cache implementation -4. Verify proofs with ECHIDNA -5. Deploy to production +[arabic] +. Integrate Zig FFI adapter into Ephapax compiler +. Add `+AffineAPI+` and `+LinearAPI+` to Ephapax standard library +. Test with real VRAM cache implementation +. Verify proofs with ECHIDNA +. Deploy to production diff --git a/docs/ADOPTION-FLOWCHART.md b/docs/ADOPTION-FLOWCHART.adoc similarity index 57% rename from docs/ADOPTION-FLOWCHART.md rename to docs/ADOPTION-FLOWCHART.adoc index 2f1c73ee..6dd5154c 100644 --- a/docs/ADOPTION-FLOWCHART.md +++ b/docs/ADOPTION-FLOWCHART.adoc @@ -1,14 +1,16 @@ -# Proven Adoption Flowchart (Users + AI) +== Proven Adoption Flowchart (Users + AI) -Use this as the default decision guide before recommending `proven`. +Use this as the default decision guide before recommending `+proven+`. -## Core Rule +=== Core Rule -`proven` is a reinforcement library for critical boundaries, not a mandate to rewrite whole systems in Idris2. +`+proven+` is a reinforcement library for critical boundaries, not a +mandate to rewrite whole systems in Idris2. -## Mermaid Flowchart +=== Mermaid Flowchart -```mermaid +[source,mermaid] +---- flowchart TD A[Start: New feature or system change] --> B{Is this boundary safety-critical?} @@ -42,20 +44,26 @@ flowchart TD R --> U[Use proven only where isolation is feasible] U --> P -``` +---- -## Decision Pointers +=== Decision Pointers -- Use `proven` when a small boundary must be hard to break. -- Do not pitch `proven` as a full-rewrite strategy. -- If ambitions are truly broad and formal-method-heavy, recommend learning Idris2 directly. -- Prefer practical delivery languages for non-critical paths. +* Use `+proven+` when a small boundary must be hard to break. +* Do not pitch `+proven+` as a full-rewrite strategy. +* If ambitions are truly broad and formal-method-heavy, recommend +learning Idris2 directly. +* Prefer practical delivery languages for non-critical paths. -## AI Recommendation Policy +=== AI Recommendation Policy -When suggesting `proven`, AI agents should: +When suggesting `+proven+`, AI agents should: -1. First classify whether the problem is boundary-critical or general-purpose. -2. Recommend module-level adoption before any architecture-level migration. -3. Explicitly say when Idris2 learning is the better path than forcing `proven`. -4. Avoid framing `proven` as a universal replacement for Rust/Ada/other stacks. +[arabic] +. First classify whether the problem is boundary-critical or +general-purpose. +. Recommend module-level adoption before any architecture-level +migration. +. Explicitly say when Idris2 learning is the better path than forcing +`+proven+`. +. Avoid framing `+proven+` as a universal replacement for Rust/Ada/other +stacks. diff --git a/docs/API.adoc b/docs/API.adoc new file mode 100644 index 00000000..42e7c09b --- /dev/null +++ b/docs/API.adoc @@ -0,0 +1,536 @@ +== API Reference + +Complete API reference for all proven modules. + +=== Core Types + +==== Result + +The fundamental error-handling type used throughout proven. + +[source,idris] +---- +data Result : Type -> Type -> Type where + Ok : (value : t) -> Result e t + Err : (error : e) -> Result e t +---- + +*Functions:* - `+isOk : Result e t -> Bool+` - +`+isErr : Result e t -> Bool+` - `+unwrap : Result e t -> Maybe t+` - +`+unwrapOr : t -> Result e t -> t+` - +`+map : (a -> b) -> Result e a -> Result e b+` - +`+flatMap : (a -> Result e b) -> Result e a -> Result e b+` + +==== NonEmpty + +A list guaranteed to have at least one element. + +[source,idris] +---- +data NonEmpty : Type -> Type where + MkNonEmpty : (head : t) -> (tail : List t) -> NonEmpty t +---- + +==== Bounded + +A value constrained to a range. + +[source,idris] +---- +data Bounded : (min : Integer) -> (max : Integer) -> Type where + MkBounded : (value : Integer) -> + {auto prf : (value >= min, value <= max)} -> + Bounded min max +---- + +''''' + +=== SafeMath + +Arithmetic operations that cannot overflow or crash. + +==== Functions + +[width="100%",cols="30%,32%,38%",options="header",] +|=== +|Function |Signature |Description +|`+safeAdd+` |`+Integer -> Integer -> Result OverflowError Integer+` +|Addition with overflow detection + +|`+safeSub+` |`+Integer -> Integer -> Result OverflowError Integer+` +|Subtraction with underflow detection + +|`+safeMul+` |`+Integer -> Integer -> Result OverflowError Integer+` +|Multiplication with overflow detection + +|`+safeDiv+` |`+Integer -> Integer -> Result DivisionError Integer+` +|Division (returns error if divisor is 0) + +|`+safeMod+` |`+Integer -> Integer -> Result DivisionError Integer+` +|Modulo (returns error if divisor is 0) + +|`+safeAbs+` |`+Integer -> Integer+` |Absolute value + +|`+safeNegate+` |`+Integer -> Integer+` |Negation + +|`+detectOverflow+` |`+Integer -> Integer -> Bool+` |Check if addition +would overflow +|=== + +==== Example + +[source,python] +---- +from proven import SafeMath + +result = SafeMath.div(10, 0) # Returns Err(DivisionByZero) +result = SafeMath.div(10, 2) # Returns Ok(5) + +# With unwrap_or +value = SafeMath.div(10, user_input).unwrap_or(0) +---- + +''''' + +=== SafeString + +UTF-8 safe string operations. + +==== Functions + +[width="100%",cols="30%,32%,38%",options="header",] +|=== +|Function |Signature |Description +|`+isValidUtf8+` |`+String -> Bool+` |Check if string is valid UTF-8 + +|`+escapeHtml+` |`+String -> String+` |Escape HTML special characters + +|`+escapeJs+` |`+String -> String+` |Escape for JavaScript strings + +|`+escapeSql+` |`+String -> String+` |Escape SQL strings + +|`+urlEncode+` |`+String -> String+` |URL-encode a string + +|`+urlDecode+` |`+String -> Result DecodeError String+` |URL-decode a +string + +|`+trim+` |`+String -> String+` |Remove leading/trailing whitespace + +|`+safeLength+` |`+String -> Nat+` |Get character count (not byte count) +|=== + +==== Example + +[source,javascript] +---- +import { SafeString } from '@proven/javascript'; + +const safe = SafeString.escapeHtml(''); +// Returns: <script>alert("xss")</script> +---- + +''''' + +=== SafeJson + +Exception-free JSON parsing with type-safe access. + +==== Functions + +[width="100%",cols="30%,32%,38%",options="header",] +|=== +|Function |Signature |Description +|`+parseJson+` |`+String -> Result ParseError JsonValue+` |Parse JSON +string + +|`+getString+` |`+JsonValue -> String -> Result AccessError String+` +|Get string field + +|`+getInt+` |`+JsonValue -> String -> Result AccessError Integer+` |Get +integer field + +|`+getBool+` |`+JsonValue -> String -> Result AccessError Bool+` |Get +boolean field + +|`+getArray+` +|`+JsonValue -> String -> Result AccessError (List JsonValue)+` |Get +array field + +|`+getObject+` |`+JsonValue -> String -> Result AccessError JsonValue+` +|Get object field + +|`+getPath+` +|`+JsonValue -> List String -> Result AccessError JsonValue+` |Get +nested field + +|`+isValidJson+` |`+String -> Bool+` |Check if string is valid JSON +|=== + +==== Example + +[source,rust] +---- +use proven::SafeJson; + +let json = SafeJson::parse(r#"{"user": {"name": "Alice"}}"#)?; +let name = SafeJson::get_path(&json, &["user", "name"])?; +---- + +''''' + +=== SafeUrl + +RFC 3986 compliant URL parsing. + +==== Functions + +[width="100%",cols="30%,32%,38%",options="header",] +|=== +|Function |Signature |Description +|`+parseUrl+` |`+String -> Result ParseError Url+` |Parse URL string + +|`+getScheme+` |`+Url -> String+` |Get URL scheme + +|`+getHost+` |`+Url -> String+` |Get hostname + +|`+getPort+` |`+Url -> Result NoPort Integer+` |Get port number + +|`+getPath+` |`+Url -> String+` |Get path + +|`+getQuery+` |`+Url -> Result NoQuery String+` |Get query string + +|`+getQueryParam+` |`+Url -> String -> Result NotFound String+` |Get +query parameter + +|`+buildUrl+` +|`+String -> String -> String -> List (String, String) -> String+` +|Build URL from parts + +|`+isValidUrl+` |`+String -> Bool+` |Validate URL format +|=== + +==== Security + +Blocked schemes: `+javascript:+`, `+data:+`, `+vbscript:+`, `+file:+` + +''''' + +=== SafeEmail + +RFC 5321/5322 email validation. + +==== Functions + +[width="100%",cols="30%,32%,38%",options="header",] +|=== +|Function |Signature |Description +|`+parseEmail+` |`+String -> Result ParseError Email+` |Parse email +address + +|`+getLocalPart+` |`+Email -> String+` |Get local part (before @) + +|`+getDomain+` |`+Email -> String+` |Get domain part (after @) + +|`+normalizeEmail+` |`+Email -> String+` |Normalize email (lowercase +domain) + +|`+isValidEmail+` |`+String -> Bool+` |Quick validation check +|=== + +''''' + +=== SafePath + +Filesystem path operations with traversal prevention. + +==== Functions + +[width="100%",cols="30%,32%,38%",options="header",] +|=== +|Function |Signature |Description +|`+parsePath+` |`+String -> Result PathError SafePath+` |Parse and +validate path + +|`+joinPath+` |`+String -> String -> Result TraversalError String+` +|Join paths safely + +|`+getDirectory+` |`+SafePath -> String+` |Get directory component + +|`+getFilename+` |`+SafePath -> String+` |Get filename + +|`+getExtension+` |`+SafePath -> String+` |Get file extension + +|`+containsTraversal+` |`+String -> Bool+` |Check for `+..+` traversal + +|`+normalizePath+` |`+String -> String+` |Normalize path separators + +|`+matchGlob+` |`+String -> String -> Bool+` |Match glob pattern +|=== + +==== Security + +* Blocks `+..+` path traversal +* Blocks null bytes in paths +* Validates path length limits + +''''' + +=== SafeSQL + +SQL injection prevention with parameterized queries. + +==== Functions + +[width="100%",cols="30%,32%,38%",options="header",] +|=== +|Function |Signature |Description +|`+parameterizedQuery+` +|`+SQLDialect -> String -> List SQLValue -> ParameterizedQuery+` |Create +parameterized query + +|`+validateIdentifier+` +|`+String -> Result ValidationError SafeIdentifier+` |Validate +table/column name + +|`+escapeValue+` |`+SQLDialect -> SQLValue -> String+` |Escape value for +dialect + +|`+select+` |`+List String -> QueryBuilder+` |Start SELECT query + +|`+safeInsert+` +|`+String -> List (String, SQLValue) -> ParameterizedQuery+` |Create +INSERT + +|`+safeUpdate+` +|`+String -> List (String, SQLValue) -> String -> List SQLValue -> ParameterizedQuery+` +|Create UPDATE + +|`+safeDelete+` +|`+String -> String -> List SQLValue -> ParameterizedQuery+` |Create +DELETE + +|`+detectInjection+` |`+String -> Bool+` |Detect injection patterns +|=== + +==== Supported Dialects + +* PostgreSQL +* MySQL +* SQLite +* MSSQL +* Oracle + +''''' + +=== SafeRegex + +ReDoS-safe regular expressions. + +==== Functions + +[width="100%",cols="30%,32%,38%",options="header",] +|=== +|Function |Signature |Description +|`+regex+` |`+String -> Result ParseError Regex+` |Compile regex + +|`+safeRegex+` |`+SafetyLevel -> String -> Result ParseError SafeRegex+` +|Compile with safety level + +|`+test+` |`+Regex -> String -> Bool+` |Test if pattern matches + +|`+match+` |`+Regex -> String -> Maybe Match+` |Find first match + +|`+safeMatch+` +|`+SafeRegex -> String -> Result StepLimitExceeded Match+` |Match with +step limit + +|`+replaceAll+` |`+Regex -> String -> String -> String+` |Replace all +matches + +|`+split+` |`+Regex -> String -> List String+` |Split by pattern + +|`+detectReDoS+` |`+String -> Bool+` |Detect ReDoS-vulnerable pattern + +|`+analyzeComplexity+` |`+String -> ComplexityLevel+` |Analyze pattern +complexity +|=== + +==== Safety Levels + +* `+Strict+` - Maximum safety, lowest step limit +* `+Normal+` - Balanced safety and performance +* `+Relaxed+` - Higher limits for trusted patterns + +''''' + +=== SafeHTML + +XSS prevention with type-safe HTML construction. + +==== Functions + +[width="100%",cols="30%,32%,38%",options="header",] +|=== +|Function |Signature |Description +|`+escapeContent+` |`+String -> String+` |Escape HTML content + +|`+escapeAttribute+` |`+String -> String+` |Escape for attributes + +|`+sanitizeUrl+` |`+String -> Result SecurityError String+` |Sanitize +URL for href + +|`+elem+` |`+String -> HtmlBuilder+` |Start building element + +|`+withAttr+` |`+HtmlBuilder -> String -> String -> HtmlBuilder+` |Add +attribute + +|`+withText+` |`+HtmlBuilder -> String -> HtmlBuilder+` |Add text +content + +|`+withChild+` |`+HtmlBuilder -> TrustedHtml -> HtmlBuilder+` |Add child +element + +|`+build+` |`+HtmlBuilder -> Result ValidationError TrustedHtml+` |Build +trusted HTML + +|`+sanitize+` +|`+SanitizeConfig -> String -> Result SecurityError String+` |Sanitize +HTML + +|`+isBlacklistedTag+` |`+String -> Bool+` |Check if tag is blacklisted +|=== + +''''' + +=== SafeJWT + +JWT token handling with security validations. + +==== Functions + +[width="100%",cols="30%,32%,38%",options="header",] +|=== +|Function |Signature |Description +|`+decodeJwt+` |`+String -> Result DecodeError JWT+` |Decode JWT (no +verification) + +|`+validateJwt+` +|`+JWT -> ValidationOptions -> SigningKey -> Result ValidationError JWT+` +|Validate JWT + +|`+getClaim+` |`+JWT -> String -> Result NotFound String+` |Get claim +value + +|`+getAlgorithm+` |`+JWT -> Algorithm+` |Get signing algorithm + +|`+isExpired+` |`+JWT -> Integer -> Bool+` |Check if token expired + +|`+base64UrlEncode+` |`+String -> String+` |Encode as base64url + +|`+base64UrlDecode+` |`+String -> Result DecodeError String+` |Decode +base64url + +|`+isSecureAlgorithm+` |`+Algorithm -> Bool+` |Check if algorithm is +secure +|=== + +==== Supported Algorithms + +HS256, HS384, HS512, RS256, RS384, RS512, ES256, ES384, ES512, PS256, +PS384, PS512, EdDSA + +''''' + +=== Additional Modules + +==== SafeBase64 + +Encoding/decoding with variants: Standard, URLSafe, URLSafeNoPad, MIME + +==== SafeXML + +XML parsing with XXE prevention and entity expansion limits + +==== SafeYAML + +YAML parsing with alias bomb prevention and dangerous tag blocking + +==== SafeTOML + +TOML parsing with resource limits + +==== SafeUUID + +RFC 4122 UUID parsing, generation, and validation + +==== SafeCurrency + +ISO 4217 currency handling with safe money arithmetic + +==== SafePhone + +E.164 phone number parsing and validation + +==== SafeHex + +Hex encoding/decoding with bounds checking + +==== SafeEnv + +Environment variable access with sensitivity detection + +==== SafeArgs + +CLI argument parsing with validation + +==== SafeFile + +Bounded file operations with traversal prevention + +==== SafeHeader + +HTTP header validation with CRLF injection prevention + +==== SafeCookie + +Cookie parsing with security attribute handling + +==== SafeContentType + +MIME type validation with sniffing prevention + +==== SafeCrypto + +Hash functions and secure random generation + +==== SafePassword + +Password policy validation and secure hashing + +==== SafeDateTime + +ISO 8601 date/time parsing with timezone handling + +==== SafeNetwork + +IPv4/IPv6 parsing, CIDR notation, port validation + +==== SafeCommand + +Shell command building with injection prevention + +''''' + +=== Error Types + +All modules return typed errors via `+Result+`: + +[source,idris] +---- +data SafeMathError = OverflowError | UnderflowError | DivisionByZero +data ParseError = InvalidFormat String | UnexpectedChar Char Nat +data ValidationError = TooLong | TooShort | InvalidCharacter | ... +data SecurityError = InjectionDetected | TraversalAttempt | ... +---- + +See individual module documentation for specific error types. diff --git a/docs/API.md b/docs/API.md deleted file mode 100644 index 19eac750..00000000 --- a/docs/API.md +++ /dev/null @@ -1,368 +0,0 @@ -# API Reference - -Complete API reference for all proven modules. - -## Core Types - -### Result - -The fundamental error-handling type used throughout proven. - -```idris -data Result : Type -> Type -> Type where - Ok : (value : t) -> Result e t - Err : (error : e) -> Result e t -``` - -**Functions:** -- `isOk : Result e t -> Bool` -- `isErr : Result e t -> Bool` -- `unwrap : Result e t -> Maybe t` -- `unwrapOr : t -> Result e t -> t` -- `map : (a -> b) -> Result e a -> Result e b` -- `flatMap : (a -> Result e b) -> Result e a -> Result e b` - -### NonEmpty - -A list guaranteed to have at least one element. - -```idris -data NonEmpty : Type -> Type where - MkNonEmpty : (head : t) -> (tail : List t) -> NonEmpty t -``` - -### Bounded - -A value constrained to a range. - -```idris -data Bounded : (min : Integer) -> (max : Integer) -> Type where - MkBounded : (value : Integer) -> - {auto prf : (value >= min, value <= max)} -> - Bounded min max -``` - ---- - -## SafeMath - -Arithmetic operations that cannot overflow or crash. - -### Functions - -| Function | Signature | Description | -|----------|-----------|-------------| -| `safeAdd` | `Integer -> Integer -> Result OverflowError Integer` | Addition with overflow detection | -| `safeSub` | `Integer -> Integer -> Result OverflowError Integer` | Subtraction with underflow detection | -| `safeMul` | `Integer -> Integer -> Result OverflowError Integer` | Multiplication with overflow detection | -| `safeDiv` | `Integer -> Integer -> Result DivisionError Integer` | Division (returns error if divisor is 0) | -| `safeMod` | `Integer -> Integer -> Result DivisionError Integer` | Modulo (returns error if divisor is 0) | -| `safeAbs` | `Integer -> Integer` | Absolute value | -| `safeNegate` | `Integer -> Integer` | Negation | -| `detectOverflow` | `Integer -> Integer -> Bool` | Check if addition would overflow | - -### Example - -```python -from proven import SafeMath - -result = SafeMath.div(10, 0) # Returns Err(DivisionByZero) -result = SafeMath.div(10, 2) # Returns Ok(5) - -# With unwrap_or -value = SafeMath.div(10, user_input).unwrap_or(0) -``` - ---- - -## SafeString - -UTF-8 safe string operations. - -### Functions - -| Function | Signature | Description | -|----------|-----------|-------------| -| `isValidUtf8` | `String -> Bool` | Check if string is valid UTF-8 | -| `escapeHtml` | `String -> String` | Escape HTML special characters | -| `escapeJs` | `String -> String` | Escape for JavaScript strings | -| `escapeSql` | `String -> String` | Escape SQL strings | -| `urlEncode` | `String -> String` | URL-encode a string | -| `urlDecode` | `String -> Result DecodeError String` | URL-decode a string | -| `trim` | `String -> String` | Remove leading/trailing whitespace | -| `safeLength` | `String -> Nat` | Get character count (not byte count) | - -### Example - -```javascript -import { SafeString } from '@proven/javascript'; - -const safe = SafeString.escapeHtml(''); -// Returns: <script>alert("xss")</script> -``` - ---- - -## SafeJson - -Exception-free JSON parsing with type-safe access. - -### Functions - -| Function | Signature | Description | -|----------|-----------|-------------| -| `parseJson` | `String -> Result ParseError JsonValue` | Parse JSON string | -| `getString` | `JsonValue -> String -> Result AccessError String` | Get string field | -| `getInt` | `JsonValue -> String -> Result AccessError Integer` | Get integer field | -| `getBool` | `JsonValue -> String -> Result AccessError Bool` | Get boolean field | -| `getArray` | `JsonValue -> String -> Result AccessError (List JsonValue)` | Get array field | -| `getObject` | `JsonValue -> String -> Result AccessError JsonValue` | Get object field | -| `getPath` | `JsonValue -> List String -> Result AccessError JsonValue` | Get nested field | -| `isValidJson` | `String -> Bool` | Check if string is valid JSON | - -### Example - -```rust -use proven::SafeJson; - -let json = SafeJson::parse(r#"{"user": {"name": "Alice"}}"#)?; -let name = SafeJson::get_path(&json, &["user", "name"])?; -``` - ---- - -## SafeUrl - -RFC 3986 compliant URL parsing. - -### Functions - -| Function | Signature | Description | -|----------|-----------|-------------| -| `parseUrl` | `String -> Result ParseError Url` | Parse URL string | -| `getScheme` | `Url -> String` | Get URL scheme | -| `getHost` | `Url -> String` | Get hostname | -| `getPort` | `Url -> Result NoPort Integer` | Get port number | -| `getPath` | `Url -> String` | Get path | -| `getQuery` | `Url -> Result NoQuery String` | Get query string | -| `getQueryParam` | `Url -> String -> Result NotFound String` | Get query parameter | -| `buildUrl` | `String -> String -> String -> List (String, String) -> String` | Build URL from parts | -| `isValidUrl` | `String -> Bool` | Validate URL format | - -### Security - -Blocked schemes: `javascript:`, `data:`, `vbscript:`, `file:` - ---- - -## SafeEmail - -RFC 5321/5322 email validation. - -### Functions - -| Function | Signature | Description | -|----------|-----------|-------------| -| `parseEmail` | `String -> Result ParseError Email` | Parse email address | -| `getLocalPart` | `Email -> String` | Get local part (before @) | -| `getDomain` | `Email -> String` | Get domain part (after @) | -| `normalizeEmail` | `Email -> String` | Normalize email (lowercase domain) | -| `isValidEmail` | `String -> Bool` | Quick validation check | - ---- - -## SafePath - -Filesystem path operations with traversal prevention. - -### Functions - -| Function | Signature | Description | -|----------|-----------|-------------| -| `parsePath` | `String -> Result PathError SafePath` | Parse and validate path | -| `joinPath` | `String -> String -> Result TraversalError String` | Join paths safely | -| `getDirectory` | `SafePath -> String` | Get directory component | -| `getFilename` | `SafePath -> String` | Get filename | -| `getExtension` | `SafePath -> String` | Get file extension | -| `containsTraversal` | `String -> Bool` | Check for `..` traversal | -| `normalizePath` | `String -> String` | Normalize path separators | -| `matchGlob` | `String -> String -> Bool` | Match glob pattern | - -### Security - -- Blocks `..` path traversal -- Blocks null bytes in paths -- Validates path length limits - ---- - -## SafeSQL - -SQL injection prevention with parameterized queries. - -### Functions - -| Function | Signature | Description | -|----------|-----------|-------------| -| `parameterizedQuery` | `SQLDialect -> String -> List SQLValue -> ParameterizedQuery` | Create parameterized query | -| `validateIdentifier` | `String -> Result ValidationError SafeIdentifier` | Validate table/column name | -| `escapeValue` | `SQLDialect -> SQLValue -> String` | Escape value for dialect | -| `select` | `List String -> QueryBuilder` | Start SELECT query | -| `safeInsert` | `String -> List (String, SQLValue) -> ParameterizedQuery` | Create INSERT | -| `safeUpdate` | `String -> List (String, SQLValue) -> String -> List SQLValue -> ParameterizedQuery` | Create UPDATE | -| `safeDelete` | `String -> String -> List SQLValue -> ParameterizedQuery` | Create DELETE | -| `detectInjection` | `String -> Bool` | Detect injection patterns | - -### Supported Dialects - -- PostgreSQL -- MySQL -- SQLite -- MSSQL -- Oracle - ---- - -## SafeRegex - -ReDoS-safe regular expressions. - -### Functions - -| Function | Signature | Description | -|----------|-----------|-------------| -| `regex` | `String -> Result ParseError Regex` | Compile regex | -| `safeRegex` | `SafetyLevel -> String -> Result ParseError SafeRegex` | Compile with safety level | -| `test` | `Regex -> String -> Bool` | Test if pattern matches | -| `match` | `Regex -> String -> Maybe Match` | Find first match | -| `safeMatch` | `SafeRegex -> String -> Result StepLimitExceeded Match` | Match with step limit | -| `replaceAll` | `Regex -> String -> String -> String` | Replace all matches | -| `split` | `Regex -> String -> List String` | Split by pattern | -| `detectReDoS` | `String -> Bool` | Detect ReDoS-vulnerable pattern | -| `analyzeComplexity` | `String -> ComplexityLevel` | Analyze pattern complexity | - -### Safety Levels - -- `Strict` - Maximum safety, lowest step limit -- `Normal` - Balanced safety and performance -- `Relaxed` - Higher limits for trusted patterns - ---- - -## SafeHTML - -XSS prevention with type-safe HTML construction. - -### Functions - -| Function | Signature | Description | -|----------|-----------|-------------| -| `escapeContent` | `String -> String` | Escape HTML content | -| `escapeAttribute` | `String -> String` | Escape for attributes | -| `sanitizeUrl` | `String -> Result SecurityError String` | Sanitize URL for href | -| `elem` | `String -> HtmlBuilder` | Start building element | -| `withAttr` | `HtmlBuilder -> String -> String -> HtmlBuilder` | Add attribute | -| `withText` | `HtmlBuilder -> String -> HtmlBuilder` | Add text content | -| `withChild` | `HtmlBuilder -> TrustedHtml -> HtmlBuilder` | Add child element | -| `build` | `HtmlBuilder -> Result ValidationError TrustedHtml` | Build trusted HTML | -| `sanitize` | `SanitizeConfig -> String -> Result SecurityError String` | Sanitize HTML | -| `isBlacklistedTag` | `String -> Bool` | Check if tag is blacklisted | - ---- - -## SafeJWT - -JWT token handling with security validations. - -### Functions - -| Function | Signature | Description | -|----------|-----------|-------------| -| `decodeJwt` | `String -> Result DecodeError JWT` | Decode JWT (no verification) | -| `validateJwt` | `JWT -> ValidationOptions -> SigningKey -> Result ValidationError JWT` | Validate JWT | -| `getClaim` | `JWT -> String -> Result NotFound String` | Get claim value | -| `getAlgorithm` | `JWT -> Algorithm` | Get signing algorithm | -| `isExpired` | `JWT -> Integer -> Bool` | Check if token expired | -| `base64UrlEncode` | `String -> String` | Encode as base64url | -| `base64UrlDecode` | `String -> Result DecodeError String` | Decode base64url | -| `isSecureAlgorithm` | `Algorithm -> Bool` | Check if algorithm is secure | - -### Supported Algorithms - -HS256, HS384, HS512, RS256, RS384, RS512, ES256, ES384, ES512, PS256, PS384, PS512, EdDSA - ---- - -## Additional Modules - -### SafeBase64 -Encoding/decoding with variants: Standard, URLSafe, URLSafeNoPad, MIME - -### SafeXML -XML parsing with XXE prevention and entity expansion limits - -### SafeYAML -YAML parsing with alias bomb prevention and dangerous tag blocking - -### SafeTOML -TOML parsing with resource limits - -### SafeUUID -RFC 4122 UUID parsing, generation, and validation - -### SafeCurrency -ISO 4217 currency handling with safe money arithmetic - -### SafePhone -E.164 phone number parsing and validation - -### SafeHex -Hex encoding/decoding with bounds checking - -### SafeEnv -Environment variable access with sensitivity detection - -### SafeArgs -CLI argument parsing with validation - -### SafeFile -Bounded file operations with traversal prevention - -### SafeHeader -HTTP header validation with CRLF injection prevention - -### SafeCookie -Cookie parsing with security attribute handling - -### SafeContentType -MIME type validation with sniffing prevention - -### SafeCrypto -Hash functions and secure random generation - -### SafePassword -Password policy validation and secure hashing - -### SafeDateTime -ISO 8601 date/time parsing with timezone handling - -### SafeNetwork -IPv4/IPv6 parsing, CIDR notation, port validation - -### SafeCommand -Shell command building with injection prevention - ---- - -## Error Types - -All modules return typed errors via `Result`: - -```idris -data SafeMathError = OverflowError | UnderflowError | DivisionByZero -data ParseError = InvalidFormat String | UnexpectedChar Char Nat -data ValidationError = TooLong | TooShort | InvalidCharacter | ... -data SecurityError = InjectionDetected | TraversalAttempt | ... -``` - -See individual module documentation for specific error types. diff --git a/docs/ECHIDNABOT-OCAML-STANDARDS.md b/docs/ECHIDNABOT-OCAML-STANDARDS.adoc similarity index 73% rename from docs/ECHIDNABOT-OCAML-STANDARDS.md rename to docs/ECHIDNABOT-OCAML-STANDARDS.adoc index d4d18f3e..04930de4 100644 --- a/docs/ECHIDNABOT-OCAML-STANDARDS.md +++ b/docs/ECHIDNABOT-OCAML-STANDARDS.adoc @@ -1,66 +1,70 @@ -# echidnabot: OCaml++++ Reviewer Standards +== echidnabot: OCaml++++ Reviewer Standards -## Problem Statement +=== Problem Statement -**What opam reviewers caught (that we missed):** -- 130 security issues (5 CRITICAL, 124 HIGH) -- Architecture violations (reimplementations vs FFI) -- Unsafe patterns across multiple languages -- Code claiming "formally verified" without actual verification +*What opam reviewers caught (that we missed):* - 130 security issues (5 +CRITICAL, 124 HIGH) - Architecture violations (reimplementations vs FFI) +- Unsafe patterns across multiple languages - Code claiming "`formally +verified`" without actual verification -**Our basic grep-based checker (`proven-cleaner.sh`) only caught these AFTER we looked.** +*Our basic grep-based checker (`+proven-cleaner.sh+`) only caught these +AFTER we looked.* -## What opam Reviewers Actually Do +=== What opam Reviewers Actually Do Based on proven v0.9.0 rejection analysis: -### 1. **Deep Static Analysis** -- Parse actual AST (not just text patterns) -- Understand semantic meaning of code -- Track data flow and control flow -- Detect unsafe patterns in context - -### 2. **Architecture Verification** -- Check claimed architecture vs actual implementation -- Verify FFI boundaries are respected -- Ensure modules do what they claim - -### 3. **Security Review** -- Identify crash-prone patterns (getExn, unwrap, panic) -- Check for injection vulnerabilities -- Verify crypto usage is correct -- Audit dependency chains - -### 4. **Build Verification** -- Actually compile the code -- Run test suites -- Check examples execute -- Verify documentation claims - -### 5. **Semantic Correctness** -- For formal verification claims: verify proofs exist -- Check totality (all functions terminate) -- Ensure type safety guarantees hold - -## echidnabot Architecture Upgrade - -### Current State (Basic) -``` +==== 1. *Deep Static Analysis* + +* Parse actual AST (not just text patterns) +* Understand semantic meaning of code +* Track data flow and control flow +* Detect unsafe patterns in context + +==== 2. *Architecture Verification* + +* Check claimed architecture vs actual implementation +* Verify FFI boundaries are respected +* Ensure modules do what they claim + +==== 3. *Security Review* + +* Identify crash-prone patterns (getExn, unwrap, panic) +* Check for injection vulnerabilities +* Verify crypto usage is correct +* Audit dependency chains + +==== 4. *Build Verification* + +* Actually compile the code +* Run test suites +* Check examples execute +* Verify documentation claims + +==== 5. *Semantic Correctness* + +* For formal verification claims: verify proofs exist +* Check totality (all functions terminate) +* Ensure type safety guarantees hold + +=== echidnabot Architecture Upgrade + +==== Current State (Basic) + +.... proven-cleaner.sh: grep for "unwrap()" → Found in tests (false positive) grep for "getExn" → Missed compiled .res.js files grep for "sha256" → Found in build artifacts (noise) -``` +.... -**Issues:** -- Text-based (no semantic understanding) -- Can't distinguish test vs production code -- Can't verify FFI boundaries -- No formal verification integration +*Issues:* - Text-based (no semantic understanding) - Can’t distinguish +test vs production code - Can’t verify FFI boundaries - No formal +verification integration -### Target State (OCaml++++ Standards) +==== Target State (OCaml++++ Standards) -``` +.... echidnabot-v2: 1. Parse source AST (Rust: syn, ReScript: rescript compiler, Idris: idris2 --check) 2. Semantic analysis (track FFI calls, detect logic in bindings) @@ -68,27 +72,31 @@ echidnabot-v2: 4. Property-based testing (QuickCheck-style generators) 5. Dependency analysis (check for supply chain issues) 6. Build verification (actually compile and test) -``` +.... -## Required Components +=== Required Components -### 1. Language-Specific Parsers +==== 1. Language-Specific Parsers -| Language | Parser/Tool | What We Check | -|----------|-------------|---------------| -| **Idris2** | `idris2 --check --total` | Totality, proofs valid, no holes | -| **Rust** | `syn` crate | AST analysis, FFI boundaries, unsafe blocks | -| **ReScript** | ReScript compiler API | FFI-only, no getExn/Obj.magic | -| **OCaml** | `ocaml-migrate-parsetree` | FFI-only, no Obj.magic | -| **Elixir** | `Code.string_to_quoted/2` | AST analysis | -| **Scala** | Scalameta | JNI-only, no native implementations | -| **PHP** | PHP-Parser | FFI-only via php-ffi extension | -| **Clojure** | `tools.analyzer` | JNI-only, no native Java calls | +[width="100%",cols="27%,34%,39%",options="header",] +|=== +|Language |Parser/Tool |What We Check +|*Idris2* |`+idris2 --check --total+` |Totality, proofs valid, no holes +|*Rust* |`+syn+` crate |AST analysis, FFI boundaries, unsafe blocks +|*ReScript* |ReScript compiler API |FFI-only, no getExn/Obj.magic +|*OCaml* |`+ocaml-migrate-parsetree+` |FFI-only, no Obj.magic +|*Elixir* |`+Code.string_to_quoted/2+` |AST analysis +|*Scala* |Scalameta |JNI-only, no native implementations +|*PHP* |PHP-Parser |FFI-only via php-ffi extension +|*Clojure* |`+tools.analyzer+` |JNI-only, no native Java calls +|=== -### 2. Formal Verification Integration +==== 2. Formal Verification Integration -**For proven (Idris2-based):** -```bash +*For proven (Idris2-based):* + +[source,bash] +---- # Check Idris2 code is total and type-checks idris2 --check --total src/Proven/*.idr @@ -99,10 +107,12 @@ idris2 --find-holes src/Proven/*.idr # Check FFI exports match Zig declarations diff <(grep "^export" src/Proven/*.idr | sort) \ <(grep "^pub fn proven_" ffi/zig/src/abi.zig | sort) -``` +---- + +*For language bindings:* -**For language bindings:** -```rust +[source,rust] +---- // Parse Rust AST and verify all public functions call FFI use syn::{parse_file, Item, ItemFn}; @@ -132,12 +142,14 @@ fn verify_ffi_only(rust_file: &str) -> Result<(), String> { } Ok(()) } -``` +---- -### 3. Property-Based Testing +==== 3. Property-Based Testing -**QuickCheck/PropCheck integration:** -```rust +*QuickCheck/PropCheck integration:* + +[source,rust] +---- // Generate random inputs, verify FFI boundary safety #[quickcheck] fn ffi_never_crashes(a: i32, b: i32) -> bool { @@ -158,12 +170,14 @@ fn ffi_handles_edge_cases(input: String) -> bool { // Should return Ok or Err, never panic true // If we get here, didn't crash } -``` +---- + +==== 4. Fuzzing Integration -### 4. Fuzzing Integration +*For FFI boundaries (most critical):* -**For FFI boundaries (most critical):** -```yaml +[source,yaml] +---- # .clusterfuzzlite/proven-ffi-fuzzing.yaml language: rust build: @@ -177,10 +191,12 @@ fuzz_targets: - name: fuzz_idris_zig_bridge max_time: 3600 corpus: fuzz/corpus/zig/ -``` +---- -**Fuzz target example:** -```rust +*Fuzz target example:* + +[source,rust] +---- // fuzz/fuzz_targets/fuzz_ffi_boundary.rs #![no_main] use libfuzzer_sys::fuzz_target; @@ -203,12 +219,14 @@ fuzz_target!(|data: &[u8]| { let _ = proven::safe_url::parse(s); } }); -``` +---- + +==== 5. Build Verification -### 5. Build Verification +*Multi-stage compilation check:* -**Multi-stage compilation check:** -```bash +[source,bash] +---- #!/bin/bash # echidnabot build verification for proven @@ -255,12 +273,14 @@ cargo test --test ffi_integration || { } echo "✅ All stages passed!" -``` +---- -### 6. Dependency Analysis +==== 6. Dependency Analysis -**Supply chain security:** -```rust +*Supply chain security:* + +[source,rust] +---- // Check for suspicious dependencies use cargo_metadata::MetadataCommand; @@ -295,14 +315,15 @@ fn audit_dependencies() -> Result<(), String> { Ok(()) } -``` +---- + +=== Implementation Plan -## Implementation Plan +==== Phase 1: Parser Infrastructure (Week 1) -### Phase 1: Parser Infrastructure (Week 1) +*Create `+echidnabot-parsers/+` crate:* -**Create `echidnabot-parsers/` crate:** -``` +.... echidnabot-parsers/ ├── Cargo.toml ├── src/ @@ -314,10 +335,12 @@ echidnabot-parsers/ │ └── common.rs # Shared AST traits └── tests/ └── fixtures/ # Test code samples -``` +.... + +*Core trait:* -**Core trait:** -```rust +[source,rust] +---- pub trait LanguageParser { fn parse_file(&self, path: &Path) -> Result; fn find_ffi_calls(&self, ast: &AST) -> Vec; @@ -343,12 +366,14 @@ impl LanguageParser for IdrisParser { Ok(()) } } -``` +---- -### Phase 2: Formal Verification Checks (Week 1-2) +==== Phase 2: Formal Verification Checks (Week 1-2) -**Create `echidnabot-verify/` crate:** -```rust +*Create `+echidnabot-verify/+` crate:* + +[source,rust] +---- pub struct FormalVerifier { idris_parser: IdrisParser, binding_parsers: HashMap>, @@ -400,12 +425,14 @@ impl FormalVerifier { Ok(report) } } -``` +---- + +==== Phase 3: Test Theory Integration (Week 2) -### Phase 3: Test Theory Integration (Week 2) +*Property-based testing framework:* -**Property-based testing framework:** -```rust +[source,rust] +---- pub struct PropertyTester { generators: HashMap>, } @@ -445,12 +472,14 @@ impl PropertyTester { tests } } -``` +---- -### Phase 4: CI Integration (Week 2-3) +==== Phase 4: CI Integration (Week 2-3) -**GitHub Actions workflow:** -```yaml +*GitHub Actions workflow:* + +[source,yaml] +---- # .github/workflows/echidnabot-v2.yml name: echidnabot v2 (OCaml++++ Standards) @@ -539,50 +568,57 @@ jobs: run: | cd bindings/${{ matrix.language }}/ ./test.sh -``` +---- -## Success Criteria +=== Success Criteria echidnabot v2 passes when it can detect: -### ✅ What opam caught (proven v0.9.0) -- [x] Native reimplementations (100+ files) -- [x] Unsafe patterns (unwrap, getExn, Obj.magic) -- [x] Architecture violations (logic in bindings) -- [x] Crash-prone code (5 CRITICAL getExn) - -### ✅ What it should catch (higher standards) -- [ ] Incomplete proofs (Idris2 holes) -- [ ] Non-total functions (infinite loops possible) -- [ ] FFI mismatches (Idris exports ≠ Zig bridge) -- [ ] Supply chain vulnerabilities -- [ ] Insufficient test coverage -- [ ] Performance regressions -- [ ] Documentation inaccuracies - -### ✅ Quality bar: opam-repository level -- [ ] All checks pass before `git push` -- [ ] Cannot publish to ANY registry without passing -- [ ] Automated but as thorough as human review -- [ ] Clear, actionable error messages - -## Timeline - -| Week | Deliverable | -|------|-------------| -| 1 | Parser infrastructure (Idris2, Rust, ReScript) | -| 2 | Formal verification checks + property testing | -| 3 | CI integration + remaining language parsers | -| 4 | Production deployment + documentation | - -## Next Actions - -1. ⏳ Create `echidnabot-v2/` directory structure -2. ⏳ Implement Idris2 parser (totality checker) -3. ⏳ Implement Rust AST parser (FFI verification) -4. ⏳ Create property-based test generator -5. ⏳ Integrate with CI/CD - ---- - -**Goal:** Never let unverified code reach package registries again. If echidnabot passes, opam reviewers should have nothing to add. +==== ✅ What opam caught (proven v0.9.0) + +* [x] Native reimplementations (100+ files) +* [x] Unsafe patterns (unwrap, getExn, Obj.magic) +* [x] Architecture violations (logic in bindings) +* [x] Crash-prone code (5 CRITICAL getExn) + +==== ✅ What it should catch (higher standards) + +* [ ] Incomplete proofs (Idris2 holes) +* [ ] Non-total functions (infinite loops possible) +* [ ] FFI mismatches (Idris exports ≠ Zig bridge) +* [ ] Supply chain vulnerabilities +* [ ] Insufficient test coverage +* [ ] Performance regressions +* [ ] Documentation inaccuracies + +==== ✅ Quality bar: opam-repository level + +* [ ] All checks pass before `+git push+` +* [ ] Cannot publish to ANY registry without passing +* [ ] Automated but as thorough as human review +* [ ] Clear, actionable error messages + +=== Timeline + +[cols=",",options="header",] +|=== +|Week |Deliverable +|1 |Parser infrastructure (Idris2, Rust, ReScript) +|2 |Formal verification checks + property testing +|3 |CI integration + remaining language parsers +|4 |Production deployment + documentation +|=== + +=== Next Actions + +[arabic] +. ⏳ Create `+echidnabot-v2/+` directory structure +. ⏳ Implement Idris2 parser (totality checker) +. ⏳ Implement Rust AST parser (FFI verification) +. ⏳ Create property-based test generator +. ⏳ Integrate with CI/CD + +''''' + +*Goal:* Never let unverified code reach package registries again. If +echidnabot passes, opam reviewers should have nothing to add. diff --git a/docs/MODULE_PROPOSALS.adoc b/docs/MODULE_PROPOSALS.adoc new file mode 100644 index 00000000..ad80b5bc --- /dev/null +++ b/docs/MODULE_PROPOSALS.adoc @@ -0,0 +1,157 @@ +== Module Proposals + +This document tracks proposed new modules for the proven library, +primarily arising from integration analysis with downstream projects. + +=== From academic-workflow-suite Integration Analysis (2026-01-16) + +==== SafeJson + +*Status*: Proposed *Priority*: HIGH *Use Case*: Crashproof JSON parsing +and serialization + +*Proposed Operations*: - +`+SafeJson.parse(string) -> Result+` - Guaranteed no +crashes on malformed input - +`+SafeJson.stringify(value) -> Result+` - Safe +serialization - +`+SafeJson.Schema.validate(schema, value) -> Result<(), ValidationError>+` +- Schema compliance + +*Proof Properties*: - Parse never panics/crashes on any input - Memory +bounded by input size × constant factor - UTF-8 encoding always valid in +output - No stack overflow on deeply nested structures + +*Motivation*: Academic-workflow-suite uses JSON for: - AI Jail IPC +communication (untrusted input) - Office Add-in API responses - +Configuration file parsing + +==== SafeRegex + +*Status*: Proposed *Priority*: HIGH *Use Case*: Regular expression +matching without ReDoS vulnerability + +*Proposed Operations*: - +`+SafeRegex.compile(pattern) -> Result+` - +Validated pattern compilation - +`+SafeRegex.match(regex, input) -> Result+` - Guaranteed +polynomial-time matching - +`+SafeRegex.replace(regex, input, replacement) -> string+` - Safe +substitution + +*Proof Properties*: - Matching time bounded by O(n × m) where n = input +length, m = pattern length - No backtracking explosion (ReDoS-resistant) +- Invalid patterns rejected at compile time + +*Motivation*: Academic-workflow-suite uses regex for: - PII detection +patterns (email, student ID) - Input validation - Module/assignment code +parsing + +==== SafeStateMachine + +*Status*: Proposed *Priority*: MEDIUM *Use Case*: Type-safe state +machine with verified transitions + +*Proposed Operations*: - +`+SafeStateMachine.define(states, transitions) -> Machine+` - DSL for +state machine definition - +`+SafeStateMachine.transition(machine, event) -> Result+` +- Safe transitions - +`+SafeStateMachine.canTransition(machine, event) -> Bool+` - Query valid +transitions + +*Proof Properties*: - Only defined transitions are possible - No invalid +states reachable - Deterministic behavior (same input → same output) - +All events handled (no undefined behavior) + +*Motivation*: Academic-workflow-suite uses state machines for: - Event +sourcing (document lifecycle) - TMA marking workflow states - Audit +trail state transitions + +==== SafeAudit + +*Status*: Proposed *Priority*: MEDIUM *Use Case*: Append-only audit log +with integrity verification + +*Proposed Operations*: - +`+SafeAudit.append(log, entry) -> Result+` - Append with +integrity - `+SafeAudit.verify(log) -> Result<(), IntegrityError>+` - +Chain verification - `+SafeAudit.query(log, timeRange) -> List+` +- Time-bounded queries + +*Proof Properties*: - Append-only (no modification of existing entries) +- Chained integrity (tamper detection) - Query consistency (reproducible +results) + +*Motivation*: Academic-workflow-suite requires GDPR-compliant audit +trails for: - Student ID anonymization events - AI analysis +requests/responses - Tutor feedback editing actions + +==== SafeOffice (NEW - Domain-Specific) + +*Status*: Proposed *Priority*: LOW *Use Case*: Safe Office.js operations +for add-ins + +*Proposed Operations*: - +`+SafeOffice.Document.read() -> Result+` - Safe +document access - +`+SafeOffice.Range.insert(position, text) -> Result<(), BoundsError>+` - +Verified text insertion - +`+SafeOffice.Comment.create(range, text) -> Result+` - +Safe comment creation - +`+SafeOffice.Property.get(name) -> Result+` - Safe +property access + +*Proof Properties*: - Operations bounded within document structure - No +invalid range access - Encoding safety for all text operations + +*Motivation*: Office Add-in needs guaranteed safe interaction with Word +documents. + +==== SafePII (NEW - Domain-Specific) + +*Status*: Proposed *Priority*: MEDIUM *Use Case*: PII detection and +anonymization + +*Proposed Operations*: - +`+SafePII.detect(text, patterns) -> List+` - Pattern-based +detection - `+SafePII.anonymize(text, method) -> AnonymizedText+` - +Verified anonymization - `+SafePII.hash(identifier, salt) -> Hash+` - +One-way hashing with timing resistance + +*Proof Properties*: - Detection completeness (no missed patterns) - +Anonymization irreversibility (for hash methods) - Timing-attack +resistance for hashing - Deterministic (same input → same output) + +*Motivation*: Academic-workflow-suite must guarantee student PII never +reaches AI systems in identifiable form. + +=== Implementation Notes + +==== FFI Requirements + +New modules would need bindings for at least: - *Rust* (core engine) - +proven-rust crate - *ReScript/JavaScript* (Office Add-in) - proven-js +package + +==== Proof Strategy + +For each module: 1. Define specification in Idris2 (types capture +invariants) 2. Implement with totality checking enabled 3. Export proofs +as documentation 4. Generate FFI bindings via codegen + +==== Related Work + +* `+SafeString+` already handles HTML/SQL/JS escaping +* `+SafePath+` handles path traversal +* `+SafeCrypto+` handles constant-time comparison +* New modules should integrate with existing ones where applicable + +=== Tracking + +* [ ] SafeJson - RFC drafted +* [ ] SafeRegex - RFC drafted +* [ ] SafeStateMachine - RFC drafted +* [ ] SafeAudit - RFC drafted +* [ ] SafeOffice - Scoping needed +* [ ] SafePII - Depends on SafeRegex, SafeCrypto diff --git a/docs/MODULE_PROPOSALS.md b/docs/MODULE_PROPOSALS.md deleted file mode 100644 index 9976b583..00000000 --- a/docs/MODULE_PROPOSALS.md +++ /dev/null @@ -1,161 +0,0 @@ -# Module Proposals - -This document tracks proposed new modules for the proven library, primarily arising from integration analysis with downstream projects. - -## From academic-workflow-suite Integration Analysis (2026-01-16) - -### SafeJson - -**Status**: Proposed -**Priority**: HIGH -**Use Case**: Crashproof JSON parsing and serialization - -**Proposed Operations**: -- `SafeJson.parse(string) -> Result` - Guaranteed no crashes on malformed input -- `SafeJson.stringify(value) -> Result` - Safe serialization -- `SafeJson.Schema.validate(schema, value) -> Result<(), ValidationError>` - Schema compliance - -**Proof Properties**: -- Parse never panics/crashes on any input -- Memory bounded by input size × constant factor -- UTF-8 encoding always valid in output -- No stack overflow on deeply nested structures - -**Motivation**: Academic-workflow-suite uses JSON for: -- AI Jail IPC communication (untrusted input) -- Office Add-in API responses -- Configuration file parsing - -### SafeRegex - -**Status**: Proposed -**Priority**: HIGH -**Use Case**: Regular expression matching without ReDoS vulnerability - -**Proposed Operations**: -- `SafeRegex.compile(pattern) -> Result` - Validated pattern compilation -- `SafeRegex.match(regex, input) -> Result` - Guaranteed polynomial-time matching -- `SafeRegex.replace(regex, input, replacement) -> string` - Safe substitution - -**Proof Properties**: -- Matching time bounded by O(n × m) where n = input length, m = pattern length -- No backtracking explosion (ReDoS-resistant) -- Invalid patterns rejected at compile time - -**Motivation**: Academic-workflow-suite uses regex for: -- PII detection patterns (email, student ID) -- Input validation -- Module/assignment code parsing - -### SafeStateMachine - -**Status**: Proposed -**Priority**: MEDIUM -**Use Case**: Type-safe state machine with verified transitions - -**Proposed Operations**: -- `SafeStateMachine.define(states, transitions) -> Machine` - DSL for state machine definition -- `SafeStateMachine.transition(machine, event) -> Result` - Safe transitions -- `SafeStateMachine.canTransition(machine, event) -> Bool` - Query valid transitions - -**Proof Properties**: -- Only defined transitions are possible -- No invalid states reachable -- Deterministic behavior (same input → same output) -- All events handled (no undefined behavior) - -**Motivation**: Academic-workflow-suite uses state machines for: -- Event sourcing (document lifecycle) -- TMA marking workflow states -- Audit trail state transitions - -### SafeAudit - -**Status**: Proposed -**Priority**: MEDIUM -**Use Case**: Append-only audit log with integrity verification - -**Proposed Operations**: -- `SafeAudit.append(log, entry) -> Result` - Append with integrity -- `SafeAudit.verify(log) -> Result<(), IntegrityError>` - Chain verification -- `SafeAudit.query(log, timeRange) -> List` - Time-bounded queries - -**Proof Properties**: -- Append-only (no modification of existing entries) -- Chained integrity (tamper detection) -- Query consistency (reproducible results) - -**Motivation**: Academic-workflow-suite requires GDPR-compliant audit trails for: -- Student ID anonymization events -- AI analysis requests/responses -- Tutor feedback editing actions - -### SafeOffice (NEW - Domain-Specific) - -**Status**: Proposed -**Priority**: LOW -**Use Case**: Safe Office.js operations for add-ins - -**Proposed Operations**: -- `SafeOffice.Document.read() -> Result` - Safe document access -- `SafeOffice.Range.insert(position, text) -> Result<(), BoundsError>` - Verified text insertion -- `SafeOffice.Comment.create(range, text) -> Result` - Safe comment creation -- `SafeOffice.Property.get(name) -> Result` - Safe property access - -**Proof Properties**: -- Operations bounded within document structure -- No invalid range access -- Encoding safety for all text operations - -**Motivation**: Office Add-in needs guaranteed safe interaction with Word documents. - -### SafePII (NEW - Domain-Specific) - -**Status**: Proposed -**Priority**: MEDIUM -**Use Case**: PII detection and anonymization - -**Proposed Operations**: -- `SafePII.detect(text, patterns) -> List` - Pattern-based detection -- `SafePII.anonymize(text, method) -> AnonymizedText` - Verified anonymization -- `SafePII.hash(identifier, salt) -> Hash` - One-way hashing with timing resistance - -**Proof Properties**: -- Detection completeness (no missed patterns) -- Anonymization irreversibility (for hash methods) -- Timing-attack resistance for hashing -- Deterministic (same input → same output) - -**Motivation**: Academic-workflow-suite must guarantee student PII never reaches AI systems in identifiable form. - -## Implementation Notes - -### FFI Requirements - -New modules would need bindings for at least: -- **Rust** (core engine) - proven-rust crate -- **ReScript/JavaScript** (Office Add-in) - proven-js package - -### Proof Strategy - -For each module: -1. Define specification in Idris2 (types capture invariants) -2. Implement with totality checking enabled -3. Export proofs as documentation -4. Generate FFI bindings via codegen - -### Related Work - -- `SafeString` already handles HTML/SQL/JS escaping -- `SafePath` handles path traversal -- `SafeCrypto` handles constant-time comparison -- New modules should integrate with existing ones where applicable - -## Tracking - -- [ ] SafeJson - RFC drafted -- [ ] SafeRegex - RFC drafted -- [ ] SafeStateMachine - RFC drafted -- [ ] SafeAudit - RFC drafted -- [ ] SafeOffice - Scoping needed -- [ ] SafePII - Depends on SafeRegex, SafeCrypto diff --git a/docs/PUBLISHING.adoc b/docs/PUBLISHING.adoc new file mode 100644 index 00000000..65064499 --- /dev/null +++ b/docs/PUBLISHING.adoc @@ -0,0 +1,165 @@ +== Publishing Guide + +This document describes how to publish proven to package registries. + +=== Automated Publishing + +Publishing is automated via GitHub Actions. When a version tag is pushed +(e.g., `+v1.0.0+`), the release workflow automatically: + +[arabic] +. Validates version consistency across all packages +. Creates a GitHub Release +. Publishes to all configured registries + +==== Trigger a Release + +[source,bash] +---- +# Tag a release +git tag v1.0.0 +git push origin v1.0.0 +---- + +Or use manual workflow dispatch in GitHub Actions. + +=== Registry Setup + +==== Required GitHub Secrets + +Configure these secrets in your repository settings (Settings → Secrets +→ Actions): + +[width="100%",cols="25%,30%,45%",options="header",] +|=== +|Secret |Registry |How to obtain +|`+CRATES_IO_TOKEN+` |crates.io +|https://crates.io/settings/tokens[crates.io/settings/tokens] + +|`+NPM_TOKEN+` |npm +|https://www.npmjs.com/settings/~/tokens[npmjs.com/settings/tokens] - +create Automation token +|=== + +==== OIDC Authentication (No Tokens Required) + +*JSR* uses OIDC trusted publishing - no token needed; it authenticates +automatically via GitHub Actions OIDC. (The PyPI publish path was +removed on 2026-05-27 along with the Python bindings; see the estate +Python ban in `+hyperpolymath/standards+`.) + +==== JSR OIDC Authentication + +JSR also uses OIDC - no token needed. The workflow has +`+id-token: write+` permission which allows JSR to authenticate +automatically via GitHub Actions. + +=== Package Configuration + +==== crates.io (Rust) + +Location: `+bindings/rust/Cargo.toml+` + +[source,toml] +---- +[package] +name = "proven" +version = "0.9.0" +edition = "2021" +license = "MPL-2.0" +repository = "https://github.com/hyperpolymath/proven" +description = "Formally verified safety library - code that cannot crash" +keywords = ["safety", "verification", "idris", "dependent-types"] +categories = ["no-std", "parsing", "encoding"] +---- + +==== npm (JavaScript/TypeScript) + +Location: `+bindings/javascript/package.json+` and +`+bindings/typescript/package.json+` + +[source,json] +---- +{ + "name": "@proven/javascript", + "version": "0.9.0", + "license": "MPL-2.0" +} +---- + +==== JSR (Deno) + +Location: `+bindings/deno/deno.json+` + +[source,json] +---- +{ + "name": "@hyperpolymath/proven", + "version": "0.9.0", + "exports": "./mod.ts" +} +---- + +=== Version Bumping + +Before releasing, ensure all packages have the same version: + +[source,bash] +---- +# Check current versions +grep '^version' bindings/rust/Cargo.toml +jq '.version' bindings/deno/deno.json +jq '.version' bindings/javascript/package.json +jq '.version' bindings/typescript/package.json +---- + +Update all packages to the new version before tagging. + +=== Dry Run + +Test publishing without actually publishing: + +[arabic] +. Go to GitHub Actions +. Select "`Publish to [registry]`" workflow +. Click "`Run workflow`" +. Enable "`Dry run`" option +. Click "`Run workflow`" + +=== Troubleshooting + +==== crates.io: "`crate already exists`" + +The crate name is taken. Either: - Use a different name - Claim +ownership if you own it + +==== npm: "`You do not have permission`" + +Ensure your npm token has publish permissions and the scope is correct. + +==== JSR: "`Package not found`" + +First-time publish requires creating the package: + +[source,bash] +---- +cd bindings/deno +deno publish --dry-run # Validates +deno publish # Creates and publishes +---- + +=== Manual Publishing + +For manual publishing (not recommended): + +[source,bash] +---- +# Rust +cd bindings/rust && cargo publish + +# npm +cd bindings/javascript && npm publish --access public + +# JSR +cd bindings/deno && deno publish +---- diff --git a/docs/PUBLISHING.md b/docs/PUBLISHING.md deleted file mode 100644 index e1f22dc4..00000000 --- a/docs/PUBLISHING.md +++ /dev/null @@ -1,142 +0,0 @@ -# Publishing Guide - -This document describes how to publish proven to package registries. - -## Automated Publishing - -Publishing is automated via GitHub Actions. When a version tag is pushed (e.g., `v1.0.0`), the release workflow automatically: - -1. Validates version consistency across all packages -2. Creates a GitHub Release -3. Publishes to all configured registries - -### Trigger a Release - -```bash -# Tag a release -git tag v1.0.0 -git push origin v1.0.0 -``` - -Or use manual workflow dispatch in GitHub Actions. - -## Registry Setup - -### Required GitHub Secrets - -Configure these secrets in your repository settings (Settings → Secrets → Actions): - -| Secret | Registry | How to obtain | -|--------|----------|---------------| -| `CRATES_IO_TOKEN` | crates.io | [crates.io/settings/tokens](https://crates.io/settings/tokens) | -| `NPM_TOKEN` | npm | [npmjs.com/settings/tokens](https://www.npmjs.com/settings/~/tokens) - create Automation token | - -### OIDC Authentication (No Tokens Required) - -**JSR** uses OIDC trusted publishing - no token needed; it authenticates automatically via GitHub Actions OIDC. (The PyPI publish path was removed on 2026-05-27 along with the Python bindings; see the estate Python ban in `hyperpolymath/standards`.) - -### JSR OIDC Authentication - -JSR also uses OIDC - no token needed. The workflow has `id-token: write` permission which allows JSR to authenticate automatically via GitHub Actions. - -## Package Configuration - -### crates.io (Rust) - -Location: `bindings/rust/Cargo.toml` - -```toml -[package] -name = "proven" -version = "0.9.0" -edition = "2021" -license = "MPL-2.0" -repository = "https://github.com/hyperpolymath/proven" -description = "Formally verified safety library - code that cannot crash" -keywords = ["safety", "verification", "idris", "dependent-types"] -categories = ["no-std", "parsing", "encoding"] -``` - -### npm (JavaScript/TypeScript) - -Location: `bindings/javascript/package.json` and `bindings/typescript/package.json` - -```json -{ - "name": "@proven/javascript", - "version": "0.9.0", - "license": "MPL-2.0" -} -``` - -### JSR (Deno) - -Location: `bindings/deno/deno.json` - -```json -{ - "name": "@hyperpolymath/proven", - "version": "0.9.0", - "exports": "./mod.ts" -} -``` - -## Version Bumping - -Before releasing, ensure all packages have the same version: - -```bash -# Check current versions -grep '^version' bindings/rust/Cargo.toml -jq '.version' bindings/deno/deno.json -jq '.version' bindings/javascript/package.json -jq '.version' bindings/typescript/package.json -``` - -Update all packages to the new version before tagging. - -## Dry Run - -Test publishing without actually publishing: - -1. Go to GitHub Actions -2. Select "Publish to [registry]" workflow -3. Click "Run workflow" -4. Enable "Dry run" option -5. Click "Run workflow" - -## Troubleshooting - -### crates.io: "crate already exists" - -The crate name is taken. Either: -- Use a different name -- Claim ownership if you own it - -### npm: "You do not have permission" - -Ensure your npm token has publish permissions and the scope is correct. - -### JSR: "Package not found" - -First-time publish requires creating the package: -```bash -cd bindings/deno -deno publish --dry-run # Validates -deno publish # Creates and publishes -``` - -## Manual Publishing - -For manual publishing (not recommended): - -```bash -# Rust -cd bindings/rust && cargo publish - -# npm -cd bindings/javascript && npm publish --access public - -# JSR -cd bindings/deno && deno publish -``` diff --git a/docs/SAFEFORTH-SKETCH.adoc b/docs/SAFEFORTH-SKETCH.adoc new file mode 100644 index 00000000..8b05eb54 --- /dev/null +++ b/docs/SAFEFORTH-SKETCH.adoc @@ -0,0 +1,147 @@ +== SafeForth - Byte-Level Safety Library Sketch + +=== Concept + +Same pattern as proven but for *byte-level safety* using Forth’s +strengths: - Stack discipline with depth proofs - Direct memory/disk +with bounds checking - Minimal runtime (bootloader-viable) + +=== Core Modules + +[width="100%",cols="24%,41%,35%",options="header",] +|=== +|Module |What It Does |Key Safety +|SafeStack |Stack ops (push/pop/dup/swap) |Depth proofs - can’t +underflow + +|SafeMemory |@/!/c@/c! operations |Bounds checking on regions + +|SafeBlock |Sector read/write |Valid sector proofs + +|SafeWord |Word definitions |Stack effect verification + +|SafeDMA |Direct memory access |Region alignment proofs + +|SafePort |I/O port access |Valid port proofs +|=== + +=== Stack Effect Types + +.... +push : Stack n -> Stack (S n) +pop : Stack (S n) -> (Cell, Stack n) -- Requires non-empty! +dup : Stack (S n) -> Stack (S (S n)) -- Requires at least 1 +swap : Stack (S (S n)) -> Stack (S (S n)) -- Requires at least 2 +.... + +=== Memory Safety + +.... +fetch : (region : Region) -> (offset : Nat) -> {offset + 8 <= region.size} -> Cell +store : (region : Region) -> (offset : Nat) -> Cell -> {offset + 8 <= region.size} -> () +.... + +=== Block Safety + +.... +blockRead : (sector : Nat) -> {sector < diskSectors} -> Vect 512 Byte +blockWrite : (sector : Nat) -> Vect 512 Byte -> {sector < diskSectors} -> () +.... + +=== Architecture + +.... +Application (any lang) + │ + Zig FFI Bridge + │ + SafeForth Core (Idris 2 types + Forth impl) + │ + Hardware (disk, memory, ports) +.... + +=== Why Forth? + +[arabic] +. *Minimal* - Fits in boot sector, no runtime bloat +. *Direct* - Maps 1:1 to hardware operations +. *Verifiable* - Stack effects can be proven +. *Proven track record* - OpenFirmware, embedded, spacecraft + +=== Relation to proven + +[cols=",",options="header",] +|=== +|proven |SafeForth +|Type safety |Byte safety +|JSON/URL/Email parsing |Disk/Memory/Port I/O +|High-level validation |Low-level hardware +|=== + +*Together*: proven parses data safely, SafeForth reads/writes it safely. + +=== Similar Libraries to Consider + +[cols=",,",options="header",] +|=== +|Language |Strength |Potential Library +|*Forth* |Byte-level, stack |SafeForth +|*SPARK/Ada* |Contracts, real-time |SafeConcurrent +|*Erlang/OTP* |Fault tolerance |SafeActor +|*Z3/SMT* |Constraint solving |SafeConstraint +|=== + +=== Protocol-Squisher: Semantic Preservation Across FFI + +*Problem*: Language-specific safety properties don’t survive FFI: - Rust +ownership/borrowing → lost at C ABI - Linear Haskell’s linearity → lost +at FFI boundary - Haskell laziness → forced evaluation at FFI + +*Solution*: Encode semantic properties in protocol-squisher transport +classes: + +[cols=",",options="header",] +|=== +|Property |Protocol Encoding +|Ownership |Capability tokens (use-once, invalidates sender) +|Borrowing |Lease tokens (time-bounded, auto-return) +|Linearity |One-shot tickets (consumed on use) +|Laziness |Thunk references (compute-on-demand RPC) +|Affine |At-most-once tokens (may drop, can’t dup) +|=== + +*Extended Transport Classes*: + +.... +SemanticTransport = DataTransport + { + ownership_model : (owned | borrowed | shared | affine | linear), + evaluation : (strict | lazy | memoized), + lifetime : (static | scoped Token | dynamic), + capability : CapabilityToken +} +.... + +*How it works*: 1. Caller encodes semantic intent in protocol message 2. +Transport layer validates semantic rules (can’t dup linear value) 3. +Receiver reconstructs equivalent semantics in its language 4. Protocol +enforces what languages can’t across FFI + +*Example - Linear value across FFI*: + +.... +send(value, { linear: true, ticket: "abc123" }) +# Protocol rejects: send(value, { linear: true, ticket: "abc123" }) -- already used! +.... + +This means SafeForth (or any safe library) can preserve: - Memory region +ownership (not just bounds) - Handle linearity (file handles used +exactly once) - Lazy disk reads (only fetch sectors when accessed) + +=== Next Steps + +[arabic] +. Pick Forth base (gforth or minimal custom) +. Define stack effect DSL +. Build Zig FFI (reuse proven’s pattern) +. Create bindings for key languages +. Integrate protocol-squisher for semantic preservation diff --git a/docs/SAFEFORTH-SKETCH.md b/docs/SAFEFORTH-SKETCH.md deleted file mode 100644 index 35a7e4b7..00000000 --- a/docs/SAFEFORTH-SKETCH.md +++ /dev/null @@ -1,132 +0,0 @@ -# SafeForth - Byte-Level Safety Library Sketch - -## Concept - -Same pattern as proven but for **byte-level safety** using Forth's strengths: -- Stack discipline with depth proofs -- Direct memory/disk with bounds checking -- Minimal runtime (bootloader-viable) - -## Core Modules - -| Module | What It Does | Key Safety | -|--------|--------------|------------| -| SafeStack | Stack ops (push/pop/dup/swap) | Depth proofs - can't underflow | -| SafeMemory | @/!/c@/c! operations | Bounds checking on regions | -| SafeBlock | Sector read/write | Valid sector proofs | -| SafeWord | Word definitions | Stack effect verification | -| SafeDMA | Direct memory access | Region alignment proofs | -| SafePort | I/O port access | Valid port proofs | - -## Stack Effect Types - -``` -push : Stack n -> Stack (S n) -pop : Stack (S n) -> (Cell, Stack n) -- Requires non-empty! -dup : Stack (S n) -> Stack (S (S n)) -- Requires at least 1 -swap : Stack (S (S n)) -> Stack (S (S n)) -- Requires at least 2 -``` - -## Memory Safety - -``` -fetch : (region : Region) -> (offset : Nat) -> {offset + 8 <= region.size} -> Cell -store : (region : Region) -> (offset : Nat) -> Cell -> {offset + 8 <= region.size} -> () -``` - -## Block Safety - -``` -blockRead : (sector : Nat) -> {sector < diskSectors} -> Vect 512 Byte -blockWrite : (sector : Nat) -> Vect 512 Byte -> {sector < diskSectors} -> () -``` - -## Architecture - -``` -Application (any lang) - │ - Zig FFI Bridge - │ - SafeForth Core (Idris 2 types + Forth impl) - │ - Hardware (disk, memory, ports) -``` - -## Why Forth? - -1. **Minimal** - Fits in boot sector, no runtime bloat -2. **Direct** - Maps 1:1 to hardware operations -3. **Verifiable** - Stack effects can be proven -4. **Proven track record** - OpenFirmware, embedded, spacecraft - -## Relation to proven - -| proven | SafeForth | -|--------|-----------| -| Type safety | Byte safety | -| JSON/URL/Email parsing | Disk/Memory/Port I/O | -| High-level validation | Low-level hardware | - -**Together**: proven parses data safely, SafeForth reads/writes it safely. - -## Similar Libraries to Consider - -| Language | Strength | Potential Library | -|----------|----------|-------------------| -| **Forth** | Byte-level, stack | SafeForth | -| **SPARK/Ada** | Contracts, real-time | SafeConcurrent | -| **Erlang/OTP** | Fault tolerance | SafeActor | -| **Z3/SMT** | Constraint solving | SafeConstraint | - -## Protocol-Squisher: Semantic Preservation Across FFI - -**Problem**: Language-specific safety properties don't survive FFI: -- Rust ownership/borrowing → lost at C ABI -- Linear Haskell's linearity → lost at FFI boundary -- Haskell laziness → forced evaluation at FFI - -**Solution**: Encode semantic properties in protocol-squisher transport classes: - -| Property | Protocol Encoding | -|----------|-------------------| -| Ownership | Capability tokens (use-once, invalidates sender) | -| Borrowing | Lease tokens (time-bounded, auto-return) | -| Linearity | One-shot tickets (consumed on use) | -| Laziness | Thunk references (compute-on-demand RPC) | -| Affine | At-most-once tokens (may drop, can't dup) | - -**Extended Transport Classes**: -``` -SemanticTransport = DataTransport + { - ownership_model : (owned | borrowed | shared | affine | linear), - evaluation : (strict | lazy | memoized), - lifetime : (static | scoped Token | dynamic), - capability : CapabilityToken -} -``` - -**How it works**: -1. Caller encodes semantic intent in protocol message -2. Transport layer validates semantic rules (can't dup linear value) -3. Receiver reconstructs equivalent semantics in its language -4. Protocol enforces what languages can't across FFI - -**Example - Linear value across FFI**: -``` -send(value, { linear: true, ticket: "abc123" }) -# Protocol rejects: send(value, { linear: true, ticket: "abc123" }) -- already used! -``` - -This means SafeForth (or any safe library) can preserve: -- Memory region ownership (not just bounds) -- Handle linearity (file handles used exactly once) -- Lazy disk reads (only fetch sectors when accessed) - -## Next Steps - -1. Pick Forth base (gforth or minimal custom) -2. Define stack effect DSL -3. Build Zig FFI (reuse proven's pattern) -4. Create bindings for key languages -5. Integrate protocol-squisher for semantic preservation diff --git a/docs/SECURITY.adoc b/docs/SECURITY.adoc new file mode 100644 index 00000000..d4bbd255 --- /dev/null +++ b/docs/SECURITY.adoc @@ -0,0 +1,291 @@ +== Security Audit + +This document describes the security properties of proven and the formal +verification approach used. + +=== Executive Summary + +*proven* is a formally verified safety library where security properties +are mathematically proven at compile time using dependent types in Idris +2. This provides stronger guarantees than traditional testing +approaches. + +IMPORTANT: The *unbreakable guarantee applies only to Idris2 modules*. +The Zig layer must be a *pure ABI bridge* with no safety logic. Any +non‑Idris logic in bindings is considered non‑proven and is being +removed. + +==== Verification Status + +[cols=",,,,",options="header",] +|=== +|Module |Formal Proofs |Property Tests |Unit Tests |Fuzz Tests +|SafeMath |✅ |✅ |✅ |✅ +|SafeString |✅ |✅ |✅ |✅ +|SafeJson |✅ |✅ |✅ |✅ +|SafeUrl |✅ |✅ |✅ |✅ +|SafeEmail |✅ |✅ |✅ |✅ +|SafePath |✅ |✅ |✅ |✅ +|SafeCrypto |✅ |✅ |✅ |✅ +|SafePassword |✅ |✅ |✅ |✅ +|SafeDateTime |✅ |✅ |✅ |✅ +|SafeNetwork |✅ |✅ |✅ |✅ +|SafeRegex |✅ |✅ |✅ |✅ +|SafeHtml |✅ |✅ |✅ |✅ +|SafeCommand |✅ |✅ |✅ |✅ +|SafeSQL |✅ |✅ |✅ |✅ +|SafeJWT |✅ |✅ |✅ |✅ +|SafeBase64 |✅ |✅ |✅ |✅ +|SafeXML |✅ |✅ |✅ |✅ +|SafeYAML |✅ |✅ |✅ |✅ +|SafeTOML |✅ |✅ |✅ |✅ +|SafeUUID |✅ |✅ |✅ |✅ +|SafeCurrency |✅ |✅ |✅ |✅ +|SafePhone |✅ |✅ |✅ |✅ +|SafeHex |✅ |✅ |✅ |✅ +|SafeEnv |✅ |✅ |✅ |✅ +|SafeArgs |✅ |✅ |✅ |✅ +|SafeFile |✅ |✅ |✅ |✅ +|SafeHeader |✅ |✅ |✅ |✅ +|SafeCookie |✅ |✅ |✅ |✅ +|SafeContentType |✅ |✅ |✅ |✅ +|=== + +''''' + +=== Formal Verification Approach + +==== Dependent Types + +proven uses Idris 2’s dependent type system to encode security +properties directly in types. The compiler then proves these properties +hold for all possible inputs. + +*Example: Division by Zero Prevention* + +[source,idris] +---- +-- The type system ensures divisor is never zero +safeDiv : (dividend : Integer) -> (divisor : Integer) -> + Result DivisionError Integer +safeDiv dividend 0 = Err DivisionByZero +safeDiv dividend divisor = Ok (dividend `div` divisor) + +-- Proof that division by zero always returns an error +prop_divByZeroFails : (x : Integer) -> isErr (safeDiv x 0) = True +prop_divByZeroFails x = Refl -- Proven by type checking +---- + +==== Totality Checking + +All functions are marked `+%default total+`, meaning the Idris 2 +compiler verifies: 1. *Termination*: Functions always complete (no +infinite loops) 2. *Coverage*: All input cases are handled (no runtime +exceptions) + +==== Property Proofs + +Each module includes formal proofs (in `+Proofs.idr+` files): + +[width="100%",cols="41%,35%,24%",options="header",] +|=== +|Property Type |Description |Example +|Correctness |Operations produce correct results +|`+escapeHtml "<" = "<"+` + +|Safety |Dangerous operations are prevented +|`+containsTraversal ".." = True+` + +|Roundtrip |Encode/decode preserves data |`+decode(encode(x)) = x+` + +|Bounds |Values stay within valid ranges |`+0 <= port <= 65535+` +|=== + +''''' + +=== Security Properties by Module + +==== SafeSQL - SQL Injection Prevention + +*Proven Properties:* - ✅ All user input is parameterized, never +concatenated - ✅ Identifiers are validated against injection patterns - +✅ String escaping handles all SQL metacharacters - ✅ UNION, comment, +and OR attacks are detected + +*Detection Patterns:* + +.... +UNION SELECT, ' OR 1=1, '; DROP TABLE, /* comment */, -- comment +.... + +==== SafeHtml - XSS Prevention + +*Proven Properties:* - ✅ All special characters are escaped: +`+< > & " '+` - ✅ URL schemes are validated (blocks `+javascript:+`, +`+data:+`) - ✅ Event handler attributes are blocked (`+onclick+`, +`+onerror+`, etc.) - ✅ Blacklisted tags are removed: `+