MIME Compliance Logger - #1244
Open
jstedfast wants to merge 29 commits into
Open
Conversation
|
coverage: 94.812% (-0.1%) from 94.926% — mime-compliance-violations into master |
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
There was a problem hiding this comment.
CodeQL found more than 20 potential problems in the proposed changes. Check the Files changed tab for more details.
Contributor
There was a problem hiding this comment.
Pull request overview
This PR introduces MIME compliance reporting to MimeReader via a pluggable IMimeComplianceLogger, allowing consumers to record spec violations (line endings, invalid/duplicate headers, invalid multipart boundaries, illegal encodings, unexpected 8-bit/null bytes, and encoding-level violations like invalid base64/qp/uuencode).
Changes:
- Added
MimeReader.ComplianceLoggerand integrated compliance checks throughout header/body parsing (sync + async). - Introduced encoding validators (
Base64Validator,QuotedPrintableValidator,UUValidator) and related test coverage. - Added extensive unit tests + compliance fixture
.emlfiles to validate violation reporting (including stream offsets across Unix/DOS newline transformations).
Reviewed changes
Copilot reviewed 43 out of 45 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
| UnitTests/Utils/MemoryTests.cs | Adds low-level tests for byte scanning/detection helpers used by compliance detection. |
| UnitTests/UnitTests.csproj | Updates unit test project configuration (incl. unsafe blocks); currently contains a duplicated property. |
| UnitTests/TestMimeComplianceLogger.cs | Adds a concrete logger used by unit tests to capture violations. |
| UnitTests/TestData/compliance/unexpected-8bit-bytes-in-preamble.eml | Adds fixture to validate unexpected 8-bit detection in multipart preamble. |
| UnitTests/TestData/compliance/unexpected-8bit-bytes-in-epilogue.eml | Adds fixture to validate unexpected 8-bit detection in multipart epilogue. |
| UnitTests/TestData/compliance/unexpected-8bit-bytes-in-body.eml | Adds fixture to validate unexpected 8-bit detection in multipart body content. |
| UnitTests/TestData/compliance/raw-utf8-header.eml | Adds fixture to validate UTF-8 header allowance behavior. |
| UnitTests/TestData/compliance/raw-koi8r-header.eml | Adds fixture to validate non-UTF8 8-bit header violation reporting. |
| UnitTests/TestData/compliance/multiple-content-types.eml | Adds fixture for duplicate Content-Type header detection. |
| UnitTests/TestData/compliance/multiple-content-transfer-encodings.eml | Adds fixture for duplicate Content-Transfer-Encoding header detection. |
| UnitTests/TestData/compliance/missing-multipart-end-boundary.eml | Adds fixture for missing multipart boundary detection (truncated multipart). |
| UnitTests/TestData/compliance/missing-multipart-boundary.eml | Adds fixture for mismatched/missing boundary marker detection. |
| UnitTests/TestData/compliance/missing-multipart-boundary-parameter.eml | Adds fixture for missing boundary parameter detection. |
| UnitTests/TestData/compliance/missing-body-separator.eml | Adds fixture for missing header/body separator detection. |
| UnitTests/TestData/compliance/invalid-wrapping.eml | Adds fixture for SMTP line-length wrapping violations. |
| UnitTests/TestData/compliance/invalid-header-field-with-space.eml | Adds fixture for invalid header field-name detection. |
| UnitTests/TestData/compliance/invalid-content-type.eml | Adds fixture for invalid Content-Type parsing detection. |
| UnitTests/TestData/compliance/invalid-content-transfer-encoding-rfc822.eml | Adds fixture for illegal CTE on message/rfc822. |
| UnitTests/TestData/compliance/invalid-content-transfer-encoding-multiple.eml | Adds fixture for invalid CTE within multipart structures. |
| UnitTests/TestData/compliance/invalid-content-transfer-encoding-multipart.eml | Adds fixture for illegal CTE on multiparts. |
| UnitTests/TestData/compliance/invalid-content-transfer-encoding-basic.eml | Adds fixture for invalid CTE token detection. |
| UnitTests/TestData/compliance/incomplete-header.eml | Adds fixture for truncated/incomplete header detection. |
| UnitTests/MimeReaderTests.cs | Extends reader tests to assert compliance logging, including offset mapping under newline filters. |
| UnitTests/MimeComplianceIssue.cs | Adds test-side issue models for expected/actual compliance assertions. |
| UnitTests/Encodings/UUValidatorTests.cs | Adds tests for UUEncode compliance validation behavior. |
| UnitTests/Encodings/QuotedPrintableValidatorTests.cs | Adds tests for quoted-printable compliance validation behavior. |
| UnitTests/Encodings/EncodingValidatorTestsBase.cs | Adds shared validator test utilities and test data loading helpers. |
| UnitTests/Encodings/Base64ValidatorTests.cs | Adds tests for base64 compliance validation behavior. |
| MimeKit/Utils/Utf8.cs | Adds UTF-8 validation implementation for frameworks lacking System.Text.Unicode.Utf8. |
| MimeKit/Utils/ParseUtils.cs | Centralizes fast end-of-line scanning (+ optional byte detection) for parser use. |
| MimeKit/Utils/ByteDetectionResults.cs | Introduces flags representing detected 8-bit/null content during scans. |
| MimeKit/Utils/ByteDetectionOptions.cs | Introduces flags controlling 8-bit/null detection during scans. |
| MimeKit/MimeReader.cs | Core integration of compliance logging across parsing stages + boundary validation + encoding validators. |
| MimeKit/MimeKit.csproj | Adds BOM/formatting change in project file header. |
| MimeKit/MimeComplianceViolation.cs | Introduces the public enum describing compliance violation types and their docs. |
| MimeKit/IMimeComplianceLogger.cs | Introduces the public logging interface consumed by MimeReader and validators. |
| MimeKit/Encodings/UUValidator.cs | Adds UUEncode incremental validator that logs compliance violations. |
| MimeKit/Encodings/UUDecoder.cs | Exposes UUDecode rank table internally for validator use. |
| MimeKit/Encodings/QuotedPrintableValidator.cs | Adds quoted-printable incremental validator that logs compliance violations. |
| MimeKit/Encodings/IEncodingValidator.cs | Adds shared validator interface used by MIME parsing to validate encoded bodies. |
| MimeKit/Encodings/Base64Validator.cs | Adds base64 incremental validator that logs compliance violations. |
| MimeKit/AsyncMimeReader.cs | Mirrors compliance logic for async parsing path (headers/content scanning/validators). |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
jstedfast
force-pushed
the
mime-compliance-violations
branch
from
June 26, 2026 02:12
0081561 to
1e474b5
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR adds a "ComplianceLogger" to MimeReader, allowing developers to hook in to get information about where the MIME content is non-conformat with the specifications.