Legal Contract Clause Extraction with DistilRoBERTa

📋 Overview

An AI-powered system that automatically extracts 6 critical clause types from legal contracts using fine-tuned DistilRoBERTa. This project demonstrates transfer learning applied to legal document processing, achieving 67% accuracy on date extraction while identifying key performance patterns.

Target Clauses:

• Governing Law
• Effective Date
• Expiration Date
• Anti-Assignment
• Cap on Liability
• License Grant

🎯 Results

Metric	Score
Overall Accuracy	50%
Date Extraction Accuracy	67%
"No Answer" Detection	96%+
Training Time	50 minutes (T4 GPU)

Key Finding: Strong inverse correlation (r=-0.89) between answer length and extraction accuracy.

🚀 Quick Start

Prerequisites

pip install transformers torch datasets accelerate scikit-learn

Model Access

Download the fine-tuned model:

• HuggingFace Link (~330MB)

Usage

from transformers import AutoTokenizer, AutoModelForQuestionAnswering

# Load model
model = AutoModelForQuestionAnswering.from_pretrained("./models/fine_tuned_distilroberta_50pct")
tokenizer = AutoTokenizer.from_pretrained("./models/fine_tuned_distilroberta_50pct")

# Extract clauses (use extract_clauses() function from notebook)
results = extract_clauses(contract_text)

📁 Project Structure

• contract_extraction.ipynb - Complete implementation
• Technical_Report.pdf - Detailed analysis
• Video_Walkthrough - Demo and explanation

🔬 Methodology

1. Dataset: CUAD (Contract Understanding Atticus Dataset) - 510 commercial contracts
2. Model: DistilRoBERTa-base (82M parameters)
3. Training: 50% of data, 2 epochs, batch size 16
4. Split: 178 train / 76 val / 77 test contracts (by contract ID to prevent leakage)

📊 Performance Analysis

Accuracy by Clause Type

Clause Type	Accuracy	Avg Length
Effective Date	66.7%	21 chars
Expiration Date	11.1%	206 chars
Governing Law	12.5%	188 chars
Anti-Assignment	12.5%	357 chars
License Grant	25.0%	292 chars
Cap on Liability	25.0%	262 chars

Insight: Model excels at short answers but struggles with long, complex legal clauses.

💡 Key Learnings

1. Answer length is critical: Performance degrades significantly for clauses >200 characters
2. Preprocessing matters: Fixed token position mapping bug that caused NaN loss
3. Efficient training: Achieved competitive results with 50% data and lightweight model
4. Hyperparameter testing: Evaluated 3 configurations (max_length, batch_size, data percentage)

🔧 Technical Highlights

• Debugged NaN validation loss by fixing answer position mapping
• Systematic error analysis identifying 4 failure categories
• Baseline comparison demonstrating clear improvement
• Production deployment with functional inference pipeline

🎯 Business Impact

Potential Value:

• 99.7% reduction in initial contract screening time
• Estimated savings: $8K-15K/month for mid-size law firms
• ROI: Model development cost ~$50, immediate deployment value

Use Cases:

• Contract metadata extraction
• Pre-screening for manual review
• Due diligence acceleration

⚠️ Limitations

• Performance degrades with answer length >200 characters
• Trained only on commercial contracts (limited domain)
• 384 token context window insufficient for long clauses
• Requires GPU for practical inference speed

🔮 Future Improvements

1. Increase context window to 512-768 tokens
2. Train on 100% of CUAD dataset
3. Evaluate RoBERTa-large or Legal-BERT
4. Implement hierarchical extraction approach
5. Add explainability features

📚 References

• CUAD Dataset: TheAtticusProject/cuad
• DistilRoBERTa: Hugging Face Model
• Paper: Hendrycks et al. (2021) - CUAD: An Expert-Annotated NLP Dataset for Legal Contract Review

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.

👤 Author

Raghu Ram

• GitHub: @raghuneu
• LinkedIn: Your Profile
• Email: shantharajamani.r@northeastern.edu

🙏 Acknowledgments

• CUAD dataset creators at The Atticus Project
• Hugging Face for the Transformers library
• Northeastern University Course: Masters in Information Systems

---

⭐ If you find this project useful, please consider giving it a star!

---

Key Improvements Made:

1. ✅ Added badges for professionalism
2. ✅ Better formatting with emojis and tables
3. ✅ Clear project structure
4. ✅ Methodology section
5. ✅ Performance breakdown table
6. ✅ Business impact quantification
7. ✅ Limitations and future work
8. ✅ References and acknowledgments
9. ✅ Author information
10. ✅ Better code formatting