openai

/

privacy-filter

like 1.46k

Follow

OpenAI 35.6k

Hi,

First of all, huge thanks for releasing the privacy-filter model! It is fantastic to see more focus and open-source contributions dedicated to PII detection and high-throughput data sanitization.

While exploring privacy-filter, we found it highly efficient for generic text sanitization. However, when applying privacy filters to Edge-Cloud collaborative AI Agents with long-term memory, we noticed that relying on a relatively coarse 8-category label system and traditional masking strategies introduces a few challenges:

1. Coarse Granularity & Semantic Loss: Grouping distinct high-risk assets (e.g., API keys, passwords, DB strings) into a broad [SECRET] or [ACCOUNT_NUMBER] tag can severely disrupt the semantic context. This often misleads cloud agents when they try to perform tool use or logical reasoning.
2. Language & Conversational Context: In realistic multi-turn dialogues, privacy is often revealed implicitly rather than explicitly. Existing models (and evaluation datasets) sometimes struggle with complex conversational contexts and non-English texts (like Chinese), leading to under-detection (leakage) or over-masking (utility loss).

To address the critical privacy-utility tradeoff specifically for Agent Memory scenarios, our team (MemTensor) independently developed and just open-sourced MemPrivacy this week (May 9). We’re excited to share it with the community here!

🌟 What makes MemPrivacy different?

• Strict 4-Level Privacy Taxonomy: Instead of flat categories, we classify data into PL1 (Preferences) to PL4 (Critical Secrets/Credentials). This fine-grained, hierarchical approach allows users to configure protection thresholds freely.
• Semantic-Typed Placeholders: Instead of replacing sensitive text with generic *** or broad tags, MemPrivacy replaces them with precise semantic placeholders (e.g., <Health_Info_1>). This preserves the structural semantics for the cloud agent while keeping the raw data securely on the edge device.
• Strong Bilingual & Contextual Support: Trained on our newly MemPrivacy-Bench, which features rich, realistic user-agent interactions (both explicit and implicit privacy exposures in English and Chinese).

📊 Experimental Highlights

1. State-of-the-Art Privacy Extraction Performance
Compared to general LLMs and specialized filters, MemPrivacy achieves significantly higher accuracy in realistic dialogues. To rigorously test its generalization, we evaluated the model not only on MemPrivacy-Bench (reaching an F1 score of 85.97) but also on a challenging Out-of-Distribution (OOD) dataset (PersonaMem-v2). Across both in-distribution and OOD evaluations, MemPrivacy demonstrates a substantial lead, effectively mitigating the limitations of coarse-grained categorization and proving its robust extraction capabilities in unseen conversational scenarios.

2. Zero Utility Loss for Cloud Memory Systems
The biggest pain point of data masking is making the Agent "dumb". We evaluated MemPrivacy across popular memory frameworks (LangMem, Mem0, Memobase). Due to our typed placeholders and local restoration architecture, MemPrivacy maintains the cloud agent's reasoning capabilities, keeping utility loss to under 1.6% (and below 0.89% if only PL4 is masked), vastly outperforming traditional irreversible masking.

We believe MemPrivacy perfectly complements high-throughput models like privacy-filter by offering a highly tailored solution for the Edge-Cloud Agent architecture.

For anyone interested in Agent privacy, feel free to check out our work:

• GitHub: https://github.com/MemTensor/MemPrivacy
• Models: https://huggingface.co/collections/IAAR-Shanghai/memprivacy

Would love to hear your thoughts and feedback! Keep up the great work! 🛡️🤖