This article explores using HCP Vault to anonymize sensitive data in retrieval augmented generation (RAG) workflows with IBM Granite and Ollama. It addresses the risk of large language models (LLMs) leaking personal identifiable information (PII) by employing Vault’s transform secrets engine for data masking and tokenization. A demo illustrates masking credit card numbers and tokenizing billing addresses for vacation rental bookings, ensuring safe data handling in a local test environment using Open WebUI. By Rosemary Wang.
Main points discussed:
- RAG and PII Risks: RAG enhances LLM output but risks exposing sensitive data like PII, a top concern in OWASP 2025 risks for LLMs.
- HCP Vault Solution: Vault’s transform secrets engine masks and tokenizes data to prevent leaks.
- Demo Setup: Uses Terraform to configure Vault, Python scripts for data generation, and Docker for local LLM testing with Ollama and Open WebUI.
- Data Protection: Masking hides credit card details (non-reversible), while tokenization with convergent encryption allows address analysis without revealing plaintext.
- Controlled Access: Authorized agents can decode tokenized data via Vault, ensuring security.
By masking or tokenizing sensitive data before augmenting a LLM with RAG, you can protect access to the data and prevent leakage of sensitive information. In this demo, an LLM in testing and other applications by default do not require access to sensitive information like credit card information or billing street address. They can still analyze and provide other information without leaking payment information. Good read!
[Read More]