docs(weave): Document autopatch_settings for PII nb

docs/docs/reference/gen_notebooks/pii.md

@@ -15,22 +15,32 @@ title: Handling and Redacting PII
 
 <!--- @wandbcode{cod-notebook} -->
 
-# How to use Weave with PII data:
+# How to use Weave with PII data
 
-In this tutorial, we'll demonstrate how to utilize Weave while ensuring your Personally Identifiable Information (PII) data remains private. Weave supports removing PII from LLM calls and preventing PII from being displayed in the Weave UI. 
+In this guide, you'll learn how to use W&B Weave while ensuring your Personally Identifiable Information (PII) data remains private. The guide demonstrates the following methods to identify, redact and anonymize PII data:
 
-To detect and protect our PII data, we'll identify and redact PII data and optionally anonymize it with the following methods:
 1. __Regular expressions__ to identify PII data and redact it.
 2. __Microsoft's [Presidio](https://microsoft.github.io/presidio/)__, a python-based data protection SDK. This tool provides redaction and replacement functionalities.
 3. __[Faker](https://faker.readthedocs.io/en/master/)__, a Python library to generate fake data, combined with Presidio to anonymize PII data.
 
-Additionally, we'll make use of _Weave Ops input/output logging customization_ to seamlessly integrate PII redaction and anonymization into the workflow. See [here](https://weave-docs.wandb.ai/guides/tracking/ops/#customize-logged-inputs-and-outputs) for more information.
+Additionally, you'll learn how to use _`weave.op` input/output logging customization_ and _`autopatch_settings`_ to integrate PII redaction and anonymization into the workflow. For more information, see [Customize logged inputs and outputs](https://weave-docs.wandb.ai/guides/tracking/ops/#customize-logged-inputs-and-outputs).
 
-For this use-case, we will leverage Anthropic's Claude Sonnet to perform sentiment analysis while tracing the LLM calls using Weave's [Traces](https://wandb.github.io/weave/quickstart). Sonnet will receive a block of text and output one of the following sentiment classifications: _positive_, _negative_, or _neutral_.
+To get started, do the following:
 
-## Overview of Weave Ops Input/Output Logging Customization
+1. Review the [Overview](#overview) section.
+2. Complete the [prerequisites](#prerequisites).
+3. Review the [available methods](#redaction-methods-overview) for identifying, redacting and anonymizing PII data.
+4. [Apply the methods to Weave calls](#apply-the-methods-to-weave-calls).
 
-Weave Ops support defining input and output postprocessing functions. These functions allow you to modify the data that is passed to your LLM call or logged to Weave, respectively.
+## Overview 
+
+The following section provides an overview of input and output logging using `weave.op`, as well as best practices for working with PII data in Weave.
+
+### Customize input and output logging using `weave.op`
+
+Weave Ops allow you to define input and output postprocessing functions. Using these functions, you can modify the data that is passed to your LLM call or logged to Weave.
+
+In the following example, two postprocessing functions are defined and passed as arguments to `weave.op()`.
 
 ```python
 from dataclasses import dataclass
@@ -60,14 +70,33 @@ def some_llm_call(a: int, hide_me: str) -> CustomObject:
     return CustomObject(x=a, secret_password=hide_me)
 ```
 
-# Setup
+### Best practices for using Weave with PII data 
+
+Before using Weave with PII data, review the best practices for using Weave with PII data.
+
+#### During testing
+- Log anonymized data to check PII detection
+- Track PII handling processes with Weave Traces
+- Measure anonymization performance without exposing real PII
+
+#### In production
+- Never log raw PII
+- Encrypt sensitive fields before logging
+
+#### Encryption tips
+- Use reversible encryption for data you need to decrypt later
+- Apply one-way hashing for unique IDs you don't need to reverse
+- Consider specialized encryption for data you need to analyze while encrypted
+
+## Prerequisites
 
-Let's install the required packages and set up our API keys. Your Weights & Biases API key can be found [here](https://wandb.ai/authorize), and your Anthropic API keys are [here](https://console.anthropic.com/settings/keys).
+1. First, install the required packages. 
 
 
 ```python
 %%capture
 # @title required python packages:
+!pip install cryptography 
 !pip install presidio_analyzer
 !pip install presidio_anonymizer
 !python -m spacy download en_core_web_lg    # Presidio uses spacy NLP engine
@@ -78,6 +107,11 @@ Let's install the required packages and set up our API keys. Your Weights & Bias
 !pip install cryptography                   # to encrypt our data
 ```
 
+2. Set up your API keys. You can find your API keys at the following links.
+
+   - [W&B](https://wandb.ai/authorize)
+   - [Anthropic](https://console.anthropic.com/settings/keys).
+
 
 ```python
 %%capture
@@ -90,6 +124,8 @@ _ = set_env("ANTHROPIC_API_KEY")
 _ = set_env("WANDB_API_KEY")
 ```
 
+3. Initialize your Weave project.
+
 
 ```python
 import weave
@@ -99,7 +135,7 @@ WEAVE_PROJECT = "pii_cookbook"
 weave.init(WEAVE_PROJECT)
 ```
 
-Let's load our initial PII data. For demonstration purposes, we'll use a dataset containing 10 text blocks. A larger dataset with 1000 entries is available.
+4. Load the demo PII dataset, which contains 10 text blocks. 
 
 
 ```python
@@ -112,11 +148,20 @@ pii_data = response.json()
 print('PII data first sample: "' + pii_data[0]["text"] + '"')
 ```
 
-# Redaction Methods Implementation
+## Redaction methods overview
 
-## Method 1: Regular Expression Filtering
+Once you've completed the [setup](#setup), you can 
 
-Our initial method is to use [regular expressions (regex)](https://docs.python.org/3/library/re.html) to identify PII data and redact it. It allows us to define patterns that can match various formats of sensitive information like phone numbers, email addresses, and social security numbers. By using regex, we can scan through large volumes of text and replace or redact information without the need for more complex NLP techniques. 
+To detect and protect our PII data, we'll identify and redact PII data and optionally anonymize it using the following methods:
+
+1. __Regular expressions__ to identify PII data and redact it.
+2. __Microsoft [Presidio](https://microsoft.github.io/presidio/)__, a Python-based data protection SDK that provides redaction and replacement functionality.
+3. __[Faker](https://faker.readthedocs.io/en/master/)__, a Python library for generating fake data.
+
+
+### Method 1: Filter using regular expressions
+
+[Regular expressions (regex)](https://docs.python.org/3/library/re.html) are the simplest method to identify and redact PII data. Regex allows you to define patterns that can match various formats of sensitive information like phone numbers, email addresses, and social security numbers. Using regex, you can scan through large volumes of text and replace or redact information without the need for more complex NLP techniques. 
 
 
 ```python
@@ -181,15 +226,10 @@ print(f"Raw text:\n\t{test_text}")
 print(f"Redacted text:\n\t{cleaned_text}")
 ```
 
-## Method 2: Microsoft Presidio Redaction
-Our next method involves complete removal of PII data using Presidio. This approach redacts PII and replaces it with a placeholder representing the PII type. 
-
-For example:
-`"My name is Alex"` becomes `"My name is <PERSON>"`.
+### Method 2: Redact using Microsoft Presidio 
+The next method involves complete removal of PII data using [Microsoft Presidio](https://microsoft.github.io/presidio/). Presidio redacts PII and replaces it with a placeholder representing the PII type. For example, Presidio replaces `Alex` in `"My name is Alex"` with `<PERSON>`.
 
-Presidio comes with a built-in [list of recognizable entities](https://microsoft.github.io/presidio/supported_entities/). We can select the ones that are important for our use case. In the below example, we redact names, phone numbers, locations, email addresses, and US Social Security Numbers.
-
-We'll then encapsulate the Presidio process into a function.
+Presidio comes with a built-in support for [common entities](https://microsoft.github.io/presidio/supported_entities/). In the below example, we redact all entities that are a `PHONE_NUMBER`, `PERSON`, `LOCATION`, `EMAIL_ADDRESS` or `US_SSN`. The Presidio process is encapsulated in a function.
 
 
 ```python
@@ -229,17 +269,17 @@ print(f"Raw text:\n\t{text}")
 print(f"Redacted text:\n\t{anonymized_text}")
 ```
 
-## Method 3: Anonymization with Replacement using Fakr and Presidio
+### Method 3: Anonymize with replacement using Faker and Presidio
 
-Instead of redacting text, we can anonymize it by swapping PII (like names and phone numbers) with fake data generated using the [Faker](https://faker.readthedocs.io/en/master/) Python library. For example:
+Instead of redacting text, you can anonymize it by using MS Presidio to swap PII like names and phone numbers with fake data generated using the [Faker](https://faker.readthedocs.io/en/master/) Python library. For example, suppose you have the following data:
 
 `"My name is Raphael and I like to fish. My phone number is 212-555-5555"` 
 
-might become
+Once the data has been processed using Presidio and Faker, it might look like:
 
 `"My name is Katherine Dixon and I like to fish. My phone number is 667.431.7379"`
 
-To effectively utilize Presidio, we must supply references to our custom operators. These operators will direct Presidio to the functions responsible for swapping PII with fake data.
+To effectively use Presidio and Faker together, we must supply references to our custom operators. These operators will direct Presidio to the Faker functions responsible for swapping PII with fake data.
 
 
 ```python
@@ -285,7 +325,7 @@ print(f"Raw text:\n\t{text_to_anonymize}")
 print(f"Anonymized text:\n\t{anonymized_results.text}")
 ```
 
-Let's consolidate our code into a single class and expand the list of entities to include the additional ones we identified earlier.
+Let's consolidate our code into a single class and expand the list of entities to include the additional ones identified earlier.
 
 
 ```python
@@ -348,19 +388,55 @@ print(f"Raw text:\n\t{text_to_anonymize}")
 print(f"Anonymized text:\n\t{anonymized_text}")
 ```
 
-# Applying the Methods to Weave Calls
+### Method 4: Use `autopatch_settings` 
+
+You can use `autopatch_settings` to configure PII handling directly during initialization for one or more of the supported LLM integrations. The advantages of this method are:
 
-In these examples we will integrate our PII redaction and anonymization methods into Weave Models, and preview the results in Weave Traces.
+1. PII handling logic is centralized and scoped at initialization, reducing the need for scattered custom logic.
+2. PII processing workflows can be customized or disabled entirely for specific intergations.
 
-We'll create a [Weave Model](https://wandb.github.io/weave/guides/core-types/models) which is a combination of data (which can include configuration, trained model weights, or other information) and code that defines how the model operates. 
+To use `autopatch_settings` to configure PII handling, define `postprocess_inputs` and/or `postprocess_output` in `op_settings` for any one of the supported LLM integrations. 
 
-In this model, we will include our predict function where the Anthropic API will be called. Additionally, we will include our postprocessing functions to ensure that our PII data is redacted or anonymized before it is sent to the LLM.
+```python 
+
+def postprocess(inputs: dict) -> dict:
+    if "SENSITIVE_KEY" in inputs:
+        inputs["SENSITIVE_KEY"] = "REDACTED"
+    return inputs
+
+client = weave.init(
+    ...,
+    autopatch_settings={
+        "openai": {
+            "op_settings": {
+                "postprocess_inputs": postprocess,
+                "postprocess_output": ...,
+            }
+        },
+        "anthropic": {
+            "op_settings": {
+                "postprocess_inputs": ...,
+                "postprocess_output": ...,
+            }
+        }
+    },
+)
+```
 
-Once you run this code you will receive a links to the Weave project page as well as the specific trace (LLM calls)you ran.
 
-## Regex Method 
+## Apply the methods to Weave calls
 
-In the simplest case, we can use regex to identify and redact PII data in the original text.
+In the following examples, we will integrate our PII redaction and anonymization methods into Weave Models and preview the results in Weave Traces.
+
+First, we'll create a [Weave Model](https://wandb.github.io/weave/guides/core-types/models). A Weave Model is a combination of information like configuration settings, model weights, and code that defines how the model operates. 
+
+In our model, we will include our predict function where the Anthropic API will be called. Anthropic's Claude Sonnet is used to perform sentiment analysis while tracing LLM calls using [Traces](https://wandb.github.io/weave/quickstart). Claude Sonnet will receive a block of text and output one of the following sentiment classifications: _positive_, _negative_, or _neutral_. Additionally, we will include our postprocessing functions to ensure that our PII data is redacted or anonymized before it is sent to the LLM.
+
+Once you run this code, you will receive a links to the Weave project page, as well as the specific trace (LLM calls) you ran.
+
+### Regex method 
+
+In the simplest case, we can use regex to identify and redact PII data from the original text.
 
 
 ```python
@@ -420,9 +496,9 @@ for entry in pii_data:
     await model.predict(entry["text"])
 ```
 
-## Presidio Redaction Method
+### Presidio redaction method
 
-Here we will use Presidio to identify and redact PII data in the original text.
+Next, we will use Presidio to identify and redact PII data from the original text.
 
 ![](../../media/pii/redact.png)
 
@@ -484,9 +560,9 @@ for entry in pii_data:
     await model.predict(entry["text"])
 ```
 
-## Faker + Presidio Replacement Method
+### Faker and Presidio replacement method
 
-Here we will have Faker generate anonymized replacement PII data and use Presidio to identify and replace the PII data in the original text.
+In this example, we use Faker to generate anonymized replacement PII data and use Presidio to identify and replace the PII data in the original text.
 
 
 ![](../../media/pii/replace.png)
@@ -551,27 +627,81 @@ for entry in pii_data:
     await model.predict(entry["text"])
 ```
 
-## Checklist for Safely Using Weave with PII Data
+### `autopatch_settings` method
 
-### During Testing
-- Log anonymized data to check PII detection
-- Track PII handling processes with Weave Traces
-- Measure anonymization performance without exposing real PII
+In the following example, we set `postprocess_inputs` for `anthropic` to the `postprocess_inputs_regex()` function () at initialization. The `postprocess_inputs_regex` function applies the`redact_with_regex` method defined in [Method 1: Regular Expression Filtering](#method-1-regular-expression-filtering). Now, `redact_with_regex` will be applied to all inputs to any `anthropic` models.
 
-### In Production
-- Never log raw PII
-- Encrypt sensitive fields before logging
 
-### Encryption Tips
-- Use reversible encryption for data you need to decrypt later
-- Apply one-way hashing for unique IDs you don't need to reverse
-- Consider specialized encryption for data you need to analyze while encrypted
+```python
+import json
+from typing import Any
+
+import anthropic
+
+import weave
+
+client = weave.init(
+    ...,
+    autopatch_settings={
+        "anthropic": {
+            "op_settings": {
+                "postprocess_inputs": postprocess_inputs_regex,
+            }
+        }
+    },
+)
+
+
+# Define an input postprocessing function that applies our regex redaction for the model prediction Weave Op
+def postprocess_inputs_regex(inputs: dict[str, Any]) -> dict:
+    inputs["text_block"] = redact_with_regex(inputs["text_block"])
+    return inputs
+
+
+# Weave model / predict function
+class sentiment_analysis_regex_pii_model(weave.Model):
+    model_name: str
+    system_prompt: str
+    temperature: int
+
+    async def predict(self, text_block: str) -> dict:
+        client = anthropic.AsyncAnthropic()
+        response = await client.messages.create(
+            max_tokens=1024,
+            model=self.model_name,
+            system=self.system_prompt,
+            messages=[
+                {"role": "user", "content": [{"type": "text", "text": text_block}]}
+            ],
+        )
+        result = response.content[0].text
+        if result is None:
+            raise ValueError("No response from model")
+        parsed = json.loads(result)
+        return parsed
+```
+
+
+```python
+# create our LLM model with a system prompt
+model = sentiment_analysis_regex_pii_model(
+    name="claude-3-sonnet",
+    model_name="claude-3-5-sonnet-20240620",
+    system_prompt='You are a Sentiment Analysis classifier. You will be classifying text based on their sentiment. Your input will be a block of text. You will answer with one the following rating option["positive", "negative", "neutral"]. Your answer should be one word in json format: {classification}. Ensure that it is valid JSON.',
+    temperature=0,
+)
+
+print("Model: ", model)
+# for every block of text, anonymized first and then predict
+for entry in pii_data:
+    await model.predict(entry["text"])
+```
+
+### (Optional) Encrypt your data 
 
-<details>
-<summary> (Optional) Encrypting our data </summary>
 ![](../../media/pii/encrypt.png)
 
-In addition to anonymizing PII, we can add an extra layer of security by encrypting our data using the cryptography library's [Fernet](https://cryptography.io/en/latest/fernet/) symmetric encryption. This approach ensures that even if the anonymized data is intercepted, it remains unreadable without the encryption key.
+In addition to anonymizing PII, you can add an extra layer of security by encrypting your data using the cryptography library's [Fernet](https://cryptography.io/en/latest/fernet/) symmetric encryption. This approach ensures that even if the anonymized data is intercepted, it remains unreadable without the encryption key.
 
 ```python
 import os
@@ -666,4 +796,3 @@ for entry in pii_data:
     encrypted_input = EncryptedSentimentAnalysisInput.encrypt(entry["text"])
     await model.predict(encrypted_input)
 ```
-</details>