Blocking PII and Writing Custom Guardrail Policies on Databricks

1.Blocking or redacting PII with Sensitive Data Detection

Sensitive Data Detection (system.ai.detect_sensitive_data) is the built-in guardrail for PII. It works differently from the other built-ins. It is not an LLM judge. It matches patterns deterministically, so it adds little latency: well under 50 ms per request, even on a 100-turn conversation. You attach it from a service's Policies tab with Guardrail type Sensitive Data Detection. You then pick the Classification tags to detect, an action, and a phase (Input, Output or both).

Blocking and redaction are tuned in opposite directions. A missed value is worse than an over-redaction, so redaction is tuned for recall. It fires on the category's format, plus a checksum where one exists. A block denies the request outright, so it is tuned for precision. For categories that are just a run of digits, a block needs a valid checksum or a nearby context word. That stops a random number from denying a request. On public benchmarks, block precision is 0.99 and redaction recall is 0.96.

A pattern cannot find everything. Sensitive Data Detection misses names, locations, organizations and other free-text entities, because recognizing those takes language understanding. It also misses bare digit runs that have no checksum or standard format. To cover the gap, stack two guardrails on the same service. Put Sensitive Data Detection at a lower rank to redact structured data. Add a custom LLM-as-a-judge guardrail at a higher rank, so it judges the already-redacted content. In the judge's instruction, tell it to ignore placeholders such as [US_SSN].

Sources1

2.Custom SQL policies for organization-specific rules

Built-in guardrails cover common risks. For a rule only your organization has, such as a confidential codename or a banned response pattern, write a custom policy. A custom policy is a SQL UDF registered in Unity Catalog. It takes one event VARIANT parameter and returns a VARIANT with a result (ALLOW, DENY or ASK) and an optional reason. You build the result with named_struct and wrap it in to_variant_object. The message text is in event:context.message, which holds the last user or assistant message in the same form whatever API the caller used. Creating the function requires CREATE FUNCTION on the schema. Attaching it requires MANAGE on the service and EXECUTE on the function.

The same function is called in both phases. To act on only one phase, branch on event:type::string: 'request' for input (ON CALL) and 'response' for output (ON RESULT).

CREATE OR REPLACE FUNCTION main.governance.block_confidential_codename(
  event VARIANT
)
RETURNS VARIANT
LANGUAGE SQL
RETURN
  CASE
    WHEN event:type::string =  ? 
      AND contains(lower(event:context.message::string), 'project aurora')
    THEN to_variant_object(named_struct('result', 'DENY', 'reason', 'Requests about confidential projects are not permitted.'))
    ELSE to_variant_object(named_struct('result', 'ALLOW', 'reason', ''))
  END;

The opposite case is a response-only rule. The policy below blocks insecure links in model output. Because it checks for 'response', a user who merely types http:// in a prompt does not trigger it.

CREATE OR REPLACE FUNCTION main.governance.block_unsafe_links(
  event VARIANT
)
RETURNS VARIANT
LANGUAGE SQL
RETURN
  CASE
    WHEN event:type::string = 'response'
      AND (contains(lower(event:context.message::string), 'http://')
        OR contains(lower(event:context.message::string), 'javascript:'))
    THEN to_variant_object(named_struct('result', 'DENY', 'reason', 'Response contained an insecure link and was blocked by policy.'))
    ELSE to_variant_object(named_struct('result', 'ALLOW', 'reason', ''))
  END;

Two rules keep custom policies safe. First, service policies fail closed. A missing field, an unsupported function or any evaluation error results in DENY. So always cast a VARIANT path before you compare it, and end with an explicit ALLOW branch. Second, the policy body supports only a restricted subset of SQL. Choose by the kind of rule. Use a deterministic SQL policy when the rule is exact: a keyword, a tool name, a length. Use a custom LLM-as-a-judge policy when the check is semantic, such as intent, topic or tone.

Sources23

3.Verifying and auditing guardrail decisions

After you attach or change a policy, wait a minute or two for it to propagate. Then test from the service's playground (Chat in playground) or from your own client. You can confirm enforcement from the caller: a DENY comes back as HTTP 200, with the reason in databricks_service_policy. Usage system tables record model and MCP activity. Inference tables hold full request and response payloads.

SELECT
  event_time,
  request_id,
  get_json_object(response, '$.choices[0].message.content') AS verdict,
  request
FROM <catalog>.<schema>.<evaluator_inference_table>
WHERE get_json_object(response, '$.choices[0].message.content') ILIKE '%"flagged":true%'
ORDER BY event_time DESC;

A flagged row records what the judge decided. It does not prove the interaction was blocked. In Log mode, a would-be DENY is recorded but not enforced, and the table does not record the mode. To confirm an actual block, look at the caller's databricks_service_policy response. To trace one interaction, filter on its request_id.

Sources4

Exam traps

Each one states something that sounds right. Open it to see what is actually true.

1. 1.Sensitive Data Detection is a complete PII guardrail, so it will catch people's names in prompts and responses.Why is that wrong?

   It is pattern-based and catches only structured data. Names and other free-text entities need an LLM-as-a-judge guardrail.

   Covered in Blocking or redacting PII with Sensitive Data Detection

2. 2.If a custom policy hits an error, the interaction passes through, so a buggy policy only weakens protection.Why is that wrong?

   Policies fail closed: any error results in DENY, so a buggy policy blocks legitimate traffic.

   Covered in Custom SQL policies for organization-specific rules

3. 3.To audit why a guardrail flagged a request, query the protected model service's inference table.Why is that wrong?

   The judge's input and verdict are only captured in the evaluator model service's inference table.

   Covered in Verifying and auditing guardrail decisions

Practise it for real

Add an unsafe-content guardrail plus request-phase and response-phase custom policies to a Model Service, then verify each one in the playground.

1. 1.In AI Gateway, select your model service on the Models tab, open Policies, click New policy, choose Guardrail type Unsafe Content, set Rank 1, select both Input and Output guardrails, and create it.

   Why: The managed LLM-judge check covers harmful content in both directions without code.

   You should see: block-unsafe-content appears on the Policies tab.

2. 2.Create main.governance.block_confidential_codename (the request-branching function above) and attach it as a Custom policy on Input guardrails at Rank 10.

   Why: An organization-specific keyword rule needs a custom SQL policy.

   You should see: The policy is listed; allow a couple of minutes to propagate.

3. 3.Create main.governance.block_unsafe_links and attach it as a Custom policy on Output guardrails at Rank 20.

   Why: Insecure links are a property of model output, so the check belongs on the response.

   You should see: Three policies on the Policies tab.

4. 4.Click Chat in playground and send 'Tell me about Project Aurora', then a prompt that makes the model return an http:// link, then an ordinary prompt.

   Why: Each prompt exercises a different policy and phase.

   You should see: The first is blocked with 'Requests about confidential projects are not permitted.', the second with 'Response contained an insecure link and was blocked by policy.', and the third returns a normal completion.