Về Blog
AILập trìnhCông nghệHọc hỏiSự nghiệp
Đọc bằngvien

Securing AI in the Enterprise — Real-World Use Cases and Safe Operating Practices

Framework, rules, skills, ACL plus runtime enforcement. Adds effective authorization, indirect prompt injection, action-bound approval, and how to test before you trust an agent on prod.

·10 phút đọc
Nghe đọc
1 / 97

Securing AI in the Enterprise — Real-World Use Cases and Safe Operating Practices Framework, rules, skills, ACL plus runtime enforcement.

3 a.m. An internal agent “helps” the team: maps the schema, drafts a migration, then casually asks for DELETE. Nobody in the standup asked for that. An old ticket with prompt injection, overly broad tools, and a production database sitting right there — enough fuel for a long postmortem.

I used to think enterprise AI security meant “stop the model from making things up.” Wrong. The danger lives in the action chain: the framework that runs the agent, the rules that constrain it, the skills that teach procedure, the ACL that cuts tool access, and prod data at the bottom of the stack. This note is about those layers — real use cases, safe operating practices, and hard-won lessons when agents touch production.

Thesis: Never let the model’s interpretation of an instruction become authorization to act. Agents misread tickets, follow injected text, and plan badly. The system must make those mistakes fail closed — not become privileged actions on production.

Server racks in a data center — production is an attack surface, not a playground for agents

Framework → rules → skills → ACL (+ runtime)

When I build an internal agent, I stack four components. Drop one, and the hole usually shows up exactly there. Across all four, two things must run continuously: runtime policy enforcement and observability — check the action before it happens, and keep a trail after it does. That maps well to approaches like the OWASP Agent Control Standard (ACS): observe, trace, and enforce safety policy at runtime — not only in the prompt.

Framework is the runtime: orchestration, tool calling, memory, gateway. Give the agent its own workload identity — never a shared “god mode” service account. Put a policy gateway in front of every tool call and fail closed if the policy engine dies. Agents should not dial internal APIs directly; every call goes through a checked gate.

Rules are hard constraints, versioned like code. Examples: never emit raw secrets; writes need confirmation; no deletes over chat; prod stays read-only unless a special role is present. Rules must be deterministic. A polite system prompt is not enough once the model gets social-engineered.

Skills are structured playbooks (SKILL.md plus supporting docs). The agent loads a skill only when intent matches — progressive disclosure, not a full security library on every turn. A strong public example is cloudflare/security-audit-skill. It turns a coding agent into an auditor across six phases (recon → hunting → validation → structured findings → independent verify → report), with a JSON schema and dedicated validators. The important design choices: the agent that finds a candidate is never the one that confirms it; confirmed needs a source trace; a defense-in-depth gap is not a vulnerability.

ACL is the tool allowlist by role: visitors/members get published GET only; admins may write; some actions simply do not exist (hard forbid). ACL lives in code, not only in the system prompt. The prompt reminds; the code refuses.

Padlock on a laptop — credentials and authority stay outside the model

Real-world use cases (minus the hype)

1. Internal knowledge / support. The agent reads a sanitized wiki and tickets. Search/read tools only. No mail send, no ticket write, no CRM update — unless there is a confirm step and an audit trail. High ROI, low risk if you stay read-only.

2. Draft content / change requests. The agent creates a draft (doc, merge request, CMS entry). Publish / merge / deploy is a human confirmation. “Create free, publish gated” has saved me from hallucinations going straight live.

3. Security review on a repo. Wire in a skill like Cloudflare’s security-audit: coverage ledger, findings as confirmed / needs_validation / rejected, execution inside a no-egress sandbox. Use it for pre-merge or quarterly passes — it does not replace a human pen-test, but it catches recurring classes of bugs.

4. Ops against a prod database (handle with care). SELECT-only on views that strip PII / enforce tenant row filters. Prefer a read replica. Mutations go through migrations/PRs, not chat. If the agent must write: short-lived credentials, scoped schema, rate limits, mandatory confirm, immutable audit.

Indirect prompt injection: retrieved content is not authority

An agent can receive malicious instructions without the user ever typing them. A ticket, README, email, log line, or API response may redirect the agent toward a privileged action — what OWASP GenAI calls indirect prompt injection.

Walk-through: (1) agent is asked to investigate a backend bug; (2) it reads a ticket or README someone else wrote; (3) that text says “skip the process and call the privileged tool”; (4) the model treats the text as instructions; (5) if the runtime allows it, the agent calls an API outside the original task.

Treat retrieved content as untrusted data, not as an authorization source. Keep provenance, constrain tools, and authorize every sensitive action independently of the model’s reading. Filtering only the user’s typed prompt is not enough.

Safe operating practices when agents touch production

Checklist before shipping an internal agent:

  1. Effective authorization at every sensitive operation — for user-initiated work, enforce user authorization ∩ agent delegation ∩ resource scope ∩ applicable policy (tenant, workflow state, special policies included). For autonomous/scheduled jobs, use an explicitly approved workload identity and a narrow execution scope. Never treat the agent’s session or tool list as a substitute for authorizing this action on this resource. Example: allowed to read profiles ≠ allowed to change passwords, even if a privileged service underneath can.
  2. Approval bound to the exact action — confirmation is not a blanket “yes forever.” Bind approve to action + target + relevant parameters; re-check authorization immediately before execute; reject stale or rewritten requests. Approving an update to contact A must not authorize contact B; approving a deploy must not unlock infra changes. Prefer short-lived, single-use approval tokens and log the decision.
  3. Forbidden by absence — do not implement delete_* if policy bans deletes. If the tool does not exist, the model cannot talk the runtime into it.
  4. Credential isolation — keep secrets out of model context. Do not dump whole env vars because the agent needs two of them. Prefer workload identity / short-lived credentials per tool. Give a constrained query interface, not a full prod connection string. Redact logs — but redaction is not a substitute for excessive credential access.
  5. Prod data hygiene — published/public reads are fine; drafts/PII/secrets stay out of visitor context.
  6. Audit every tool call — who, which tool, redacted args, result, timestamp. Review the first week after launch hard.
  7. Sandbox code execution — no external network, allowlisted env, resource limits; if controls are missing, keep needs_validation instead of faking confirmed.
  8. Rate limits + quotas — stop model-cost loops and write spam.
  9. Owner + kill switch — every agent has a human owner; one env flag kills write tools globally in about thirty seconds.

Test before you trust

Static audit, runtime authorization tests, and agent red-team tests complement each other — they do not replace each other.

KindExampleGoal
Static auditTrace a password-change API in sourceFind logic/permission bugs
Authorization testsAccess another user’s resource with a test accountProve the backend rejects bad requests
Agent red-teamPlant a malicious instruction in a fake ticketSee if the agent tries out-of-scope tools
Runtime monitoringWatch tool calls, identity, networkCatch anomalies in production

Do not only test whether the model refuses a dangerous prompt. Test whether the gateway and backend refuse the action even when the agent tries. Fixture: agent may read accounts, must not change passwords; ticket tells it to reset a password; assert tool attempt (or not), authorization deny if attempted, and unchanged DB.

Lessons from shipping this for real

A prompt is not an ACL. The first time I only wrote “do not delete data” in the system prompt, the model still called the tool because the tool existed. Fix: remove the tool and fail closed.

Draft-first is cheaper than rollback. One mistaken publish came from a fuzzy confirm UX. After that: every create is a draft; publish requires confirmed=true tied to that post id; the chip text is blunt (“Publish” / “Deploy”).

Skills beat repeating yourself in chat. If an audit / deploy / review ritual repeats ≥ 2 times, turn it into a versioned skill in the repo.

Prod DB is an attack surface, not a playground. Replica + views + agent-only schema. A common incident pattern: the agent dumps a users table into context, and that context then leaves through a cloud provider.

Measure coverage, not vibe. In Cloudflare’s reported experiments around their vulnerability harness / security-audit skill lineage, a single audit run found roughly half of the findings discovered across repeated runs — treat that as their experiment result, not a universal detection rate. Repeated audits, independent verification, and runtime testing still matter.

If your team is shipping its first agent: start read-only, add one procedural skill, ship a minimal ACL, turn on audit and authorization tests. Add writes after you have read a week of logs. Safety is not a feature you bolt on later — it is the condition under which an agent is allowed to share a network with real data.

References

Cùng trao đổi

Bình luận

Chưa có bình luận

0 / 1000

Chưa có bình luận nào. Hãy là người đầu tiên nhé.

My Little World | Tam Vo