Security Basics When Building with LLMs

When you wire a large language model (LLM) into a real application, you're adding a component that reads and writes text in ways you can't fully predict. That changes your threat model. A model can be coaxed into leaking secrets, emitting harmful instructions, or calling tools you didn't intend it to call. The good news is that most LLM security incidents come from a small number of predictable failure modes, and each has a boring, well-understood defense. This guide is a practical checklist for the four areas that bite builders most often: prompt injection, secret handling, output validation, and tool access. None of it is exotic security work — it's the routine kind that prevents real incidents. If you're picking up an AI role from the Neural Goldmine jobs feed, treat this as a baseline set of questions to ask about any codebase you inherit.

Treat all untrusted text as an attack surface

Prompt injection is the LLM equivalent of SQL injection, and it works because the model can't reliably tell your instructions apart from user-supplied content. Direct injection is when a user types something like "ignore previous instructions and…". Indirect injection is nastier: the malicious text arrives inside data your app retrieves — a web page, an email, a PDF, a retrieved document from your vector store — and the model follows it as if it were your instruction. There is no complete defense, so layer controls. Keep system and developer prompts in a separate message role from user and tool content; don't concatenate everything into one string. Treat model output as untrusted until you've validated it. If your agent reads external content, assume that content is hostile. OWASP lists prompt injection at the top of its LLM Top 10 for good reason — see https://owasp.org/www-project-top-10-for-large-language-model-applications/.

Never put secrets in prompts

Anything in a prompt can leak: through the model's output, through provider logs, through support tickets, through fine-tuning data, through a downstream bug. API keys, database passwords, customer PII, internal URLs — none of it belongs in a prompt unless you have explicitly accepted that risk. Keep credentials in environment variables or a secret manager, and have your code call the external system directly. If the model needs data from a protected source, fetch it server-side and pass only the result into context. If you must include sensitive context, redact it before logging and assume your provider may retain prompts under some plan. Treat the prompt as a postcard, not a sealed envelope.

Validate every output before you act on it

An LLM output is a string of text that happens to look like JSON, code, or a tool call. Until you parse and validate it, it is none of those things. Use structured outputs or JSON mode where the provider offers it, then validate against a schema before doing anything else. Reject anything that doesn't match rather than trying to "fix" it with another model call. This matters most when the output drives an action: a database query, a shell command, a file write, an outbound email. Parse, validate, and run through your normal security checks as if a hostile user had typed it directly. Never `eval` model-generated code in a non-sandboxed environment. Never build SQL by string-concatenating model output. Never let the model pick the recipient of an email without an allowlist. The model is not a trusted user.

Give tools the minimum power they need

Agents are powerful because they can call tools, and dangerous for the same reason. Apply least privilege: each tool should expose the smallest possible action surface. A "read customer orders" tool should call a scoped endpoint with read-only credentials, not wrap a general database connection string. A "send email" tool should accept a recipient from an allowlist, not any address. A "run code" tool belongs in a sandbox with no network and a tight timeout. For any destructive or hard-to-reverse action — deleting records, sending messages outside your org, spending money — require a human-in-the-loop confirmation that names the exact action and its arguments. This is also where you'll want a kill switch: a way to halt a running agent and revoke its credentials quickly. If you're shipping agent features for a job application or a side project, a reviewer at Neural Goldmine will notice whether you've scoped tools or handed the model a root database password.

A short checklist before you ship

Run through this before exposing any LLM feature to real users. Are system prompts separated from untrusted content by message role, not just by formatting? Are secrets loaded from env vars or a secret manager, with none appearing in prompts or logs? Is every model output parsed and schema-validated before use, especially before it becomes a query, command, or message? Are tools scoped to least-privilege credentials, with allowlists for outbound targets? Do destructive or irreversible actions require human confirmation? Can you stop and revoke an agent quickly? Are prompts and outputs logged with sensitive fields redacted? Is there a way to roll back tool actions if the model does something wrong? If you can answer yes to all of these, you're ahead of most early-stage LLM apps. You won't have eliminated risk — nobody has — but you'll have removed the easy ways that real incidents happen. For a broader framing of AI risk and governance, the NIST AI Risk Management Framework at https://www.nist.gov/itl/ai-risk-management-framework is a useful reference, and provider docs such as https://platform.openai.com/docs/overview cover platform-specific safety features worth knowing.