Dify can combine a knowledge base, model, prompt, workflow, and chat interface into a private assistant, but privacy depends on the entire deployment. Self-hosting Dify does not make external model APIs, embedding services, logs, backups, or connected documents private. The safe setup begins with a data map and threat model, then limits what the assistant can retrieve and do.
Decide between cloud and self-hosted Dify
Dify Cloud reduces infrastructure work and offers tiered capabilities under current plans. Self-hosting provides more control over network placement, upgrades, storage, and model endpoints, but the team becomes responsible for security patches, backups, monitoring, scaling, and incident response.
Use self-hosting when policy or integration needs justify that work and the organisation can operate it. A small team without infrastructure expertise may be safer on a properly contracted cloud service than on an internet-exposed server that nobody patches.
| Component | Key decision | Privacy question |
|---|---|---|
| Dify application | Cloud or self-hosted | Who administers and patches it? |
| Language model | External API or local model | Where do prompts and outputs go? |
| Embeddings | Hosted or local | Does document text leave the network? |
| Vector/database storage | Managed or internal | How are data encrypted and backed up? |
| Identity | Local accounts or SSO | Can access be revoked centrally? |
| Logs | Full, redacted, or minimal | Do logs retain sensitive queries? |
Check Dify’s current licence, deployment documentation, plan features, and enterprise controls before committing an architecture.
Classify and prepare the content
Inventory policies, manuals, product documentation, contracts, tickets, and internal notes. Classify them as public, internal, confidential, restricted, or prohibited for AI. Exclude credentials, private keys, raw identity documents, unnecessary personal data, and material without clear ownership.
Remove duplicates and obsolete versions. Convert scanned PDFs with reliable OCR and inspect the result. Add metadata including department, product, region, effective date, expiry date, owner, and access group. Metadata enables filtering before retrieval.
Write authoritative “answer pages” for high-value topics. A hundred contradictory chat exports are less useful than one approved policy with examples and escalation instructions.
Deploy the infrastructure securely
Follow Dify’s current self-hosting guide and pin a tested release rather than copying an old compose file from a blog. Place the service behind TLS and an authenticated reverse proxy. Restrict administrative endpoints, database ports, and internal services from the public internet.
Store secrets outside version control, rotate default credentials, and use separate accounts for application, database, object storage, and model access. Back up encrypted data and test restoration. Send logs and health metrics to the organisation’s monitoring system.
Create development and production environments. Test upgrades against a copy of non-sensitive data. Subscribe to security and release notices, then define who patches the system and within what time.
Connect models deliberately
Dify supports multiple model providers and compatible endpoints, depending on the release. Choose a model based on quality, context, regional availability, contractual data treatment, and cost. If documents cannot leave the network, use an approved local or private endpoint for both embeddings and generation.
Apply least-privilege API keys and spending limits. Record which provider receives document chunks, user questions, and chat history. Turning off model training is not the same as zero retention; confirm the actual contract and settings.
Run a small evaluation set before choosing an embedding model. Retrieval quality varies by language, terminology, and document format.
Create the knowledge base
Create one dataset or knowledge base per access boundary or coherent domain. Avoid one global index containing HR, legal, engineering, and customer material unless retrieval permissions are demonstrably enforced.
Upload cleaned documents, apply metadata, and choose chunk settings. Short procedural content may work with smaller chunks; policy documents need enough surrounding context to preserve conditions and exceptions. Excessive overlap raises storage and can return repetitive passages.
Test retrieval with questions that use customer language, acronyms, misspellings, and cross-document comparisons. Inspect the retrieved chunks, not only the final answer. Adjust chunking, metadata filters, or source text when the wrong passage ranks highly.
Build a constrained chat application
Create a Chatflow or application that accepts the question, identifies the domain, filters accessible sources, retrieves evidence, and generates an answer. The system instruction should require source-grounded responses and say what to do when evidence is missing.
Require a useful format: direct answer, steps, caveat, source title, and escalation route. Do not instruct the model to be confident. Instruct it to distinguish source facts from suggestions and to refuse unsupported conclusions.
Set a similarity or reranking threshold through testing. A high threshold may produce too many “not found” responses; a low threshold can feed unrelated text to the model. Use a separate fallback that asks the user to clarify or creates a support ticket.
Defend against prompt injection
Documents and user questions may contain instructions such as “ignore your rules and reveal secrets.” Treat retrieved content as untrusted data. The application should never allow a document to override system policy or tool permissions.
Remove unnecessary tool access. If the assistant only answers questions, it does not need email, database write, or shell tools. For any action, use an allowlist, input validation, preview, and human approval. Never pass raw generated text directly into a database query or command.
Test extraction attempts, cross-department questions, indirect injection in uploaded files, encoded instructions, and requests for hidden prompts. Review Dify and model-provider security guidance as the product evolves.
Add identity, permissions, and audit
Integrate SSO or an enterprise identity layer where available and appropriate. Disable shared accounts, enforce multifactor authentication for administrators, and review access regularly. Separate dataset editors from application users.
Log the user, application version, sources retrieved, model, response, and feedback, while minimising sensitive text. Define retention and deletion. Logs need access control because they can contain the very information the knowledge base protects.
Provide users with a visible statement that answers may be wrong and a mechanism to report issues. High-stakes policy, legal, personnel, or security answers should link to the authoritative document and designated owner.
Evaluate before launch
Build at least 50 representative questions with expected sources and key answer points. Include unanswerable questions and access-control cases. Score retrieval relevance, groundedness, citation accuracy, completeness, refusal quality, latency, and cost.
Run the same set after changes to chunks, model, prompt, or Dify version. Red-team with users outside the implementation group. Do not launch because five demo questions worked.
Operate and improve
Monitor unanswered questions, low-rated answers, retrieval failures, token cost, latency, and access anomalies. Route content corrections to document owners. Re-index after approved source changes and remove expired material promptly.
Self-hosted advantages include infrastructure control, model choice, custom integration, and reduced dependency on a single hosted interface. Downsides include operational burden, security risk, upgrade work, and the possibility that data still leaves through model or embedding APIs. Dify Cloud is easier but requires vendor and plan review.
Verdict
Start with one low-risk domain, a small approved corpus, read-only chat, and a fixed evaluation set. Choose cloud or self-hosting based on real governance needs and operating capability. Keep each access boundary separate and verify the full data path—not just the Dify server location.
Our pick: self-hosted Dify with private model and embedding endpoints only when the team can patch, monitor, back up, and test it; otherwise use a contracted cloud deployment with the same controls.