Dify.ai, a self-hosted prompt-management tool

[ekevu123]

For all developers who work with large language models and prompts, there are graphical user interfaces that can make life easier.

Dify.ai is especially good and, that's the best part, it can be self-hosted.
Self-hosting documentation: https://docs.dify.ai/getting-started/install-self-hosted

Feature overview:

• Building visual prompt workflows
• Supports multiple LLMs, incl. open-source ones
• Supports RAGs and vector databases
• It has an API that you can use to integrate into your own application
• You can log and evaluate your user's requests

I searched for a long time for a good open solution, and came to the conclusion that this is my favourite.

I would love to see this on cloudron and maybe other developers who work with LLMs might agree.

[taowang]

I am looking into this as well. I have been using flowise, a ui on top of langchain.
Dify is not based on langchain It could be another alternative to langchain.

[umnz]

From a quick scan it isn't very clear from the pricing page whether the self hosted community edition is feature complete or has certain features missing. Has anyone found a price/feature/comparison page? Otherwise I agree this looks very solid.

[taowang]

@umnz with the community self-hosted version, you can't modify their branding ("powered by dify" and logo) unless you obtain a license from them. If you just use dify as backend, you don't need to purchase license.

[umnz]

@taowang seems like a good candidate for Cloudron in that case!

[LoudLemur]

via ai:

"Importance of AI Prompt Management:

Consistency: Ensures consistent outputs across multiple interactions or users.
Efficiency: Saves time by reusing and refining effective prompts.
Performance optimization: Helps in fine-tuning prompts for better results.
Collaboration: Enables teams to share and improve prompts collectively.
Version control: Tracks changes and improvements in prompts over time.
Compliance: Helps maintain adherence to guidelines and ethical standards.

Issues in Prompt Management:

Versioning: Keeping track of different prompt versions and their effectiveness.
Organization: Managing a large number of prompts across various projects or use cases.
Sharing: Efficiently sharing prompts among team members or across departments.
Testing: Systematically testing and comparing different prompt variations.
Documentation: Maintaining clear records of prompt purposes, contexts, and performance.
Security: Ensuring sensitive prompts are not exposed to unauthorized users.
Scalability: Managing prompts as the number of AI applications and users grows.

How dify.ai can help:

Centralized repository: Provides a single location to store and organize all prompts.
Version control: Tracks changes to prompts over time, allowing for easy rollbacks if needed.
Collaboration features: Enables team members to work together on prompt development and refinement.
Testing and optimization: Offers tools to systematically test and compare different prompt versions.
Integration capabilities: Can potentially integrate with various AI models and platforms for seamless workflow.
User management: Allows for access control and permission settings for different team members.
Analytics: Provides insights into prompt performance and usage patterns.
Templating: Offers reusable prompt templates to maintain consistency across similar use cases.
API access: Likely provides API endpoints for programmatic access to prompts in applications.
Documentation: Facilitates adding context, notes, and performance data to each prompt.
"

[taowang]

This app is extremely complicated. It includes at least 9 services....

[zonzonzon]

vote for this app!

[timconsidine]

@zonzonzon er, you didn't vote

[taowang]

For folks who need to self host this app, I created a comprehensive tutorial.

Create a fresh server, update and upgrade the system, install docker and docker compose with this one command.

sudo apt-get update -y &&
sudo apt-get upgrade -y &&
sudo apt-get install -y apt-transport-https ca-certificates curl software-properties-common gnupg lsb-release &&
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg &&
echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null &&
sudo apt-get update -y &&
sudo apt-get install -y docker-ce &&
sudo usermod -aG docker $USER &&
sudo curl -L https://github.com/docker/compose/releases/download/v2.26.1/docker-compose-`uname -s-uname -m` -o /usr/local/bin/docker-compose &&
sudo chmod +x /usr/local/bin/docker-compose && \

Clone the Repo

git clone https://github.com/langgenius/dify.git

Customize the .env file

cd dify/docker
cp .env.example .env

I only customized the following variables.
APP_WEB_URL=your_domain_name
NGINX_HTTPS_ENABLED=true
NGINX_SSL_CERT_FILENAME=cert.pem
NGINX_SSL_CERT_KEY_FILENAME=key.pem

Ctrl S to save, Ctrl X to exit the nano editor.
cd -

Create SSL Certificate

Create ssl certificate at Cloudflare (open SSL/TSL tab and open Original Server)

nano dify/docker/nginx/ssl/cert.pem
Copy and paste the Cloudflare certificate, then Ctrl S, Ctrl X

nano dify/docker/nginx/ssl/key.pem
Copy and paste the Cloudflare SSL key, then Ctrl S, Ctrl X

Deploy

cd dify/docker
docker compose up -d

Upgrade

cd dify/docker
docker compose down
git pull origin main
docker compose pull
docker compose up -d

Setting Up Automated Backups for Dify Using Restic and Cloudflare R2

Step 1: Install Restic

1. Connect to your server via SSH.
2. Update your package list and install Restic:

   sudo apt update
   sudo apt install restic
   

Step 2: Prepare Your Cloudflare R2 Bucket

1. Create a new R2 bucket in your Cloudflare account if you haven't already.
2. Note down the following information from your Cloudflare R2 setup:
   • Bucket name
   • Access Key ID
   • Secret Access Key
   • R2 Endpoint URL

Step 3: Create the Backup Script

1. Create a new directory for the Restic script:

   mkdir -p /root/restic
   

2. Create and open the backup script using nano:

   nano /root/restic/backup.sh
   

3. Copy and paste the following content into the file:

   #!/bin/bash
   
   # Set environment variables for Restic
   export RESTIC_REPOSITORY="s3:YOUR_R2_ENDPOINT_URL/YOUR_BUCKET_NAME"
   export AWS_ACCESS_KEY_ID="YOUR_ACCESS_KEY_ID"
   export AWS_SECRET_ACCESS_KEY="YOUR_SECRET_ACCESS_KEY"
   
   # Set the backup source
   BACKUP_SOURCE="/root/dify/docker/volumes"
   
   # Perform the backup
   restic backup $BACKUP_SOURCE
   
   # Prune old snapshots (keep last 7 daily, last 4 weekly, and last 12 monthly backups)
   restic forget --keep-daily 7 --keep-weekly 4 --keep-monthly 12 --prune
   
   # Check the repository for errors
   restic check
   

4. Replace the following placeholders with your actual Cloudflare R2 information:

   • YOUR_R2_ENDPOINT_URL: Your Cloudflare R2 endpoint URL
   • YOUR_BUCKET_NAME: Your R2 bucket name
   • YOUR_ACCESS_KEY_ID: Your R2 Access Key ID
   • YOUR_SECRET_ACCESS_KEY: Your R2 Secret Access Key

5. Save the file and exit nano:

   • Press Ctrl + X
   • When prompted to save, press Y
   • Press Enter to confirm the file name

6. Make the script executable:

   chmod +x /root/restic/backup.sh
   

7. Secure the script (as it contains sensitive information):

   chmod 700 /root/restic/backup.sh
   

Step 4: Initialize the Restic Repository

1. Set the environment variables (replace with your actual R2 information):

   export RESTIC_REPOSITORY="s3:YOUR_R2_ENDPOINT_URL/YOUR_BUCKET_NAME"
   export AWS_ACCESS_KEY_ID="YOUR_ACCESS_KEY_ID"
   export AWS_SECRET_ACCESS_KEY="YOUR_SECRET_ACCESS_KEY"
   

2. Initialize the Restic repository:

   restic init
   

Step 5: Set Up Daily Backups

1. Open the crontab file:

   crontab -e
   

2. If prompted to select an editor, choose nano by entering the corresponding number.

3. Add the following line at the end of the file to run the backup daily at 3 AM:

   0 3 * * * /root/restic/backup.sh > /root/restic/backup.log 2>&1
   

4. Save the file and exit nano:

   • Press Ctrl + X
   • When prompted to save, press Y
   • Press Enter to confirm

Verifying the Setup

To ensure everything is set up correctly:

1. Run the backup script manually:

   /root/restic/backup.sh
   

2. Check the Restic repository for snapshots:

   restic snapshots
   

Restoring from a Backup

To restore your data:

1. List available snapshots:

   restic snapshots
   

2. Restore the latest snapshot:

   restic restore latest --target /path/to/restore
   

   Replace /path/to/restore with the directory where you want to restore the data.

Done.