Langfuse - Open Source LLM Engineering Platform

[Lanhild]

Traces, evals, prompt management and metrics to debug and improve your LLM application.

Tools for the full development workflow
All Langfuse features are tightly integrated with Langfuse Tracing.

Open source LLM engineering platform: Observability, metrics, evals, prompt management, playground, datasets. Integrates with LlamaIndex, Langchain, OpenAI SDK, LiteLLM, and more. YC W23

---

Hi,

I started a package for Langfuse. Everything works well, the only missing element is the startup of the application. I have the following error:

Jun 21 17:22:43 prisma:error <30>1 2024-06-21T21:22:43Z my.numi.tools 14b3836e-aeca-46c2-bd02-d454bfec8b80 1022 14b3836e-aeca-46c2-bd02-d454bfec8b80 - Invalid `prisma.$queryRaw()` invocation: <30>1 2024-06-21T21:22:43Z my.numi.tools 14b3836e-aeca-46c2-bd02-d454bfec8b80 1022 14b3836e-aeca-46c2-bd02-d454bfec8b80 - <30>1 2024-06-21T21:22:43Z my.numi.tools 14b3836e-aeca-46c2-bd02-d454bfec8b80 1022 14b3836e-aeca-46c2-bd02-d454bfec8b80 - <30>1 2024-06-21T21:22:43Z my.numi.tools 14b3836e-aeca-46c2-bd02-d454bfec8b80 1022 14b3836e-aeca-46c2-bd02-d454bfec8b80 - Prisma Client could not locate the Query Engine for runtime "debian-openssl-3.0.x". <30>1 2024-06-21T21:22:43Z my.numi.tools 14b3836e-aeca-46c2-bd02-d454bfec8b80 1022 14b3836e-aeca-46c2-bd02-d454bfec8b80 - <30>1 2024-06-21T21:22:43Z my.numi.tools 14b3836e-aeca-46c2-bd02-d454bfec8b80 1022 14b3836e-aeca-46c2-bd02-d454bfec8b80 - This happened because Prisma Client was generated for "linux-musl-openssl-3.0.x", but the actual deployment required "debian-openssl-3.0.x". <30>1 2024-06-21T21:22:43Z my.numi.tools 14b3836e-aeca-46c2-bd02-d454bfec8b80 1022 14b3836e-aeca-46c2-bd02-d454bfec8b80 - Add "debian-openssl-3.0.x" to `binaryTargets` in the "schema.prisma" file and run `prisma generate` after saving it: <30>1 2024-06-21T21:22:43Z my.numi.tools 14b3836e-aeca-46c2-bd02-d454bfec8b80 1022 14b3836e-aeca-46c2-bd02-d454bfec8b80 - <30>1 2024-06-21T21:22:43Z my.numi.tools 14b3836e-aeca-46c2-bd02-d454bfec8b80 1022 14b3836e-aeca-46c2-bd02-d454bfec8b80 - generator client { <30>1 2024-06-21T21:22:43Z my.numi.tools 14b3836e-aeca-46c2-bd02-d454bfec8b80 1022 14b3836e-aeca-46c2-bd02-d454bfec8b80 - provider = "prisma-client-js" <30>1 2024-06-21T21:22:43Z my.numi.tools 14b3836e-aeca-46c2-bd02-d454bfec8b80 1022 14b3836e-aeca-46c2-bd02-d454bfec8b80 - binaryTargets = ["native", "debian-openssl-3.0.x"] <30>1 2024-06-21T21:22:43Z my.numi.tools 14b3836e-aeca-46c2-bd02-d454bfec8b80 1022 14b3836e-aeca-46c2-bd02-d454bfec8b80 - } <30>1 2024-06-21T21:22:43Z my.numi.tools 14b3836e-aeca-46c2-bd02-d454bfec8b80 1022 14b3836e-aeca-46c2-bd02-d454bfec8b80 - <30>1 2024-06-21T21:22:43Z my.numi.tools 14b3836e-aeca-46c2-bd02-d454bfec8b80 1022 14b3836e-aeca-46c2-bd02-d454bfec8b80 - The following locations have been searched: <30>1 2024-06-21T21:22:43Z my.numi.tools 14b3836e-aeca-46c2-bd02-d454bfec8b80 1022 14b3836e-aeca-46c2-bd02-d454bfec8b80 - /app/code/langfuse/node_modules/.pnpm/@prisma+client@5.13.0_prisma@5.13.0/node_modules/.prisma/client
Jun 21 17:22:43 prisma:error <30>1 2024-06-21T21:22:43Z my.numi.tools 14b3836e-aeca-46c2-bd02-d454bfec8b80 1022 14b3836e-aeca-46c2-bd02-d454bfec8b80 - Invalid `prisma.$queryRaw()` invocation: <30>1 2024-06-21T21:22:43Z my.numi.tools 14b3836e-aeca-46c2-bd02-d454bfec8b80 1022 14b3836e-aeca-46c2-bd02-d454bfec8b80 - <30>1 2024-06-21T21:22:43Z my.numi.tools 14b3836e-aeca-46c2-bd02-d454bfec8b80 1022 14b3836e-aeca-46c2-bd02-d454bfec8b80 - <30>1 2024-06-21T21:22:43Z my.numi.tools 14b3836e-aeca-46c2-bd02-d454bfec8b80 1022 14b3836e-aeca-46c2-bd02-d454bfec8b80 - Prisma Client could not locate the Query Engine for runtime "debian-openssl-3.0.x". <30>1 2024-06-21T21:22:43Z my.numi.tools 14b3836e-aeca-46c2-bd02-d454bfec8b80 1022 14b3836e-aeca-46c2-bd02-d454bfec8b80 - <30>1 2024-06-21T21:22:43Z my.numi.tools 14b3836e-aeca-46c2-bd02-d454bfec8b80 1022 14b3836e-aeca-46c2-bd02-d454bfec8b80 - This happened because Prisma Client was generated for "linux-musl-openssl-3.0.x", but the actual deployment required "debian-openssl-3.0.x". <30>1 2024-06-21T21:22:43Z my.numi.tools 14b3836e-aeca-46c2-bd02-d454bfec8b80 1022 14b3836e-aeca-46c2-bd02-d454bfec8b80 - Add "debian-openssl-3.0.x" to `binaryTargets` in the "schema.prisma" file and run `prisma generate` after saving it: <30>1 2024-06-21T21:22:43Z my.numi.tools 14b3836e-aeca-46c2-bd02-d454bfec8b80 1022 14b3836e-aeca-46c2-bd02-d454bfec8b80 - <30>1 2024-06-21T21:22:43Z my.numi.tools 14b3836e-aeca-46c2-bd02-d454bfec8b80 1022 14b3836e-aeca-46c2-bd02-d454bfec8b80 - generator client { <30>1 2024-06-21T21:22:43Z my.numi.tools 14b3836e-aeca-46c2-bd02-d454bfec8b80 1022 14b3836e-aeca-46c2-bd02-d454bfec8b80 - provider = "prisma-client-js" <30>1 2024-06-21T21:22:43Z my.numi.tools 14b3836e-aeca-46c2-bd02-d454bfec8b80 1022 14b3836e-aeca-46c2-bd02-d454bfec8b80 - binaryTargets = ["native", "debian-openssl-3.0.x"] <30>1 2024-06-21T21:22:43Z my.numi.tools 14b3836e-aeca-46c2-bd02-d454bfec8b80 1022 14b3836e-aeca-46c2-bd02-d454bfec8b80 - } <30>1 2024-06-21T21:22:43Z my.numi.tools 14b3836e-aeca-46c2-bd02-d454bfec8b80 1022 14b3836e-aeca-46c2-bd02-d454bfec8b80 - <30>1 2024-06-21T21:22:43Z my.numi.tools 14b3836e-aeca-46c2-bd02-d454bfec8b80 1022 14b3836e-aeca-46c2-bd02-d454bfec8b80 - The following locations have been searched: <30>1 2024-06-21T21:22:43Z my.numi.tools 14b3836e-aeca-46c2-bd02-d454bfec8b80 1022 14b3836e-aeca-46c2-bd02-d454bfec8b80 - /app/code/langfuse/node_modules/.pnpm/@prisma+client@5.13.0_prisma@5.13.0/node_modules/.prisma/client <30>1 2024-06-21T21:22:43Z my.numi.tools 14b3836e-aeca-46c2-bd02-d454bfec8b80 1022 14b3836e-aeca-46c2-bd02-d454bfec8b80 - /app/code/langfuse/node_modules/.pnpm/@prisma+client@5.13.0_prisma@5.13.0/node_modules/@prisma/client <30>1 2024-06-21T21:22:43Z my.numi.tools 14b3836e-aeca-46c2-bd02-d454bfec8b80 1022 14b3836e-aeca-46c2-bd02-d454bfec8b80 - /app/node_modules/.pnpm/@prisma+client@5.13.0_prisma@5.13.0/node_modules/@prisma/client <30>1 2024-06-21T21:22:43Z my.numi.tools 14b3836e-aeca-46c2-bd02-d454bfec8b80 1022 14b3836e-aeca-46c2-bd02-d454bfec8b80 - /tmp/prisma-engines <30>1 2024-06-21T21:22:43Z my.numi.tools 14b3836e-aeca-46c2-bd02-d454bfec8b80 1022 14b3836e-aeca-46c2-bd02-d454bfec8b80 - /app/code/langfuse/packages/shared/prisma
Jun 21 17:22:43 /app/code/langfuse/node_modules/.pnpm/@prisma+client@5.13.0_prisma@5.13.0/node_modules/@prisma/client <30>1 2024-06-21T21:22:43Z my.numi.tools 14b3836e-aeca-46c2-bd02-d454bfec8b80 1022 14b3836e-aeca-46c2-bd02-d454bfec8b80 - /app/node_modules/.pnpm/@prisma+client@5.13.0_prisma@5.13.0/node_modules/@prisma/client <30>1 2024-06-21T21:22:43Z my.numi.tools 14b3836e-aeca-46c2-bd02-d454bfec8b80 1022 14b3836e-aeca-46c2-bd02-d454bfec8b80 - /tmp/prisma-engines <30>1 2024-06-21T21:22:43Z my.numi.tools 14b3836e-aeca-46c2-bd02-d454bfec8b80 1022 14b3836e-aeca-46c2-bd02-d454bfec8b80 - /app/code/langfuse/packages/shared/prisma
Jun 21 17:22:43 Telemetry, unexpected error: PrismaClientInitializationError: <30>1 2024-06-21T21:22:43Z my.numi.tools 14b3836e-aeca-46c2-bd02-d454bfec8b80 1022 14b3836e-aeca-46c2-bd02-d454bfec8b80 - Invalid `prisma.$queryRaw()` invocation: <30>1 2024-06-21T21:22:43Z my.numi.tools 14b3836e-aeca-46c2-bd02-d454bfec8b80 1022 14b3836e-aeca-46c2-bd02-d454bfec8b80 - <30>1 2024-06-21T21:22:43Z my.numi.tools 14b3836e-aeca-46c2-bd02-d454bfec8b80 1022 14b3836e-aeca-46c2-bd02-d454bfec8b80 - <30>1 2024-06-21T21:22:43Z my.numi.tools 14b3836e-aeca-46c2-bd02-d454bfec8b80 1022 14b3836e-aeca-46c2-bd02-d454bfec8b80 - Prisma Client could not locate the Query Engine for runtime "debian-openssl-3.0.x". <30>1 2024-06-21T21:22:43Z my.numi.tools 14b3836e-aeca-46c2-bd02-d454bfec8b80 1022 14b3836e-aeca-46c2-bd02-d454bfec8b80 - <30>1 2024-06-21T21:22:43Z my.numi.tools 14b3836e-aeca-46c2-bd02-d454bfec8b80 1022 14b3836e-aeca-46c2-bd02-d454bfec8b80 - This happened because Prisma Client was generated for "linux-musl-openssl-3.0.x", but the actual deployment required "debian-openssl-3.0.x". <30>1 2024-06-21T21:22:43Z my.numi.tools 14b3836e-aeca-46c2-bd02-d454bfec8b80 1022 14b3836e-aeca-46c2-bd02-d454bfec8b80 - Add "debian-openssl-3.0.x" to `binaryTargets` in the "schema.prisma" file and run `prisma generate` after saving it: <30>1 2024-06-21T21:22:43Z my.numi.tools 14b3836e-aeca-46c2-bd02-d454bfec8b80 1022 14b3836e-aeca-46c2-bd02-d454bfec8b80 - <30>1 2024-06-21T21:22:43Z my.numi.tools 14b3836e-aeca-46c2-bd02-d454bfec8b80 1022 14b3836e-aeca-46c2-bd02-d454bfec8b80 - generator client { <30>1 2024-06-21T21:22:43Z my.numi.tools 14b3836e-aeca-46c2-bd02-d454bfec8b80 1022 14b3836e-aeca-46c2-bd02-d454bfec8b80 - provider = "prisma-client-js" <30>1 2024-06-21T21:22:43Z my.numi.tools 14b3836e-aeca-46c2-bd02-d454bfec8b80 1022 14b3836e-aeca-46c2-bd02-d454bfec8b80 - binaryTargets = ["native", "debian-openssl-3.0.x"] <30>1 2024-06-21T21:22:43Z my.numi.tools 14b3836e-aeca-46c2-bd02-d454bfec8b80 1022 14b3836e-aeca-46c2-bd02-d454bfec8b80 - } <30>1 2024-06-21T21:22:43Z my.numi.tools 14b3836e-aeca-46c2-bd02-d454bfec8b80 1022 14b3836e-aeca-46c2-bd02-d454bfec8b80 - <30>1 2024-06-21T21:22:43Z my.numi.tools 14b3836e-aeca-46c2-bd02-d454bfec8b80 1022 14b3836e-aeca-46c2-bd02-d454bfec8b80 - The following locations have been searched: <30>1 2024-06-21T21:22:43Z my.numi.tools 14b3836e-aeca-46c2-bd02-d454bfec8b80 1022 14b3836e-aeca-46c2-bd02-d454bfec8b80 - /app/code/langfuse/node_modules/.pnpm/@prisma+client@5.13.0_prisma@5.13.0/node_modules/.prisma/client <30>1 2024-06-21T21:22:43Z my.numi.tools 14b3836e-aeca-46c2-bd02-d454bfec8b80 1022 14b3836e-aeca-46c2-bd02-d454bfec8b80 - /app/code/langfuse/node_modules/.pnpm/@prisma+client@5.13.0_prisma@5.13.0/node_modules/@prisma/client <30>1 2024-06-21T21:22:43Z my.numi.tools 14b3836e-aeca-46c2-bd02-d454bfec8b80 1022 14b3836e-aeca-46c2-bd02-d454bfec8b80 - /app/node_modules/.pnpm/@prisma+client@5.13.0_prisma@5.13.0/node_modules/@prisma/client <30>1 2024-06-21T21:22:43Z my.numi.tools 14b3836e-aeca-46c2-bd02-d454bfec8b80 1022 14b3836e-aeca-46c2-bd02-d454bfec8b80 - /tmp/prisma-engines <30>1 2024-06-21T21:22:43Z my.numi.tools 14b3836e-aeca-46c2-bd02-d454bfec8b80 1022 14b3836e-aeca-46c2-bd02-d454bfec8b80 - /app/code/langfuse/packages/shared/prisma <30>1 2024-06-21T21:22:43Z my.numi.tools 14b3836e-aeca-46c2-bd02-d454bfec8b80 1022 14b3836e-aeca-46c2-bd02-d454bfec8b80 - at In.handleRequestError (/app/code/langfuse/node_modules/.pnpm/@prisma+client@5.13.0_prisma@5.13.0/node_modules/@prisma/client/runtime/library.js:122:7154) <30>1 2024-06-21T21:22:43Z my.numi.tools 14b3836e-aeca-46c2-bd02-d454bfec8b80 1022 14b3836e-aeca-46c2-bd02-d454bfec8b80 - at In.handleAndLogRequestError (/app/code/langfuse/node_modules/.pnpm/@prisma+client@5.13.0_prisma@5.13.0/node_modules/@prisma/client/runtime/library.js:122:6188) <30>1 2024-06-21T21:22:43Z my.numi.tools 14b3836e-aeca-46c2-bd02-d454bfec8b80 1022 14b3836e-aeca-46c2-bd02-d454bfec8b80 - at In.request (/app/code/langfuse/node_modules/.pnpm/@prisma+client@5.13.0_prisma@5.13.0/node_modules/@prisma/client/runtime/library.js:122:5896) <30>1 2024-06-21T21:22:43Z my.numi.tools 14b3836e-aeca-46c2-bd02-d454bfec8b80 1022 14b3836e-aeca-46c2-bd02-d454bfec8b80 - at process.processTicksAndRejections (node:internal/process/task_queues:95:5) <30>1 2024-06-21T21:22:43Z my.numi.tools 14b3836e-aeca-46c2-bd02-d454bfec8b80 1022 14b3836e-aeca-46c2-bd02-d454bfec8b80 - at async l (/app/code/langfuse/node_modules/.pnpm/@prisma+client@5.13.0_prisma@5.13.0/node_modules/@prisma/client/runtime/library.js:127:11167) <30>1 2024-06-21T21:22:43Z my.numi.tools 14b3836e-aeca-46c2-bd02-d454bfec8b80 1022 14b3836e-aeca-46c2-bd02-d454bfec8b80 - at async A (/app/code/langfuse/web/.next/server/pages/api/public/health.js:1:5002) <30>1 2024-06-21T21:22:43Z my.numi.tools 14b3836e-aeca-46c2-bd02-d454bfec8b80 1022 14b3836e-aeca-46c2-bd02-d454bfec8b80 - at async a (/app/code/langfuse/web/.next/server/pages/api/public/health.js:1:4690)
Jun 21 17:22:43 at async A (/app/code/langfuse/web/.next/server/pages/api/public/health.js:1:1119) <30>1 2024-06-21T21:22:43Z my.numi.tools 14b3836e-aeca-46c2-bd02-d454bfec8b80 1022 14b3836e-aeca-46c2-bd02-d454bfec8b80 - at async /app/code/langfuse/node_modules/.pnpm/@sentry+nextjs@8.8.0_@opentelemetry+api@1.9.0_@opentelemetry+core@1.25.0_@opentelemetry+instr_g4e53b5uljxdtjcdarqr3up6tm/node_modules/@sentry/nextjs/cjs/common/wrapApiHandlerWithSentry.js:90:43 <30>1 2024-06-21T21:22:43Z my.numi.tools 14b3836e-aeca-46c2-bd02-d454bfec8b80 1022 14b3836e-aeca-46c2-bd02-d454bfec8b80 - at async K (/app/code/langfuse/node_modules/.pnpm/next@14.2.3_@babel+core@7.24.7_@opentelemetry+api@1.9.0_@playwright+test@1.43.1_react-dom@18.2.0_react@18.2.0/node_modules/next/dist/compiled/next-server/pages-api.runtime.prod.js:20:16853) { <30>1 2024-06-21T21:22:43Z my.numi.tools 14b3836e-aeca-46c2-bd02-d454bfec8b80 1022 14b3836e-aeca-46c2-bd02-d454bfec8b80 - clientVersion: '5.13.0', <30>1 2024-06-21T21:22:43Z my.numi.tools 14b3836e-aeca-46c2-bd02-d454bfec8b80 1022 14b3836e-aeca-46c2-bd02-d454bfec8b80 - errorCode: undefined <30>1 2024-06-21T21:22:43Z my.numi.tools 14b3836e-aeca-46c2-bd02-d454bfec8b80 1022 14b3836e-aeca-46c2-bd02-d454bfec8b80 - }

I'm guessing it's because the environment the app's been built on is different than the runtime's.

Resources

• https://github.com/langfuse/langfuse
• https://langfuse.com/
• https://github.com/Lanhild/langfuse-cloudron

[LoudLemur]

• Licence: Somewhat unusual
• Demo: https://langfuse.com/docs/demo
• Free Tier: available

Langfuse is an Open Source observability tool for developers who use LLMs (Large Language Models) in their applications.

YC W23 (Y Combinator Winter 2023) https://www.ycombinator.com/blog/meet-the-yc-winter-2023-batch

Thanks to @Lanhild for discovering Langfuse, recognizing its value and rapidly coding to help move it more swiftly to Cloudron.

The CEO of Anthropic recently stated that one of the breakthroughs now is interpretability of AI models: being able to see why a language model made the decisions it made. They are just beginning to be able to "see" concepts like e.g. hedging or a genre of music. This has just started having practical applications.

With AI applications, which often require lots of VRAM, I wonder if anybody has a preferred hosting platform.

Maybe you could add an "ai" tag to this to help people discover it in searches.

[Lanhild]

@girish

I have successfully packaged Langfuse for Cloudron, available here.

The one thing that does not work half works is startup. When executing node /app/code/langfuse/web/server.js, the app does start and says it listens on port 3000, but healthcheck never resolves/says connection to port 3000 is blocked.

[LoudLemur]

@Lanhild

Thank you for that.

Also, it looks like the licence has been very pleasantly upgraded to GNU GPL v3.0

[girish]

@Lanhild that usually happens when the app is not listening on 0.0.0.0 . Most likely it's only listening on localhost .

[Lanhild]

@girish Right. Forgot about this, using upstream's HOSTNAME variable to 0.0.0.0 works.

Though, I spoke too fast about the package working, I still have a runtime error after this

Investigating.

[jagan]

Hi @Lanhild sorry to hijack this thread, but can you please have a look at packaging LiteLLM please?
LiteLLM already has Langfuse built in, so this might be interesting to you.

Link to App Wishlist: https://forum.cloudron.io/topic/13141/litellm-openrouter-self-hosted-alternative-proxy-provides-access-to-openai-bedrock-anthropic-gemini-etc?_=1737171723819