Vellum is an AI development platform for product and engineering teams. It provides tooling for experimentation, evaluation, deployment, monitoring, and collaboration to take AI products from early-stage idea to production-grade feature.

Vellum

Vellum is a developer platform for building production-worthy applications on LLMs like OpenAI’s GPT-4 or Anthropic’s Claude. Use Vellum to save hours on prompt engineering, iterate on prompts in production confidently, and continuously fine-tune for even better results.

I’ve talked with thousands of companies working with AI over the past year. Lately, most of our conversations have been about agentic workflows and how to build them.

We decided to create a go-to resource that highlights new architectures, design patterns, infrastructure, and lessons on building agentic workflows. 

This article is packed with the latest research and insights from AI consultants, founders, and engineers. A special thanks to Yohei Nakajima, Zac Harris, Eduardo Ordax, Armand Ruiz, Erik Wikander, Vasilije Markovic, and Anton Eremin for their valuable input.

Here’s what you’ll find inside:
- Emerging architectures
- Workflow components
- Design patterns (single and multi-level agents)
- The agentic stack
- Lessons learned

If you’re building one and need help, my DMs are open! Happy to help. Link to report in comments.

Fresh from the oven! 🍪 Vellum now integrates with Groq! 

Now, you can access and run all of Groq's hosted models on Vellum at incredible inference speeds. 
The current speed of Llama 3 70B is 280 tokens/second 🤯...

Here's a fun one for all of you: Recursively generating a prompt that generates flirtatious and NSFW messages.

As part of Vellum.ai's offsite, I wanted to experiment with a different way of evaluating prompts: pitting two prompts against each other.

Check out the full blog post for some cool learnings!

https://lnkd.in/ezirpyWY

"We start with Vellum and end with Vellum." - Max Bryan, VP of Technology and Design at Rentgrata

A piece of advice I got from a fellow founder in the early days of Vellum was to understand what people do right before using vellum, and right after using vellum --> and build product to make that whole journey easier.

I'm so excited to share that Ben Margolit, Max and their team at Rentgrata built their game changing real estate chatbot Ari using our platform end-to-end. Property management companies can now gain insights into resident satisfaction using Ari instead of having to dig through reams of text.

Using Vellum throughout their entire AI development lifecycle - experimentation, evaluation to production monitoring, Rentgrata achieved 100% accuracy, accelerated their development timeline by 50%, and enabled their customers to make data-driven decisions to boost resident happiness.

In one powerful example, Rentgrata's insights helped a company avoid a costly $300k window renovation and instead allocate funds more effectively based on the true resident priorities.

We're proud to partner with leaders like Max, Ben and the Rentgrata team to bring AI to the proptech industry. Excited to see how Ari continues to revolutionize the renter and property manager experience!

Read more about Rentgrata's test-driven journey from prototype to production here:

👇 Let's talk about logprobs: A powerful tool for building LLM apps which have asymmetric risk exposure

Say you operate in a regulated industry and are worried about the impact your LLM chatbot could have if it's not compliant with the law. A common architecture we suggest is a "Guardrail prompt" towards the end of your chatbot. Guardrail prompt(s) check for specific criteria (e.g., is the response compliant? is the tone appropriate etc.) and help with a safe exit when things are not shaping up as expected. As always, remember to have a fast LLM like GPT 3.5 Turbo or Claude 3 Haiku as your Guardrail prompt because it will usually be in the critical path.

You set up a good Guardrail prompt, and you get a response that your chatbot is indeed compliant. But when you look closer, the Guardrail prompt should have said that it's not compliant. The evaluator is incorrect!

Here's where Logprobs come in. Logprobs is an additional property returned in the LLM API response which shows the full probability distribution of each token -> and you can use it to your advantage.

Look at the example in the image. Our simple guardrail prompt is only instructed to output a single token: whether the support agent is in compliance with HIPAA or not. The guardrail prompt is only 70% sure in its answer and when there's an asymmetric risk in being wrong, you may want to override the Guardrail prompt's response. Sometimes we recommend our customers take Guardrail prompt responses only when the prompt is 99%+ confident.

How do you set up your prompts and logic to get your systems working reliably in production?

🤯 Vellum's changelog for March blows my mind 🤯 
https://lnkd.in/dy_Gt6Mu

A key factor in the success of all startups is the rate at which they 🚢 . I'm extremely impressed by the work of our engineering team Noa Flaherty Sidd Seethepalli David Vargas Jackson Hardy Pei 🤖 Li Maddie Abboud Nihala T. Ashlee Radka

Here’s a quick rundown of my favorites:
⛓️ Subworkflow Nodes: Easily reuse groups of nodes across multiple Workflows.
🛑 Workflow Error Nodes: Terminate workflows and raise errors for better control.
📖 Read-only Workflow Diagrams: Visualize workflow executions for better debugging.
📐 Workflow Node Mocking: Speed up development by mocking specific nodes.
🧪 Easier Node Debugging: Test elements individually without running the entire workflow.
✂️ Customizable Chunk Settings: Optimized Document Indexes to improve search results.

Hoping for a huge April and beyond! 🚀

AI:VELLUM

Vellum