Manot Inc. collects personal information via its website to provide services and enhance user experience. The company customarily discloses personal information to service providers.

Feedback Intelligence

Manot Inc.

Feedback Intelligence | 904 followers on LinkedIn. Your users understand their needs best. 
Let them optimize your LLMs for you. | Feedback Intelligence is an enterprise workspace for AI PMs and Devs to personalize LLMs to user needs. Gain user signals that optimize LLMs through real-world usage.

Head of Product @ Pure App | Advisor

Stefan Radisavljevic

Enterprise Accounts at AWS | Startup Advisor | Mentor

Maria Kitaigora

I help Startups grow through smart marketing | Startup Marketing Advisor

Araks Nalbandyan

Product / AI

Haig Douzdjian

As we step into 2024, we are excited to share a new case study authored by Chinar Movsisyan and Erik Harutyunyan on Emotion recognition application. In this post, you will learn about: 🔄 A Comparative Look at Active Learning versus Manot's Actionable Insights 📚 Manot’s comprehensive guidance on Identifying Edge Cases 📈 Experiment Results on SOTA DAN Architecture Your feedback is super helpful for us. Take a moment to read, share, and leave your comments. #emotionrecognition #activelearning #modelevaluation #actionableinsights

The main challenge with LLM applications is not building them but evaluating their effectiveness, accuracy, and value. if you want to know how companies should evaluate their production LLM applications, make sure to join my conversation with Chinar Movsisyan , CEO & Founder of Feedback Intelligence . Here's a hint, it's implicit feedback 😜 Check out the episode: https://lnkd.in/euVWvzzg #Genai #llms #Evaluation #production #software #machinelearning #AI

Serop B.

Had a great discussion with Jakub about implicit feedback and tools like Amplitude , which got me thinking... and writing! I pulled together an article to break it down: 🤖 What exactly is implicit feedback in the context of LLM apps? 💡 Why does it matter, and how can we actually capture it? 👥 How’s it different from tools like Amplitude or Mixpanel ? Now, I’m heading to Austin for the MLOps World conference, where I’ll be sharing real examples and case studies ✨ If you’re attending, stop by our booth in the startup corner! We’ve got some cool merch (and even cooler conversations) waiting for you 🦖

Chinar Movsisyan

MLOps World 's 5th conference is just around the corner, and here’s a sneak peek at the agenda: 🛠️ How to move from PoC to production-ready LLMs 💡 Solving real business problems with LLMs 🎯 Making sure LLM apps are truly aligned with business goals It’s exciting to see AI maturing beyond the labs and hobby projects into real-world impact — and the stakes just keep getting higher! I’ll be speaking about optimizing LLM-powered agents through implicit feedback — think of it as marrying Amplitude ’s user insights with RLHF’s feedback loop. Looking forward to connecting with fellow founders and friends there! Simba Khadder Featureform Shreya Rajpal Guardrails AI Aporia Alon Gubkin Faraz Thambi David Scharbach Azat Harutyunyan Feedback Intelligence MLOps World

Large Language Model (LLM) applications require continuous improvement to deliver exceptional user experiences. However, AI teams often struggle to gather actionable insights from real-world usage. Chinar Movsisyan , founder and CEO of Feedback Intelligence will address this challenge head-on. Key takeaways: - Overcoming common obstacles in optimizing LLM applications - Harnessing implicit feedback from user behavior for measurable improvements - Accelerating the development of smarter, more responsive LLM applications Why it matters: As LLM-powered applications become increasingly prevalent, fine-tuning and optimising these systems is crucial. This talk offers a unique perspective on harnessing real-world usage data to create smarter, more responsive AI applications. Whether you're an AI practitioner, product manager, or simply curious about the future of LLM technology, this session promises valuable insights for anyone working with or interested in AI. Chinar brings over 8 years of deep learning experience, from research labs to venture-backed startups. Her expertise in mission-critical AI applications across healthcare, drones, and satellites makes her uniquely qualified to address the complexities of LLM optimization. https://lnkd.in/dBWq9t6V #AIOptimization #MachineLearning MLOps World Generative AI World

Faraz Thambi

When building LLM apps, dev and data science teams focus on one set of metrics, while PMs, POs, and business teams focus on another aka “dev metrics” and “business metrics.” Dev and DS teams typically zero in on technical metrics like accuracy, precision, recall, or F1 scores. These are crucial to making sure the AI system works well technically, but they don’t always tell you how it impacts the business. On the flip side, business teams care about things like customer satisfaction, user intent, correctness, or helpfulness - metrics that tie directly to business goals. One big challenge? Translating dev metrics into business metrics - and the other way around. Take this example: Imagine a chatbot used by agents at an insurance company. On paper, it’s performing great - high accuracy, F1 scores, etc. - all things the dev team tracks. But when agents use it, they’re getting a lot of unexpected outputs on the specific questions they’re asking. This highlights the real challenge: bridging that gap between dev and business metrics 🌉 Figuring out how to connect them is key to moving from just a PoC to a full-scale AI solution that solves actual business problems. I’d love to hear how you approach this!

After pivoting to LLMs, I've spoken to hundreds of practitioners and companies to understand their LLM app development processes and pinpoint where they struggle most. Here’s the TL;DR: When AI teams start building LLM applications for a specific use case, they typically begin with a small data set (100-300 data points) to evaluate the app (whether it’s a chatbot, conversational AI, or an agent) using familiar metrics like correctness and relevance. Often, they rely on open-source libraries for this (e.g., Ragas). After some analysis, they make a decision to put the app into production, asking users to provide feedback—essentially to let them know when things go wrong. But here’s where things fall short: This strategy doesn’t yield significant ROI. Users may leave a rating or give a thumbs-up/down, but they rarely read lengthy responses in detail, and they often lack any real incentive to give explicit feedback. The takeaway? Instead of hoping for explicit feedback, they start to focus on implicit feedback, capturing user preferences based on behavior rather than direct input. By designing a robust implicit feedback function for LLM applications, they gain deeper insights into tasks, user needs, and ethical considerations. This approach allows for more accurate optimization by continuously capturing app usage patterns, breakpoints, and evolving user needs. It’s like traditional application testing followed by ongoing monitoring and maintenance—distinct phases, each with its own set of stakeholders. It’s interesting to see the industry moving towards investing in implicit feedback functions at the enterprise level 🔄

User feedback is always important. Tools like Amplitude and Mixpanel are excellent for capturing user experiences and improving products, ultimately boosting satisfaction and helping businesses stay user-centric. When it comes to AI, improving models is all about learning, and RLHF plays a big role in optimizing performance over time. With LLMs, traditional deterministic evaluations don’t work as well, and RLHF is typically applied only at the foundation model level. A good example is ChatGPT: its performance improved gradually as they collected data from real usage aka interactions. While users can give explicit feedback, most people don’t tend to do it. That’s why ChatGPT sometimes offers two response options to nudge users into providing feedback, which then helps fine-tune the model. However, traditional RLHF doesn’t apply well to the application layer of LLMs, such as in RAG, agents, or prompt engineering. But there’s potential to rethink this. Combining RLHF with tools like Mixpanel or Amplitude could create a more effective approach to improving LLM applications. Placing users at the center of evaluation and optimization is essential — they’re the ones who know their tasks and needs best.

Agents are being built to automate various tasks, and LangChain is the go-to tool. But once your agent is in production, the real challenge begins—automatically tracking which tools are invoked for user queries and flagging when similar queries trigger different tools… 😵💫 and the list goes on. Observing the step-by-step process your agent takes is crucial for refining its performance and optimizing through real-world usage! Mels has created a step-by-step guide on how to build an agent using Langchain and optimize it with Feedback Intelligence 🛠️ 💡 Interested? Let’s chat!

If you're planning to build a RAG system this weekend and need to improve the performance continously, we have put together step-by-step guidance on 🤖 implementing a RAG model from zero using GPT-4o as a base model ✂️ splitting the GDPR open-source dataset into chunks ⚙️ vectorizing and storing these chunks in ChromaDB 🚀 launching the chatbot and 🧠 automatically improving the performance using the usage aka optimizing the chatbot using human feedback Would love your thoughts!

MANO

AI:MANOT

Manot Inc.

locale

market

industry

website

crunchbase

linkedin

peer of

Metrics