Gentrace focuses on systematizing the development of large language model (LLM) applications, self-described as facilitating streamlined application development.

Gentrace

Maplebear (Instacart) operates a leading grocery delivery platform in the United States and Canada. The company partners with various regional and national grocers, which offer their selection of food and other goods to customers through Instacart's ubiquitous platform. Once an item is ordered through Instacart's site, the item is picked and delivered to the customer's home by one of the platform's 600,000 shoppers, who are classified as independent contractors. Instacart earns fees based on average order value and leverages its platform's high usage to sell advertising, mainly to consumer-packaged goods companies. Instacart currently has about 8 million monthly active users (or orderers) on its platform.

Maplebear Inc. Common Stock

Gentrace | 1,665 followers on LinkedIn. Testing and monitoring tailored to your generative AI application. | Testing and monitoring tailored to your AI app. Get from MVP to production. Gentrace systematizes LLM app development.

GTM Engine o11y OG

Jeff Cestra

Co-founder, COO at Gentrace

Daniel C. Liem

Co-founder, CTO at Gentrace

Vivek Nair

Account Executive

Nick Enthoven

Gentrace is expanding our team. We're looking for senior software engineers in NYC and SF who want to build tools to help the most advanced companies in the world make their generative AI systems reliable and predictable. Learn more at https://gentrace.ai/eng

Most engineers approach LLM-as-a-judge all wrong. The usual high-level metrics like hallucination or safety rarely tell you if your app actually works as intended. Even with human evaluators, general metrics won’t help them judge your app’s performance in a useful way. Good evals are built on something specific to your app—a unique “unfair advantage” that gives the LLM clear criteria. LLM-as-a-judge is a widely used approach where an LLM grades another LLM’s output. But without an unfair advantage, it can fall short in quality. LLM-as-a-judge problems: • Circular reasoning: How can an LLM grade what it itself generated? • Poor initial results lead teams back to vibes and manual grading. Imagine an LLM app is tasked with writing emails. A bad eval would ask the LLM to rate itself on how well it followed the prompt—a circular question that adds little signal and won’t help improve the model. Instead, give your LLM unfair advantages. Here’s how: Add tailored asserts: specific criteria your model should meet. For instance, in our email example: • No footer • Directly asks a question • Includes recipient’s email Another approach is comparison to a reference output. You provide a high-quality reference email for comparison, so the LLM can check for: • Missing or extra information • Conciseness vs. verbosity To build reliable AI evaluations, always ask: how can I create an unfair advantage for the LLM These targeted methods ensure your LLM-as-a-judge evals give you actionable insights into the quality of your app. At Gentrace , we’re making it easy for AI engineers to build their own high-quality evals. My unfair advantage blog post shares more examples for using LLM-as-a-judge effectively: https://lnkd.in/gavtDhCY

Doug Safreno

New release, featuring production evaluator graphs and local evaluations / local test data. Production evaluator graphs Production evaluators now automatically create graphs to show how performance is trending over time. For example, you can create a "Safety" evaluator which uses LLM-as-a-judge to score whether an output is compliant with your AI safety policy. Then, you can see how the average output "Safety" trends over time. Local evaluations / local test data Gentrace now allows you to more easily define local evaluations and use completely local data / datasets. This makes Gentrace work better with existing unit testing frameworks and patterns. It also makes Gentrace incrementally adoptable into homegrown testing stacks. More: • User-specific view settings can be saved and overridden from a URL • Filter test runs by their input values • Added explicit compare button • Pinecone v3 (Node) support • o1 support • Fixed 68 bugs

Quizlet takes unstructured text and builds flash cards, syllabi, and other learning tools for students with generative AI. With Gentrace, they increased testing 40x and reduced test duration to 1 minute. Learn more: https://lnkd.in/gF_Zi7Cd

Faire built an AI agent that reviews PRs. They use an AI evaluator in Gentrace to review the reviewer. Learn more about how Faire systematically develops their AI features in their blog post: https://lnkd.in/gf_arpB9

New release, featuring datasets. Datasets Gentrace now supports datasets, which organize test data into separate groups within a pipeline. This is particularly helpful when multiple engineers are working on the same pipeline and want to test different pieces of functionality. Test result settings memory Settings in the test result pages (eg hiding or re-ordering fields) are now saved across results in the same pipeline. This makes it easier to see exactly what you want (and only exactly what you want), without redoing your work every time. More: • Performance improvements on the live view • Tweaked new user experience during pipeline setup • Added bulk archive functionality for test results • gpt-4o-mini supported as an evaluator • Fixed 37 bugs

New release, featuring revamped & realtime test results. Gentrace has three new views to facilitate better testing: 1. Aggregate comparison The new aggregate view shows the statistical differences between different versions of your LLM-based feature. 2. List of test cases with evaluations The new list view shows all of the scores for different test cases. When doing a comparison, it highlights what changed between the different modes. 3. Drilldown per test case The new drilldown view presents a clear picture of the output(s) for a specific test case. It includes the JSON representation, evaluations, and timeline. Other changes • Performance: Gentrace is much faster in many views • gpt-4o supported in playground beta & as a grader • Processor edits show which evaluators will be affected • Improved human notes experience • Fixed 83 bugs

New release, featuring comparative (head-to-head) evaluators, classification evaluators, a better code editor with Python support, creating test data from live runs, and more: Comparative (head-to-head) evaluators Gentrace now supports evaluators that compare two outputs for head-to-head types of analysis. Use them to determine which of two experiments is better using an LLM or heuristics. Classification evaluators Gentrace now supports classification evaluators that do a simple comparison of classification outputs and generate more robust aggregate metrics like precision, recall, and F1. Better code editor and Python support We've upgraded the Gentrace code editor to dramatically improve the code writing experience. Gentrace also now supports Python for writing processors and evaluators. Create test data from live runs Gentrace supports creating test data directly from your live runs. When you create test data from live data, Gentrace allows you to edit the data and choose the pipeline to which it gets saved. More: • Performance work • Revamped / more filters • Prod eval daily AI limit is now per evaluator • Evaluator outputs stream during testing • Save test case data on blur • Boolean metadata type • CSV export of results includes expected values • Fixed 73 bugs

New release, featuring image evaluation, evaluator templates, ISO 27001 compliance, new blog / docs, and more: Image evaluation Gentrace now supports evaluating images using GPT-4 with Vision. Example use cases: • Validate that a generated webpage has certain visual characteristics • Compare a generated image to a baseline for higher quality Evaluator templates Gentrace added evaluator and processor templates. If you create the perfect factualness check or a very useful conciseness evaluator, create a template and share with your organization. ISO 27001 compliance Gentrace has completed our audit and been certified as ISO 27001 compliant. New blog & docs To share our thinking on best practices in generative AI development, we've created a blog. We've also re-organized our docs and added an AI-powered search. More: • Grader role: can only manually grade outputs • During evaluator testing, streaming outputs • Rerun missing, errored, all evaluations • Collapse left bar • Filter live data by metadata • Download live data as CSV • Fixed 34 bugs

We recommend 4 main approaches in detecting AI hallucination: • Automated comparison to an expected value / ground truth • Automated specific hallucination checks • Expert review • End-user feedback Learn more about these strategies on our blog: https://lnkd.in/grWA6tKi

Metric
Company Name	Gentrace	Maplebear Inc. Common Stock
Symbol		CART
MSH ID	AI:GENTRACE	COM:INSTACART
Market	PRIVATE	STOCKS
Sector	Technology	Consumer Cyclical
Industry	Software	Specialty Retail
Country	US	US
Stage	Angel
Employee Count		3.38K
Website	gentrace.ai	instacart.com
LinkedIn	gentrace
Founders

Metric
Market Cap		11.16B
Enterprise Value
Monthly Web Traffic	46
Web Traffic Growth	16.50
Valuation
Raised Capital	250K

Metric
Revenue		3.04B
Revenue (LTM)
Revenue (NTM)
Gross Profit		2.26B
EBITDA		-2.09B
Operating Income		-2.15B
Net Income		-1.62B
EPS		-5.81
Diluted EPS		-5.81
Revenue 2022
Revenue 2023
Revenue 2024
Revenue 2025
Revenue 2026

Metric
Revenue Growth (LTM)
Revenue Growth (NTM)
Gross Profit Growth (LTM)
EBITDA Growth (LTM)
Net Income Growth (LTM)
14-Day Growth Score		95.02
28-Day Growth Score		38.82
90-Day Growth Score		50.36
365-Day Growth Score		48.74

Metric
Gross Profit Margin		0.75
Operating Profit Margin		-0.68
EBITDA Margin		-0.69
Net Profit Margin		-0.52
Return on Equity		-0.48
Return on Assets		-0.34
Return on Capital Employed		-0.54

GENT

AI:GENTRACE

Gentrace

Company Overview

Market Metrics

Financial Performance

Growth Metrics

Profitability Ratios

Valuation Multiples

Operational Metrics

Cash Flow Metrics

Balance Sheet Metrics

Efficiency Metrics

Metric
P/E Ratio		-1.52
P/B Ratio		0.66
Revenue Multiple (LTM)
Revenue Multiple (NTM)
EBITDA Multiple (LTM)
EBITDA Multiple (NTM)
Price to Sales Ratio		3.50
EV Multiple		-4.60

Metric
Days Sales Outstanding		102.46
Days Payables Outstanding		27.78
Days Inventory Outstanding
Operating Cycle		107.86
Cash Conversion Cycle		85.70
Asset Turnover		0.80

Metric
Operating Cash Flow		586M
Free Cash Flow		530M
Cash Flow to Debt		21.66
Operating Cash Flow/Sales		0.22
Free Cash Flow Yield		0.05

Metric
Cash & Equivalents		2.27B
Accounts Receivable		-33M
Inventory
Goodwill		318M
Debt to Capitalization		0.01
Debt to Assets		0.01
Current Ratio		4.50
Quick Ratio		4.33

Metric
Rule of 40 (LTM)
Rule of 40 (NTM)
Stock Comp to Revenue		0.88
R&D to Revenue		0.75
SG&A to Revenue		0.26