Refuel AI provides a no-code LLM training platform for fine-tuning language models purpose-built for specific tasks.

Refuel AI

Generate, annotate, clean and enrich datasets for all your AI needs with Refuel's LLM-powered platform. Simply instruct Refuel on the datasets you need, and let LLMs do the work of creating and labeling data. 

Refuel

For all the progress in data science, one of the most stubborn problems that’s persisted has been resume parsing.

Resume parsing can be complex — major variations in what titles/skills mean, discrepancies in file format, and jargon changing between industries.

In fact, a recent study found that traditional ATS algorithms and rules-based parsers were only able to attain 60-70% accuracy, leading to talent mismatch, lost opportunities, and wasted effort.

We put Refuel and an LLM based approach to the test, and realized higher accuracy (95% vs 60-70%), significant time/cost savings, and a flexible output schema.

Here’s how we did it  ⬇

This is huge.

For the first time, we have an open-source model that’s state-of-the-art and outperforming GPT-4o, on most evals.

Today, Meta announced the release of the Llama 3.1 set of models, including Llama 3.1 405B, their largest open-source model to date.

This is a paradigm shift for a few reasons:

1. Sam Altman was indeed correct when he said that the cost of intelligence is going to 0.

There are multiple competing providers for that intelligence, democratized for everyone.

2. You can now customize and manage the models/weights/infra yourself without compromising on performance, with a super-permissive license.

3. TCO is typically lower — assuming you manage this well, you can get cheap hardware and deploy on your own, so no premiums to be paid to any model providers

4. You get the benefit of complete ownership and security over your data.

We’ve already begun spinning up Llama 3.1 into the Refuel platform, and plan on making it available to all of our customers soon.

What excites you the most about the Llama 3.1 release?

We just benchmarked OpenAI's newest model, GPT-4o mini. Here’s what we learned:

1. GPT-4o mini looks to be OpenAI’s replacement for GPT-3.5-turbo.

Amongst our customers, very few were using GPT-3.5-turbo and instead opted for Claude Haiku.

However, GPT-4o mini appears to be smarter AND cheaper than Haiku — which is a big deal when competing for the simpler LLM workloads


2. Large, frontier models (ex. GPT-4-turbo/ Claude Opus / Sonnet 3.5) are excellent at complex reasoning, but slow and expensive.

Meanwhile, smaller models are a faster and cheaper approach for simpler tasks - extraction, simple summarization, Q&A, etc.


3. Given the huge cost reduction, it’s hard to imagine that this will make any money (if at all) for OpenAI.

Could this be a loss leader for OpenAI to make it harder for enterprise leaders to justify open source approaches?


In either case, the win for LLM consumers is that we’re at the start of a race to the bottom for the cost of intelligence.

This is a great time to build with LLMs.

AMD’s $665M acquisition of Silo AI is a bigger deal than most think, and strategically positions them to take on the 800 pound gorilla in the room - NVIDIA.

NVIDIA has long followed an approach of building models, frameworks, benchmarks and other products to showcase how enterprises can leverage NVIDIA as their hardware provider for AI workloads.

This is evidenced through NVDIA’s Omniverse, NIM, and Optix products - all of which have helped them get a leg up in the AI arms race.

AMD’s acquisition of Silo AI now allows them to play the same ballgame.

By making it easier to build solutions on top of their hardware (commoditizing their complements), AMD can simultaneously compete with Nvidia's strategy while generating additional demand for their hardware.

Moreover, buying Silo AI allows AMD to quickly acquire AI talent that's familiar with the AMD stack (Silo AI runs LLMs on an AMD-based cluster).

What’s your prediction for AMD 24 months from now?

LLM benchmarks can be misleading. So much so, that Anthropic and OpenAI are investing millions to try and address this challenge.

The natural instinct is to pick the model with the highest eval % and call it a day, right? Not exactly.

1. Public benchmarks use datasets that are not reflective of common usage by consumers or enterprise users - check out MMLU and Hellaswag for yourself.

2. Moreover, a recent study from Surge AI found that a third of these datasets contain typos and “nonsensical” writing.

3. Additionally, there’s no way to tell if the LLM is actually reasoning, or merely regurgitating an answer that the model was previously trained on - resulting in contamination.

The bottom line - No benchmarks are going to be reflective of YOUR data. For you to trust AI models on your data and tasks, you’ll have to create your own evaluation datasets.

The value of benchmarks increases the more specific they are. For example, Anthropic just announced an initiative to fund the development of new types of benchmarks (cyber attacks, manipulation, deception etc.)

In Refuel's case, we’ve developed use case specific and industry specific benchmarks, such as in financial services and retail (pictured below). We’ve worked with our customers to carefully construct benchmarks with significant involvement from domain experts that are as close to real world performance as possible.

Is your business using the right evals?

“[𝑞𝑢𝑎𝑙𝑖𝑡𝑦 𝑑𝑎𝑡𝑎] 𝑖𝑠 𝑡ℎ𝑒 𝑏𝑖𝑔𝑔𝑒𝑠𝑡 𝑖𝑛ℎ𝑖𝑏𝑖𝑡𝑜𝑟 𝑓𝑜𝑟 𝑐𝑜𝑚𝑝𝑎𝑛𝑖𝑒𝑠 𝑡ℎ𝑎𝑡 ℎ𝑎𝑣𝑒 𝑎𝑙𝑟𝑒𝑎𝑑𝑦 𝑖𝑛𝑣𝑒𝑠𝑡𝑒𝑑 𝑛𝑜𝑤 𝑖𝑛 𝐿𝐿𝑀𝑠, 𝑎𝑟𝑐ℎ𝑖𝑡𝑒𝑐𝑡𝑢𝑟𝑒 𝑎𝑛𝑑 𝑝𝑒𝑜𝑝𝑙𝑒”. This was the quote that stood out the most in CB Insights “Enterprise AI Report” released last week.

A few interesting insights and takeaways: 

🚀  𝟏. 𝐓𝐡𝐞 𝐩𝐞𝐫𝐟𝐨𝐫𝐦𝐚𝐧𝐜𝐞 𝐠𝐚𝐩 𝐛𝐞𝐭𝐰𝐞𝐞𝐧 𝐨𝐩𝐞𝐧 𝐬𝐨𝐮𝐫𝐜𝐞 𝐚𝐧𝐝 𝐜𝐥𝐨𝐬𝐞𝐝 𝐬𝐨𝐮𝐫𝐜𝐞 𝐢𝐬 𝐜𝐥𝐨𝐬𝐢𝐧𝐠 𝐟𝐚𝐬𝐭: 

Meta’s open-source Llama-3-70B recently outperformed Anthropic’s Claude-3-Sonnet according to the MMLU benchmark (although Claude-3.5-Sonnet is back to being stronger than the Llama models).

As business leaders grapple with financial constraints, they will have to find the sweet spot between performance, cost, and flexibility while considering the ROI of open source models.

⭐️  𝟐. 𝐁𝐢𝐠𝐠𝐞𝐫 𝐢𝐬𝐧’𝐭 𝐚𝐥𝐰𝐚𝐲𝐬 𝐛𝐞𝐭𝐭𝐞𝐫:

Smaller language models (SLMs) built for specific use cases are not only often faster and cheaper, but can also outperform LLMs

For example, Microsoft Phi-3 with 7B parameters outperformed ChatGPT 3.5 trained on 20B parameters, as measured by MMLU. 

And of course, Refuel-LLM-2, our purpose-built model, outperforms GPT-4-Turbo on data labeling, cleaning and enrichment benchmarks.

Domain-specific-models are not an opportunity enterprise buyers should shy away from, and should be explored for task specific applications.

📈  𝟑. 𝐏𝐫𝐨𝐩𝐫𝐢𝐞𝐭𝐚𝐫𝐲 𝐚𝐧𝐝 𝐜𝐥𝐞𝐚𝐧 𝐝𝐚𝐭𝐚 𝐚𝐫𝐞 𝐞𝐯𝐞𝐫𝐲𝐭𝐡𝐢𝐧𝐠:

Clean data minimizes downstream AI effects and proprietary data drives differentiated business outcomes.

As the quote below aptly alludes to, curating quality data and developing the supporting infrastructure will become the lifeblood of product development and the determinant of success in the era of Gen AI.

We’re lucky to see this in action every day with our customers and partners — good data strategy, the curiosity and bravery to try task-specific models and focus on ROI — 𝐭𝐡𝐞𝐬𝐞 𝐚𝐫𝐞 𝐭𝐡𝐞 𝐢𝐧𝐠𝐫𝐞𝐝𝐢𝐞𝐧𝐭𝐬 𝐭𝐨 𝐬𝐮𝐜𝐜𝐞𝐬𝐬 𝐰𝐢𝐭𝐡 𝐀𝐈 𝐭𝐨𝐝𝐚𝐲. 

Which takeaway stood out to you the most?

AI:REFUEL

Refuel AI

Analysis Not Available