Diffbot is a company that reads and transforms data from 1.2 billion public websites into usable data, aiming to create the world's largest compendium of human knowledge.

Diffbot

LeadGraph

We Structure the World's Knowledge.

Diffbot is a world-class group of AI engineers building a universal database of structured information, to provide knowledge as a service to all intelligent applications. Whether you are building an app that uses web content, an enterprise business application, or a smart robotic assistant, we've got you covered. Thousands of leading companies rely on Diffbot data for their enterprise and consumer applications.

Matrix

Scott Thompson

Bloomberg Beta

StartX (Stanford-StartX Fund)

Tencent

Amplify Partners

Felicis

Andy Bechtolsheim

Bill Lee

Georges Harik

Call me a hater but I don't see the value in AI search. All they appear to be doing is summarizing the same results as Google search. 

I rant about how AI search can do better 👇

The Knowledge Graph now includes Research Papers. 

At Diffbot, our mission is to structure the world's knowledge, enabling a new generation of intelligent applications grounded in trusted information. Our knowledge graph is based on our own crawl of the public web, already comparable in coverage to Google's web crawl. However a big portion of human knowledge is not contained on webpages but in academic research papers - often buried in hard to parse PDF files, deep in the navigational structure of website, or on a PhD student's homepage.

That's why we are excited to announce we've added a new entity type to the Diffbot Knowledge Graph, type:Research, backed by a new data source: OpenAlex.org.  OpenAlex is a non-profit that has indexed over 250M research papers from over 250k sources, continuously tracking and connecting works, authors, institutions, and concepts. This integration means that Diffbot users now have direct access to an unprecedented range of research publications, datasets, books, and more, significantly enriching the ecosystem of data available for intelligent applications.

The open and accessible nature of OpenAlex, from its CC0-licensed data to its transparent operations, resonates with our commitment to democratizing data access. This integration facilitates an environment where developers, researchers, and innovators can freely explore, connect, and analyze data in a knowledge graph structure, fostering a culture of open innovation.

Our initial integration augments the metadata from OpenAlex with the full text of the publication (when it is publicly accessible), analyzed with Diffbot's NLP tags (similar to our Article API) to resolve Diffbot entities.  We plan to further link the other fields like the authors, publications, and institutions using Diffbot's entity resolution technology. 

You're not the only one asking ChatGPT for medical questions.  According to recent post by the Stanford Institute for Human-Centered AI, one in 10 doctors also turn to ChatGPT in their day-to-day work.

Despite the surprisingly rapid adoption of LLMs in medicine, a recent study found that over 30% of individual statements produced by the LLM (using a GPT4 + RAG system), were hallucinated citations, citing resources that did not support the statements in the response.  Systems that did not use RAG, or used (Claude/Mistral/Gemini Pro) as their base LLM did even worse, with Gemini producing 80% of citations that were unsubstantiated.

The study wasn't all bad news though.  They found they were able to substantially increase the validity of citations  by just sending the response through another pass of GPT-4 with a prompt like "Your task is to verify the accuracy of a reference statement against a provided source text.."

Providing accuracy heath information is one of the most important use cases of generative AI where you want Verified Intelligence and where hallucinations could have, literally fatal consequences.  It's these hallucinations that have led FDA Commissioner Robert Califf to publicly state that the agency is "struggling" to regulate generative AI.  The convenience and access to LLM means it is going to be used no matter what regulators do, so this underscores the urgency of the research in getting these systems right.

Link to the blog post, and the Arxiv pre-print of the study, in the comments.

As the rush to implement generative AI into the applications we use for work and daily life continues, companies are recognizing the importance of checking the LLM against external sources of truth in order to build reliable experiences.  We call this approach Verified Intelligence. 

The Wall Street journal recently published a piece about high-profile education startup Khan Academy's AI tutor for kids is teaching kids wrong math. Their AI product Khanmigo was one of the earliest showcase examples at OpenAI's GPT-4 launch.  The AI hallucinated answers to basic arithmetic questions and provided incorrect feedback when the student came up with the wrong answer.  

Of course it's easy in retrospect to understand how this happened given how generative models work, and the fix--having the LLM have access to a calculator so that it can actually do math--is pretty clear. It's a good example of how both organizations can get carried away in the hype early on, as well as a preview of how many production applications will add grounding to external knowledge sources in the coming years for a use case as important as educating our next generation.

Read the full article here: 

https://lnkd.in/d_8GzeQM


Enterprises implementing generative AI are increasingly aware of how insufficient it is to simply present the raw generation of a language model in any user-facing system.  Hence, the popularity of techniques like Retrieval-Augmented Generation (RAG) that attempt to "ground" the answer with external knowledge sources being used in systems like Perplexity, Bing Chat, and ChatGPT4 itself.  

Sounds good in principle, but have you noticed these citations themselves are often hallucinated, either citing an irrelevant article that doesn't actually  relate to the claim, or is missing entirely?

This is because a lot of such systems are still built at the prompt engineering layer by "instructing" the LLM to cite, instead of tackling the fundamentally harder problems of search relevance, citation authority, and validating the strength of the citation.  We're still a ways away from an AI system that is able to build a references section like an advanced grad student.  

However, researchers at University of Washington and Singapore have developed a method that uses a more specialized language model that's trained to recognize logical entailment (NLI) together with a fine-grained (i.e. sentence-level) reward function to fine-tune a language model that outperforms ChatGPT in precision and recall on generating answers with citations across three different datasets. 

Link to paper in comments.

COM:DIFFBOT

Diffbot