Using AI to turn the Web into a database
Advancements in AI are making previously impossible products a reality. As I discussed previously, How AI will make the Semantic Web possible, and now is the perfect time to revisit some of the original ideas that failed.
One of the most exciting ideas from the Semantic Web was the creation of "Semantic Web Agents" that would navigate the web and do tasks on your behalf:
The Semantic Web will bring structure to the meaningful content of Web pages, creating an environment where software agents roaming from page to page can readily carry out sophisticated tasks for users. Such an agent coming to the clinic's Web page will know not just that the page has keywords such as "treatment, medicine, physical, therapy" (as might be encoded today) but also that Dr. Hartman works at this clinic on Mondays, Wednesdays and Fridays and that the script takes a date range in yyyy-mm-dd format and returns appointment times. And it will "know" all this without needing artificial intelligence on the scale of 2001's Hal or Star Wars's C-3PO. Instead these semantics were encoded into the Web page when the clinic's office manager (who never took Comp Sci 101) massaged it into shape using off-the-shelf software for writing Semantic Web pages along with resources listed on the Physical Therapy Association's site.
- Tim Berners-Lee, The Semantic Web
A typical web research task is to create a list of things. You are not looking for a particular thing but things that fit specific criteria. For example, "**books that won the Hugo Award for best novel and also featured hackers." Unless someone has already compiled this list into a page somewhere on the web, you will need multiple web searches and copy and paste many results to form a final list.
With the current development of modern AI, a new window of opportunity has opened. Could it be possible to build functional software agents without having semantics encoded into the web pages? I decided to create a prototype to test this hypothesis.
My first challenge was using modern LLMs (Large Language Models) to turn unstructured text on the web into structured information. My approach was to turn everything into a Question and Answering problem.
The list "Books that won the Hugo Award for best novel and also featured hackers" is a list of "things" or "entities." that have some properties. We can rewrite those properties as the following sub-questions:
Is it a book?
Did it win the Hugo Award for best novel?
Did it feature hackers?
If all those questions are true about an "entity," it should be on the list of "Books that won the Hugo Award for best novel and featured hackers". Given a text, AI is good enough today to answer those questions well. I decided then to create a prototype that would be able to:
Break the query into sub-questions
Find potential entities
Validate those entities using the sub-questions
The prototype can break the query into subquestions, search for relevant information and answer the questions. Based on those responses, it creates the final list. Here is a video of the results:
How does it work? It first generates a list of potential entities that answer the query. Then for each one of the candidates, it will answer the sub-questions by searching the web and compiling evidence before giving a final answer.
One of the biggest problems in using LLMs for search is that the LLM can hallucinate fake responses. I encountered this many times while testing the prototype. As you can see in the video, the system hallucinated a response. It did not always do that, and with some prompt engineering, I was able to reduce the problem, but it was still frequent enough to be an issue.
Hallucination remains a problem in building the semantic web using LLMs. However, by reducing the questions to yes or no, we limit the space of possible answers and could better verify the results. I think this could be a good avenue to eliminate the hallucination problem in the future. I believe it would be possible to fine-tune an LLM to answer Yes/No questions using only the evidence provided
This type of structured search will be a good sweet spot for combining search engines and LLMs, and I expect to see an advanced version of it in Bing and Bard soon.
Very interesting thoughts, and a promising approach and proof of concept! The semantic web has been an idea 'out there' that could finally get implemented right with powerful LLMs+ KBs. you are on to something with how you draw out structure from the data on the web. Combining LLMs with knowledge-base info via prompt engineering, chaining, fine-tuning, and additional database/knowledge-base integration is the way to get to structured information. It's a game changer IMHO.