THE BLOG

Antony Slumbers 4/14/24 Antony Slumbers 4/14/24

Three Ways to Incorporate Your Own Data When Working with Generative AI

A quick guide for business people.

Generative AI, via the likes of Open AI's GPT-4, Google's Gemini and Anthropic's Claude, is revolutionising the way businesses interact with and utilise artificial intelligence. These models can generate human-like text, assist with various tasks, and provide intelligent responses to user queries. However, to truly harness the power of generative AI for your business, it will often be the case that you wish to incorporate your own data into the process. In this article, we'll explore three methods to integrate your company's knowledge with generative AI and discuss the pros and cons of each approach.

Method 1: Uploading Documents within the Context of a Question

The simplest way to incorporate your own data when working with generative AI is to upload relevant documents within the context of a question. This approach involves providing the AI with specific excerpts or documents that contain the information needed to answer a given query. By doing so, you can ensure that the AI has access to the most relevant and up-to-date information from your company's knowledge base.

Pros:

Easy to implement and doesn't require any additional technical setup
Allows for targeted information sharing based on the specific question
Provides a straightforward way to control the information the AI uses to generate responses

Cons:

Limited by the size of the AI's context window, which may not accommodate larger documents or extensive knowledge bases
Requires manual effort to select and upload relevant documents for each question
The AI may still generate hallucinations if the provided context is incomplete or if it makes assumptions beyond the given information

Method 2: Connecting a ChatGPT Model to an Internal Directory of Documents

Another approach is to connect a model to an internal directory of documents. This method involves setting up a system where the AI can access and retrieve information from a centralised repository of your company's documents, such as reports, presentations, and knowledge base articles. One might do this via creating a 'GPT' via ChatGPT, or by interacting with a models API.

Pros:

Enables the AI to access a larger volume of information compared to uploading documents within the context of a question
Allows for a more automated and scalable way of incorporating your company's knowledge into the AI's responses
Provides a centralised and organised way to manage and update your company's knowledge base

Cons:

Requires technical setup and maintenance to connect the AI model to the document directory and ensure smooth information retrieval
The relevance of the retrieved information may be limited by traditional keyword-based search techniques
The AI may generate hallucinations if the retrieved documents are not entirely relevant or if there are gaps in the information

Method 3: Using a Vector Database and Retrieval-Augmented Generation (RAG) with Claude or A N Other model.

The most advanced and effective way to incorporate your own data when working with generative AI is to use a vector database and Retrieval-Augmented Generation (RAG). This approach involves converting your company's knowledge into high-dimensional vectors and storing them in a vector database. The RAG technique is then used to retrieve the most relevant information from the vector database and integrate it seamlessly with the AI's generation process.

Pros:

Offers the most accurate and relevant information retrieval based on semantic similarity
Enables a tight integration between the retrieved information and the AI's generation process, minimising the risk of hallucinations
Provides the highest level of customisation and fine-tuning to suit your specific business needs

Cons:

Requires significant technical expertise and resources to implement and maintain
Involves additional costs for setting up and running a vector database
Requires careful data preparation, preprocessing, and vectorisation to ensure optimal performance

Choosing the Right Approach

The best approach for incorporating your own data when working with generative AI depends on your business requirements, available resources, and the desired level of customisation and accuracy. If you prioritise ease of implementation and have a limited amount of data to incorporate, uploading documents within the context of a question may be sufficient. If you have a larger knowledge base and want a more automated solution, connecting a ChatGPT model to an internal directory of documents could be a good choice.

However, if your primary goal is to minimise hallucinations, maximise the relevance of the AI's responses, and have the highest level of customisation, using a vector database and RAG with one of the foundational models (GPT, Claude et al) is the most suitable option. This approach ensures that the AI is always grounded in your company's factual information and provides the most accurate and reliable responses.

Regardless of the approach you choose, it's essential to regularly review and update your company's knowledge base, monitor the AI's performance, and have human oversight to identify and correct any potential inaccuracies or hallucinations.

By incorporating your own data into generative AI, you can unlock the full potential of this technology and create AI-powered solutions that are tailored to your business needs.

—————————————-

A Note on Hallucinations:

When it comes to minimising hallucinations and ensuring accurate information (such as with a customer service chatbot), the choice of approach can have a significant impact.

A) Uploading documents within the context of a question:

Hallucination risk: Moderate to High

Explanation: While providing relevant documents within the context can help ground the responses in factual information, there is still a risk of hallucinations if the model makes assumptions or extrapolates beyond the provided context. The limited size of the context window may also lead to incomplete information, increasing the chances of hallucinations.

B) Connecting a ChatGPT model to an internal directory of documents:

Hallucination risk: Moderate

Explanation: By connecting the model to a directory of documents, you can provide a larger volume of information for the model to draw from. This can help reduce hallucinations compared to the first approach. However, the effectiveness of this approach depends on the quality and relevance of the documents retrieved by the keyword-based search. If the retrieved documents are not entirely relevant or if there are gaps in the information, the model may still generate hallucinations.

C) Using a vector database and RAG with Claude:

Hallucination risk: Low

Explanation: The combination of a vector database and RAG offers the best chance of minimising hallucinations. The vector database enables more accurate and relevant information retrieval based on semantic similarity, reducing the chances of missing important context. The RAG technique ensures that the retrieved information is tightly integrated with the generation process, keeping the model grounded in the factual information stored in the vector database. This approach allows for more control over the information the model uses to generate responses, minimising the risk of hallucinations.

Antony Slumbers 4/14/24 Antony Slumbers 4/14/24

Why Real Estate Needs a New Mindset to Succeed with Generative AI

STOP!!!

I need you to take a deep breath and work through this step by step.

Those of you who, like me, spend too long talking to large language models, will recognise this counterintuitive phrase.

An instruction to an inanimate object to do something we know it cannot, but that weirdly, does produce superior outcomes.

What is going on? What delusion is being played out here? It’s a machine yet telling it to take a deep breath makes it work better.

What’s going on is that you ARE dealing with software, but not software that behaves as you have come to expect.

And that’s why, most likely, you are making mistakes in your plans to incorporate it into your business.

Historically software has been very deterministic. It does A, then B, then C. And it does the same thing over and over again. So we can easily decide what we want it to do and then expect it to do just that.

But large language models DO NOT behave like this. You don’t know if they will do A, then B, then C and you certainly do not know if they will do the same thing twice.

You also don’t know if they are going to tell you the truth. Ask the wrong sort of question, in the wrong sort of way, and they’ll give you an answer, a very plausible one. But it will have little to do with the truth.

Conversely, they will also make things up when you WANT them to - ‘give me 10 ideas about XYZ’.

So is that 'hallucination' bug actually a feature?

Can you have creativity without imagining things?

Either way, unless you have deeply internalised the actualité of what you are dealing with, you are going to come a cropper.

And you cannot deeply internalise the actualité of large language models unless you have used them extensively yourself, in a myriad of ways.

Which brings us back to STOP!!!

I am hearing and reading many accounts of real estate companies saying they are building this or that application, but doing so in a manner that suggests only a tendentious familiarity with the above. Not least of all a general lack of awareness as to what is generative and what is not generative AI (Generative AI is a creative tool, whereas Predictive AI is an analytical tool. That's quite some difference)

Which is of course understandable. Everyone is forever under pressure to deliver. And when we’re talking ‘flavour of the month’ the pressure can be intense.

But the outcomes will not be good. We need to approach generative AI from a different direction.

And that involves two things; practice and patience.

Practice in that you need to have as many people as possible within your company trying to incorporate using a language model in everything they do. You need to see what emerges from this. What works, what does not, what is surprising, what is disappointing. Where ‘might’ these things REALLY be useful? What should we get them to do, what should we do together, and what should we keep for ourselves?Where will they allow us to do something faster, better, cheaper? And much more importantly, where might they enable us to do something we could not do before? And what might that mean?

And patience in that Claude 3 has just been released but soon (in weeks maybe months, but not a year) we will have ChatGPT 5 and Gemini 2. And we’ve already been told, by Sam Altman himself, not to try and make certain hard and complicated things work today that the next version, coming soon, will happen right out of the box.

Bloomberg spend a fortune on BloombergGPT, their own customised version of GPT 3.5, only to find that today’s GPT 4 Turbo beats it for $20 a month.

So practice and be patient.

This is the fastest developing technology in history but it is more like an alien being than anything else we are familiar with. There is no user guide because the developers do not know what it can, or cannot, do. It is an emerging technology.

Ultimately, use of these tools will be embedded in everything your company does, and everyone in your company will use them in everything they do. But for now, the best advice is to use them, learn about them and see what comes from that. Only then should you be trying to productise them.

So ….. take a deep breath and let’s work through this step by step.

Antony Slumbers 4/9/24 Antony Slumbers 4/9/24

Why Real Estate Needs a New Mindset to Succeed with Generative AI