How to build LLM applications using Go

Since LLMS (large language models) and adjacent tools such as models have grown dramatically over the past year, more and more developers are thinking of merging LLMS into their applications.

Since LLMS often requires custom devices and large calculations, they are more common while filling them as network services that provide applications programming facades to access. This is the way you work for LLMS to drive apps like Openai or Google Gemini; Even your running tools like OLLAMA Roll LLM in the REST applications interface for local consumption. Moreover, developers who benefit from LLMS in their applications often require complementary tools such as veil databases, which are commonly published as network services as well.

In other words, LLM powder applications are very similar to modern cloud modern applications: they require excellent support for RIST and RPC protocols, synchronization and performance. These only coincide with the areas that excel in it, making it a great language to write LLM applications.

This blog post works through an example of using Go for the simple application that works with LLM. It begins to describe the problem that the experimental application solves, and continues by providing many variables from the application that accomplishes the same task, but using different packages to implement it. All code of illustrations for this post is available online.

A rag servant for questions and answers

LLM shared app technology is Rac -RAC -Vival Generation. RAG is one of the most developed ways to customize the LLM knowledge base for field interactions.

We will build a Rag In going. This http server provides two users:

• Add a document to the base of knowledge
• Ask a question about this base of knowledge

In the real world scenario, users will add a set of documents to the server, and proceed with asking questions to it. For example, the company can fill the base of the Rag Server’s knowledge with the internal documents and use them to provide the capabilities of questions and answers with LLM for internal users.

Here is a scheme that shows the interactions of our servant with the outside world:

In addition to the user who sends HTTP requests (the process described above), the server interacts with:

• Form of inclusion to calculate the vectors of the documents provided and the user’s questions.
• A vector database for storing and recovering implications efficiently.
• LLM to ask questions based on the context collected from the base of knowledge.

Tangantly, the server displays HTTP end points for users:

/add/: POST {"documents": [{"text": "..."}, {"text": "..."}, ...]}: It offers a series of text documents to the server, to add it to the base of its knowledge. For this request, the server:

1. It is calculated by the perfection of each document using the inclusion form.
2. Store documents along with the vector guarantees in the DB vector.

/query/: POST {"content": "..."}Provide a question to the server. For this request, the server:

1. Calculate the iotament of the question using the inclusion form.
2. The similarity research is used in Vector DB to find the most relevant documents in the knowledge database.
3. It uses a simple guided engineering to reformulate the question with the most relevant documents in the step (2) as a context, send it to LLM, and re -answer it to the user.

The services that our demonstration uses are:

It should be very easy to replace it with other equivalent services. In fact, this is what the second and third variables revolve around the server! We will start with the first variable that uses these tools directly.

Using the GEMINI and Weavia applications interface directly

Both Gemini API and Weavia have SDKS Go (customer libraries), and our first server variable uses this directly. A complete symbol of this alternative is in this guide.

We will not completely reproduce the code in this blog post, but below some of the notes that must be taken into account while reading:

buildingThe code brown will be familiar to anyone who wrote HTTP server in Go. Customer libraries are prepared for Gemini and Weaviate and customers are stored in the value of the situation that is passed to HTTP processors.

Path registrationHTTP methods for our server trivial to prepare it using Guidance improvements It was presented at GO 1.22:

mux := http.NewServeMux()
mux.HandleFunc("POST /add/", server.addDocumentsHandler)
mux.HandleFunc("POST /query/", server.queryHandler)

Synchronization: Our servant’s HTTP processors reach other services over the network and wait for the response. This is not a problem for Go, because each HTTP processor works simultaneously in its Goroutine. This RAG server can deal with a large number of simultaneous requests, the code of each linear and coincidence processor.

Introduction programming facades: since /add/ The order may provide a large number of documents to add it to the base of knowledge, and the server returns Introduction programming facades For both implications (embModel.BatchEmbedContents) And db (rs.wvClient.Batch) For efficiency.

Using Langchain to go

The second Rag Servant Langchaningo is used to accomplish the same task.

Langchain is a popular Python frame for building LLM applications. Langchaningo is the equivalent. The frame contains some tools needed to create applications of normative components, and supports many LLM providers and vector databases in a common application programming interface. This allows developers to write a code that may work with any provider and change service providers very easily.

The complete symbol of this alternative is present in this guide. You will notice two things when reading the symbol:

First, it is somewhat shorter than the previous variable. Langchaningo takes care of the fully applications programming facades of the vectors’ databases in shared interfaces, and a lower code is needed to prepare and deal with Weavia.

Second, the Langchaningo applications interface makes it easy to switch service providers. Let’s say that we want to replace Weavia with another DB carrier; In our previous variable, we have to rewrite all the code that communicates with the DB vector to use a new application programming interface. With a frame like Langchaningo, we no longer need to do so. As long as Langchaningo supports the new DB vector that we are interested in, we must be able to replace a few lines of code in our servant, because all DBS implement a common interface:

type VectorStore interface {
    AddDocuments(ctx context.Context, docs []schema.Document, options ...Option) ([]string, error)
    SimilaritySearch(ctx context.Context, query string, numDocuments int, options ...Option) ([]schema.Document, error)
}

Using GENKIT for Go

Earlier this year, Google Genkit introduced a new open-up-based newly-used framework framework. Genkit shares some characteristics with Langchain, but it differs in other aspects.

Like Langchain, it provides common facades that can be implemented by different service providers (as additional clothes), thus making the switch from one to another simpler. However, he does not try to describe how different LLM components interact; Instead, it focuses on production features such as the administration of claims and engineering, and publishing with integrated developers tools.

Our Rag Thred Servant uses Genkit to go to accomplish the same task. Its full symbol in this guide.

This variable is somewhat similar to Langchaningo One – common facades for LLMS, and the implications and DBS are used instead of direct provider applications, making it easy to switch from one to another. In addition, spreading the LLM application to production is much easier with Genkit; We do not implement this in our alternative, but do not hesitate to read documents if you are interested.

Summary – Go to LLM applications that work

Samples in this post provide just a taste for what is possible to build LLM applications. It shows the simplicity of building a strong rag servant with a relatively small symbol; More importantly, samples pack a large degree of preparation for production due to some GO features.

Working with LLM services often means sending RIST or RPC requests to a network service, awaiting response, sending new requests to other services accordingly, etc. It surpasses all of these, providing great tools for managing synchronization and complexing the Tawfiq Network services.

In addition, Go’s wonderful and reliable performance as a protected language that makes it a natural choice for implementing the most essential basic building blocks of the LLM ecosystem. For some examples, see projects like ollama, Localai, Weavia or Milvus.

Credits: Elie Bandsky

Verne ho photography on Unsplash