deepu-sasidharan

Deepu is a polyglot developer, Java Champion, and OSS aficionado. He mainly works with Java, JS, Rust, and Golang. He co-leads JHipster and created the JDL Studio and KDash. He's a Staff Developer Advocate at Okta. He is also an international speaker and a published author.

Staff Developer Advocate

Deepu K Sasidharan

Learn how to implement fine-grained authorization in Java RAG systems using LangChain4j and Auth0 FGA to secure your AI applications.

Artificial Intelligence is evolving at a breathtaking speed. AI agents will soon become the standard for building applications, and they are going to be the new Web 2.0. With all that comes the critical need for robust security measures and checks. Retrieval-Augmented Generation (RAG) systems have become a cornerstone in building AI agents and applications. Agents that can access and utilize domain-specific knowledge are more accurate and more useful. However, as these systems become more prevalent, ensuring they only access and expose authorized information has become a critical challenge.

[Sensitive Information Disclosure](https://genai.owasp.org/llmrisk/llm022025-sensitive-information-disclosure/) is a common issue that plagues RAG-based systems. We don't want the Large Language Model (LLM) to accidentally access or expose sensitive data from a database. Traditional Role-Based Access Control (RBAC) systems are not enough to secure RAG applications and agents. This is where Fine-Grained Authorization (FGA) or Relationship-Based Access Control (ReBAC) shines as an Authorization solution for RAG.

## The Security Challenge in RAG Systems

RAG systems present some unique security challenges that traditional Role-Based Access Control (RBAC) systems struggle to address:

1. **Dynamic Context**: RAG systems need to make real-time decisions about which documents and tools can be used for context enhancement.
2. **Complex Relationships**: Documents often have intricate relationships with users, teams, and projects, forming a complex graph that simple role-based systems can't model effectively.
3. **Granular Control**: Different parts of documents may have different sensitivity levels requiring fine-grained control.
4. **Performance Requirements**: Authorization checks must be fast to maintain the AI system's responsiveness.

This is where FGA shines, it allows us to:

- Model complex relationships between users, groups, and documents
- Handle hierarchical and transitive permissions naturally
- Scale authorization rules efficiently

Let's look at some real-world scenarios where sensitive data could be at risk:

- A knowledge base assistant exposing confidential documents to employees
- A support AI revealing sensitive user information
- An AI assistant leaking proprietary information about a company
- A chatbot exposing private data to unauthorized users

## Why Auth0 FGA?

[Auth0 FGA](https://fga.dev/) is the SaaS version of [OpenFGA](https://openfga.dev/). OpenFGA is an open-source Fine-Grained Authorization system, created by Auth0, that allows you to manage access control in your applications. It's currently a CNCF Sandbox Project. OpenFGA is a powerful tool that allows you to define complex access control policies and enforce them in your application. It is a great choice for securing RAG applications as it solves most of the security challenges in RAG systems, including the ones mentioned above.

If you're new to RAG and access control, I recommend checking out this introductory post on [RAG and Access Control: Where Do You Start?](https://auth0.com/blog/rag-and-access-control-where-do-you-start/).

<include
 src="LinkCard"
 title="RAG and Access Control: Where Do You Start?"
 link="https://auth0.com/blog/rag-and-access-control-where-do-you-start/"
 description="Learn how to get started with Auth0 FGA for a RAG application."
img="https://images.ctfassets.net/23aumh6u8s0i/79NCborGcFIrAMeaV7G8oi/f9c588c2d05f0828d447e72e1fe825cc/Latam01.jpg"
/>

## Why Java and LangChain4j?

Java isn't far behind when it comes to building AI applications. It has a large developer community and a rich ecosystem of open-source frameworks, and enterprises are adopting these frameworks.

[LangChain4j](https://github.com/langchain4j/langchain4j) is one of the popular Java AI frameworks. It is a powerful open-source framework that provides capabilities similar to LangChain and LlamaIndex used in the Python/JS ecosystem. It is becoming the Spring Boot of the Java AI ecosystem. The framework offers a comprehensive set of features for building AI applications. Some of the key features include:

- Easy integration with major commercial and open-source LLMs
- Unified API for different model providers
- Tools like prompt templating, chat memory management, and function calls.
- High-level patterns, abstractions, and implementations for building Agents and RAG applications

## Building a Secure RAG System

LangChain4j makes it simple to get started with the "Easy RAG" feature while providing advanced capabilities for complex use cases. Hence, in this guide, we will look at a secure Java RAG system using LangChain4j and Auth0 FGA. We'll focus on Relationship-Based Access Control (ReBAC), which prevents unauthorized access to sensitive information while maintaining the flexibility and power of RAG systems.

### Prerequisites

> **This guide was created with the following tools and services**:
> 
> - [Java 21](https://adoptium.net/)
> - [Gradle 8.12](https://gradle.org/)
> - An FGA instance ([Create an Auth0 FGA store](https://dashboard.fga.dev) or self-hosted OpenFGA)
> - An [OpenAI API key](https://platform.openai.com) or a local [Ollama](https://ollama.com/) instance

### Setting up the project

To get started, clone the [**auth0-ai-samples**](https://a0.to/auth0-ai-samples) repository from GitHub:

```bash
git clone https://github.com/auth0-samples/auth0-ai-samples.git
cd auth0-ai-samples/authorization-for-rag/langchain4j-java
```

The application uses Gradle as the build system and is structured as follows:

- `src/main/java/rag/RagApplication.java`: The main entry point of the application that defines the RAG pipeline.
- `src/main/java/rag/FGARetriever.java`: The FGA-aware retriever that retrieves documents from the vector store based on the user's permissions.
- `src/main/java/rag/FGAInit.java`: Script to create the FGA store and models.
- `src/main/resources/docs/*.md`: Sample markdown documents that will be used as context for the LLM. We have public and private documents for demonstration purposes.
- `build.gradle`: The Gradle build file that defines the dependencies and tasks.

Let us look at the important bits and pieces before we run the application.

### RAG pipeline

The RAG pipeline configures the underlying LLM and defines the retrievers for data. The pipeline is defined as follows:

```java
/** src/main/java/rag/RagApplication.java **/
final ChatLanguageModel CHAT_MODEL_OPENAI = OpenAiChatModel
               .builder()
               .apiKey(System.getProperty("OPENAI_API_KEY"))
               .modelName(GPT_4_O_MINI)
               .build();

var user = "user2";
// 1. Read and load documents from the assets folder
var documents = FileSystemDocumentLoader
                 .loadDocuments("src/main/resources/docs");
// 2. Create an in-memory vector store from the documents.
InMemoryEmbeddingStore<TextSegment> embeddingStore = new InMemoryEmbeddingStore<>();
EmbeddingStoreIngestor.ingest(documents, embeddingStore);
// 3. Create a base retriever
ContentRetriever baseRetriever = EmbeddingStoreContentRetriever.from(embeddingStore);
// 4. Create the FGA retriever that wraps the base retriever
ContentRetriever fgaRetriever = FGARetriever.create(
       baseRetriever,
       // FGA tuple to query for the user's permissions
       content -> new ClientBatchCheckItem()
               .user("user:" + user)
               .relation("viewer")
               ._object(
                 "doc:" +
                 content.textSegment().metadata()
                 .getString("file_name").split("\\.")[0]
               )
       );
// 5. Create an assistant with the FGA retriever
var assistant = AiServices.builder(Assistant.class)
       .chatLanguageModel(CHAT_MODEL_OPENAI)
       .chatMemory(MessageWindowChatMemory.withMaxMessages(10))
       .contentRetriever(fgaRetriever)
       .build();
// 6. Query the retrieval chain with a prompt
var query = "Show me forecast for ZEKO?";
var answer = assistant.chat(query);
```

> If you want to use Ollama running locally, uncomment the `CHAT_MODEL_OLLAMA` variable and comment out the `CHAT_MODEL_OPENAI` line. Be sure to update the base URL and model name to match your local Ollama instance.

Here is a visual representation of the RAG architecture

![LangChain4j RAG architecture](https://images.ctfassets.net/23aumh6u8s0i/1X37t9DnD6btkPzE6pwGOg/68a80ce358980bc6a2ac9e0f13c08fb1/image1.jpg)

### FGA retriever

The `FGARetriever` class implements `ContentRetriever` and filters documents based on the authorization model defined in the FGA store. This retriever is ideal for scenarios where you already have documents in a vector store and want to filter the vector store results based on the user's permissions. Assuming the vector store has already narrowed down the documents to a few, the FGA retriever will further narrow down the documents to only the ones to which the user has access.

Here is the important part of the `FGARetriever` class simplified for readability:

```java
/** src/main/java/rag/FGARetriever.java **/
// Create a client to interact with the FGA store
new OpenFgaClient(new ClientConfiguration()
       .apiUrl(/*...*/)
       .storeId(/*...*/)
       // credentials only required if using Auth0 FGA,
       // comment out for local OpenFGA
       .credentials(/*...*/)
)

// Check permissions for the user and object
private Map<String, Boolean> checkPermissions(List<ClientBatchCheckItem> checks) {
   var options = //...
   var request = //...
   var response = fgaClient.batchCheck(request, options).get();

   Map<String, Boolean> permissionMap = new HashMap<>();
   for (var result : response.getResult()) {
       var checkKey = getCheckKey(result.getRequest());
       permissionMap.put(checkKey, result.isAllowed());
   }
   return permissionMap;
}

// retrieve the documents and filter them using checkPermissions function
public List<Content> retrieve(Query query) {
   // First, get relevant documents from the base retriever
   List<Content> relevantContent = baseRetriever.retrieve(query);

   // Create data structures to track checks and document mappings
   // var checks, documentToObject, seenChecks = ...

   // Process each document to build checks
   for (Content doc : relevantContent) {
       ClientBatchCheckItem check = buildQuery.apply(doc);
       var checkKey = getCheckKey(check);
       documentToObject.put(doc, checkKey);

       // Skip duplicate checks for same user, object, and relation
       if (!seenChecks.contains(checkKey)) {
           /*...*/
       }
   }

   var permissionsMap = checkPermissions(checks);
   // filter based on permission
   return relevantContent.stream()
           .filter(doc -> permissionsMap.get(documentToObject.get(doc)))
           .collect(Collectors.toList());
}
```

## Set up An FGA Instance

Visit the [Auth0 FGA dashboard](https://dashboard.fga.dev/), navigate to **Settings**, and in the **Authorized Clients** section, click **+ Create Client**. Give your client a name, mark all three client permissions, and then click **Create**. Once your client is created, you’ll see a modal containing Store ID, Client ID, and Client Secret. Click **Continue** to see the `FGA_API_URL` and `FGA_API_AUDIENCE`.

> To run OpenFGA locally with Docker, use the following command:
>
> ```bash
> docker pull openfga/openfga && \
> docker run -p 8080:8080 -p 8081:8081 -p 3000:3000 openfga/openfga run
> ```

Add `.env` file with the following content to the root of the project.

```sh
# OpenAI key. Not required if using Ollama
OPENAI_API_KEY=<your-openai-api-key>

# Open FGA
FGA_STORE_ID=<your-fga-store-id>
# Required only for Auth0 FGA
FGA_CLIENT_ID=<your-fga-store-client-id>
FGA_CLIENT_SECRET=<your-fga-store-client-secret>
FGA_API_URL=https://api.xxx.fga.dev
FGA_API_AUDIENCE=https://api.xxx.fga.dev/
```

> Check the instructions [here](https://help.openai.com/en/articles/4936850-where-do-i-find-my-openai-api-key) to find your OpenAI API key.

### Create the FGA models and tuples

We will use a simple model that defines just an owner and viewer relation for docs:

```sh
model
 schema 1.1

type user

type doc
 relations
   define owner: [user]
   define viewer: [user, user:*]
   define can_view: owner or viewer
```

> Check out [this documentation](https://docs.fga.dev/modeling) to learn more about creating an authorization model in FGA.

Now, to have access to the public information, we will need the following tuple on FGA.

- **User**: `user:*`
- **Object**: select doc and add `public-doc` in the ID field
- **Relation** : `viewer`

A tuple signifies a user’s relation to a given object. For example, the above tuple implies that all users can view the `public-doc` object.

Now, to have access to the private information, we will need the following tuple on FGA.

- **User**: `user:user1`
- **Object**: select doc and add `private-doc` in the ID field
- **Relation** : `viewer`

The store, models, and tuples can be created programmatically using the [OpenFGA Java SDK](https://github.com/openfga/java-sdk). This is defined in `src/main/java/rag/FGAInit.java` and can be run using the following command:

```bash
./gradlew runFGAInit
```

Once done, copy the store ID from the console and update the `.env` file with the store ID.

> You can also use the Auth0 FGA dashboard to create the store, models, and tuples. Navigate to **Model Explorer**. You’ll need to update the model information with the above schema.
> Next, navigate to the **Tuple Management** section and click **+ Add Tuple**, fill in the details for both tuples detailed above.

## Test the Application

Now that you have set up the application and the FGA store, you can run the application using the following command:

```bash
./gradlew run
```

The application will start with the query, `Show me forecast for ZEKO?` Since this information is in a private document, and `user2` does not have access to this document, the application will not be able to retrieve it. The FGA retriever will filter out the private document from the vector store results and, hence, print a similar output.

```bash
The provided context does not include specific forecasts or projections for 
Zeko Advanced Systems Inc. ...
```

If you change the query to something that is available in the public document, the application will be able to retrieve the information.

Now change the user to `user1` and run the application again:

```bash
./gradlew run
```

This time, you should see a response containing the forecast information since we have a tuple that defines the `viewer` relation for `user1` to the `private-doc` object.

Congratulations! You have run a simple RAG application using LangChain4j and secured it using OpenFGA.

## Implementation Patterns for Securing RAG Systems

Here are some common patterns for securing RAG systems using FGA:

1.  **Pre-filtering Pattern**: Check permissions before searching the vector database. This can be done using the [OpenFGA list objects request](https://openfga.dev/docs/getting-started/perform-list-objects) and metadata filtering feature of supported vector databases.
2.  **Post-filtering Pattern**: Perform a vector database search first and then filter the results by permissions. This can be done using the [OpenFGA batch check request](https://openfga.dev/docs/getting-started/perform-check#03-calling-batch-check-api) and custom retrievers or re-rankers. This is what we just did in the example above.
3.  **Hybrid Pattern**: Combine pre-filtering and post-filtering patterns to achieve the best of both worlds.

## Learn More About Auth0 for GenAI, Auth0 FGA and GenAI

In this post, you learned how to secure a LangChain4j-based RAG application using [OpenFGA](https://fga.dev/). We invite you to check out the [OpenFGA code on GitHub](https://a0.to/fga_repo). Visit [Auth for GenAI](https://a0.to/ai-content) to learn more about how Auth0 can help you secure your GenAI applications.

> [Sign up for Auth for GenAI](https://a0.to/ai-content) in Developer Preview.

Before you go, we have some great news to share: we are working on more content and sample apps in collaboration with amazing GenAI frameworks like [LlamaIndex](https://www.llamaindex.ai/), [LangChain](https://www.langchain.com/), [CrewAI](https://www.crewai.com/), [Vercel AI SDK](https://sdk.vercel.ai/), and [GenKit](https://firebase.google.com/docs/genkit). [Auth for GenAI](https://auth0.com/blog/auth-for-genai/) is our upcoming product to help you protect your users' information in GenAI-powered applications. Make sure to join the [Auth0 Lab Discord server](http://a0.to/auth0-lab-discord) to hear more and ask questions.

Secure Java AI Agents: Authorization for RAG Using LangChain4j and Auth0 FGA

jess-author

Jessica Temporal is a Senior Developer Advocate at Auth0 and co-founder of the first Brazilian data science podcast called <a target='_blank' href='https://pizzadedados.com'><strong>Pizza de Dados</strong></a>. She’s the author of <a target='_blank' href='https://www.amazon.com/dp/B0CDNX6NS7/ref=sr_1_1?&_encoding=UTF8&tag=jesstempora0e-20&linkCode=ur2&linkId=efc4229f6b816609dfce4f185781d99a&camp=1789&creative=9325' ><strong>The Big Git Microbook</strong></a>, creator of <a target='_blank' href='https://gitfichas.com/en'><strong>GitFichas.com</strong></a>, and a part of <a target='_blank' href='https://www.linkedin.com/learning/instructors/jessica-temporal'><strong>LinkedIn Learning</strong></a> instructors team. Jessica helped develop the AI that identifies possible unlawful expenses from Brazilian politicians. She is part of the PyLadies initiative that works to improve diversity and inclusion in technology. In her free time, she likes to knit, play video games, use her 3D printer, and is learning to create 3D models.

Sr. Developer Advocate

Jessica Temporal

Generative AI is everywhere, and since not every user has access to all data, learn how to use retrieval-augmented generation with fine-grained authorization to ensure that private information stays private.

Build Trustworthy AI: Implementing Access Control for RAG Systems Using FGA

I stream, blog, and make youtube videos about tech stuff. I love coding, I love AI, and I love building stuff!

Juan Cruz Martinez

Learn how to use Okta FGA to secure your LangChain RAG agent in Python.

Retrieval-Augmented Generation (RAG) is a great way to create AI-driven applications that deliver more accurate, context-rich answers. Instead of relying purely on pre-trained knowledge, a RAG system retrieves domain-specific data (like PDFs, knowledge bases, or markdown files) and passes it into a language model. This is *awesome* when your data is public or freely shareable. But what happens if some of that data is *restricted* or *confidential*? That’s where fine-grained authorization comes in!

In this post, we’ll walk through:

1. **Why** authorization is crucial in RAG
2. **What** OpenFGA is, and how it helps you enforce fine-grained permissions
3. **How** to build a secure pipeline that filters out unauthorized documents **before** they reach your language model

## The Challenge of Authorization in RAG Systems

RAG systems retrieve relevant information from large datasets and use that information to enhance the AI model responses. While this functionality is powerful, it raises a significant challenge: **ensuring that each user only accesses the information they are authorized to see**. Authorization must be:

* **Accurate**: Only correct data gets through.
* **Performant**: Works quickly, even with large datasets.
* **Adaptive**: Updates with dynamic organizational changes, role updates, or any changes in the relationship between the information and the users.

A secure RAG system needs to enforce fine-grained access control without sacrificing speed or scalability. Roles might change, projects can be reassigned, and permissions could evolve over time. Handling all this efficiently is key to building a truly secure and robust RAG application.

## Proposed Solution: Secure Retrieval with LangChain, OpenAI, and OpenFGA

The solution employs a workflow that integrates document retrieval, user-specific authorization filtering, and Large Language Model (LLM) response generation.

We’ll build a pipeline that uses:

* **LangChain**: A helpful toolkit for chaining the steps of a RAG pipeline (loading data, embeddings, retrieval, prompting).
* **LLMs** (OpenAI or any other): For both embeddings (similarity search) and text generation.
* **OpenFGA**: A fine-grained authorization service that checks, for each document, if a user has the “can_view” relationship.

### LangChain

LangChain is a library that streamlines building LLM applications by assembling different components—like loading documents, vector stores, and prompt templates. This structure makes it easier to read, maintain, and modify your RAG pipeline.

### LLMs

Although we’ll demonstrate OpenAI’s embeddings and Chat APIs, you can easily replace them with other providers (e.g., Anthropic, LLaMA, or Azure OpenAI). The main tasks here are:

1. **Embedding** documents so you can do similarity searches.
2. **Generating** text from the retrieved documents to answer the user’s query.

### FGA and Okta FGA

**FGA (Fine-Grained Authorization)** is about controlling _who can do what_ with _which resources_, down to an individual level. In a typical role-based system, you might say, “Admins can see everything, and Regular Users can see some subset.” But in a real-world app—especially one that deals with many documents—this might not be flexible enough.

**OpenFGA (and Okta FGA)** addresses this by letting you define authorization _relationships_. The relationships defined in the authorization model can be either direct or indirect. Simply put, direct relationships are directly assigned between a consumer and a resource (we call them user and object) and stored in the database. Indirect relationships are the relationships we can infer based on the data and the authorization model.

If you would like to learn the basics of using FGA for RAG, check out this blog post on [RAG and Access Control: Where Do You Start?](https://auth0.com/blog/rag-and-access-control-where-do-you-start/).

<include
 src="LinkCard"
 title="RAG and Access Control: Where Do You Start?"
 link="https://auth0.com/blog/rag-and-access-control-where-do-you-start/"
 description="Learn how to get started with Okta FGA for a RAG application."
 img="https://images.ctfassets.net/23aumh6u8s0i/79NCborGcFIrAMeaV7G8oi/f9c588c2d05f0828d447e72e1fe825cc/Latam01.jpg"
/>

## Implementation: Step-by-Step

Below is a sample setup in Python. We’ll keep it simple so you can see the big picture. Feel free to adapt for your chosen LLM or a more robust data store.

### Set Up Prerequisites

To follow this tutorial and secure your application, you’ll need the following:

* Python 3.8.1 or newer.
* An Okta FGA account. If you don’t have one, you can [create one for free](https://dashboard.fga.dev/).
* An [OpenAI account and API key](https://platform.openai.com/).

### Download and install the sample code

To get started, clone the[ auth0-ai-samples](https://a0.to/auth0-ai-samples) repository from GitHub:

```
git clone https://github.com/auth0-samples/auth0-ai-samples.git
cd auth0-ai-samples/authorization-for-rag/langchain-python
# Create a virtual env
python -m venv venv
# Activate the virtual env
source ./venv/bin/activate
# Install dependencies
pip install -r requirements.txt
```

The application is written in Python and is structured as follows:

* `main.py` &mdash; The main entry point of the application, and it is where we define the RAG pipeline
* `docs/*.md` &mdash; Sample markdown files to be used as context for the LLM. There are two types of docs, public and private. Private documents are only accessible to certain individuals.
* `helpers/memory_store.py` &mdash; Creates an in-memory vector store that acts as the base retriever in the chain.
* `helpers/read_documents.py` &mdash; Utility to read the markdown files from the `docs` folder.
* `scripts/fga_init.py` &mdash; Utility to initialize the OktaFGA authorization model and sample data.

### RAG Pipeline

The `main.py` file defines the RAG pipeline using `Langchain` to interact with the underlying LLM model and retrieve data from our context. In your project, your data may be sourced from different platforms and systems, make sure you check the proper documentation about [loaders in the Langchain ecosystem](https://python.langchain.com/v0.1/docs/modules/data_connection/).

The following diagram represents the RAG architecture we are defining:

![LangChain RAG architecture](https://images.ctfassets.net/23aumh6u8s0i/4ReiQ2qnjzYRR18KujkHwe/4c4738866ad275b26aed2d7b9c8d3661/langchain-rag.png)

Now let’s talk Python:

```python
class RAG:
    def __init__(self):
        documents = read_documents()
        self.vector_store = MemoryStore.from_documents(documents)
        self.prompt = ChatPromptTemplate.from_template(
            """You are an assistant for question-answering tasks. Use the following pieces of retrieved context to answer the question. If you don't know the answer, just say that you don't know. Use three sentences maximum and keep the answer concise.\\nQuestion: {question}\\nContext: {context}\\nAnswer:"""
        )

        self.llm = ChatOpenAI(model="gpt-4o-mini")

    def query(self, user_id: str, question: str):
        chain = (
            {
                "context": FGARetriever(
                    retriever=self.vector_store.as_retriever(),
                    build_query=lambda doc: ClientBatchCheckItem(
                        user=f"user:{user_id}",
                        object=f"doc:{doc.metadata.get('id')}",
                        relation="viewer",
                    ),
                ),
                "question": RunnablePassthrough(),
            }
            | self.prompt
            | self.llm
            | StrOutputParser()
        )

        return chain.invoke(question)
```

Let’s break that down.

The `RAG` class first initializes a vector store and reads the documents using the helper functions. It also defines a system prompt for our use case and the LLM model in use, in our case, `gpt-40-mini`.

The class also defines a method query that builds the chain by piping the `FGARetriever` instance with the prompt and the LLM model. `FGARetriever` is provided by the [Auth0 AI SDK for Python](https://github.com/auth0-lab/auth0-ai-python).

The `FGARetriever` is designed to abstract the base retriever from the FGA query logic. The `build_query` argument lets us specify how to query our FGA model, in this case, by asking if the user is a viewer of the document.

```python
build_query=lambda doc: ClientBatchCheckItem(
    user=f"user:{user_id}",
    object=f"doc:{doc.metadata.get('id')}",
    relation="viewer",
),
```

With this design, you can plug any Langchain retriever, combined with checks for any FGA model query.

### Create an OktaFGA Account

If you already have an [Auth0 account](https://auth0.com/), you can use the same credentials to log in to the Okta FGA dashboard at [https://dashboard.fga.dev](https://dashboard.fga.dev/). If you don't have an Auth0 account, hop over to [https://dashboard.fga.dev](https://dashboard.fga.dev/) and create a free account.

Once you are logged in, you should see a dashboard similar to the one below.

![The Okta FGA Dashboard getting started page](https://images.ctfassets.net/23aumh6u8s0i/38P8Vu4qsBvCGrZ4ePew3C/090d068c68b785311a00aa788b92579b/fga-get-started.png)

> When you log into the Okta FGA dashboard for the first time, you may be asked to create a new store. This store will serve as the home for your authorization model and all the data the engine requires to make authorization decisions. Simply pick a name and create your store to get started.

### Create an OktaFGA Client

Once you are in the dashboard, you’ll need a client to make API calls to OktaFGA. To create a client, navigate to **Settings**, and in the **Authorized Clients** section, click **Create Client**. Give your client a name, mark all three client permissions, and then click **Create**.

![Create FGA client](https://images.ctfassets.net/23aumh6u8s0i/2KQc8ACR0MDTqftUgUSXpX/5b779bcece07a69ac775de0fc760c208/Uploaded_from_RAG_and_Access_Control__Where_Do_You_Start)

When you create the client, OktaFGA will provide you with some data like a Store ID, a Client ID and a Client Secret. Don’t close that yet, you’ll need those values next.

At the root of the project, there’s a `.env.example` file. Copy the file and paste it as `.env`. Then, open the file and edit the three FGA-related variables with the values provided by OktaFGA. When you are ready, click continue, and the modal will display the values for the missing variables (`FGA_API_URL` and `FGA_API_AUDIENCE`).

At this step, you can also add your [OpenAI API Key](https://help.openai.com/en/articles/4936850-where-do-i-find-my-openai-api-key), which you’ll need to run the demo.

### Configure the OktaFGA model

Now that the application is set up, you can run the provided script to initialize the model and some sample data. To set things up, run:

```
python ./scripts/fga_init.py
```

You can verify that the script worked by navigating to the model explorer page in OktaFGA. The following model should now have been created:

```
model
 schema 1.1

type user

type doc
 relations
   define owner: [user]
   define viewer: [user, user:*]
```

> You can visit the OktaFGA documentation to learn more about [modeling OktaFGA and creating an authorization model](https://docs.fga.dev/modeling).

On top of the model, the script also created two tuples. Tuples in OktaFGA define the relationships between the types.

First, it defined a tuple to give all users access to the public doc:

- **User** :` user:*`
- **Object** : `public-doc`
- **Relation** : `viewer`

Then, it created a second tuple to give the admin user access to the private doc:

- **User** :` user:admin`
- **Object** : `private-doc`
- **Relation** : `viewer`

> You can visit the OktaFGA documentation to learn more about [tuples and how to create them](https://docs.fga.dev/content/getting-started/new-getting-started#add-tuples).

### Query the Chain

Invoke the chain to process a query and generate a response.

```python
rag = RAG()

question = "What is the forecast for ZEKO?"

# Juan only has access to public docs
response = rag.query("juan", question)
print("Response to Juan:", response)

# Admin has access to all docs
response = rag.query("admin", question)
print("Response to Admin:", response)
```

Fantastic! You now know how to build and scale a secure RAG with Python and LangChain. It’s time to test things out.

To run the chain, simply call the `main.py` file:

```
python main.py
```

If you follow the steps, you’ll see a response like the following:

```
Response to Juan: The retrieved context does not provide any specific forecast or predictions for ZEKO (Zeko Advanced Systems Inc.). It mainly outlines the company's mission, technologies, and products without detailing any financial or market forecasts. Therefore, I don't know the forecast for ZEKO.

Response to Admin: The forecast for Zeko Advanced Systems Inc. (ZEKO) for fiscal year 2025 is generally bearish. Projected revenue growth is expected to remain subdued at 2-3%, with net income growth projected at 1-2%, primarily due to margin pressures and competitive challenges. Investors should be cautious, given the potential headwinds the company faces.
```

As you can see, `Juan` couldn’t retrieve any “protected” data, while the user `admin` got an accurate response to the prompt.

## Conclusion

You’ve seen how to build a **secure** RAG system that respects user permissions at the **document level** using **OpenFGA**. This approach prevents data leaks, meets security and compliance requirements, and stays flexible—perfect for businesses dealing with a large pool of sensitive information.

Feel free to adapt this flow for other vector stores, different LLM providers, or your own organizational rules. By coupling RAG’s retrieval power with fine-grained authorization, you’ll build AI-driven apps that are both insightful and safe.

Before you go, we have some great news to share: we are working on more content and sample apps in collaboration with amazing GenAI frameworks like [LlamaIndex](https://www.llamaindex.ai/), [LangChain](https://www.langchain.com/),[ CrewAI](https://www.crewai.com/), [Vercel AI](https://sdk.vercel.ai/), and others. 

[Auth for GenAI](https://a0.to/ai-content) is our upcoming product to help you protect your user's information in GenAI-powered applications.

Make sure to join the [Auth0 Lab Discord server](http://a0.to/auth0-lab-discord) to hear more and ask questions.

Happy coding!

Building a Secure RAG with Python, LangChain, and OpenFGA

Learn how to use Auth0 FGA to secure your LlamaIndex RAG agent on Node.js.

Generative Artificial Intelligence (GenAI) has massively changed the software landscape and AI agents are all the rage now. AI agents can also be used for complex Retrieval-Augmented Generation (RAG) systems where additional context can be provided to the Large Language Model (LLM) by retrieving updated or domain-specific data from a database or a search engine. This technique can reduce hallucinations and improve accuracy of the LLM. AI agents can also use RAG as a tool to perform complex workflows.

[Sensitive Information Disclosure](https://genai.owasp.org/llmrisk/llm022025-sensitive-information-disclosure/) is a common issue that plagues RAG-based systems. We don't want the LLM to accidentally access or expose sensitive data from a database. Traditional Role-Based Access Control (RBAC) systems are not enough to secure RAG applications and agents. This is where [Fine-Grained Authorization (FGA)](https://fga.dev/) shines as a solution.

If you would like to learn the basics of using FGA for RAG, check out this blog post on [RAG and Access Control: Where Do You Start?](https://auth0.com/blog/rag-and-access-control-where-do-you-start/).

<include
  src="LinkCard"
  title="RAG and Access Control: Where Do You Start?"
  link="https://auth0.com/blog/rag-and-access-control-where-do-you-start/"
  description="Learn how to get started with Auth0 FGA for a RAG application."
  img="https://images.ctfassets.net/23aumh6u8s0i/79NCborGcFIrAMeaV7G8oi/f9c588c2d05f0828d447e72e1fe825cc/Latam01.jpg"
/>

In this tutorial, we will build a simple RAG agent using the [LlamaIndex.TS](https://ts.llamaindex.ai/), which is the JavaScript version of the popular [LlamaIndex](https://www.llamaindex.ai/) framework, and secure it using Auth0 FGA.

## Prerequisites

> **This tutorial was created with the following tools and services**:
>
> - [NodeJS v20](https://nodejs.org/en/download)
> - An Auth0 FGA account. [Create one](https://dashboard.fga.dev).
> - An OpenAI account and API key. [Create one](https://platform.openai.com).

## What Does LlamaIndex Do?

[LlamaIndex](https://www.llamaindex.ai/) is a flexible framework for building AI agents and RAG applications. It provides Python and JavaScript SDKs to interact with a variety of LLMs and databases. LlamaIndex can be used to build AI agents and workflows that can interact with the user, retrieve data from a database, and generate responses using an LLM. It also provides tools like [LlamaParse](https://www.llamaindex.ai/llamaparse) to transform unstructured data into LLM-optimized formats and LlamaCloud to store and retrieve LLM-ready data from the cloud.

## Set up a LlamaIndex RAG Application

To get started, clone the [**auth0-ai-samples**](https://a0.to/auth0-ai-samples) repository from GitHub:

```bash
git clone https://github.com/auth0-samples/auth0-ai-samples.git
cd auth0-ai-samples/authorization-for-rag/llamaindex-agentic-js
# install dependencies
npm install
```

The application is written in TypeScript for the Node.js platform and is structured as follows:

- `index.ts`: The main entry point of the application that defines the RAG pipeline.
- `assets/docs/*.md`: Sample markdown documents that will be used as context for the LLM. We have public and private documents for demonstration purposes.
- `scripts/fga-init.ts`: Utility to initialize the Auth0 FGA authorization model.

Let us look at the important bits and pieces before we run the application.

### RAG pipeline

The RAG pipeline is defined in `index.ts`. It uses LlamaIndex Agents to interact with the underlying LLM and retrieve data from the database. The pipeline is defined as follows:

```typescript
/** index.ts **/
const user = 'user1';
// 1. Read and load documents from the assets folder
const documents = await new SimpleDirectoryReader().loadData('./assets/docs');
// 2. Create an in-memory vector store from the documents using the default OpenAI embeddings
const vectorStoreIndex = await VectorStoreIndex.fromDocuments(documents);
// 3. Create a retriever that uses FGA to gate fetching documents on permissions.
const retriever = FGARetriever.create({
  // Set the similarityTopK to retrieve more documents as SimpleDirectoryReader creates chunks
  retriever: vectorStoreIndex.asRetriever({ similarityTopK: 30 }),
  // FGA tuple to query for the user's permissions
  buildQuery: (document) => ({
    user: `user:${user}`,
    object: `doc:${document.metadata.file_name.split('.')[0]}`,
    relation: 'viewer',
  }),
});
// 4. Create a query engine and convert it into a tool
const queryEngine = vectorStoreIndex.asQueryEngine({ retriever });
const tools = [
  new QueryEngineTool({
    queryEngine,
    metadata: {
      name: 'zeko-internal-tool',
      description: `This tool can answer detailed questions about ZEKO.`,
    },
  }),
];

// 5. Create an agent using the tools array and OpenAI GPT-4 LLM
const agent = new OpenAIAgent({ tools });

// 6. Query the agent
let response = await agent.chat({ message: 'Show me forecast for ZEKO?' });
```

Here is a visual representation of the RAG architecture

![LlamaIndex RAG architecture](https://images.ctfassets.net/23aumh6u8s0i/lGssMbzaSU4J0zXG4bFcC/21f2abbda55a13640350d637609469f7/llamaindex-rag.jpg)

### FGA retriever

The `FGARetriever` class filters documents based on the authorization model defined in Auth0 FGA and will be available as part of the [**auth0-ai-js**](https://github.com/auth0-lab/auth0-ai-js) SDK. This retriever is a post-search filter ideal for scenarios where you already have documents in a vector store and want to filter the vector store results based on the user's permissions. Assuming the vector store already narrows down the documents to a few, the FGA retriever will further narrow down the documents to only the ones to which the user has access.

The alpha version of the retriever can be installed using the following command:

```bash
npm install "https://github.com/auth0-lab/auth0-ai-js.git#alpha-2" --save
```

The build query function is used to construct the query to the FGA store. The query is constructed using the user, object, and relation. The user is the user ID, the object is the document ID or the document name, and the relation is the permission that the user must have on the document.

```typescript
buildQuery: (document) => ({
  user: `user:${user}`,
  object: `doc:${document.metadata.file_name.split(".")[0]}`,
  relation: "viewer",
}),
```

### Retrieval Agent

1. The `queryEngine` is created from the vector store index and configured to use our custom FGA retriever. The query engine handles searching through documents and retrieving relevant information based on user queries.

2. The `tools` array contains a `QueryEngineTool` that wraps our query engine. The tool provides a structured interface for the agent to access the query engine's capabilities.

3. The `agent` is created using OpenAI's GPT-4 model and the tools array. The agent acts as an intelligent interface between the user and the tools - it understands natural language queries, determines when to use the query engine tool, and formulates responses based on the retrieved information.

```typescript
/** index.ts **/
const queryEngine = vectorStoreIndex.asQueryEngine({ retriever });
const tools = [
  new QueryEngineTool({
    queryEngine,
    metadata: {
      name: 'zeko-internal-tool',
      description: `This tool can answer detailed questions about ZEKO.`,
    },
  }),
];

// 5. Create an agent using the tools array and OpenAI GPT-4 LLM
const agent = new OpenAIAgent({ tools });
```

## Set up an FGA Store

In the [Auth0 FGA dashboard](https://dashboard.fga.dev/), navigate to **Settings**, and in the **Authorized Clients** section, click **+ Create Client**. Give your client a name, mark all three client permissions, and then click **Create**.

![Create FGA client](https://images.ctfassets.net/23aumh6u8s0i/2KQc8ACR0MDTqftUgUSXpX/5b779bcece07a69ac775de0fc760c208/Uploaded_from_RAG_and_Access_Control__Where_Do_You_Start)

Once your client is created, you’ll see a modal containing Store ID, Client ID, and Client Secret.

Add `.env` file with the following content to the root of the project. Click **Continue** to see the `FGA_API_URL` and `FGA_API_AUDIENCE`.

```
# OpenAI
OPENAI_API_KEY=<your-openai-api-key>

# Auth0 FGA
FGA_STORE_ID=<your-fga-store-id>
FGA_CLIENT_ID=<your-fga-store-client-id>
FGA_CLIENT_SECRET=<your-fga-store-client-secret>
# Required only for non-US regions
FGA_API_URL=https://api.xxx.fga.dev
FGA_API_AUDIENCE=https://api.xxx.fga.dev/
```

[Check the instructions here to find your OpenAI API key](https://help.openai.com/en/articles/4936850-where-do-i-find-my-openai-api-key).

Next, navigate to **Model Explorer**. You’ll need to update the model information with this:

```text
model
  schema 1.1

type user

type doc
  relations
    define owner: [user]
    define viewer: [user, user:*]
```

Remember to click **Save**.

> Check out [this documentation to learn more about creating an authorization model in FGA](https://docs.fga.dev/modeling).

Now, to have access to the public information, you’ll need to add a tuple on FGA. Navigate to the **Tuple Management** section and click **+ Add Tuple**, fill in the following information:

- **User** : `user:*`
- **Object** : select doc and add `public-doc` in the ID field
- **Relation** : `viewer`

A tuple signifies a user’s relation to a given object. For example, the above tuple implies that all users can view the `public-doc` object.

> Alternatively, you can use the `scripts/fga-init.ts` script to initialize the FGA store with the model and tuple.
> Run the `npm run fga:init` command after setting up the `.env` file.

## Test the Application

Now that you have set up the application and the FGA store, you can run the application using the following command:

```bash
npm start
```

The application will start with the query, `Show me forecast for ZEKO?` Since this information is in a private document, and we haven't defined a tuple with access to this document, the application will not be able to retrieve it. The FGA retriever will filter out the private document from the vector store results and, hence, print a similar output.

```text
The provided context does not include specific forecasts or projections for Zeko Advanced Systems Inc. ...
```

If you change the query to something that is available in the public document, the application will be able to retrieve the information.

Now, to have access to the private information, you’ll need to update your tuple list. Go back to the Auth0 FGA dashboard in the **Tuple Management** section and click **+ Add Tuple**, fill in the following information:

- **User** : `user:user1`
- **Object** : select doc and add `private-doc` in the ID field
- **Relation** : `viewer`

Now click **Add Tuple** and then run the script again:

```bash
npm start
```

This time, you should see a response containing the forecast information since we added a tuple that defines the `viewer` relation for `user1` to the `private-doc` object.

Congratulations! You have run a simple RAG application using LlamaIndex and secured it using Auth0 FGA.

## Learn More about Auth for GenAI, Auth0 FGA and GenAI

In this post, you learned how to secure a LlamaIndex-based RAG application using Auth0's [Auth for GenAI](https://a0.to/ai-content) and [Auth0 FGA](https://fga.dev/). Auth0 FGA is built on top of OpenFGA, which is open-source. We invite you to check out the [OpenFGA code on GitHub](https://a0.to/fga_repo).

Before you go, we have some great news to share: we are working on more content and sample apps in collaboration with amazing GenAI frameworks like [LlamaIndex](https://www.llamaindex.ai/), [LangChain](https://www.langchain.com/), [CrewAI](https://www.crewai.com/), [Vercel AI SDK](https://sdk.vercel.ai/), and [GenKit](https://firebase.google.com/docs/genkit). [Auth for GenAI](https://auth0.com/blog/auth-for-genai/) is our upcoming product to help you protect your user's information in GenAI-powered applications. Make sure to join the [Auth0 Lab Discord server](http://a0.to/auth0-lab-discord) to hear more and ask questions.

Build a Secure RAG Agent Using LlamaIndex and Auth0 FGA on Node.js

Learn how to use Auth0 FGA to secure your LangChain RAG agent on Node.js

Like it or not, Generative Artificial Intelligence (GenAI) has massively changed the software landscape. Retrieval-Augmented Generation (RAG) is a cornerstone of GenAI-powered applications. It is an architecture that improves the accuracy of Large Language Models (LLMs) by providing more context to the model. This is done by retrieving updated or domain-specific data from a database or a search engine. This is a powerful technique that can be used to improve the accuracy of LLMs in a variety of applications.

Security is even more paramount when it comes to RAG-based systems, as we don't want the LLM to accidentally access or expose sensitive data. Traditional Role-Based Access Control (RBAC) systems are not enough to secure RAG applications. This is where [Fine-Grained Authorization (FGA)](https://fga.dev/) comes to the rescue.

If you would like to learn the basics of using FGA for RAG, check out this blog post on [RAG and Access Control: Where Do You Start?](https://auth0.com/blog/rag-and-access-control-where-do-you-start/)

<include
  src="LinkCard"
  title="RAG and Access Control: Where Do You Start?"
  link="https://auth0.com/blog/rag-and-access-control-where-do-you-start/"
  description="Learn how to get started with Auth0 FGA for a RAG application."
  img="https://images.ctfassets.net/23aumh6u8s0i/79NCborGcFIrAMeaV7G8oi/f9c588c2d05f0828d447e72e1fe825cc/Latam01.jpg"
/>

In this tutorial, we will build a simple RAG agent using the [LangChain](https://www.langchain.com/) framework and secure it using Auth0 FGA.

<AmpContent>
<amp-youtube
    data-videoid="u81b6nsDoPA"
    layout="responsive"
    width="480" height="270">
</amp-youtube>
</AmpContent>

<NonAmpContent>
<div
  className='embed-container'
  style="position: relative; padding-bottom: 56.25%; height: 0; overflow: hidden; max-width: 100%;margin-bottom:40px;">
    <iframe
      style="position: absolute; top: 0; left: 0; width: 100%; height: 100%;"
      src='https://www.youtube.com/embed/u81b6nsDoPA'
      frameborder='0' allowfullscreen>
    </iframe>
</div>
</NonAmpContent>


## Prerequisites

> **This tutorial was created with the following tools and services**:
>
> - [NodeJS v20](https://nodejs.org/en/download)
> - An Auth0 FGA account. [Create one](https://dashboard.fga.dev).
> - An OpenAI account and API key. [Create one](https://platform.openai.com).

## Why LangChain?

[LangChain](https://www.langchain.com/) is one of the leading frameworks for building context-aware reasoning applications, including RAG applications. It is open-source and provides a simple API to interact with the underlying LLMs. It supports a variety of LLMs, including OpenAI's GPT models, Google VertexAI, Anthropic, and more. It also provides a simple way to retrieve data from a database or a search engine to provide more context to the LLM. For this tutorial, we will use [LangGraph](https://www.langchain.com/langgraph) from LangChain to build an agentic RAG application. LangGraph is a powerful tool that allows you to build agents that can handle complex tasks, including RAG applications. It provides advanced features like workflows, persisted context, and streaming support.

## Set up a LangGraph RAG application

To get started, clone the [**auth0-ai-samples**](https://a0.to/auth0-ai-samples) repository from GitHub:

```bash
git clone https://github.com/auth0-samples/auth0-ai-samples.git
cd auth0-ai-samples/authorization-for-rag/langgraph-agentic-js
# install dependencies
npm install
```

The application is written in TypeScript for the Node.js platform and is structured as follows:

- `index.ts`: The main entry point of the application that defines the RAG pipeline.
- `assets/docs/*.md`: Sample markdown documents that will be used as context for the LLM. We have public and private documents for demonstration purposes.
- `helpers/read-documents.ts`: Utility to load markdown files as documents.
- `helpers/langchain.ts`: Defines the LangGraph retrieval agent.
- `scripts/fga-init.ts`: Utility to initialize the Auth0 FGA authorization model.

Let us look at the important bits and pieces before we run the application.

### RAG pipeline

The RAG pipeline is defined in `index.ts`. It uses the LangGraph framework to interact with the underlying LLM and retrieve data from the database. The pipeline is defined as follows:

```js
/** index.ts **/
const user = 'user1';
// 1. Read and load documents from the assets folder
const documents = await readDocuments();
// 2. Create an in-memory vector store from the documents for OpenAI models.
const vectorStore = await MemoryVectorStore.fromDocuments(
  documents,
  new OpenAIEmbeddings({ model: 'text-embedding-3-small' }),
);
// 3. Create a retriever that uses FGA to gate fetching documents on permissions.
const retriever = FGARetriever.create({
  retriever: vectorStore.asRetriever(),
  // FGA tuple to query for the user's permissions
  buildQuery: (doc) => ({
    user: `user:${user}`,
    object: `doc:${doc.metadata.id}`,
    relation: 'viewer',
  }),
});
// 4. Convert the retriever into a tool for an agent.
const fgaTool = retriever.asJoinedStringTool();
// 5. The agent will call the tool, rephrasing the original question and
// populating the "query" argument, until it can answer the user's question.
const retrievalAgent = RetrievalAgent.create([fgaTool]);
// 6. Query the retrieval agent with a prompt
const answer = await retrievalAgent.query('Show me forecast for ZEKO?');
```

Here is a visual representation of the RAG architecture:

![LangChain RAG architecture](https://images.ctfassets.net/23aumh6u8s0i/707VuhSJ2FqDBTrQNP6rTl/0fcd90a881441369e1466b109c02686c/langgraph-rag.jpg)

### FGA retriever

The `FGARetriever` class filters documents based on the authorization model defined in Auth0 FGA and will be available as part of the [**auth0-ai-js**](https://github.com/auth0-lab/auth0-ai-js) SDK. This retriever is ideal for scenarios where you already have documents in a vector store and want to filter the vector store results based on the user's permissions. Assuming the vector store already narrows down the documents to a few, the FGA retriever will further narrow down the documents to only the ones that the user has access to. 

The alpha version of the retriever can be installed using the following command:

```bash
npm install "https://github.com/auth0-lab/auth0-ai-js.git#alpha-1" --save
```

The build query function is used to construct the query to the FGA store. The query is constructed using the user, object, and relation. The user is the user ID, the object is the document ID, and the relation is the permission that the user has on the document.

```typescript
buildQuery: (doc) => ({
  user: `user:${user}`,
  object: `doc:${doc.metadata.id}`,
  relation: 'viewer',
}),
```

### Retrieval Agent

The `RetrievalAgent` class configures a LangGraph retrieval agent that uses the FGA retriever tool and an OpenAI model to answer the user's question. The agent will call the tool to get context documents in this case.

```typescript
/** helpers/langchain.ts **/
static create(tools: StructuredToolInterface[]) {
  // Create a retrieval agent that has access to the retrieval tool.
  const retrievalAgent = createReactAgent({
    llm: new ChatOpenAI({ temperature: 0, model: 'gpt-4o-mini' }),
    tools,
    stateModifier: [
      "Answer the user's question only based on context retrieved from provided tools.",
      'Only use the information provided by the tools.',
      'If you need more information, ask for it.',
    ].join(' '),
  });
  return new RetrievalAgent(retrievalAgent);
}
```

## Set up an FGA store

In the [Auth0 FGA dashboard](https://dashboard.fga.dev/), navigate to **Settings**, and in the **Authorized Clients** section, click **+ Create Client**. Give your client a name, mark all three client permissions, and then click **Create**.


![Create FGA client](https://images.ctfassets.net/23aumh6u8s0i/2KQc8ACR0MDTqftUgUSXpX/5b779bcece07a69ac775de0fc760c208/Uploaded_from_RAG_and_Access_Control__Where_Do_You_Start)


Once your client is created, you’ll see a modal containing Store ID, Client ID, and Client Secret.

Add `.env` file with the following content to the root of the project. Click **Continue** to see the `FGA_API_URL` and `FGA_API_AUDIENCE`.

```
# OpenAI
OPENAI_API_KEY=<your-openai-api-key>

# Auth0 FGA
FGA_STORE_ID=<your-fga-store-id>
FGA_CLIENT_ID=<your-fga-store-client-id>
FGA_CLIENT_SECRET=<your-fga-store-client-secret>
# Required only for non-US regions
FGA_API_URL=https://api.xxx.fga.dev
FGA_API_AUDIENCE=https://api.xxx.fga.dev/
```

[Check the instructions here to find your OpenAI API key](https://help.openai.com/en/articles/4936850-where-do-i-find-my-openai-api-key).

Next, navigate to **Model Explorer**. You’ll need to update the model information with this:

```text
model
  schema 1.1

type user

type doc
  relations
    define owner: [user]
    define viewer: [user, user:*]
```

Remember to click **Save**.

> Check out [this documentation to learn more about creating an authorization model in FGA](https://docs.fga.dev/modeling).

Now, to have access to the public information, you’ll need to add a tuple on FGA. Navigate to the **Tuple Management** section and click **+ Add Tuple**, fill in the following information:

- **User**: `user:*`
- **Object**: select doc and add `public-doc` in the ID field
- **Relation**: `viewer`

A tuple signifies a user’s relation to a given object. For example, the above tuple implies that all users can view the `public-doc` object.

> Alternatively, you can use the `scripts/fga-init.ts` script to initialize the FGA store with the model and tuple.
> Run the `npm run fga:init` command after setting up the `.env` file.

## Test the application

Now that you have set up the application and the FGA store, you can run the application using the following command:

```bash
npm start
```

The application will start with the query, `Show me forecast for ZEKO?` Since this information is in a private document, and we haven't defined a tuple with access to this document, the application will not be able to retrieve it. The FGA retriever will filter out the private document from the vector store results and, hence, print a similar output.

```text
The provided context does not include specific forecasts or projections for Zeko Advanced Systems Inc. ...
```

If you change the query to something that is available in the public document, the application will be able to retrieve the information.

Now, to have access to the private information, you’ll need to update your tuple list. Go back to the Auth0 FGA dashboard in the **Tuple Management** section and click **+ Add Tuple**, fill in the following information:

- **User**: `user:user1`
- **Object**: select doc and add `private-doc` in the ID field
- **Relation**: `viewer`

Now click **Add Tuple** and then run the script again:

```bash
npm start
```

This time, you should see a response containing the forecast information since we added a tuple that defines the `viewer` relation for `user1` to the `private-doc` object.

Congratulations! You have run a simple RAG application using LangChain and secured it using Auth0 FGA.

## Learn more about Auth0 for GenAI, Auth0 FGA and GenAI

In this post, you learned how to secure a LangChain-based RAG application using Auth0's [Auth for GenAI](https://a0.to/ai-content) and [Auth0 FGA](https://fga.dev/). Auth0 FGA is built on top of OpenFGA, which is open-source. We invite you to check out the [OpenFGA code on GitHub](https://a0.to/fga_repo).

Before you go, we have some great news to share: we are working on more content and sample apps in collaboration with amazing GenAI frameworks like [LlamaIndex](https://www.llamaindex.ai/), [LangChain](https://www.langchain.com/), [CrewAI](https://www.crewai.com/), [Vercel AI SDK](https://sdk.vercel.ai/), and [GenKit](https://firebase.google.com/docs/genkit). [Auth for GenAI](https://auth0.com/blog/auth-for-genai/) is our upcoming product to help you protect your user's information in GenAI-powered applications. Make sure to join the [Auth0 Lab Discord server](http://a0.to/auth0-lab-discord) to hear more and ask questions.


Build a Secure LangChain RAG Agent Using Auth0 FGA and LangGraph on Node.js

Learn how to use Auth0 FGA to secure your LangGraph RAG agent in Python.

Building Retrieval-Augmented Generation (RAG) applications are all the rage, and for a good reason. They let you tap into vast knowledge bases to answer questions, generate content, and power all sorts of AI-driven experiences. But what happens when you need to control who can access what information? That's where things get interesting.

We're diving deep into building a secure RAG pipeline using LangGraph, Python, and Auth0 FGA. Forget simple vector lookups; we're talking about a graph-based approach that lets you orchestrate complex workflows, integrate tools, and enforce fine-grained access control.

## Proposed Solution: Secure Retrieval with LangGraph, OpenAI, and Auth0 FGA

The solution employs a workflow that integrates document retrieval, user-specific authorization filtering, and LLM response generation.

We’ll build a pipeline that uses:

* **LangChain & LangGraph**: A helpful toolkit for chaining the steps of an RAG pipeline (loading data, creating embeddings, retrieving, and prompting).
* **LLMs** (OpenAI or any other): For both embeddings (similarity search) and text generation.
* **Auth0 FGA**: A fine-grained authorization service that checks, for each document, if a user has the “viewer” relationship.

### LangChain & LangGraph

LangChain and LangGraph are two complementary tools that simplify the development of LLM applications, each bringing unique strengths to the table.

LangChain is a library that streamlines the assembly of components—such as document loaders, vector stores, and prompt templates—into a robust and cohesive framework. This makes it easier to build, maintain, and scale your RAG pipeline efficiently.

LangGraph takes a different approach by enabling you to visualize and construct your workflows as a graph. With its ability to handle **complex graphs, branching logic, and advanced flow structures**, LangGraph empowers developers to design sophisticated pipelines that go beyond linear processes. By mapping out the connections between components visually, it enhances modularity and simplifies debugging and optimization. Read more here to get started: [https://langchain-ai.github.io/langgraph/](https://langchain-ai.github.io/langgraph/).

### LLMs

Although we’ll demonstrate OpenAI’s embeddings and Chat APIs, you can easily replace them with other providers (e.g., Anthropic, LLaMA, or Azure OpenAI). The main tasks here are:

1. **Embedding** documents so you can do similarity searches.
2. **Generating** text from the retrieved documents to answer the user’s query.

### FGA, OpenFGA and Auth0 FGA

**FGA (Fine-Grained Authorization)** is about controlling *who can do what* with *which resources*, down to an individual level. In a typical role-based system, you might say, “Admins can see everything, and Regular Users can see some subset.” But in a real-world app—especially one that deals with many documents—this might not be flexible enough.

**OpenFGA (and Auth0 FGA)** addresses this by letting you define authorization *relationships.* The relationships defined in the authorization model can be either direct or indirect. Simply put, direct relationships are directly assigned between a consumer and a resource (we call them user and object) and stored in the database. Indirect relationships are the relationships we can infer based on the data and the authorization model.

If you would like to learn the basics of using FGA for RAG, check out this blog post on [RAG and Access Control: Where Do You Start?](https://auth0.com/blog/rag-and-access-control-where-do-you-start/).

<include
   src="LinkCard"
   title="RAG and Access Control: Where Do You Start?"
   link="https://auth0.com/blog/rag-and-access-control-where-do-you-start/"
   description="Learn how to get started with Auth0 FGA for a RAG application."  img="https://images.ctfassets.net/23aumh6u8s0i/79NCborGcFIrAMeaV7G8oi/f9c588c2d05f0828d447e72e1fe825cc/Latam01.jpg"
/>

## Implementation: Step-by-Step

Below is a sample setup in Python. We’ll keep it simple so you can see the big picture. Feel free to adapt for your chosen LLM or a more robust data store.

### Set Up Prerequisites

To follow this tutorial and secure your application, you’ll need the following:

* Python 3.9 or newer.
* An Auth0 FGA account. If you don’t have one, you can [create one for free](https://dashboard.fga.dev/).
* An [OpenAI account and API key](https://platform.openai.com/).

### Download and install the sample code

To get started, clone the[ auth0-ai-samples](https://a0.to/auth0-ai-samples) repository from GitHub:

```bash
git clone https://github.com/auth0-samples/auth0-ai-samples.git
cd auth0-ai-samples/authorization-for-rag/langgraph-agentic-python
# Create a virtual env
python -m venv venv
# Activate the virtual env
source ./venv/bin/activate
# Install dependencies
pip install -r requirements.txt
```

The application is written in Python and is structured as follows:

* `main.py` &mdash; The main entry point of the application, and it is where we define the RAG pipeline
* `docs/*.md` &mdash; Sample markdown files are to be used as context for the LLM. There are two types of docs, public and private. Private documents are only accessible to certain individuals.
* `helpers/memory_store.py` &mdash; Creates an in-memory vector store that acts as the base retriever in the chain.
* `helpers/read_documents.py` &mdash; Utility to read the markdown files from the `docs` folder.
* `scripts/fga_init.py` &mdash; Utility to initialize the Auth0 FGA authorization model and sample data.

### RAG Pipeline

The `main.py` file defines the RAG graph using `LangGraph` to interact with the underlying LLM model and retrieve data from our context. In your project, your data may be sourced from different platforms and systems, make sure you check the proper documentation about [loaders in the Langchain ecosystem](https://python.langchain.com/v0.1/docs/modules/data_connection/).

The following diagram represents the RAG architecture we are defining:

![LangGraph RAG architecture](https://images.ctfassets.net/23aumh6u8s0i/NwbnsTvKszhWZIng1VjwH/b8b5d467787cbdbb0ac6a850738f4922/langgraph-python-example.png)

## Python Implementation

Let’s now explore the code at `main.py`, which is where all the magic happens. We’ll start splitting the code into relevant chunks, so it will be easier to follow and understand.

Let’s start by defining a LangChain tool that is responsible for querying the RAG for context by querying the vector database and filtering the relevant documents based on the user access:

```python
@tool
def agent_retrieve_context_tool(query: str):
    """Call to get information about a company, e.g., What is the financial outlook for ZEKO?"""
    documents = read_documents()
    vector_store = MemoryStore.from_documents(documents)

    user_id = "admin"

    retriever = FGARetriever(
        retriever=vector_store.as_retriever(),
        build_query=lambda doc: ClientBatchCheckItem(
            user=f"user:{user_id}",
            object=f"doc:{doc.metadata.get('id')}",
            relation="viewer",
        ),
    )

    relevant_docs = retriever.invoke(query)

    if len(relevant_docs) > 0:
        return "\n\n".join([doc.page_content for doc in relevant_docs])

    return "I don't have any information on that."

tools = [agent_retrieve_context_tool]
```

The `FGARetriever` defined in the `retrieve` node is designed to abstract the base retriever from the FGA query logic. The `build_query` argument lets us specify how to query our FGA model, in this case, by asking if the user is a viewer of the document.

```python
build_query=lambda doc: ClientBatchCheckItem(
    user=f"user:{user_id}",
    object=f"doc:{doc.metadata.get('id')}",
    relation="viewer",
),
```

Then we need to define the nodes in the Graph, each node will process unique functions, so we will explore them separately, starting with the `agent_node`, which is responsible for querying the LLM model based on the user’s input.

Each node in LangGraph receives state as an object and can return state updates, for example:

```python
def agent_node(state: State):
    """
    Generate the response from the agent.
    """
    llm_response = llm.invoke(state["messages"])
    return {"messages": [llm_response]}
```

After the `agent_node` runs, we are using a conditional edge to determine the next step. In our particular case, the node` agent_should_continue` will finalize the execution of the graph if the LLM response didn’t trigger a tool call. But if we detect a tool call, then we continue the graph execution with the `tools` node.

```python
def agent_should_continue(state: State):
    """
    Determines whether the conversation should continue based on the user input.
    """
    last_message = state["messages"][-1]
    if last_message.tool_calls:
        return "tools"

    return END
```

The `tools` node is a bit different from the others, as it builds on top of the Tool we declared at the start and uses the `ToolNode` method to simplify the logic. This method will wrap our tool into a node that adds a response from the tool into the chat messages state.

But that’s not all, as after we process the tool call, and we retrieve the proper context to answer the user’s input, we need to call, once again, the LLM model to generate an answer.

```python
def generate_response_node(state: State):
    """
    Generate the response from the agent based on the result of the RAG tool.
    """
    prompt = PromptTemplate(
        template="""You are an assistant for question-answering tasks. Use the following pieces of retrieved-context to answer the question. If you don't know the answer, just say that you don't know. Use three sentences maximum and keep the answer concise. Question: {question}. Context: {context}. Answer:""",
        input_variables=["question", "context"],
    )

    question = state["messages"][0].content
    context = state["messages"][-1].content

    chain = prompt | llm

    llm_response = chain.invoke(
        {"question": question, "context": context}, prompt=prompt
    )

    return {"messages": [llm_response]}
```

If you read the previous article, this may be familiar, but this is a simple chain that uses a prompt with a question and context to provide a good and relevant answer to the user.

Now, with all the parts in place, we can build the graph:

```python
# Create the OpenAI chat tool
llm = ChatOpenAI(model="gpt-4o-mini").bind_tools(tools)

# Build the graph
graph_builder = StateGraph(State)
tool_node = ToolNode(tools)

# Define the nodes
graph_builder.add_node("agent", agent_node)
graph_builder.add_node("tools", tool_node)
graph_builder.add_node("generate_response", generate_response_node)

# Define the edges
graph_builder.add_edge(START, "agent")
graph_builder.add_conditional_edges(
    "agent",
    agent_should_continue,
    ["tools", END],
)
graph_builder.add_edge("tools", "generate_response")

# Compile the graph
graph = graph_builder.compile()
```

That’s awesome! Now, if you set up all the environment variables, you could run the code. If you are not sure how to set them up, don’t worry; we will do that next.

### Create an Auth0 FGA Account

If you already have an [Auth0 account](https://auth0.com/), you can use the same credentials to log in to the Auth0 FGA dashboard at [https://dashboard.fga.dev](https://dashboard.fga.dev/). If you don't have an Auth0 account, hop over to [https://dashboard.fga.dev](https://dashboard.fga.dev/) and create a free account.

Once you are logged in, you should see a dashboard similar to the one below.

![The Auth0 FGA Dashboard getting started page](https://images.ctfassets.net/23aumh6u8s0i/38P8Vu4qsBvCGrZ4ePew3C/090d068c68b785311a00aa788b92579b/fga-get-started.png)

> When you log into the Auth0 FGA dashboard for the first time, you may be asked to create a new store. This store will serve as the home for your authorization model and all the data the engine requires to make authorization decisions. Simply pick a name and create your store to get started.

### Create an Auth0 FGA Client

Once you are in the dashboard, you’ll need a client to make API calls to Auth0 FGA. To create a client, navigate to **Settings**, and in the **Authorized Clients** section, click **Create Client**. Give your client a name, mark all three client permissions, and then click **Create**.

![Create FGA client](https://images.ctfassets.net/23aumh6u8s0i/2KQc8ACR0MDTqftUgUSXpX/5b779bcece07a69ac775de0fc760c208/Uploaded_from_RAG_and_Access_Control__Where_Do_You_Start)

When you create the client, Auth0 FGA will provide you with some data like a Store ID, a Client ID and a Client Secret. Don’t close that yet, you’ll need those values next.

At the root of the project, there’s a `.env.example` file. Copy the file and paste it as `.env`. Then, open the file and edit the three FGA-related variables using the values provided by Auth0 FGA. When you are ready, click continue, and the modal will display the values for the missing variables (`FGA_API_URL` and `FGA_API_AUDIENCE`).

At this step, you can also add your [OpenAI API Key](https://help.openai.com/en/articles/4936850-where-do-i-find-my-openai-api-key), which you’ll need to run the demo.

### Configure the Auth0 FGA model

Now that the application is set up, you can run the provided script to initialize the model and some sample data. To set things up, run:

```
python ./scripts/fga_init.py
```

You can verify that the script worked by navigating to the model's page in Auth0 FGA. The following model should now have been created:

```
model
 schema 1.1

type user

type doc
 relations
   define owner: [user]
   define viewer: [user, user:*]
```

> You can visit the Auth0 FGA documentation to learn more about[ modeling Auth0 FGA and creating an authorization model](https://docs.fga.dev/modeling).

On top of the model, the script also created two tuples. Tuples in Auth0 FGA define the relationships between the types.

First, it defined a tuple to give all users access to the public doc:

- **User** &mdash; `user:*`
- **Object** &mdash; `public-doc`
- **Relation** &mdash; `viewer`

Then, it created a second tuple to give the admin user access to the private doc:

- **User** &mdash; `user:admin`
- **Object** &mdash; `private-doc`
- **Relation** &mdash; `viewer`

> You can visit the Auth0 FGA documentation to learn more about [tuples and how to create them](https://docs.fga.dev/content/getting-started/new-getting-started#add-tuples).

### Query the Graph

Invoke the graph to process a query and generate a response.

```python
# Run the graph
result = graph.invoke(
    {"messages": [("human", "What is the financial outlook for ZEKO?")]}
)
print(result["messages"][-1].content)
```

Fantastic! You now know how to build and scale a secure RAG with Python and LangGraph. It’s time to test things out.

To run the graph, simply call the `main.py` file:

```
python main.py
```

If you follow the steps, you’ll see responses like the following:

```
Example with no access:
The retrieved context does not provide any specific forecast or predictions for ZEKO (Zeko Advanced Systems Inc.). It mainly outlines the company's mission, technologies, and products without detailing any financial or market forecasts. Therefore, I don't know the forecast for ZEKO.

Example with access:
The forecast for Zeko Advanced Systems Inc. (ZEKO) for fiscal year 2025 is generally bearish. Projected revenue growth is expected to remain subdued at 2-3%, with net income growth projected at 1-2%, primarily due to margin pressures and competitive challenges. Investors should be cautious, given the potential headwinds the company faces.
```

Depending on the user you use in the `agent_retrieve_context_tool`, you’ll see different results, experiment also by adding and removing privileges from the Auth0 FGA dashboard.

## Learn More about Auth for GenAI, Auth0 FGA, and GenAI

Before you go, we have some great news to share: we are working on more content and sample apps in collaboration with amazing GenAI frameworks like [LlamaIndex](https://www.llamaindex.ai/), [LangGraph](https://www.langchain.com/), [CrewAI](https://www.crewai.com/), [Vercel AI](https://sdk.vercel.ai/), and others. 

[Auth for GenAI](https://a0.to/ai-content) is our upcoming product to help you protect your user's information in GenAI-powered applications.

Make sure to join the [Auth0 Lab Discord server](http://a0.to/auth0-lab-discord) to hear more and ask questions.

Happy coding!

"rag"