AmazonQ - building a GenAI assistant

I’ve been working on a proof of concept (POC) using Amazon Q for business for several weeks. These are my preliminary thoughts and some challenges I encountered while creating our first Generative AI assistant.

First, let me set the context for the POC. This work was not for a customer. It was to familiarize myself with the service and to get a sense of what we could do in future. I purposely kept it as simple as I could. I was more interested in getting something to work than trying all the bells and whistles.

I worked on this project during our weekly Twitch streams, and you can watch the sessions on-demand on my YouTube page.

• Building a GenAI Assistant with Amazon Q - session 1
• Building a GenAI Assistant with Amazon Q - session 2

Key Concepts and Terms

An Amazon Q application is made up of several components. Here are the ones that are relevant to our POC.

• Application: The main component of your chat solution.
• Index: This is where Amazon Q stores documents. It supports a native retriever, or you can connect it to Amazon Kendra. For the POC, I used a native retriever.
• Retriever: The component that pulls data from an index during a conversation.
• Data source connector: Crawls data and stores it in your Q index. I was impressed with the number of data source connectors that Q supports during the preview. For our project, we used an Amazon Q Web Crawler.
• Web experience: This is the chat interface you use to interact with the assistant.
• Identity Provider (IdP): This centralized service stores and verifies identities. We integrated the web experience with the AWS IAM Identity Center for the POC.

There are a couple of other important concepts that are not related directly to Amazon Q but are important to know:

• Retrieval Augmented Generation (RAG): This is a natural language processing (NLP) solution. Amazon Q has a built-in RAG.
• Large Language Model (LLM): This is a language-based machine learning model. Amazon Q uses an LLM as part of the RAG.

Deployment Notes

At the time of writing, Amazon Q did not support CloudFormation, so for the POC, we shifted from our regular usage of YAML templates to Python code.

As usual, the AWS SDKs are well-documented and easy to follow. Creating the components - application, index, and retriever was relatively easy. Below are code snippets for the various components.

Application

    try:
        response = client.create_application(
            description=q_app_desc,
            displayName=q_app_name,
            roleArn=q_app_role_arn,
            tags=[
                {'key': 'owner', 'value': 'CuriosOrbit'},
                {'key': 'ownerEmail', 'value': 'twitch@curiousorbit.com'},
                {'key': 'environment', 'value': q_environment},
            ]
        )
        return response['applicationArn'], response['applicationId']
    except botocore.exceptions.ClientError as error:
        if error.response['Error']['Code'] == 'ConflictException':
            print('A Q business application with that name already exists. Exiting.')
            exit(1)
        else:
            print(error.response['Error']['Code'])

Index

    try:
        response = client.create_index(
            applicationId=q_app_id,
            capacityConfiguration={
                'units': q_index_cap_units
            },
            description=q_index_desc,
            displayName=q_index_name,
            tags=[
                {'key': 'owner', 'value': 'CuriosOrbit'},
                {'key': 'ownerEmail', 'value': 'twitch@curiousorbit.com'},
                {'key': 'environment', 'value': q_environment},
            ]
        )
        return response['indexArn'], response['indexId']
    except botocore.exceptions.ClientError as error:
        print(error.response['Error']['Code'])

Retriever

    try:
        response = client.create_retriever(
            applicationId=q_app_id,
            configuration={
                'nativeIndexConfiguration': {
                    'indexId': q_index_id
                }
            },
            displayName=q_retriever_name,
            type=q_retriever_type,
            tags=[
                {'key': 'owner', 'value': 'CuriosOrbit'},
                {'key': 'ownerEmail', 'value': 'twitch@curiousorbit.com'},
                {'key': 'environment', 'value': q_environment},
            ]
        )
        return response['retrieverArn'], response['retrieverId']
    except botocore.exceptions.ClientError as error:
        print(error.response['Error']['Code'])
    except botocore.exceptions.ParamValidationError as error:
        print(error.response['Error']['Code'])

If you’d like to try it out, I’ve made the code available on GitHub. Remember, before deploying any services on AWS, make sure you understand the pricing models!

The data source connector and the Idp integration were the two application components I didn’t deploy via Boto3. Depending on your connector (or connectors), you’ll have to make several configuration choices. To get something working, I used the management console to create the web crawler. With the IdP integration, I needed to copy XML documents and thought it would be easier to do via the console.

Challenges

Ironically, the component I deployed via the management console - the web crawler - ended up being the most challenging component to get working.

When you create a crawler you need to decide on how to crawl the domain. There are four different options, and I went with ‘Source URLs’ because it seemed to be the easiest one to implement. All I needed to do was select the domain and the crawler started grabbing data.

The issue I had was that for some reason, when configured with ‘Source URLs’ the web crawler only ever managed to crawl 23 pages of our website. We don’t have a large site, but I know there are more than 23 pages. 🙂

After a conversation with AWS support (fantastic as always), we decided to test the web crawler using the ‘Source sitemaps’ option. Making this change fixed the crawling issue, and we got all our site data into the index.

A couple of final thoughts on the web crawler piece:

1. The issue could end up being our site. I pointed a ‘Source URL’ crawler at another website - we had permission to crawl - and it worked without issue.
2. I plan on getting the crawler piece into Python - in a future stream.

Deploying the Web Experience

Integrating the GenAI assistant with the IAM Identity Center solution was straightforward and well-documented. I got the integration up and running on the first attempt by following the instructions.

Once completed, I opened a session by following the ‘Deployed URL’ was authenticated through the Identity Center and started conversing with our new AI Assistant.

What’s next?

Now that I have a functioning AI assistant, I’d like to connect it to other data sources and see how useful it is. I’d want to connect it to JIRA and Slack in the short term. A longer-term goal would be to see if we could attach it to our GitLab instance.

The GitLab integration could prove extremely useful since all our Python code and CloudFormation templates are stored there. For a new team member, this has the potential to improve their productivity significantly.