{"id":8468,"date":"2024-12-10T13:48:00","date_gmt":"2024-12-10T13:48:00","guid":{"rendered":"https:\/\/rax-stg-wp-aws.com\/blogs\/create-custom-chatbot-with-azure-openai-and-azure-ai-search\/"},"modified":"2026-11-03T19:29:38","modified_gmt":"2026-11-03T19:29:38","slug":"create-custom-chatbot-with-azure-openai-and-azure-ai-search","status":"publish","type":"blogs","link":"https:\/\/www.rackspace.com\/en-gb\/blogs\/create-custom-chatbot-with-azure-openai-and-azure-ai-search\/","title":{"rendered":"Create Custom Chatbot with Azure OpenAI and Azure AI Search"},"content":{"rendered":"<p><strong>Introduction<\/strong><\/p>\n<p class=\"MsoNormal\">Nowadays, we hear a lot about generative AI (GenAI) and large language models (LLMs) from various providers for different use cases. Among them, OpenAI models such as GPT-4o and GPT-3.5 Turbo are some of the most widely used.&nbsp;<\/p>\n<p class=\"MsoNormal\">In this article, I will show you how to create a personalized chatbot that integrates with your own data source using OpenAI models in Azure Cloud.&nbsp;<\/p>\n<p class=\"MsoNormal\"><strong>Topics&nbsp;<\/strong><\/p>\n<ol>\n<li><em>Overview of&nbsp;Azure AI services&nbsp;<\/em><\/li>\n<li><em>Introduction to&nbsp;Azure AI Search&nbsp;and&nbsp;Azure OpenAI<\/em><\/li>\n<li><em>The retrieval-augmented generation (RAG) solution and its advantages<\/em><\/li>\n<li><em>Architecture diagram<\/em><\/li>\n<li><em>Create Azure resources and deploy the&nbsp;GPT-4o model with Terraform&nbsp;<\/em><\/li>\n<li><em>Adding knowledge base data to&nbsp;Azure AI Search for indexing&nbsp;<\/em><\/li>\n<li><em>Integrating&nbsp;external data sources&nbsp;with the LLM model&nbsp;<\/em><\/li>\n<li><em>Accessing the prompt and querying an external data source&nbsp;<\/em><\/li>\n<li><em>Deploying the&nbsp;OpenAI GPT model&nbsp;on Azure App Service with a custom domain<\/em><\/li>\n<\/ol>\n<p class=\"MsoNormal\"><strong>1. Overview of Azure AI services:&nbsp;<\/strong><\/p>\n<p class=\"MsoNormal\">Azure offers a wide range of AI services that developers can use to build intelligent applications. These services enable you to add capabilities like vision, speech, language, and decision-making to your apps through APIs and SDKs.<\/p>\n<ul>\n<li><strong>Vision:<\/strong> Provides image and video analysis services<\/li>\n<li><strong>Speech:<\/strong> Includes capabilities for speech recognition, translation and synthesis<\/li>\n<li><strong>Language:<\/strong> Supports natural language processing (NLP) for understanding and analyzing text<\/li>\n<li><strong>Decision: <\/strong>Offers AI services for anomaly detection and decision-making<\/li>\n<\/ul>\n<p class=\"MsoNormal\"><strong>2. Introduction to Azure AI Search and Azure OpenAI<\/strong><\/p>\n<p class=\"MsoNormal\">Azure AI Search is a powerful search platform that enables you to create a rich search experience and combine it with large language models (LLMs) to build generative AI applications using your enterprise data. Azure OpenAI provides access to models like o1-preview, o1-mini, and GPT-4o mini\/turbo\/audio through REST APIs, enabling content generation, summarization, semantic search, code generation, and more.<\/p>\n<p class=\"MsoNormal\">Available models in Azure OpenAI Studio include:<\/p>\n<ul>\n<li><strong>GPT-4o<\/strong>: Improves upon its predecessor, GPT-3.5<\/li>\n<li><strong>Text-embedding-3-large\/small:<\/strong> Converts text into numerical vectors for vector search<\/li>\n<li><strong>DALL-E:<\/strong> Generates original images from natural language prompts<\/li>\n<li><strong>Whisper:<\/strong> Transcribes and translates speech to text<\/li>\n<li><strong>Text to Speech:<\/strong> Synthesizes speech from text<\/li>\n<\/ul>\n<p class=\"MsoNormal\"><strong>3. What is RAG (Retrieval-Augmented Generation)?<\/strong>&nbsp;<\/p>\n<p class=\"MsoNormal\">RAG&nbsp;is an AI solution that combines search-based retrieval (like Azure AI Search) with generative models (like OpenAI) to create more&nbsp;accurate&nbsp;and context-aware responses.&nbsp;<\/p>\n<p class=\"MsoNormal\">Imagine a customer support chatbot that can search your company&#8217;s knowledge base through Azure AI Search and generate a response using Azure OpenAI. This helps create precise answers that are both accurate and contextually rich.&nbsp;<\/p>\n<p class=\"MsoNormal\"><strong>Advantages of RAG:&nbsp;<\/strong><\/p>\n<ul>\n<li>Accurate responses: OpenAI generates up-to-date answers using data retrieved by Azure AI Search.<\/li>\n<li>Contextual information: Azure AI Search retrieves the most relevant documents, helping OpenAI generate responses tailored to the user&#8217;s query.<\/li>\n<li>Reduced hallucinations: RAG works with a base of knowledge instead of relying solely on training data, which reduces hallucinations.<\/li>\n<li>Enhanced personalization: RAG models offer personalized responses based on user preferences, past interactions, and historical data.<\/li>\n<li>Improved user experience: By merging search-based retrieval with generative AI, RAG delivers comprehensive answers, preventing users from sifting through multiple sources.<\/li>\n<\/ul>\n<p><strong>4.&nbsp;<\/strong><strong>Architecture diagram<\/strong><br \/>\nThe architecture diagram illustrates a workflow involving Azure Cognitive Search and Azure OpenAI.<\/p>\n<p><img decoding=\"async\" src=\"\/wp-content\/uploads\/2026\/blog_article_images\/27287-body-Picture1_0.png\" alt=\"Azure AI Chatbot Pic 1\" \/><\/p>\n<p class=\"MsoNormal\"><strong>5. Creating Azure resources and deploying the GPT-4o model with Terraform<\/strong><\/p>\n<p class=\"MsoNormal\">You need to create the following resources using Terraform or from the Azure portal:<\/p>\n<ul>\n<li>Create a resource group: Organize your resources in Azure<\/li>\n<li>Create an Azure OpenAI service: Set up GPT-4o or GPT-3.5 models<\/li>\n<li>Create an Azure AI Search service: Index and retrieve documents<\/li>\n<li>Create a storage account and container: For uploading documents<\/li>\n<\/ul>\n<p class=\"MsoNormal\">I created the Terraform code in my GitHub repository to deploy these resources; you can clone or fork it from <a href=\"https:\/\/github.com\/anoopkum\/openai-terraform\/\">here<\/a>.<\/p>\n<p class=\"MsoNormal\">Go to the Azure Portal to see the resources deployed under Resource Group&nbsp;\u201cRG-openAI-demo&#8221;&nbsp;as below screenshot:&nbsp;<\/p>\n<p><img decoding=\"async\" src=\"\/wp-content\/uploads\/2026\/blog_article_images\/27287-body-Picture2.png\" alt=\"Azure AI Chatbot Pic 2\" \/><\/p>\n<p class=\"MsoNormal\"><strong>6.&nbsp;<\/strong><strong>Adding knowledge base data to Azure AI Search for indexing<\/strong><\/p>\n<p>Go to the <strong>Azure AI Search<\/strong> resource.&nbsp;<\/p>\n<p>Click on the &#8220;Import Data&#8221; option and select either a sample dataset (I used the sample dataset \u201chotels-sample\u201d in this blog) or your data.<\/p>\n<p><img decoding=\"async\" src=\"\/wp-content\/uploads\/2026\/blog_article_images\/27287-body-Picture3.png\" alt=\"Azure AI Chatbot Pic 3\" \/><\/p>\n<ul>\n<li>Import your own data from Azure Blob Storage.&nbsp;<\/li>\n<\/ul>\n<p><img decoding=\"async\" src=\"\/wp-content\/uploads\/2026\/blog_article_images\/27287-body-Picture4.png\" alt=\"Azure AI Chatbot Pic 4\" \/><\/p>\n<p class=\"MsoNormal\"><strong>&nbsp;<\/strong>The following data files are uploaded in the Azure Blob Storage container named &#8220;democontainer&#8221; as knowledge base.&nbsp;<\/p>\n<p><img decoding=\"async\" src=\"\/wp-content\/uploads\/2026\/blog_article_images\/27287-body-Picture5.png\" alt=\"Azure AI Chatbot Pic 5\" \/><\/p>\n<ul>\n<li>Follow the prompts to configure and create an index&nbsp;and an indexer.<\/li>\n<\/ul>\n<p><img decoding=\"async\" src=\"\/wp-content\/uploads\/2026\/blog_article_images\/27287-body-Picture6.png\" alt=\"Azure AI Chatbot Pic 6\" \/><\/p>\n<ul>\n<li>Go to Azure AI Search resource &gt; Search Management &gt; Indexers to verify the status.&nbsp;<\/li>\n<\/ul>\n<p><img decoding=\"async\" src=\"\/wp-content\/uploads\/2026\/blog_article_images\/27287-body-Picture7.png\" alt=\"Azure AI Chatbot Pic 7\" \/><\/p>\n<p class=\"MsoNormal\"><strong>7.&nbsp;<\/strong><strong>Integrating external data sources with the LLM model:&nbsp;<\/strong>&nbsp;<\/p>\n<ul>\n<li>Now open Azure&nbsp;OpenAI&nbsp;resource&nbsp;<strong>\u201c<\/strong><strong>openai-rxt-demo&#8221;&nbsp;<\/strong>and click on Azure&nbsp;OpenAI&nbsp;Studio to access the Azure OpenAI&nbsp;playground.&nbsp;<\/li>\n<\/ul>\n<p><img decoding=\"async\" src=\"\/wp-content\/uploads\/2026\/blog_article_images\/27287-body-Picture8.png\" alt=\"Azure AI Chatbot Pic 8\" \/><\/p>\n<ul>\n<li>In Azure OpenAI studio, click on \u201c<strong>Deployments\u201d <\/strong>to access the LLM models.&nbsp;<\/li>\n<\/ul>\n<p><img decoding=\"async\" src=\"\/wp-content\/uploads\/2026\/blog_article_images\/27287-body-Picture9.png\" alt=\"Azure AI Chatbot Pic 9\" \/><\/p>\n<ul>\n<li>Choose the GPT model deployed <strong>GPT-4o <\/strong>and navigate to the&nbsp;Playground.&nbsp;<\/li>\n<li>In the&nbsp;Chat Playground, you can start interacting with the model.&nbsp;<\/li>\n<\/ul>\n<p><img decoding=\"async\" src=\"\/wp-content\/uploads\/2026\/blog_article_images\/27287-body-Picture10.png\" alt=\"Azure AI Chatbot Pic 10\" \/><\/p>\n<ul>\n<li>Configure important settings such as the system message (instructions for the model) and parameters (e.g., temperature, token limit) to customize how the chatbot responds.<\/li>\n<\/ul>\n<p><img decoding=\"async\" src=\"\/wp-content\/uploads\/2026\/blog_article_images\/27287-body-Picture11.png\" alt=\"Azure AI Chatbot Pic 11\" \/><\/p>\n<ul>\n<li>To integrate external data, select&nbsp;<strong>&#8220;<\/strong><strong>Add Your Data\u201d.<\/strong><\/li>\n<\/ul>\n<p><img decoding=\"async\" src=\"\/wp-content\/uploads\/2026\/blog_article_images\/27287-body-Picture12.png\" alt=\"Azure AI Chatbot Pic 12\" \/><\/p>\n<ul>\n<li>Choose Azure AI Search as the data source.&nbsp;<\/li>\n<\/ul>\n<p><img decoding=\"async\" src=\"\/wp-content\/uploads\/2026\/blog_article_images\/27287-body-Picture13.png\" alt=\"Azure AI Chatbot Pic 13\" \/><\/p>\n<ul>\n<li>Once you submit the request, the data source will be configured as Azure AI Search.<\/li>\n<\/ul>\n<p><img decoding=\"async\" src=\"\/wp-content\/uploads\/2026\/blog_article_images\/27287-body-Picture14.png\" alt=\"Azure AI  Chatbot Pic 14\" \/><\/p>\n<ul>\n<li>You will need to request a quota increase for the GPT-4o model. There is no need to request a rate limit increase for GA versions. Such as GPT-3.5\/turbo. Once Microsoft approves your request, you will be able to use GPT-4o with your own data.&nbsp;<\/li>\n<li>\n<p class=\"MsoNormal\">If you would like to further increase the default rate limit, go <a href=\"https:\/\/aka.ms\/oai\/quotaincrease\/\">here<\/a>.<\/p>\n<\/li>\n<\/ul>\n<p><img decoding=\"async\" src=\"\/wp-content\/uploads\/2026\/blog_article_images\/27287-body-Picture15.png\" alt=\"Azure AI Chatbot Pic 15\" \/><\/p>\n<p class=\"MsoNormal\"><strong>8.&nbsp;<\/strong><strong>Accessing the prompt and querying the external data source:&nbsp;<\/strong><\/p>\n<ul>\n<li>In the&nbsp;Chat Playground, you can now ask questions that the model will answer based on the&nbsp;external data source.&nbsp;<strong>&nbsp;<\/strong><\/li>\n<li>For example, I asked the chatbot to <strong>&#8220;find me the best hotels in New York and visiting places&#8221;<\/strong>&nbsp;using the indexed knowledge <a>base.<\/a><\/li>\n<\/ul>\n<p><img decoding=\"async\" src=\"\/wp-content\/uploads\/2026\/blog_article_images\/27287-body-Picture16.png\" alt=\"Azure AI Chatbots Pic 16\" \/><\/p>\n<ul>\n<li>I also asked a question outside the scope of the data source, but the model was unable to provide an answer, as it can only retrieve information from the specified dataset.<\/li>\n<\/ul>\n<p><img decoding=\"async\" src=\"\/wp-content\/uploads\/2026\/blog_article_images\/27287-body-Picture17.png\" alt=\"Azure AI chatbot Pic 17\" \/><\/p>\n<ul>\n<li>When I unchecked &#8220;Limit response to your data content&#8221; in Advanced settings to allow the model to respond to queries beyond the provided data source, like I asked \u201ctell me about the Rackspace Technology\u201d, it responded from Internet.<\/li>\n<\/ul>\n<p><img decoding=\"async\" src=\"\/wp-content\/uploads\/2026\/blog_article_images\/27287-body-Picture18.png\" alt=\"Azure AI Chatbots Pic 18\" \/><\/p>\n<p class=\"MsoNormal\"><strong>9.&nbsp;<\/strong><strong>Deploying the OpenAI GPT model on Azure App Service with a custom domain<\/strong><\/p>\n<p class=\"MsoNormal\">You can deploy the model to a web app for custom domain access. To do this, follow the steps shown in the screenshot to link it to Azure App Service.<\/p>\n<p><img decoding=\"async\" src=\"\/wp-content\/uploads\/2026\/blog_article_images\/27287-body-Picture19.png\" alt=\"Azure AI chatbot pic 19\" \/><\/p>\n<p><img decoding=\"async\" src=\"\/wp-content\/uploads\/2026\/blog_article_images\/27287-body-Picture20.png\" alt=\"Azure AI Chatbot Pic 20\" \/><\/p>\n<p><strong>&nbsp;<em>Note: Enabling chat history will create a Cosmos DB database to store your chat records, which may increase your costs.<\/em><\/strong><\/p>\n<p class=\"MsoNormal\"><strong>&nbsp;<\/strong>After deployment, you can access the chatbot via a custom URL.<\/p>\n<p><img decoding=\"async\" src=\"\/wp-content\/uploads\/2026\/blog_article_images\/27287-body-Picture21.png\" alt=\"Azure AI chatbot Pic 21\" \/><\/p>\n<p class=\"MsoNormal\">I hope this article helps you create a custom chatbot using Azure OpenAI and Azure AI Search. Feel free to clone the Terraform code and share any feedback or suggestions.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Learn how to build a custom chatbot with Azure OpenAI and Azure AI Search, from deployment and data integration to creating personalized user responses.<\/p>\n","protected":false},"author":2,"featured_media":8470,"menu_order":0,"comment_status":"open","ping_status":"open","template":"","format":"standard","meta":{"content-type":"","nid":"27287","content_type":"blog","created_by":"10130","authored_on":"2024-12-05T13:48:00+00:00","field_article_industries":"","tag_category":"Technical","field_blog_author_text":"by Anoop Kumar, Professional Services Delivery Engineer, Rackspace Technology","seo_description":"Learn how to create a custom chatbot using Azure OpenAI and Azure AI Search to deliver personalized responses with integrated data sources.","field_featured_image":"","insights_image":"\/wp-content\/uploads\/2026\/blog_technical_images\/27287-insights_image-microsoft-technical-blog-1-insights.jpg","field_partners":"Microsoft","rxt_platforms":"Microsoft Azure","product":"","publisheddate":"2024-12-10T13:48:00","rxt_solutions":"Cloud, Artificial Intelligence & Machine Learning","opengraph_tag":"","x_tag":"","teaser_flag":"Technical Blog","seo_titile":"Create a Custom Chatbot with Azure OpenAI and AI Search | Rackspace Technology","field_topics":"AI","field_slug":"custom-chatbot-azure-openai-azure-ai-search","footnotes":""},"categories":[178],"tags":[179,182,204],"class_list":["post-8468","blogs","type-blogs","status-publish","format-standard","has-post-thumbnail","hentry","category-blog","tag-ai","tag-microsoft","tag-technical"],"_links":{"self":[{"href":"https:\/\/www.rackspace.com\/en-gb\/wp-json\/wp\/v2\/blogs\/8468","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.rackspace.com\/en-gb\/wp-json\/wp\/v2\/blogs"}],"about":[{"href":"https:\/\/www.rackspace.com\/en-gb\/wp-json\/wp\/v2\/types\/blogs"}],"author":[{"embeddable":true,"href":"https:\/\/www.rackspace.com\/en-gb\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.rackspace.com\/en-gb\/wp-json\/wp\/v2\/comments?post=8468"}],"version-history":[{"count":3,"href":"https:\/\/www.rackspace.com\/en-gb\/wp-json\/wp\/v2\/blogs\/8468\/revisions"}],"predecessor-version":[{"id":10123,"href":"https:\/\/www.rackspace.com\/en-gb\/wp-json\/wp\/v2\/blogs\/8468\/revisions\/10123"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.rackspace.com\/en-gb\/wp-json\/wp\/v2\/media\/8470"}],"wp:attachment":[{"href":"https:\/\/www.rackspace.com\/en-gb\/wp-json\/wp\/v2\/media?parent=8468"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.rackspace.com\/en-gb\/wp-json\/wp\/v2\/categories?post=8468"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.rackspace.com\/en-gb\/wp-json\/wp\/v2\/tags?post=8468"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}