Generative AI (genAI) offers extraordinary promise, but turning that promise into tangible business results demands more than technology. It requires a strategy rooted in real-world challenges, guided by a deep understanding of what works and what doesn’t.

This article is based on our white paper How to Create High-Value GenAI Apps: A Nearsure Guide. Here, we will share highlights about creating flexible, future-proof architectures and identifying use cases where generative AI delivers meaningful value­. Along the way, we’ll examine the missteps that companies often make and the lessons learned from those who succeed.

What is generative AI application development?

Generative AI is a branch of artificial intelligence (AI) that generates new, original content based on learned patterns and context. Applications utilizing generative AI leverage large language models (LLMs), vision models, and other advanced techniques to provide functionalities such as conversational chatbots, document analysis, AI agents, and image recognition systems.

Traditional AI and machine learning (ML) typically involve training models for specific tasks like classification, regression, or anomaly detection. In contrast, generative AI applications often utilize pre-trained large-scale models (like GPT, Gemini, or Claude) to understand and generate contextually relevant outputs. This capability allows generative AI systems to dynamically interact with users, synthesize insights from extensive document collections, and interpret complex visual data.

Key components of a successful genAI app

In the current landscape, we have found that the Retrieval-Augmented Generation (RAG) approach is the most powerful for solving business problems. To solve most of these issues, we supplement the basic RAG app architecture with a variety of other components.

What is RAG architecture?

RAG is an AI framework that improves LLMs by combining real-time data retrieval from external sources with generative models. It’s the link between static LLM knowledge and dynamic data, and it’s perfect for domain-specific tasks. This translates into more accurate, context-aware responses for AI apps.

The structure of RAG genAI apps is shown in the following graphic:

This model works extremely well for business applications because LLMs are built on vast amounts of publicly available data, making them powerful but inherently generalized. They don’t understand the specifics of your business, so we use RAG to bridge the gap.

The key to making the RAG approach work is the process of deciding what company-specific content to add to the prompt. Searching and finding the most relevant documents is a crucial task, one that is often powered by vector databases that store embeddings of company-specific data and documents.

The importance of contextual data

RAG plays a vital role in making your business-specific data accessible to LLMs. When the right documents are identified and added to the original prompts, RAG gives the LLM access to the precise context it needs to respond with insights that align directly with your operations and goals.

This involves building a vector database that acts as a knowledge repository, enabling the LLM to provide accurate and context-aware responses. Think of it as creating a specialized knowledge base that allows the LLM to speak your business language.

Step-by-step generative AI app development process

Identifying the highest value use cases

Selecting the right use cases is the foundation of any successful generative AI implementation. Focus first on areas where generative AI can deliver immediate, tangible value. These early successes provide a roadmap for scaling generative AI across your organization and refining its role as your needs evolve.

Some examples are:

• Streamlining workflows
• Democratizing access to information
• Enhancing data management
• Automating content generation

The best use cases for your organization will depend on its unique goals and challenges.

Set up tech-agnostic architecture

Adaptability is the foundation of a successful generative AI application. Think of your system as a vessel built for discovery, ready to chart uncharted waters. Its navigation systems, sails, and rigging are modular, allowing you to adjust course and adapt as conditions change or better tools become available.

With this approach, you can:

• Plan for evolution from day 0
• Confidently compare models like Azure, OpenAI, or Gemini
• Control costs
• Adapt to your industry’s demands

A modular system allows you to evaluate and adopt models based on the factors that matter most: data governance, security, performance, cost, or industry-specific requirements.

Healthcare applications, for instance, often demand a degree of precision that leaves no room for error. In contrast, marketing may prioritize creativity and speed over absolute accuracy. Each use case presents its own challenges, and your architecture must adapt to meet them without compromise.

Test and compare results across models in your architecture

To find the right fit for your use case, start with a clear definition of your desired outcomes. Rigorous benchmarking is essential. Test different models against real-world scenarios to determine which delivers the best results.

Custom LLMs may seem appealing but are rarely necessary. Publicly available models, such as Azure OpenAI, ChatGPT-4o, or those offered by Anthropic, are improving rapidly and often present significant cost advantages. Focus your efforts on evaluating these options in controlled settings rather than investing in tailored solutions prematurely.

We emphasize the value of evaluating model performance under real-world conditions. For example, in one project for character recognition, we tested different AI services for use by a gas meter company, including Azure OpenAI ChatGPT-4o and another Microsoft model built specifically for OCR. This hands-on approach revealed which model could handle the unique demands of the application most effectively—it was actually the more generalized model that did the best job. Such evaluations are critical for aligning technology choices with business goals.

Mature genAI apps for business use cases

To support high-value use cases in a business context, the core RAG applications are generally supplemented by the following layers to provide a complete solution.

The architecture of a mature genAI app to support high-value business use cases goes beyond a simple RAG application and must include a variety of elements as shown above. For a genAI app to go fully into production for mission-critical use cases, all these elements must work together.

Best tools for generative AI software development

The landscape of genAI development tools is rapidly evolving, with a growing ecosystem of frameworks and services designed to accelerate the creation of AI-powered applications. While traditional ML engineering remains largely Python-centric, generative AI development has seen a strong rise in TypeScript and Node.js adoption, particularly among web developers. Frameworks like LangChain and the OpenAI SDK offer robust support for both Python and JavaScript ecosystems, enabling flexibility across different tech stacks.

Next.js, developed by Vercel, has emerged as a powerful framework for building AI-driven web applications. Vercel’s AI SDK further enhances this by providing streamlined integration with LLMs and rich UI components for conversational interfaces.

Major cloud providers such as AWS (Bedrock) and Azure (Azure AI) are also actively expanding their toolsets to support generative AI development. These platforms offer comprehensive services for deploying, managing, and scaling AI agents, and they integrate well with vector databases—an essential component in building RAG systems.

Common pitfalls to avoid

We have identified several critical missteps that businesses often encounter when adopting generative AI. Avoiding these mistakes can mean the difference between a costly experiment and a breakthrough success.

Skipping proof of concept

Jumping straight into full implementation without validating requirements can lead to costly missteps.

A well-designed POC is an opportunity to test assumptions about the problem you’re solving and the results you expect to achieve. By working with a controlled dataset and clearly defining success metrics, you can evaluate whether generative AI delivers the value you’re after.

This step also helps identify potential issues, such as insufficient data, and accurately estimate costs before scaling up.

Building unnecessary custom models

Custom models may seem like the ultimate solution, but they come with steep costs and complexity. In most scenarios, public LLMs paired with RAG techniques provide powerful, cost-effective alternatives.

Reinventing the wheel should only be considered when essential to meeting unique business requirements.

Ignoring hallucination protection

Hallucinations happen when LLMs provide answers that are incorrect because the LLM made up an answer. In May 2024, an article from the Association for Computing Machinery reported that a hallucination leaderboard maintained by Vectara showed the rate of hallucinations was estimated at 2.5 to 8.5 percent for popular LLMs. In April 2025, that same leaderboard shows that the current rate is less than 3 percent for most of the big LLM vendors. 

For some applications, a certain level of hallucinations may be tolerable. For others where mission-critical answers are being generated, it makes sense to implement protective measures such as:

• Adding high-quality, relevant information to a prompt using RAG methods.
• Using advanced LLM prompting techniques such as chain-of-thought prompting, which encourage the model to break down complex reasoning into smaller steps, making its thought process more transparent and reducing errors. 
• Fine-tuning the LLM on a dataset specifically tailored to your application domain to improve accuracy and reduce hallucinations within that context.
• Using confidence scoring for each generated response, allowing you to filter outputs with low confidence levels. 
• Implementing fact-checking and validation checks to verify the factual accuracy of LLM outputs, especially when dealing with critical information. 

Real business impact: Use cases that work

The excitement around generative AI sometimes eclipses the need for a concrete strategy. Without a clear understanding of the business problem generative AI is meant to solve or the specific ROI it is expected to deliver, projects risk becoming directionless and expensive. Success starts with knowing why the technology is being implemented.

Using a genAI app to find the right talent

We collaborated with Microsoft’s AI Co-Innovation Labs to develop and integrate an AI-powered assistant that processes thousands of resumes. Our solution automatically ranks candidates based on their fit for job criteria. It allowed us to reduce hiring process time by 30% and cut delivery time in half for first-time viable candidates. Read more about this use case here.

Transforming medical practice incorporation

For a healthcare client headquartered in Houston, Texas, we developed an AI-powered copilot that transformed the process of medical practice incorporation. What once took weeks was reduced to days or mere hours. This streamlined automation led to a measurable increase in the number of new practices established, delivering immediate value to the client.

LLM for entertainment applications

In the entertainment industry, we worked with GrayMeta to design an application for media content management. Leveraging machine learning models and APIs, the solution introduced features like visual recognition, audio classification, and language detection, empowering companies such as HBO and Disney to navigate their media libraries with greater efficiency.

Call centers as fertile ground for successful genAI apps

Using generative AI to transcribe and summarize customer service calls in call centers leads to immediate and significant cost savings. Generative AI also improves customer follow-up by detecting cases where customers are dissatisfied.

We use generative AI alongside other techniques to help categorize and cluster calls by the reasons customers are calling and the problems they are experiencing. With this system, we can quickly detect if customers are experiencing a problem with a particular product or service.

Final thoughts: Building for ROI and scale

Evaluating potential ROI for genAI apps

A crucial part of focusing on the highest value use cases is being able to accurately estimate the ROI before development and to track it precisely after deployment.

Calculating the ROI for generative AI requires a broader and more dynamic approach compared to traditional ROI measurements. This is due to the unique characteristics of generative AI, such as its continuous evolution, qualitative benefits, and the need for ongoing maintenance.

Traditional ROI estimates typically focus on "hard" metrics, measuring cost savings and productivity gains over a set period. Our work in the field has taught us how generative AI must be measured differently because the apps create impact in multiple dimensions beyond typical cost savings or financial gains.

• Competitive differentiation: It is crucial to include qualitative measures, such as customer satisfaction and employee engagement, as part of the indicators.
• Faster analytics: The ability to quickly process and analyze large volumes of data has a qualitative effect on numerous processes.
• Cumulative benefits: Unlike traditional ROI, generative AI's benefits often build over time.This can improve brand image and customer retention.

The importance of planning for versioning, costs, and updates

On the cost side of the equation, ROI estimates must include the fact that genAI applications generally need maintenance and updates upon the release of new model versions.

Also, human oversight is required to address and rectify potential errors or inconsistencies within applications. Thus, recurring measurements are necessary to track their evolution, considering the pace of changes in generative AI capabilities relative to benefits versus maintenance costs.

How to Create High-Value GenAI Apps: A Nearsure Guide

In this guide, we delve deeper into the generative AI development world. Discover insights forged in practice and the valuable lessons we learned. Take a look!

‍