RAG vs Fine-Tuning vs Off-the-Shelf AI: An Enterprise Decision Framework

Enterprise AI adoption follows a predictable pattern: teams reach for sophisticated architectures before validating simpler ones. A six-level ladder, from direct chat at L1 to fine-tuning at L6, lines those options up against the business need behind them. The default sits lower than most roadmaps assume. L2, a system prompt with uploaded knowledge files, covers more internal use cases than it gets credit for, and you climb only when a measured limitation pushes you up.

Two constraints override that default. PII is a hard architectural gate: once personal data enters the flow, L1 and L2 are off the table no matter how well they otherwise fit. Where that gate does not apply, moving up the ladder is a cost and complexity decision you can weigh case by case. Financial services carry the second constraint, adding audit trails, explainability, and human oversight as regulatory obligations on top of GDPR and KVKK.

Six Levels, From a Browser Tab to a Trained Model

Each rung up buys capability and pays for it in complexity, lead time, and operational surface.

The Browser Tab and the Uploaded File (L1 and L2)

L1 is a browser tab: ChatGPT, Claude, or a similar service, used directly. No integration, no customization, context pasted in by hand. It costs $20-60 per user per month with zero development time, which makes it the right home for individual productivity work: writing, brainstorming, code review, research on public information, prototyping prompts, ad-hoc technical questions.

The limits are the flip side of that simplicity. Nothing persists across sessions, there is no audit trail, nothing connects to your business systems, and everything you paste, PII included, lands on a third-party provider’s infrastructure.

// When L1 is sufficient
// Scenario: Developer needs algorithm optimization help

// User simply pastes into Claude:
const prompt = `
Here's my sorting function that's running slowly on large arrays.
Can you suggest optimizations?

function bubbleSort(arr) {
  for (let i = 0; i < arr.length; i++) {
    for (let j = 0; j < arr.length - i - 1; j++) {
      if (arr[j] > arr[j + 1]) {
        [arr[j], arr[j + 1]] = [arr[j + 1], arr[j]];
      }
    }
  }
  return arr;
}
`;

// No API needed, no infrastructure, no development time
// This is the right level for this use case

L2 keeps the same browser tab and adds two things: a custom system prompt and uploaded knowledge files. That turns the generic assistant into a specialized one, in 2-8 hours of setup on a Team or Enterprise tier at $25-60 per user per month. Internal knowledge bases with stable content, public-policy compliance Q&A, onboarding assistants, documentation lookup, and product FAQ systems all fit here.

# Example Claude Project Configuration
Name: "Compliance Policy Assistant"
System Prompt: |
  You are a compliance assistant for our organization.
  Your knowledge is limited to the uploaded policy documents.

  Rules:
  - Only answer questions based on the uploaded documents
  - If information isn't in the documents, say so clearly
  - Always cite the source document and section
  - Never make up policies or procedures
  - For questions outside scope, direct to [email protected]

Knowledge Files:
  - employee-handbook-2025.pdf (150 pages)
  - anti-money-laundering-policy.pdf (80 pages)
  - data-protection-guidelines.pdf (45 pages)

Context Window Usage:
  - System prompt: ~500 tokens
  - Knowledge retrieval: ~50,000 tokens (dynamically loaded)
  - Conversation history: ~20,000 tokens
  - Available for response: ~129,500 tokens (Claude 200K)

L2 Sufficiency Checklist:

• Content is mostly static (updates less than weekly)
• No PII or sensitive business data required
• Knowledge base fits within token limits
• No need for real-time system integration
• Team size under 50 users
• No regulatory audit trail requirements

AI Calls Inside n8n, Make, and Zapier (L3)

At L3, workflow platforms put AI calls inside automations, connecting models to business systems without custom development. Expect $50-600 per month for the platform plus API costs, and one to two weeks of setup.

Platform comparison:

Feature	n8n	Make	Zapier
Self-hosting	Yes	No	No
SOC 2	Yes (Cloud)	Yes	Yes
GDPR Compliance	Yes (self-host)	Yes	Yes
Min Team Cost	$25/month	$16/month	$20/month
Best For	Control, complex flows	Balance	Simplicity

The level earns its keep on high-volume repetitive tasks, multi-system orchestration, and event-driven responses, especially for teams without dedicated AI engineering capacity.

// n8n workflow example: Support ticket classification
const ticketClassificationWorkflow = {
  // Node 1: Webhook receives new Zendesk ticket
  trigger: {
    type: "webhook",
    source: "zendesk"
  },

  // Node 2: AI classification
  aiClassification: {
    prompt: `
      Classify this support ticket into one category:
      - billing: Payment, invoices, subscription issues
      - technical: Product bugs, API errors, integration problems
      - account: Login, permissions, profile updates
      - sales: Pricing questions, upgrades, enterprise inquiries

      Ticket Subject: {{ticket.subject}}
      Ticket Description: {{ticket.description}}

      Return JSON: {"category": "...", "urgency": "low|medium|high"}
    `
  },

  // Node 3: Route based on classification
  routing: {
    billing: { queue: "billing-team", sla: "24h" },
    technical: { queue: "engineering-support", sla: "4h" },
    account: { queue: "customer-success", sla: "12h" },
    sales: { queue: "sales-team", sla: "2h" }
  }
};

// Cost for 5,000 tickets/month:
// n8n Cloud: $25 + OpenAI API ~$10 = $35/month
// vs. manual routing: 2+ hours daily of human time

Building Retrieval Yourself at L4

L4 is the first rung that involves real engineering: retrieval-augmented generation you build yourself, with a vector database, an embedding model, and orchestration code. Full control over the pipeline comes at $500-2000 per month in infrastructure plus 4-8 weeks of development.

Architecture overview:

AWS Bedrock Knowledge Bases implementation:

import {
  BedrockAgentRuntimeClient,
  RetrieveAndGenerateCommand
} from "@aws-sdk/client-bedrock-agent-runtime";

interface RAGResponse {
  answer: string;
  citations: Array<{
    source: string;
    content: string;
    score: number;
  }>;
}

async function queryKnowledgeBase(
  question: string,
  knowledgeBaseId: string
): Promise<RAGResponse> {
  const client = new BedrockAgentRuntimeClient({ region: "eu-west-1" });

  const command = new RetrieveAndGenerateCommand({
    input: { text: question },
    retrieveAndGenerateConfiguration: {
      type: "KNOWLEDGE_BASE",
      knowledgeBaseConfiguration: {
        knowledgeBaseId,
        modelArn: "arn:aws:bedrock:eu-west-1::foundation-model/anthropic.claude-sonnet-4-5-20250929-v1:0",
        retrievalConfiguration: {
          vectorSearchConfiguration: {
            numberOfResults: 10,
            overrideSearchType: "HYBRID"
          }
        },
        generationConfiguration: {
          promptTemplate: {
            textPromptTemplate: `
You are a helpful assistant answering questions based on the provided context.

Context:
$search_results$

Question: $query$

Instructions:
- Answer only based on the provided context
- If the context doesn't contain the answer, say so
- Always cite the source document
- Be concise but thorough
`
          }
        }
      }
    }
  });

  const response = await client.send(command);

  return {
    answer: response.output?.text || "No response generated",
    citations: response.citations?.map(c => ({
      source: c.retrievedReferences?.[0]?.location?.s3Location?.uri || "Unknown",
      content: c.retrievedReferences?.[0]?.content?.text || "",
      score: c.retrievedReferences?.[0]?.score || 0
    })) || []
  };
}

The work becomes necessary once the knowledge base outgrows L2: past 200K tokens or 20 files, documents that change daily, chunking or retrieval logic you need to control yourself. The remaining triggers are compliance-shaped, and any one of them is enough: a mandatory audit trail of queries and responses, data residency you have to prove, or more than a thousand queries a day.

Monthly cost breakdown (100K queries/month):

Component	Service	Cost
Vector DB	OpenSearch Serverless (2 OCU)	$350
Embeddings	Titan (100K queries x 500 tokens)	$1
LLM	Claude Sonnet (100K x 2K tokens)	$600
Storage	S3 (100GB documents)	$3
Lambda	Query processing	$20
Total		~$980/month

L5 Agents With Tools Over MCP

L5 gives the model hands: agents with tool access over the Model Context Protocol (MCP), able to reason, plan, and act across systems.

Architecture pattern:

MCP Server implementation example:

// Note: This example uses MCP SDK v1.x patterns
import { McpServer } from "@modelcontextprotocol/sdk/server/mcp.js";
import { StdioServerTransport } from "@modelcontextprotocol/sdk/server/stdio.js";
import { z } from "zod";

const server = new McpServer({
  name: "customer-support-tools",
  version: "1.0.0"
});

// Tool: Look up customer by email (returns non-PII only)
server.tool(
  "lookup_customer",
  {
    email: z.string().email().describe("Customer email address")
  },
  async ({ email }) => {
    const customer = await db.customers.findByEmail(email);

    if (!customer) {
      return {
        content: [{
          type: "text",
          text: JSON.stringify({ found: false })
        }]
      };
    }

    // Return non-sensitive customer info only
    return {
      content: [{
        type: "text",
        text: JSON.stringify({
          found: true,
          customer_id: customer.id,
          tier: customer.subscription_tier,
          account_status: customer.status
          // Note: No PII like full name, address, payment info
        })
      }]
    };
  }
);

// Tool: Create ticket (high-priority requires human approval)
server.tool(
  "create_ticket",
  {
    customer_id: z.string(),
    subject: z.string(),
    description: z.string(),
    category: z.enum(["billing", "technical", "account", "other"]),
    priority: z.enum(["low", "medium", "high"])
  },
  async ({ customer_id, subject, description, category, priority }) => {
    // High priority or billing = require human approval
    if (priority === "high" || category === "billing") {
      return {
        content: [{
          type: "text",
          text: JSON.stringify({
            status: "pending_approval",
            message: "This ticket requires human approval"
          })
        }]
      };
    }

    const ticket = await db.tickets.create({
      customer_id, subject, description, category, priority,
      created_by: "ai-agent"
    });

    // Audit log for compliance
    await auditLog.write({
      action: "ticket_created_by_agent",
      ticket_id: ticket.id,
      timestamp: new Date()
    });

    return {
      content: [{
        type: "text",
        text: JSON.stringify({ status: "created", ticket_id: ticket.id })
      }]
    };
  }
);

async function main() {
  const transport = new StdioServerTransport();
  await server.connect(transport);
}

main();

The signal for L5 is a workflow no single retrieval pass can cover: multi-step plans, tool selection that depends on context, one query touching several systems, conditional decision trees, or a human approval step in the middle of the flow.

Fine-Tuning, and Why L6 Usually Waits

L6 means training on your own data, for behavior that prompting alone cannot produce. It starts around $2000-10000 per month and assumes ML expertise in-house, which is why most of the cases people bring to it belong one rung lower.

When fine-tuning actually makes sense:

Scenario	Why Fine-tuning	Try First
Specialized terminology	Model doesn’t understand jargon	Few-shot prompting
Consistent output format	Strict formatting requirements	Output parsing
Reduced latency	Single inference vs. RAG	Model distillation
Cost at scale	High volume, per-token expensive	Smaller model
Proprietary knowledge	Can’t use external APIs	On-premises RAG

When to avoid fine-tuning:

• Problem solvable with better prompting (try few-shot first)
• Data changes frequently (re-training is expensive)
• Small dataset (fewer than 1000 examples) - overfitting risk
• Budget constraints (under $1000/month for AI)
• Team lacks ML expertise for training data curation

PII: The Hard Architectural Gate

PII (Personally Identifiable Information) changes what the architecture has to do. Every requirement below comes from law, which means missing one leaves you with a compliance failure no matter how well the system performs.

PII handling requirements by level:

L3 with PII (minimum viable):

interface L3PIIConfig {
  platform: "n8n-self-hosted" | "enterprise-tier-with-dpa";

  aiProvider: {
    // Data Processing Agreement required
    dataProcessingAgreement: string;
    dataResidency: "eu" | "us" | "specific-region";
  };

  security: {
    encryptionAtRest: true;
    encryptionInTransit: true;
    auditLogging: true;
  };

  compliance: {
    retentionPolicy: "30-days" | "as-required";
    deletionProcedure: "documented-and-tested";
  };
}

L4 with PII (recommended):

interface L4PIIArchitecture {
  vectorDatabase: {
    // Self-hosted or with appropriate DPA
    provider: "opensearch-self-hosted" | "pgvector" | "qdrant-private";
    encryption: {
      atRest: "AES-256";
      inTransit: "TLS-1.3";
      keyManagement: "AWS-KMS" | "HashiCorp-Vault";
    };
  };

  llmProvider: {
    // AWS Bedrock with VPC endpoint - data doesn't traverse public internet
    type: "aws-bedrock";
    vpcEndpoint: true;
    modelInvocationLogging: true;
  };

  dataHandling: {
    // PII should be tokenized before embedding
    preprocessing: "tokenization";
    tenantIsolation: true;
    rowLevelSecurity: true;
  };
}

When PII is likely but not yet present, design for it anyway: L4-grade infrastructure, audit logging as a core feature, and a data residency decision made up front. Any of those controls can be relaxed later if the classification never changes.

What Finance Adds on Top of GDPR

Financial services carry AI requirements that general GDPR compliance does not cover.

Regulatory framework:

Jurisdiction	Key Regulations	AI-Specific Requirements
EU	GDPR, AI Act, MiFID II	Explainability, human oversight
US	GLBA, FCRA, state laws	Fair lending, adverse action notices
UK	UK GDPR, FCA rules	Consumer Duty, operational resilience
Turkey	KVKK, BDDK regulations	Data localization (sector-specific, stricter for banking), special categories

At L1-L2, finance teams can still do internal research on public data, review non-customer code, draft general business writing, and build training material. Customer data analysis, transaction monitoring, credit decisions, and investment advice are generally prohibited at those levels.

Finance-specific L4+ requirements:

interface FinanceAIRequirements {
  auditTrail: {
    inputLogging: true;
    modelVersionLogging: true;
    outputLogging: true;
    retentionPeriod: "7-years"; // Regulatory minimum
  };

  explainability: {
    humanReadableExplanations: true;
    featureImportance: true;
    adverseActionNotices: true; // For credit decisions
  };

  humanOversight: {
    materialThreshold: 10000; // Transactions > $10K
    appealProcess: true;
    escalationPath: true;
  };

  modelRiskManagement: {
    // Per SR 11-7 / OCC 2011-12
    modelValidation: "independent-team";
    ongoingMonitoring: true;
    performanceTesting: "quarterly";
  };
}

GDPR/KVKK Pre-Implementation Checklist:

• Legal basis identified (consent, contract, legitimate interest)
• Data Protection Impact Assessment conducted for high-risk processing
• Technical measures implemented (encryption, access controls, audit logging)
• Data Processing Agreement signed with AI provider
• Data subject rights procedures documented (access, deletion, portability)
• Processing activity recorded in ROPA
• Privacy notice updated to include AI processing

None of this bolts on after launch. Model risk documentation, designed-in explainability, a human in the loop for material decisions, a seven-year audit trail, and independent validation all have to shape the first design pass.

From Requirement to Level

The PII gate, the finance requirements, and the per-level limits compress into one flowchart:

Level selection matrix:

Use Case	Recommended Level	Upgrade Signal
Personal productivity	L1	Team needs shared access
Internal FAQ (small)	L2	Content exceeds limits
Internal FAQ (large)	L4	Need multi-system data
Support ticket triage	L3	Complex routing logic
Support agent with actions	L5	None - this is the right fit
Compliance document check	L2-L3	Audit trail required
Document analysis	L4	Domain-specific accuracy
Transaction classification	L6	Latency/cost critical at scale

Deploy the L2 version first, measure accuracy and user satisfaction, and write down the specific limits you hit. Build L4 only for the cases where L2 demonstrably fails, and let the L2 assistant keep serving the simple queries it already answers well.

Two Projects, Opposite Mistakes

A company wanted an AI assistant for its 500-page employee handbook, and the engineering proposal was a full L4 build: OpenSearch, a custom embedding pipeline, an 8-week timeline. The requirement analysis said otherwise:

• 500 pages = ~250K tokens (within Claude’s context)
• Updates: quarterly handbook revisions
• Users: 200 employees
• No audit trail requirement

An L2 Claude Project covered it: 2 hours of setup, $5,000 per month (200 users on the $25 Team plan), accuracy sufficient for handbook Q&A. The savings were 8 weeks of development plus infrastructure that would have run indefinitely.

The opposite mistake carries harder consequences. A fintech startup ran customer transaction pattern analysis through L1 ChatGPT because it deployed fast and needed no infrastructure. Transaction data is PII: there was no data processing agreement with OpenAI, no audit trail for a regulatory examination, and the data was potentially leaving the jurisdiction with every query. Each of those gaps carries GDPR risk on its own. The floor for that workload is L4 on AWS Bedrock: a VPC endpoint so data stays inside AWS, model invocation logging for the audit trail, an EU region for residency.

Monthly Cost at 10K Queries

Estimates for a mid-size enterprise:

Level	Infrastructure	API/Usage	Dev Time (One-time)	Monthly Total
L1	$0	$400 (20 users)	0	$400
L2	$0	$500 (20 users)	8 hours	$500
L3	$100	$50	40 hours	$150
L4	$500	$300	160 hours	$800
L5	$1,000	$800	320 hours	$1,800
L6	$2,500	$500	400 hours	$3,000

Development costs are one-time; ongoing maintenance adds to the monthly total from L4 upward.

Model Choice Within a Level

Choosing a level fixes the architecture, but the unit economics depend on which model runs inside it. Premium models often get pointed at work a cheaper one handles just as well.

Model Prices, January 2026

Anthropic Claude Models:

Model	Input (/1M)	Output (/1M)	Context	Best For
Opus 4.5	$5.00	$25.00	200K	Complex reasoning, critical decisions
Sonnet 4.5	$3.00	$15.00	200K-1M	Code analysis, RAG, general purpose
Haiku 4.5	$1.00	$5.00	200K	Fast tasks, classification, simple Q&A
Haiku 3.5	$0.80	$4.00	200K	Budget tasks, high volume

OpenAI Models:

Model	Input (/1M)	Output (/1M)	Context	Best For
GPT-4.1	$2.00	$8.00	1M	General purpose, large context
o3	$2.00	$8.00	200K	Complex reasoning, math, coding
o4-mini	$1.10	$4.40	200K	Fast reasoning tasks
GPT-4o	$2.50	$10.00	128K	Multimodal, general purpose
GPT-4o-mini	$0.15	$0.60	128K	Budget tasks, simple operations

Google Gemini Models:

Model	Input (/1M)	Output (/1M)	Context	Best For
Gemini 2.5 Pro	$1.25-2.50	$10-15	1M	Coding, complex prompts
Gemini 2.5 Flash	$0.30	$2.50	1M	Fast, cost-efficient
Gemini 2.5 Flash-Lite	$0.10	$0.40	1M	Highest efficiency
Gemini 2.0 Flash	$0.10	$0.40	1M	Ultra-fast, budget option

Routing Tasks to the Cheapest Model That Works

The common mistake is using premium models for tasks that don’t require them:

Task Type	Wrong Choice	Right Choice	Cost Savings
Simple Q&A, FAQ	Opus 4.5 ($5)	Haiku 4.5 ($1)	5x
Document classification	Sonnet 4.5 ($3)	GPT-4o-mini ($0.15)	20x
Text summarization	GPT-4o ($2.50)	Gemini Flash ($0.30)	8x
Code review	Haiku ($1)	Sonnet 4.5 ($3)	Quality improvement
Financial analysis	Haiku ($1)	Opus/o3 ($5)	Risk reduction
Complex reasoning	Sonnet ($3)	o3 ($2)	Better accuracy

In production that mapping belongs in code, with a cheap classifier deciding where each request goes:

interface ModelRouter {
  // Classify incoming request complexity
  classifier: {
    model: "haiku-4.5"; // Use cheap model to classify
    categories: ["simple", "medium", "complex", "critical"];
  };

  // Route to appropriate model
  routing: {
    simple: {
      model: "gpt-4o-mini",
      costPer1M: 0.15,
      useCases: ["FAQ", "formatting", "classification"]
    };
    medium: {
      model: "sonnet-4.5",
      costPer1M: 3.00,
      useCases: ["summarization", "code-review", "analysis"]
    };
    complex: {
      model: "o3",
      costPer1M: 2.00,
      useCases: ["reasoning", "math", "multi-step"]
    };
    critical: {
      model: "opus-4.5",
      costPer1M: 5.00,
      useCases: ["financial-decisions", "compliance", "legal"]
    };
  };
}

Cutting the Token Bill

Two levers need no code change. Both Anthropic and OpenAI offer 50% discounts on batch processing, which fits document pipelines, nightly analysis jobs, and bulk classification. Anthropic prices cache reads at 10% of the base input price, which pays off on repeated system prompts, shared context blocks, and RAG over a stable knowledge base.

The third lever is a cascade: start with the cheapest model, escalate only on failure.

async function cascadeQuery(prompt: string): Promise<string> {
  // Try cheap model first
  const haiku = await query("haiku-4.5", prompt);
  if (haiku.confidence > 0.8) return haiku.response;

  // Escalate to mid-tier
  const sonnet = await query("sonnet-4.5", prompt);
  if (sonnet.confidence > 0.9) return sonnet.response;

  // Final escalation for complex cases
  return await query("opus-4.5", prompt);
}

Context size is the quiet one. Most chatbot interactions never leave a 128K window (GPT-4o-mini), document Q&A sits comfortably in the 200K Claude models, and the 1M options (Gemini Pro, GPT-4.1) earn their price only on work like full codebase analysis.

Which Model at Which Level

Level	Budget Model	Standard Model	Premium Model
L1	ChatGPT Free	Claude Pro ($20/mo)	ChatGPT Plus ($20/mo)
L2	-	Claude Team ($25/user)	ChatGPT Business ($30/user)
L3	GPT-4o-mini API	Sonnet 4.5 API	o3 API
L4	Haiku + Titan Embed	Sonnet + Titan	Opus + Cohere
L5	Haiku for routing	Sonnet for agents	Opus for critical
L6	Fine-tuned small	Fine-tuned medium	Custom large

In practice most production traffic can run on the budget column, with the premium column reserved for the requests that need it.

Where the Default Ends

L2 holds as the default for internal knowledge work with stable content, no PII, and a team under 50 users. Override it upward when the evidence is concrete: retrieval quality that fails your own test set, a knowledge base past the context limit, an audit trail a regulator will ask for, or a data classification that brings PII into scope. Override it downward when an L4 pipeline turns out to answer questions a single uploaded PDF would have covered.

References

• Model Context Protocol Specification - Official MCP specification for tool-augmented LLM agent integration
• MCP TypeScript SDK - Reference implementation for building MCP servers and clients
• OpenTelemetry Specification - Observability standard for tracing AI agent workflows
• AWS Secrets Manager - Managed secrets for enterprise AI integration credentials
• Amazon Cognito User Pools - Identity management for multi-tenant enterprise AI applications
• OAuth 2.0 Authorization Framework - RFC 6749 - Authorization standard for API-level AI integrations requiring delegated access
• AWS KMS Key Management Service - Customer-managed encryption keys for PII and finance data compliance
• Amazon Bedrock Knowledge Bases - Managed RAG service behind the L4 retrieval example, including vector store and ingestion options
• Anthropic Model Pricing - Per-token list prices for the Claude model family used in the selection tables
• OpenAI API Pricing - Per-token pricing and context limits for the GPT and o-series models
• Gemini API Pricing - Per-token pricing tiers for the Gemini model family
• GDPR Full Text (EUR-Lex) - Official Regulation (EU) 2016/679 text covering legal basis, impact assessments, and data subject rights