AI Agents in Workflows

Integrate AI models and intelligent agents into your workflows for advanced text processing, classification, extraction, and generation.

AI Gateway Node

The AI Gateway node connects your workflows to language models for inference and generation.

Supported Models

Access models through your configured AI Gateway:

Self-Hosted Models

• Llama, Mistral, Phi, Gemma
• DeepSeek, Qwen, Yi
• Custom fine-tuned models

Third-Party Providers

• OpenAI (GPT-4, GPT-3.5)
• Anthropic (Claude)
• Google (Gemini)
• Cohere, Groq, Together AI

Learn more about AI Gateway →

Configuration

Setting	Description
Model Selection	Choose from available models
System Prompt	Define model behavior and context
User Prompt	The input message or query
Temperature	Randomness (0.0 - 2.0)
Max Tokens	Maximum response length
Top P	Nucleus sampling parameter
Stop Sequences	Strings that halt generation

Basic Example

Configuration:

{
  "model": "gpt-4",
  "systemPrompt": "You are a helpful assistant that summarizes documents.",
  "userPrompt": "Summarize the following text:\n\n{{ pdfParser.text }}",
  "temperature": 0.7,
  "maxTokens": 500
}

Accessing Output:

{{ aiGateway.response }}
{{ aiGateway.tokenCount }}
{{ aiGateway.model }}

Advanced Features

Structured Output

Request JSON-formatted responses:

{
  "systemPrompt": "Extract information in JSON format.",
  "userPrompt": "Extract person name, email, and phone from:\n{{ formData.message }}",
  "responseFormat": {
    "type": "json_object",
    "schema": {
      "name": "string",
      "email": "string",
      "phone": "string"
    }
  }
}

Function Calling

Enable models to call external functions:

{
  "functions": [
    {
      "name": "get_weather",
      "description": "Get current weather for a location",
      "parameters": {
        "location": "string",
        "unit": "celsius | fahrenheit"
      }
    }
  ],
  "functionCall": "auto"
}

Multi-Turn Conversations

Use Conversation Memory for context:

Trigger
  → Conversation Memory (Load history)
  → AI Gateway (Generate response)
  → Conversation Memory (Save exchange)
  → Destination

Streaming Responses

Enable streaming for real-time output:

{
  "stream": true,
  "onChunk": "{{ chunk }}"
}

Prompt Engineering

Best Practices:

1. Be Specific: Clearly describe the desired output
2. Provide Context: Include relevant background information
3. Use Examples: Show the model what you want (few-shot)
4. Set Constraints: Define format, length, style requirements
5. Handle Errors: Include fallback instructions

Example Prompts:

// Classification
`Classify the sentiment of this review as positive, negative, or neutral:

Review: {{ review.text }}

Classification:`

// Extraction
`Extract all email addresses from the following text. Return as JSON array.

Text: {{ emailParser.body }}

Emails:`

// Generation
`Write a professional response to this customer inquiry:

Inquiry: {{ trigger.formData.message }}

Response guidelines:
- Be friendly and helpful
- Address their specific question
- Keep under 150 words
- End with a call to action

Response:`

// Summarization
`Summarize the key points from this document in 3-5 bullet points:

Document: {{ pdfParser.text }}

Summary:`

Cost Management

Token Usage Tracking:

{{ aiGateway.promptTokens }}
{{ aiGateway.completionTokens }}
{{ aiGateway.totalTokens }}
{{ aiGateway.estimatedCost }}

Optimization Strategies:

1. Use smaller models for simple tasks
2. Set appropriate maxTokens limits
3. Cache common responses
4. Batch similar requests
5. Use context windows efficiently

Agent Nodes

Specialized agents for common AI tasks.

Entity Extraction Agent

Extract structured entities from unstructured text.

Use Cases:

• Extract names, dates, locations from documents
• Parse contact information from emails
• Identify products and prices from invoices
• Extract skills from resumes

Configuration:

{
  "text": "{{ documentParser.content }}",
  "entityTypes": [
    "PERSON",
    "ORGANIZATION",
    "LOCATION",
    "DATE",
    "MONEY",
    "EMAIL",
    "PHONE"
  ],
  "customEntities": [
    {
      "type": "PRODUCT_ID",
      "pattern": "PRD-\\d{6}"
    }
  ]
}

Output Format:

{
  "entities": [
    {
      "text": "John Smith",
      "type": "PERSON",
      "confidence": 0.95,
      "start": 0,
      "end": 10
    },
    {
      "text": "john@example.com",
      "type": "EMAIL",
      "confidence": 0.99,
      "start": 25,
      "end": 41
    }
  ]
}

Accessing Results:

// Get all entities
{{ entityExtraction.entities }}

// Filter by type
{{ entityExtraction.entities }}.filter(e => e.type === 'PERSON')

// Get first entity
{{ entityExtraction.entities[0].text }}

Document Classification Agent

Categorize documents into predefined classes.

Use Cases:

• Classify support tickets by priority/category
• Sort emails into folders
• Tag documents by department
• Identify document types (invoice, receipt, contract)

Configuration:

{
  "text": "{{ pdfParser.text }}",
  "categories": [
    "invoice",
    "receipt",
    "contract",
    "report",
    "other"
  ],
  "multiLabel": false,
  "threshold": 0.5
}

Training with Examples:

{
  "trainingData": [
    {
      "text": "Invoice #12345 Amount Due: $500",
      "label": "invoice"
    },
    {
      "text": "This agreement is entered into on...",
      "label": "contract"
    }
  ]
}

Output Format:

{
  "predictions": [
    {
      "category": "invoice",
      "confidence": 0.92
    },
    {
      "category": "receipt",
      "confidence": 0.05
    }
  ],
  "topCategory": "invoice",
  "topConfidence": 0.92
}

Accessing Results:

// Get top category
{{ classification.topCategory }}

// Get confidence score
{{ classification.topConfidence }}

// Check if confident
{{ classification.topConfidence > 0.8 }}

Memory Nodes

Manage conversation context and knowledge.

Conversation Memory

Store and retrieve chat history for multi-turn conversations.

Configuration:

{
  "sessionId": "{{ trigger.userId }}",
  "maxMessages": 10,
  "includeSystemMessages": true
}

Operations:

Operation	Description
Load	Retrieve conversation history
Save	Store new message exchange
Clear	Reset conversation
Search	Find specific messages

Usage Pattern:

1. Load conversation history
2. Include history in AI prompt
3. Generate AI response
4. Save exchange to memory

Context Buffer

Manage token budgets for large contexts.

Configuration:

{
  "maxTokens": 4000,
  "strategy": "sliding_window",
  "preserveFirst": true
}

Strategies:

Strategy	Behavior
Sliding Window	Keep most recent messages
Summarize	Compress old messages
Truncate	Remove oldest messages

Knowledge Base

Query vector databases for semantic search (RAG).

Configuration:

{
  "vectorDB": "milvus",
  "collection": "product_docs",
  "query": "{{ trigger.question }}",
  "topK": 5,
  "similarityThreshold": 0.7
}

RAG Pattern:

Trigger (User question)
  → Knowledge Base (Retrieve relevant docs)
  → AI Gateway (Generate answer with context)
  → Response

AI Workflow Patterns

Pattern: Document Q&A

PDF Upload Trigger
  → PDF Parser (Extract text)
  → Knowledge Base (Store embeddings)
  → REST API Trigger (Questions)
    → Knowledge Base (Retrieve relevant sections)
    → AI Gateway (Answer with context)
    → API Return

Pattern: Email Triage

Email Trigger
  → Email Parser
  → Entity Extraction (Extract key info)
  → Document Classification (Categorize)
  → Conditional (Route by category)
    → [Urgent] → AI Gateway (Generate response) → Gmail
    → [Normal] → MongoDB (Queue)
    → [Spam] → Delete

Pattern: Content Generation

Schedule Trigger
  → REST API (Fetch topics)
  → Loop (Each topic)
    → AI Gateway (Generate content)
    → Document Classification (Quality check)
    → Conditional (Quality > 0.8)
      → [Pass] → MongoDB (Save)
      → [Fail] → AI Gateway (Regenerate)

Pattern: Sentiment Analysis Pipeline

Form Trigger (Customer feedback)
  → AI Gateway (Analyze sentiment)
  → Conditional (Sentiment)
    → [Negative] → Alert team + Generate response
    → [Neutral] → Store for review
    → [Positive] → Thank you email + Store

Best Practices

Model Selection

1. Task Complexity: Use smaller models for simple tasks
2. Latency Requirements: Faster models for real-time workflows
3. Cost Considerations: Balance quality and cost
4. Accuracy Needs: Larger models for critical decisions

Prompt Design

1. Test Iteratively: Refine prompts based on outputs
2. Version Control: Track prompt changes
3. Use Templates: Create reusable prompt patterns
4. Validate Outputs: Check response format and quality

Error Handling

1. Retry Logic: Handle temporary API failures
2. Fallback Models: Switch to backup model on error
3. Output Validation: Verify response format
4. Timeouts: Set appropriate timeout values

Performance

1. Batch Requests: Process multiple items together
2. Cache Results: Store common responses
3. Async Processing: Don't block on long generations
4. Stream Responses: Use streaming for better UX

Security

1. Input Sanitization: Clean user inputs
2. Output Filtering: Remove sensitive data
3. Rate Limiting: Prevent abuse
4. Audit Logging: Track AI usage

Monitoring AI Workflows

Track key metrics:

Metric	Purpose
Token Usage	Cost management
Response Time	Performance optimization
Error Rate	Reliability monitoring
Quality Scores	Output validation
User Feedback	Continuous improvement

Testing AI Workflows

Always test AI workflows with diverse inputs to ensure robust performance across edge cases.

Next Steps

• Explore MCP tools
• Test AI workflows
• Deploy to production
• See AI workflow examples