# Resource Manager Tutorial
This tutorial covers how to use the Resource Manager for allocating and managing resources like VLANs, ports, IP addresses, and hostnames.
## Table of Contents
1. [Template Definition (YAML)](#1-template-definition-yaml)
2. [Integer Resource Types](#2-integer-resource-types)
3. [String Resource Types & DSL](#3-string-resource-types--dsl)
4. [Workflows for Allocation/Deallocation](#4-workflows-for-allocationdeallocation)
5. [Resource Manager External API](#5-resource-manager-external-api)
---
## 1. Template Definition (YAML)
Resource templates define how resources should be allocated. Templates use YAML/JSON format:
**⚠️ Note: Template management will eventually move to Git-based versioning. Currently, templates are managed via gRPC APIs.**
**⚠️ Note: The template's format will be changed in the CURRENT version. We'll update all relevant people after this change is made, and/or fix existing templates to the new format.**
### Basic Template Structure
```yaml
name: "vlan-pool"
scope: "datacenter-east"
description: "VLAN ID allocation for network interfaces"
resources:
- name: "vlan_id"
base_type: "vlan"
resource_type: "int"
external: false # Resource Manager provides candidates and stores values
constraints:
start: 100
end: 4000
workflows:
allocation: "allocate-vlan-workflow"
deallocation: "deallocate-vlan-workflow"
```
### Template Fields
**User-Provided Fields:**
- **name**: Unique template identifier
- **scope**: Allocation scope (e.g., datacenter, environment)
- **description**: Human-readable description
- **version**: Initial version (user provides at creation, backend manages thereafter)
- **status**: Initial status (user provides at creation, backend manages thereafter)
- **resources**: Array of resource definitions
### Resource Definition Fields
**User-Provided Fields:**
- **name**: Resource field name
- **base_type**: Resource identifier (e.g., "vlan", "port", "hostname")
- **resource_type**: Pool type ("int" or "string")
- **external**: Boolean indicating resource ownership (OPTIONAL, default: `false`)
- `true`: Resource Manager provides candidates
- `false`: External system provides values
- **constraints**: Type-specific constraints (start/end/step for int, pattern for string)
- **workflows**: Allocation and deallocation workflow names (OPTIONAL)
- `allocation`: Workflow for allocation (default: built-in allocator selects first available value)
- `deallocation`: Workflow for cleanup (no default, must be specified if needed)
**Backend-Managed Fields:**
- **id**: Unique template ID (auto-generated)
- **version**: Template version (user provides initial, backend manages updates)
- **status**: Template status (user provides initial, backend manages lifecycle: `active`, `inactive`, `deprecated`)
- **created_at**: Creation timestamp (auto-generated)
- **updated_at**: Last modification timestamp (auto-generated)
### Default Values Summary
| Field | Default Value | Description |
|-------|---------------|-------------|
| `external` | `false` | External system provides candidates |
| `step` (integer) | `1` | Increment by 1 |
| `start` (integer) | `0` | Start from 0 |
| `exclude` (integer) | `[]` | No values excluded |
| `exclude` (string) | `[]` | No values excluded |
| `allocation` workflow | Built-in allocator | Selects first available value |
| `deallocation` workflow | None | Must specify if cleanup needed |
### Custom Scope Templates
Templates can use `scope="custom"` for runtime scoping, allowing a single template to support multiple isolated resource pools.
**Use Case**: Define one template that can be used for multiple instances without creating separate templates.
**Example**: Router interfaces where each router instance needs its own isolated interface pool.
```yaml
name: "router-interfaces"
scope: "custom" # Enables runtime scoping
description: "Interface allocation for router instances"
resources:
- name: "interface_name"
base_type: "string"
resource_type: "interface"
constraints:
pattern: "eth{seq:start=0:end=47:reusable=true}"
```
**Allocation Request with Custom Scope:**
```json
{
"template_name": "router-interfaces",
"template_scope": "custom",
"allocation_key": "router-clli-12345", // Required for scope="custom"
"request_id": "interface-req-1"
}
```
**How Custom Scoping Works:**
- Each `allocation_key` creates an isolated resource pool
- Resources are unique within each allocation key scope
- Different allocation keys can reuse the same resource values
- Example: Router "clli-12345" gets `eth0`, Router "clli-67890" also gets `eth0`
**Requirements:**
- Templates with `scope="custom"` require `allocation_key` in allocation requests
- Templates with fixed scope cannot use `allocation_key`
---
## 2. Integer Resource Types
Integer resources are used for numeric allocations like VLANs, ports, or IDs.
### Integer Constraints
```yaml
resources:
- name: "vlan_id"
base_type: "vlan"
resource_type: "int"
external: false # Resource ownership (default: true)
constraints:
start: 100 # Minimum value (default: 0)
end: 4000 # Maximum value (default: MaxInt64)
step: 1 # Increment step (default: 1)
exclude: [150, 200] # Values to exclude from allocation (optional)
```
### Integer Examples
**VLAN Pool:**
```yaml
name: "vlan-pool"
resources:
- name: "vlan_id"
base_type: "vlan"
resource_type: "int"
# external: false (default)
constraints:
start: 100
end: 4000
# workflows: uses default allocation workflow (selects first available)
```
**Simple Example (All Defaults):**
```yaml
name: "simple-ids"
resources:
- name: "id"
base_type: "vlan"
resource_type: "id"
constraints:
start: 1
end: 1000
# Uses all defaults:
# - external: false (RM provides candidates)
# - step: 1 (increment by 1)
# - allocation workflow: built-in (selects first available)
# - no deallocation workflow
```
**Custom Step Size:**
```yaml
name: "even-numbers"
resources:
- name: "even_id"
base_type: "vlan"
resource_type: "even_id"
constraints:
start: 1000
end: 2
step: -2 # Only even numbers
```
---
## 3. String Resource Types & DSL
String resources use a powerful DSL for generating complex identifiers.
### String Constraints
```yaml
resources:
- name: "hostname"
base_type: "string"
resource_type: "hostname"
external: false # Resource ownership (default: true)
constraints:
pattern: "srv-{choice:options=web,api,db}-{seq:start=1:end=999:reusable=false}"
exclude: ["srv-web-1", "srv-api-5"] # Values to exclude from allocation (optional)
```
**String Constraint Fields:**
- **pattern**: DSL pattern specification (REQUIRED)
- **exclude**: Array of values to exclude from allocation (OPTIONAL)
**Exclude Constraint Behavior:**
- **Template excludes**: Permanently excluded values defined in template
- **Filter excludes**: Runtime excludes passed via workflow filters (when supported)
- **Reusable sequences**: Work well with excludes (multiple candidates available)
- **Non-reusable sequences**: Problematic with excludes (single candidate may be excluded)
**⚠️ Exclude Limitations:**
- **Non-reusable sequences + excludes**: May cause allocation failures when sequence hits excluded value
- **Recommendation**: Avoid excludes with `reusable=false` sequences
- **Filter support**: Available via workflow filters (requires custom workflow implementation)
**Pattern Field:**
The `pattern` field uses the Resource Manager's Domain Specific Language (DSL) to define how string values should be generated. The DSL supports multiple pattern types that can be combined to create complex identifiers.
### DSL Pattern Types
The DSL uses explicit named parameters for all pattern types. **Positional parameters are NOT supported.**
### DSL Escape Sequences
The DSL uses special characters for pattern syntax. When you need these characters as literals in your output, you must escape them with a backslash (`\`).
#### **Special Characters**
| Character | Purpose | Must Escape When |
|-----------|---------|------------------|
| **`{`** | Pattern start | You want literal `{` in static text |
| **`}`** | Pattern end | You want literal `}` in static text |
| **`:`** | Parameter separator | You want literal `:` in static text |
| **`\`** | Escape character | You want literal `\` anywhere |
#### **Escape Sequences**
| Escape | Result | Usage |
|--------|--------|-------|
| **`\{`** | `{` | Literal opening brace |
| **`\}`** | `}` | Literal closing brace |
| **`\:`** | `:` | Literal colon |
| **`\\`** | `\` | Literal backslash |
#### **Where Escaping Applies**
**In Static Text (outside patterns):**
- **`:`** must be escaped → `\:`
- **`{`** and **`}`** must be escaped → `\{` and `\}`
- **`\`** must be escaped → `\\`
**Inside Pattern Parameters:**
- Same escaping rules apply when defining custom character sets or parameter values
- Example: `{str:charset=custom:chars=A\:B\{C\}:length=3}` includes `:`, `{`, `}` in the character set
#### **Validation**
The parser will reject patterns with unescaped special characters and provide clear error messages indicating the position and required escape sequence.
#### **1. Number Patterns** - `{num:...}`
Generates random numeric values within specified ranges with optional zero-padding.
**Parameters:**
| Parameter | Required | Default | Description |
|-----------|----------|---------|-------------|
| `start=N` | Yes | None | Minimum value (inclusive) |
| `end=N` | Yes | None | Maximum value (inclusive) |
| `length=N` | No | No padding | Zero-padding length |
**Parameter Details:**
**start=N:**
- **Purpose**: Lower bound of the numeric range (inclusive)
- **Required**: YES - no default value
- **Examples**: `start=1`, `start=0`, `start=-10`
- **Validation**: Can be negative, must be ≤ end value
**end=N:**
- **Purpose**: Upper bound of the numeric range (inclusive)
- **Required**: YES - no default value
- **Examples**: `end=100`, `end=4094`, `end=65535`
- **Validation**: Must be ≥ start value
**length=N:**
- **Purpose**: Zero-pad numbers to specified length
- **Required**: NO
- **Default**: No padding (numbers as-is)
- **Examples**: `length=3` → "042", `length=5` → "00123"
- **Validation**: Must be > 0 if specified
**Examples:**
```yaml
{num:start=1:end=100} # Random 1-100: "42", "87", "3"
{num:start=1:end=999:length=5} # Zero-padded: "00042", "00123", "00999"
{num:start=-10:end=10} # Negative range: "-5", "0", "7"
{num:start=1:end=100:length=3} # Padded range: "042", "087", "003"
{num:start=8000:end=9999} # Port range: "8000", "8001", "8002"
```
**Common Use Cases:**
| Use Case | Configuration | Result Examples |
|----------|---------------|-----------------|
| **Port Numbers** | `{num:start=8000:end=9999}` | "8000", "8001", "8002" |
| **VLAN IDs** | `{num:start=100:end=4000}` | "100", "101", "102" |
| **Zero-Padded IDs** | `{num:start=1:end=999:length=4}` | "0001", "0042", "0999" |
| **Small Ranges** | `{num:start=1:end=10}` | "1", "2", "3" |
#### **2. String Patterns** - `{str:...}`
Generates random strings from predefined or custom character sets.
**Parameters:**
| Parameter | Required | Default | Description |
|-----------|----------|---------|-------------|
| `charset=NAME` | Yes | None | Character set type |
| `length=N` | Yes | None | String length in characters |
| `chars=CUSTOM` | No | None | Custom character set (sets charset=custom) |
**Parameter Details:**
**charset=NAME:**
- **Purpose**: Specifies which character set to use for generation
- **Required**: YES - no default charset
- **Options**: `alpha`, `numeric`, `alphanumeric`, `lowercase`, `uppercase`, `hex`, `custom`
- **Validation**: Must be a supported charset name
**length=N:**
- **Purpose**: Length of generated string in characters
- **Required**: YES - no default length
- **Examples**: `length=5`, `length=8`, `length=12`
- **Validation**: Must be > 0
**chars=CUSTOM:**
- **Purpose**: Define custom character set for random selection
- **Required**: NO (only when using custom charset)
- **Auto-behavior**: Automatically sets `charset=custom`
- **Examples**: `chars=abc123`, `chars=ABCDEF0123456789`
- **Validation**: Cannot be empty if specified
**Supported Character Sets:**
| Charset | Characters | Use Cases |
|---------|------------|-----------|
| **alpha** | `abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ` | General identifiers |
| **numeric** | `0123456789` | Numeric codes |
| **alphanumeric** | Letters + digits (62 chars) | Mixed identifiers |
| **lowercase** | `abcdefghijklmnopqrstuvwxyz` | Lowercase identifiers |
| **uppercase** | `ABCDEFGHIJKLMNOPQRSTUVWXYZ` | Uppercase identifiers |
| **hex** | `0123456789abcdef` | Hexadecimal values |
| **custom** | User-defined via `chars=` | Specific character requirements |
**Examples:**
```yaml
{str:charset=alpha:length=5} # "AbCdE", "xyZaB"
{str:charset=numeric:length=6} # "123456", "987654"
{str:charset=alphanumeric:length=10} # "aB3xY7zQ9m"
{str:charset=custom:chars=abc123:length=4} # "abc", "cab", "1a2", "3cb"
{str:charset=hex:length=8} # "a1b2c3d4", "f9e8d7c6"
{str:charset=lowercase:length=6} # "abcdef", "ghijkl"
{str:charset=uppercase:length=4} # "ABCD", "EFGH"
```
**Common Use Cases:**
| Use Case | Configuration | Result Examples |
|----------|---------------|-----------------|
| **Server Names** | `{str:charset=lowercase:length=6}` | "websvr", "apisvr", "dbsvr" |
| **Hex Tokens** | `{str:charset=hex:length=8}` | "a1b2c3d4", "f9e8d7c6" |
| **Mixed IDs** | `{str:charset=alphanumeric:length=10}` | "aB3xY7zQ9m", "X2nP8vK4sR" |
| **Custom Codes** | `{str:charset=custom:chars=ABCDEF123:length=5}` | "AB1C2", "3DEFF", "1A2B3" |
#### **3. Choice Patterns** - `{choice:...}`
Selects values from predefined lists with equal or weighted probabilities.
**Parameters:**
| Parameter | Required | Default | Description |
|-----------|----------|---------|-------------|
| `options=LIST` | Yes | None | Comma-separated list of choices |
| `type=TYPE` | No | `simple` | Choice type: `simple` or `weighted` |
**Parameter Details:**
**options=LIST:**
- **Purpose**: Define the list of possible values to choose from
- **Required**: YES - no default options
- **Format**: Comma-separated values (no spaces around commas)
- **Examples**: `options=dev,test,prod`, `options=active,inactive`
- **Validation**: Cannot be empty, must have at least one option
**type=TYPE:**
- **Purpose**: Controls selection probability distribution
- **Required**: NO
- **Default**: `simple` (equal probability for all options)
- **Options**: `simple` or `weighted`
- **Validation**: Must be exactly `simple` or `weighted`
**Choice Types:**
**Simple Choice (`type=simple` or default):**
- **Behavior**: Equal probability for all options
- **Format**: `options=option1,option2,option3`
- **Probability**: Each option gets `1/N` chance (where N = number of options)
- **Examples**:
- `{choice:options=dev,test,prod}` → 33.3% chance each
- `{choice:options=active,inactive}` → 50% chance each
**Weighted Choice (`type=weighted`):**
- **Behavior**: Custom probability distribution
- **Format**: `options=option1-weight1,option2-weight2,option3-weight3`
- **Weights**: Numbers after dash (`-`) specify relative probability
- **Examples**:
- `{choice:type=weighted:options=dev-70,prod-30}` → 70% dev, 30% prod
- `{choice:type=weighted:options=standard-60,premium-30,enterprise-10}` → 60%, 30%, 10%
**Examples:**
```yaml
# Simple choice (equal probability)
{choice:options=dev,test,prod} # 33.3% each: "dev", "test", "prod"
{choice:options=active,inactive} # 50% each: "active", "inactive"
{choice:type=simple:options=enabled,disabled} # Explicit simple type
# Weighted choice (custom probability)
{choice:type=weighted:options=dev-70,prod-30} # 70% dev, 30% prod
{choice:type=weighted:options=standard-60,premium-30,enterprise-10} # Custom distribution
{choice:type=weighted:options=web-80,api-15,db-5} # Server type distribution
```
**Common Use Cases:**
| Use Case | Configuration | Result Examples |
|----------|---------------|-----------------|
| **Environment Types** | `{choice:options=dev,test,prod}` | "dev", "test", "prod" |
| **Server Roles** | `{choice:type=weighted:options=web-60,api-30,db-10}` | "web" (60%), "api" (30%), "db" (10%) |
| **Status Values** | `{choice:options=active,inactive,pending}` | "active", "inactive", "pending" |
| **Service Tiers** | `{choice:type=weighted:options=standard-80,premium-20}` | "standard" (80%), "premium" (20%) |
#### **4. Sequence Patterns** - `{seq:...}`
Generates sequential numeric values with gap-filling or monotonic behavior. **Supports multi-dimensional sequences** for complex combinations.
**Parameters:**
| Parameter | Required | Default | Description |
|-----------|----------|---------|-------------|
| `start=N` | No | `1` | Starting value for the sequence |
| `end=N` | No | Auto-calculated | Ending value (max 10M values total) |
| `step=N` | No | `1` | Increment step between values |
| `reusable=BOOL` | No | `true` | Whether values can be reused after deallocation |
**Understanding Reusability:**
**Reusable (`reusable=true`) - Gap-Filling Behavior:**
- **What it means**: Values can be **reused after deallocation**
- **Behavior**: Fills gaps in the sequence when values are freed
- **Perfect for**: Interface names, VLAN IDs, port numbers, temporary resources
- **Example**:
- Allocate: `lo0`, `lo1`, `lo2`
- Deallocate: `lo1`
- Next allocation: `lo1` (reuses the freed value)
- **Implementation**: Uses `generate_series` with availability checking
**Non-Reusable (`reusable=false`) - Monotonic Behavior:**
- **What it means**: Values **NEVER reused, even after deallocation**
- **Behavior**: Sequence continues monotonically, always increments
- **Perfect for**: Transaction IDs, audit trails, user IDs, permanent identifiers
- **Example**:
- Allocate: `tx_1`, `tx_2`, `tx_3`
- Deallocate: `tx_2`
- Next allocation: `tx_4` (tx_2 never used again)
- **Implementation**: Uses PostgreSQL sequences for atomic increments
**Parameter Details:**
**start=N:**
- **Purpose**: Where the sequence begins counting
- **Default**: `1`
- **Examples**: `start=0` (lo0, lo1...), `start=100` (vlan100, vlan101...)
**end=N:**
- **Purpose**: Maximum value in the sequence range
- **Default**: Auto-calculated to ensure max 10,000,000 values
- **Examples**: `end=4094` (VLAN range), `end=65535` (port range)
- **Validation**: Must be ≥ start value
**step=N:**
- **Purpose**: Increment between consecutive values
- **Default**: `1`
- **Examples**: `step=2` (even numbers: 2,4,6...), `step=10` (10,20,30...)
- **Validation**: Must be > 0
**reusable=BOOL:**
- **Purpose**: Controls gap-filling vs monotonic behavior
- **Default**: `true` (gap-filling)
- **Values**: `true` (reusable) or `false` (monotonic)
**Examples:**
```yaml
{seq} # Default: start=1, step=1, reusable=true, end=auto
{seq:start=0:end=4094:reusable=true} # Interface names: lo0, lo1, lo2... (fills gaps)
{seq:start=1:end=10000:reusable=false} # Transaction IDs: tx_1, tx_2, tx_3... (never reuses)
{seq:step=2:reusable=false} # Even numbers: 2, 4, 6, 8... (monotonic)
{seq:start=100:step=5:reusable=true} # Custom increment: 100, 105, 110... (gap-filling)
{seq:start=1000:end=9999:step=1:reusable=false} # User IDs: 1000, 1001, 1002... (permanent)
```
**Multi-Dimensional Sequences:**
Multiple reusable sequences in a single pattern create **multi-dimensional combinations**:
```yaml
# 2D sequences - generates all combinations
"ge{seq:start=0:end=2:reusable=true}/{seq:start=0:end=2:reusable=true}"
# Results: ge0/0, ge0/1, ge0/2, ge1/0, ge1/1, ge1/2, ge2/0, ge2/1, ge2/2
# 3D sequences - generates 3×3×3 = 27 combinations
"ge100-{seq:start=0:end=2:reusable=true}/{seq:start=0:end=2:reusable=true}/{seq:start=0:end=2:reusable=true}"
# Results: ge100-0/0/0, ge100-0/0/1, ge100-0/0/2, ..., ge100-2/2/2
# Descending sequences with negative steps
"item_{seq:start=5:end=1:step=-1:reusable=true}_{seq:start=10:end=5:step=-1:reusable=true}"
# Results: item_5_10, item_5_9, item_5_8, ..., item_1_5
```
**⚠️ Sequence Limitations:**
- **✅ Multiple reusable sequences**: Supported (generates all combinations)
- **❌ Multiple non-reusable sequences**: NOT supported (would waste sequence values)
- **❌ Mixed sequence types**: Cannot mix `reusable=true` and `reusable=false` in same pattern
- **❌ Excludes with non-reusable sequences**: Problematic (may cause allocation failures)
**Common Use Cases:**
| Use Case | Configuration | Behavior |
|----------|---------------|----------|
| **Interface Names** | `{seq:start=0:end=4094:reusable=true}` | lo0, lo1, lo2... fills gaps when freed |
| **VLAN IDs** | `{seq:start=100:end=4000:reusable=true}` | vlan100, vlan101... reuses freed VLANs |
| **Transaction IDs** | `{seq:start=1:reusable=false}` | tx_1, tx_2, tx_3... never reuses |
| **User IDs** | `{seq:start=1000:reusable=false}` | user_1000, user_1001... permanent IDs |
| **Even Port Numbers** | `{seq:start=8000:step=2:reusable=true}` | 8000, 8002, 8004... even ports only |
| **Multi-Dimensional GE Interfaces** | `ge{seq:0:2:reusable=true}/{seq:0:2:reusable=true}` | ge0/0, ge0/1, ge0/2, ge1/0... all combinations |
| **3D Network Topology** | `rack{seq:1:3:reusable=true}-slot{seq:1:4:reusable=true}-port{seq:1:8:reusable=true}` | rack1-slot1-port1, rack1-slot1-port2... 96 combinations |
#### **5. UUID Patterns** - `{uuid:...}`
Generates UUID-based identifiers with configurable formats.
**Parameters:**
| Parameter | Required | Default | Description |
|-----------|----------|---------|-------------|
| No parameters | No | `full` | Full UUID format (36 characters) |
| `length=N` | No | None | Custom length (1-32 characters, no dashes) |
**Parameter Details:**
**No Parameters (`{uuid}`):**
- **Purpose**: Generate full UUID with dashes
- **Required**: NO - this is the default behavior
- **Format**: "550e8400-e29b-41d4-a716-446655440000" (36 characters)
- **Use case**: When you need full UUID uniqueness
**length=N:**
- **Purpose**: Generate custom-length UUID substring (no dashes)
- **Required**: NO
- **Range**: 1-32 characters
- **Format**: First N characters of UUID without dashes
- **Examples**: `length=8` → "550e8400", `length=12` → "550e8400e29b"
- **Validation**: Must be between 1 and 32
**Predefined Formats:**
| Format | Configuration | Length | Example Output |
|--------|---------------|--------|----------------|
| **Full UUID** | `{uuid}` or `{uuid:full}` | 36 chars | "550e8400-e29b-41d4-a716-446655440000" |
| **Short UUID** | `{uuid:short}` | 8 chars | "550e8400" |
| **Custom Length** | `{uuid:length=12}` | 12 chars | "550e8400e29b" |
| **Minimal** | `{uuid:length=4}` | 4 chars | "550e" |
**Examples:**
```yaml
{uuid} # Full: "550e8400-e29b-41d4-a716-446655440000"
{uuid:full} # Explicit full (same as no params)
{uuid:short} # Short: "550e8400" (8 characters)
{uuid:length=12} # Custom: "550e8400e29b" (12 characters)
{uuid:length=16} # Medium: "550e8400e29b41d4" (16 characters)
{uuid:length=4} # Minimal: "550e" (4 characters)
```
**Common Use Cases:**
| Use Case | Configuration | Result Examples |
|----------|---------------|-----------------|
| **Unique Resource IDs** | `{uuid}` | "550e8400-e29b-41d4-a716-446655440000" |
| **Short Identifiers** | `{uuid:short}` | "550e8400", "a1b2c3d4" |
| **Database Keys** | `{uuid:length=16}` | "550e8400e29b41d4", "a1b2c3d4e5f6g7h8" |
| **Compact IDs** | `{uuid:length=8}` | "550e8400", "a1b2c3d4" |
#### **6. Static Text**
Static text appears literally in generated strings.
**Examples:**
```yaml
"srv-production-001" # Fixed string
"server_{num:start=1:max=100}_suffix" # Combined with patterns
```
#### **7. Time Patterns** - `{time:...}`
Generates current timestamp values in various formats when resources are allocated.
**Parameters:**
| Parameter | Required | Default | Description |
|-----------|----------|---------|-------------|
| Format type | Yes | None | Timestamp format specification |
| `timezone=TZ` | No | `UTC` | Timezone for formatting |
**Parameter Details:**
**Format Type (First Parameter):**
- **Purpose**: Specifies how to format the current timestamp
- **Required**: YES - no default format
- **Options**: See supported formats table below
- **Examples**: `unix`, `iso8601`, `date`, `yyyymmdd`
- **Validation**: Must be a supported format type
**timezone=TZ:**
- **Purpose**: Timezone for timestamp formatting
- **Required**: NO
- **Default**: `UTC`
- **Examples**: `timezone=EST`, `timezone=PST`, `timezone=UTC`
- **Note**: Applies to ISO/date formats, not unix timestamps
**Supported Formats:**
| Format | Description | Example Output | Use Cases |
|--------|-------------|----------------|-----------|
| **unix** | Unix timestamp (seconds) | "1705327800" | System logs, epoch-based systems |
| **unix_ms** | Unix timestamp (milliseconds) | "1705327800000" | High-precision tracking |
| **iso8601** | ISO 8601 format | "2024-01-15T14:30:00Z" | Standard timestamps |
| **rfc3339** | RFC 3339 format (same as ISO) | "2024-01-15T14:30:00Z" | Internet standards |
| **date** | Date only | "2024-01-15" | Daily operations |
| **time** | Time only | "14:30:00" | Time-specific resources |
| **yyyymmdd** | Compact date | "20240115" | File naming |
| **yyyymmddhhmmss** | Compact datetime | "20240115143000" | Detailed file naming |
| **yyyy_mm_dd** | Underscore date | "2024_01_15" | Path-friendly dates |
**Examples:**
```yaml
{time:unix} # Current unix timestamp: "1705327800"
{time:unix_ms} # Current timestamp in ms: "1705327800000"
{time:iso8601} # Current ISO time: "2024-01-15T14:30:00Z"
{time:date} # Current date: "2024-01-15"
{time:time} # Current time: "14:30:00"
{time:yyyymmdd} # Compact date: "20240115"
{time:yyyymmddhhmmss} # Compact datetime: "20240115143000"
{time:yyyy_mm_dd} # Underscore date: "2024_01_15"
{time:iso8601:timezone=EST} # With timezone: "2024-01-15T09:30:00Z"
```
**Common Use Cases:**
| Use Case | Configuration | Result Examples |
|----------|---------------|-----------------|
| **System Logs** | `{time:unix}` | "1705327800", "1705327801" |
| **Backup Files** | `{time:yyyymmdd}` | "20240115", "20240116" |
| **Session Tracking** | `{time:unix_ms}` | "1705327800000", "1705327800050" |
| **ISO Timestamps** | `{time:iso8601}` | "2024-01-15T14:30:00Z" |
| **File Naming** | `{time:yyyymmddhhmmss}` | "20240115143000" |
**Real-World Pattern Examples:**
```yaml
# System log files with timestamps
"log_{time:unix}_{choice:options=error,warn,info}"
# Result: "log_1705327800_error"
# Daily backup file names
"backup_{time:yyyymmdd}_{seq:start=1:end=999:reusable=false}_{uuid:short}"
# Result: "backup_20240115_1_a1b2c3d4"
# Session tracking with millisecond precision
"session_{time:unix_ms}_user_{num:start=1000:max=9999}"
# Result: "session_1705327800000_user_5432"
# Certificate management with date stamps
"cert_{time:date}_{str:charset=hex:length=8}"
# Result: "cert_2024-01-15_a1b2c3d4"
```
#### **8. Dynamic Patterns** - `{dynamic:...}`
Runtime placeholders for client substitution (e.g., Python `.format()`).
**Parameters:**
| Parameter | Required | Default | Description |
|-----------|----------|---------|-------------|
| Parameter name | Yes | None | Placeholder name for runtime substitution |
**Parameter Details:**
**Parameter Name (First Parameter):**
- **Purpose**: Name of the placeholder that will be substituted at runtime
- **Required**: YES - no default parameter name
- **Format**: Simple identifier (letters, numbers, underscores)
- **Examples**: `env`, `region`, `customer_id`, `datacenter`
- **Validation**: Must be valid identifier, cannot be empty
**How Dynamic Patterns Work:**
1. **Template Definition**: Use `{dynamic:param_name}` in pattern
2. **Resource Generation**: System generates template with `{param_name}` placeholder
3. **Client Substitution**: Python/client uses `.format()` to fill placeholder
4. **Final Result**: Completed string with actual values
**Examples:**
```yaml
{dynamic:env} # Generates: "{env}" placeholder
{dynamic:region} # Generates: "{region}" placeholder
{dynamic:customer_id} # Generates: "{customer_id}" placeholder
{dynamic:datacenter} # Generates: "{datacenter}" placeholder
```
**Complete Workflow Example:**
```yaml
# Template pattern:
"app_{dynamic:env}_{dynamic:region}_{seq:start=1:end=100:reusable=false}"
# Step 1: Resource Manager generates template
# Result: "app_{env}_{region}_42"
# Step 2: Python workflow fills placeholders
result = "app_{env}_{region}_42"
final_result = result.format(env="production", region="us-west")
# Final result: "app_production_us-west_42"
```
**Common Use Cases:**
| Use Case | Configuration | Template Generated | After Client Substitution |
|----------|---------------|-------------------|---------------------------|
| **Environment-Specific** | `{dynamic:env}` | "server_{env}_001" | "server_production_001" |
| **Regional Resources** | `{dynamic:region}` | "lb_{region}_web" | "lb_us-west_web" |
| **Customer Resources** | `{dynamic:customer}` | "vm_{customer}_db" | "vm_acme-corp_db" |
| **Multi-Tenant** | `{dynamic:tenant}_{dynamic:env}` | "app_{tenant}_{env}" | "app_customer1_prod" |
**Pattern Examples:**
```yaml
# Environment-aware applications
"app_{dynamic:env}_{dynamic:region}_{uuid:short}"
# Template: "app_{env}_{region}_a1b2c3d4"
# Result: "app_production_us-west_a1b2c3d4"
# Customer-specific resources
"customer_{dynamic:customer_id}_vm_{seq:start=1:end=999:reusable=false}"
# Template: "customer_{customer_id}_vm_42"
# Result: "customer_acme-corp_vm_42"
# Multi-datacenter nastartg
"dc_{dynamic:datacenter}_rack_{num:start=1:max=50:length=2}"
# Template: "dc_{datacenter}_rack_05"
# Result: "dc_east_rack_05"
```
### Complex String Examples
**Network Interfaces (Gap-Filling):**
```yaml
name: "interface-names"
resources:
- name: "interface"
base_type: "string"
resource_type: "interface"
constraints:
pattern: "lo{seq:start=0:end=4094:reusable=true}"
# Result: "lo0", "lo1", "lo2", ..., fills gaps when interfaces freed
```
**Transaction IDs (Monotonic):**
```yaml
name: "transaction-ids"
resources:
- name: "transaction_id"
base_type: "string"
resource_type: "transaction_id"
constraints:
pattern: "tx_{seq:start=1:end=999999:reusable=false}_{uuid:short}"
# Result: "tx_1_a1b2c3d4", "tx_2_e5f6g7h8", never reuses numbers
```
**Server Hostnames (Complex):**
```yaml
name: "server-hostnames"
resources:
- name: "hostname"
base_type: "string"
resource_type: "hostname"
constraints:
pattern: "{choice:type=weighted:options=web-60,api-30,db-10}_{str:charset=lowercase:length=4}_{num:start=1:max=999:length=3}"
# Result: "web_abcd_042", "api_efgh_127", "db_ijkl_891"
```
**Timestamped System Resources:**
```yaml
name: "log-files"
resources:
- name: "log_filename"
base_type: "string"
resource_type: "logfile"
constraints:
pattern: "log_{time:unix}_{choice:options=error,warn,info,debug}"
# Result: "log_1705327800_error", "log_1705327801_warn"
# Each allocation gets current timestamp when allocated
```
**Daily Backup Files:**
```yaml
name: "backup-files"
resources:
- name: "backup_filename"
base_type: "string"
resource_type: "backup_file"
constraints:
pattern: "backup_{time:yyyymmdd}_{seq:start=1:end=999:reusable=false}_{uuid:short}"
# Result: "backup_20240115_1_a1b2c3d4", "backup_20240115_2_e5f6g7h8"
# Date changes daily, sequence continues within each day
```
**Session Identifiers:**
```yaml
name: "user-sessions"
resources:
- name: "session_id"
base_type: "string"
resource_type: "session"
constraints:
pattern: "session_{time:unix_ms}_user_{num:start=1000:max=9999}"
# Result: "session_1705327800000_user_5432", "session_1705327800050_user_7891"
# Millisecond precision for unique session tracking
```
### DSL Limitations & Validation
#### **Parameter Requirements:**
| Pattern Type | Required Parameters | Optional Parameters |
|--------------|-------------------|-------------------|
| **Number** | `start=N`, `end=N` | `length=N` |
| **String** | `charset=NAME`, `length=N` | `chars=CUSTOM` |
| **Choice** | `options=LIST` | `type=simple/weighted` |
| **Sequence** | None | `reusable=true/false`, `start=N`, `end=N`, `step=N` |
| **UUID** | None | `length=N` |
| **Time** | Format type | `timezone=TZ` |
| **Static** | None | None |
| **Dynamic** | Parameter name | None |
#### **Validation Rules:**
- **All parameters must be explicitly named** - no positional parameters
- **Required parameters must be provided** - no silent defaults
- **Empty values are rejected** - `start=`, `start=`, `charset=` all fail
- **Duplicate parameters are rejected** - `start=1:start=2` fails
- **Unknown parameters are rejected** - unrecognized parameters fail
- **Maximum 10,000,000 values** per sequence (step-aware validation)
#### **Common Limitations:**
- **✅ Multiple reusable sequences** - generates all combinations
- **❌ Multiple non-reusable sequences** - NOT supported (would waste sequence values unnecessarily)
- **❌ Mixed sequence types** - Cannot mix `reusable=true` and `reusable=false` in same pattern
- **⚠️ Excludes with non-reusable sequences** - Problematic (may cause allocation failures)
- **✅ String pool exclude filters** - **NOW SUPPORTED** (template + runtime excludes)
- **Unicode characters** - Limited to ASCII character sets
- **Custom charset** - Cannot be empty
#### **Common Error Examples:**
```
Error: "positional parameters not supported: 1 (use explicit: start=N, end=N, step=N, reusable=true/false)"
Fix: Use {seq:start=1:end=100:reusable=false} instead of {seq:1:100:false}
Error: "start parameter is required (use start=N)"
Fix: Use {num:start=1:end=100} instead of {num:end=100}
Error: "charset parameter is required (use charset=alpha, charset=numeric, etc.)"
Fix: Use {str:charset=alpha:length=5} instead of {str:length=5}
Error: "sequence will generate 50000000 values, exceeds maximum allowed 10000000"
Fix: Reduce range or increase step: {seq:start=1:end=1000000:step=5}
Error: "multiple non-reusable sequences (reusable=false) are not supported - this would waste sequence values unnecessarily"
Fix: Use only one non-reusable sequence per pattern
Error: "mixing reusable and non-reusable sequences is not supported - use either all reusable=true or all reusable=false"
Fix: Make all sequences in pattern have same reusable setting
Error: "start parameter cannot be empty (use start=N)"
Fix: Provide actual values for all parameters
```
---
## 4. Workflows for Allocation/Deallocation
Workflows handle the allocation process and integration with external systems.
### Workflow Configuration
```yaml
resources:
- name: "vlan_id"
base_type: "vlan"
resource_type: "int"
external: false # Default: true
constraints: { ... }
workflows:
allocation: "allocate-vlan-workflow" # Custom workflow (optional)
deallocation: "deallocate-vlan-workflow" # Cleanup workflow (optional)
```
### Workflow Defaults
**Allocation Workflow:**
- **Default behavior**: Built-in allocator selects first available value
- **Custom workflow**: Specify workflow name for custom selection logic
- **When to use custom**:
- Custom candidate selection (odd/even values, priority-based, pattern matching)
- External system integration and validation
- Approval processes and business rules
- Multi-step allocation with rollback capability
**Deallocation Workflow:**
- **No default**: Must specify workflow if cleanup is needed
- **When required**: External system cleanup, resource release notifications
- **Self-managed resources**: Often don't need deallocation workflows
### Workflow API
The Resource Manager exposes two APIs for workflows:
#### **GetCandidates API**
Called by workflows to get candidate values for allocation:
```protobuf
message GetCandidatesRequest {
string correlation_id = 1; // Unique allocation session ID
int32 limit = 2; // Number of candidates requested
string filters = 3; // JSON filters (optional)
}
message GetCandidatesResponse {
google.protobuf.Value candidates = 1; // Array of candidate values
}
```
**Example Workflow Call:**
```python
# Python workflow activity - Integer pool filters
response = resource_manager.GetCandidates(
correlation_id="alloc-12345",
limit=10,
filters=json.dumps({"start": 200, "end": 300})
)
candidates = json.loads(response.candidates) # [201, 203, 205, ...]
# Python workflow activity - String pool exclude filter
response = resource_manager.GetCandidates(
correlation_id="alloc-67890",
limit=5,
filters=json.dumps({"exclude": ["lo1", "lo5", "lo10"]})
)
candidates = json.loads(response.candidates) # ["lo0", "lo2", "lo3", "lo4", "lo6"]
```
#### **SelectCandidate API**
Called by workflows to select a candidate or report failure:
```protobuf
message SelectCandidateRequest {
string correlation_id = 1; // Same ID from GetCandidates
bool success = 2; // true for success, false for failure
google.protobuf.Value selected_value = 3; // Structured selected value (when success=true)
string error_message = 4; // Error description (if success=false)
}
```
**Success Example:**
```python
# Workflow successfully allocated value
resource_manager.SelectCandidate(
correlation_id="alloc-12345",
success=True,
selected_value=json.dumps(205),
error_message=""
)
```
**Failure Example:**
```python
# Workflow failed to allocate
resource_manager.SelectCandidate(
correlation_id="alloc-12345",
success=False,
selected_value="",
error_message="External system rejected allocation"
)
```
### Resource Management Patterns
The `external` field determines who controls the resource allocation:
#### **Three Main Allocation Patterns:**
1. **Default Allocation**: Built-in allocator selects first candidate (no custom workflow)
2. **Custom Selection**: Custom workflow chooses specific candidate from RM-provided options
3. **External Integration**: External system provides values, RM tracks for bookkeeping
#### **Self-Managed Resources (`external: false`)**
Resource Manager provides candidates and stores values:
```yaml
name: "simple-vlan"
resources:
- name: "vlan_id"
base_type: "vlan"
resource_type: "int"
external: false # RM provides candidates and stores values
constraints: { start: 100, end: 4000 }
workflows:
allocation: "simple-allocate-workflow"
```
**How it Works:**
- **Resource Manager**: Generates candidate values using constraints/patterns
- **Workflow**: Calls `GetCandidates()` to get available values from RM
- **Resource Manager**: Tracks allocated values in internal tables
- **Use Cases**: VLANs, internal IDs, generated hostnames
**Workflow Patterns:**
**Simple Allocation (Default Behavior):**
1. Call `GetCandidates()` to get RM-generated candidates
2. Select first candidate (no custom logic needed)
3. Call `SelectCandidate(success=true, selected_value=candidate)`
**Custom Selection Logic:**
1. Call `GetCandidates()` to get RM-generated candidates
2. Apply custom business logic to select preferred candidate
3. Call `SelectCandidate(success=true, selected_value=selected_candidate)`
**Example - Regional VLAN Selection:**
```python
# Workflow: vlan-east-allocation
candidates = resource_manager.GetCandidates(correlation_id) # [100, 101, 102, 103, 104, ...]
# Custom logic: Choose odd VLANs for east datacenter
selected_vlan = None
for candidate in candidates:
if candidate % 2 == 1: # Odd VLAN
selected_vlan = candidate
break
if selected_vlan:
resource_manager.SelectCandidate(
correlation_id=correlation_id,
success=True,
selected_value=json.dumps(selected_vlan) # e.g., 101
)
else:
resource_manager.SelectCandidate(
correlation_id=correlation_id,
success=False,
error_message="No odd VLANs available in current batch"
)
```
```python
# Workflow: vlan-west-allocation
# Custom logic: Choose even VLANs for west datacenter
for candidate in candidates:
if candidate % 2 == 0: # Even VLAN
selected_vlan = candidate
break
```
#### **Externally-Managed Resources (`external: true`)**
External system provides values, Resource Manager does bookkeeping:
```yaml
name: "cloud-instances"
resources:
- name: "instance_id"
base_type: "string"
resource_type: "instance_id"
external: true # External system provides values, RM tracks only
constraints: {} # No constraints needed for external values
workflows:
allocation: "provision-cloud-instance-workflow"
deallocation: "cleanup-cloud-instance-workflow"
```
**How it Works:**
- **External System**: Generates/provides the actual resource values
- **Workflow**: Calls external system directly to get values
- **Resource Manager**: Only tracks allocated values for bookkeeping
- **Use Cases**: Cloud resource IDs, external service tokens, third-party allocations
**Allocation Workflow Pattern:**
1. Workflow calls external system directly (no `GetCandidates()` to RM)
2. External system provides value (e.g., cloud instance ID)
3. Call `SelectCandidate(success=true, selected_value=external_value)`
4. If failed: Call `SelectCandidate(success=false, error_message=reason)`
**Deallocation Workflow Pattern:**
1. Workflow triggered when resource is released
2. Clean up external system (deprovision instance)
3. Resource Manager automatically removes from allocation table
#### **Hybrid Example**
Some templates may have both self-managed and externally-managed resources:
```yaml
name: "server-deployment"
resources:
- name: "hostname"
base_type: "string"
resource_type: "hostname"
external: false # RM generates hostname
constraints:
pattern: "srv_{seq:start=1:end=999:reusable=false}"
- name: "cloud_instance_id"
base_type: "string"
resource_type: "instance_id"
external: true # Cloud provider generates instance ID
constraints: {}
workflows:
allocation: "deploy-server-workflow"
deallocation: "cleanup-server-workflow"
```
---
## 5. Resource Manager External API
The Resource Manager provides gRPC APIs for managing templates and allocations.
### Template Management API
**⚠️ Note: Template gRPC APIs will eventually be replaced with Git-based versioning. Until that migration is complete, we use the gRPC APIs below.**
#### **Create Template**
```protobuf
// gRPC call
rpc CreateTemplate(CreateTemplateRequest) returns (TemplateResponse);
message CreateTemplateRequest {
string name = 1;
string version = 2;
string scope = 3;
string description = 4;
string status = 5;
repeated ResourceDefinition resources = 6;
}
message ResourceDefinition {
string name = 1;
string base_type = 2; // Resource identifier: "vlan", "port", "hostname", etc.
string resource_type = 3; // Pool type: "int" or "string"
google.protobuf.Struct constraints = 4;
LifecycleWorkflows workflows = 5;
bool external = 6; // true: RM provides candidates, false: external system provides values
}
```
**Example gRPC Call:**
```python
request = resource_manager_pb2.CreateTemplateRequest(
name="vlan-pool",
version="1.0.0", # User-provided initial version
scope="datacenter-east",
description="VLAN allocation pool",
status="active", # User-provided initial status
resources=[
resource_manager_pb2.ResourceDefinition(
name="vlan_id",
base_type="vlan",
resource_type="int",
external=False, # default: false
constraints=Struct(fields={
"start": Value(number_value=100),
"end": Value(number_value=4000),
}),
workflows=resource_manager_pb2.LifecycleWorkflows(
allocation="allocate-vlan-workflow",
deallocation="deallocate-vlan-workflow"
)
)
]
)
response = client.CreateTemplate(request)
# Note: Subsequent version management will be handled by backend
```
#### **Get Template**
```protobuf
rpc GetTemplate(GetTemplateRequest) returns (TemplateResponse);
message GetTemplateRequest {
string name = 1;
string scope = 2;
string version = 3;
}
```
#### **List Templates**
```protobuf
rpc ListTemplates(ListTemplatesRequest) returns (ListTemplatesResponse);
message ListTemplatesRequest {
// No parameters
}
message ListTemplatesResponse {
repeated ResourceTemplate templates = 1;
}
```
#### **Update Template**
```protobuf
rpc UpdateTemplate(UpdateTemplateRequest) returns (google.protobuf.Empty);
message UpdateTemplateRequest {
string name = 1;
string scope = 2;
string version = 3; // Current version to update
string newVersion = 4; // New version (backend-managed after creation)
string newDescription = 5; // Updated description
string newStatus = 6; // New status (backend-managed after creation)
}
```
#### **Delete Template**
```protobuf
rpc DeleteTemplate(DeleteTemplateRequest) returns (google.protobuf.Empty);
message DeleteTemplateRequest {
string name = 1;
string scope = 2;
string version = 3;
}
```
### Allocation Management API
#### **Create Allocation**
```protobuf
rpc Allocate(AllocationRequest) returns (Allocation);
message AllocationRequest {
string request_id = 1;
string template_name = 2;
string template_scope = 3;
string template_version = 4;
google.protobuf.Struct parameters = 5;
string allocation_key = 6; // Custom scope identifier for templates with scope='custom'
}
message Allocation {
int64 id = 1;
int64 template_id = 2;
string template_name = 3;
string template_scope = 4;
string template_version = 5;
string request_id = 6;
string status = 7;
google.protobuf.Struct parameters = 8;
repeated AllocatedResource resources = 9;
google.protobuf.Timestamp created_at = 10;
google.protobuf.Timestamp updated_at = 11;
}
message AllocatedResource {
string name = 1;
string base_type = 2; // Resource identifier: "vlan", "port", "hostname", etc.
string resource_type = 3; // Pool type: "int" or "string"
google.protobuf.Value value = 4;
}
```
**Example gRPC Call (Fixed Scope):**
```python
request = resource_manager_pb2.AllocationRequest(
request_id="req-12345",
template_name="vlan-pool",
template_scope="datacenter-east",
template_version="1.0.0", # Optional - defaults to latest active
parameters=Struct(fields={
"datacenter": Value(string_value="east"),
"environment": Value(string_value="production")
})
)
response = client.Allocate(request)
```
**Example gRPC Call (Custom Scope):**
```python
request = resource_manager_pb2.AllocationRequest(
request_id="interface-req-1",
template_name="router-interfaces",
template_scope="custom",
allocation_key="router-clli-12345", # Required for scope='custom'
template_version="1.0.0"
)
response = client.Allocate(request)
```
#### **Get Allocation**
```protobuf
rpc GetAllocation(GetAllocationRequest) returns (Allocation);
message GetAllocationRequest {
int64 allocation_id = 1;
}
```
#### **List Allocations**
```protobuf
rpc ListAllocations(google.protobuf.Empty) returns (ListAllocationsResponse);
message ListAllocationsResponse {
repeated Allocation allocations = 1;
}
```
#### **Delete Allocation**
```protobuf
rpc DeleteAllocation(DeleteAllocationRequest) returns (google.protobuf.Empty);
message DeleteAllocationRequest {
int64 allocation_id = 1;
}
```
### Workflow Callback API (Internal)
These APIs are used internally by workflows during allocation:
#### **GetCandidates (Internal)**
```protobuf
rpc GetCandidates(GetCandidatesRequest) returns (GetCandidatesResponse);
message GetCandidatesRequest {
string correlation_id = 1; // Unique allocation session ID
int32 limit = 2; // Number of candidates requested
string filters = 3; // JSON filters (optional)
}
message GetCandidatesResponse {
google.protobuf.Value candidates = 1; // Array of candidate values
}
```
**Example gRPC Call:**
```python
request = resource_manager_pb2.GetCandidatesRequest(
correlation_id="alloc-12345",
limit=10,
filters='{"start": 200, "end": 300}'
)
response = workflow_callback_client.GetCandidates(request)
candidates = json.loads(response.candidates)
```
#### **SelectCandidate (Internal)**
```protobuf
rpc SelectCandidate(SelectCandidateRequest) returns (google.protobuf.Empty);
message SelectCandidateRequest {
string correlation_id = 1; // Same ID from GetCandidates
bool success = 2; // true for success, false for failure
google.protobuf.Value selected_value = 3; // Structured selected value (if success=true)
string error_message = 4; // Error description (if success=false)
}
```
**Example gRPC Call:**
```python
request = resource_manager_pb2.SelectCandidateRequest(
correlation_id="alloc-12345",
success=True,
selected_value=json.dumps(205),
error_message=""
)
response = workflow_callback_client.SelectCandidate(request)
```
### Workflow Engine Tasks
**All Resource Manager APIs are exposed as workflow tasks!** This is how customers should use the Resource Manager in practice.
#### **Two Levels of Workflow Integration**
**Level 1: Pre-built Allocation Tasks (Standard Use Case)**
Use these for standard resource allocation without custom logic:
- **ResourceManagerAllocateResources** - Complete allocation (built-in selection logic)
- **ResourceManagerDeallocateResources** - Complete deallocation with cleanup
- **ResourceManagerCreateTemplate** - Template management
**Level 2: Workflow Callback Tasks (Custom Logic)**
Use these when you need custom allocation/deallocation workflows:
- **GetCandidates** - Get candidates from Resource Manager
- **SelectCandidate** - Select specific candidate with custom logic
#### **When to Use Each Level**
**Use Pre-built Tasks when:**
- Standard allocation (first available value is fine)
- No custom selection logic needed
- Simple deallocation without complex cleanup
**Use Workflow Callback Tasks when:**
- Custom candidate selection (odd/even values, priority rules)
- Complex validation with external systems
- Multi-step allocation with rollback
- Custom deallocation workflows
#### **Available Workflow Tasks**
**Pre-built Allocation Tasks:**
- `ResourceManagerAllocateResources` - Complete end-to-end allocation
- `ResourceManagerDeallocateResources` - Complete deallocation with cleanup
- `ResourceManagerCreateTemplate` - Template creation and management
**Workflow Callback Tasks:**
- `GetCandidates` - Retrieve candidate values from Resource Manager
- `SelectCandidate` - Select specific candidate or report failure
#### **Usage Recommendations**
**Use Pre-built Tasks for:**
- Standard allocation workflows where first available value is acceptable
- Simple resource management without custom logic
- Quick allocation/deallocation without business rules
**Use Callback Tasks for:**
- Custom candidate selection logic (regional preferences, priority rules)
- Multi-step allocation workflows with validation
- Complex integration with external systems
- Custom deallocation workflows with specific cleanup requirements
All tasks are available in the workflow builder UI and handle gRPC communication automatically.
#### **Why Use Workflow Tasks?**
**Simplified Integration:**
- ✅ **Drag & drop** in workflow builder UI
- ✅ **No coding** required for basic resource allocation
- ✅ **Built-in error handling** and retry logic
- ✅ **Automatic logging** and monitoring
**vs. Direct gRPC:**
- ❌ **Complex setup** - gRPC client configuration, proto imports
- ❌ **Manual error handling** - implement retry logic yourself
- ❌ **More coding** - serialize/deserialize data manually
**When to Use Each:**
- **Workflow Tasks**: Most common use case - visual workflows, standard patterns
- **Direct gRPC**: Advanced integrations, custom applications, external systems
---
## Complete Examples
### Example 1: VLAN Pool Template
```yaml
name: "datacenter-vlans"
scope: "datacenter-east"
description: "VLAN allocation for east datacenter"
# version and status are managed by backend
resources:
- name: "vlan_id"
base_type: "vlan"
resource_type: "int"
constraints:
start: 100
end: 4000
step: 1
workflows:
allocation: "vlan-allocation-workflow"
deallocation: "vlan-cleanup-workflow"
```
### Example 2: Server Hostname Template
```yaml
name: "server-hostnames"
scope: "production"
description: "Production server hostname generation"
# version and status are managed by backend
resources:
- name: "hostname"
base_type: "string"
resource_type: "hostname"
constraints:
pattern: "srv_{choice:options=web,api,db}_{seq:start=100:end=999:reusable=false}"
workflows:
allocation: "provision-server-workflow"
deallocation: "deprovision-server-workflow"
```
### Example 3: Network Interface Template (Self-Managed)
```yaml
name: "interface-names"
scope: "network-config"
description: "Network interface naming"
# version and status are managed by backend
resources:
- name: "interface_name"
base_type: "string"
resource_type: "interface"
external: false # RM generates interface names
constraints:
pattern: "lo{seq:start=0:end=128:reusable=true}"
exclude: ["lo0", "lo1"] # Reserve management interfaces
workflows:
allocation: "configure-interface-workflow"
deallocation: "cleanup-interface-workflow"
```
### Example 3b: Multi-Dimensional GE Interface Template
```yaml
name: "ge-interfaces-multidim"
scope: "network-topology"
description: "Multi-dimensional GE interface allocation"
# version and status are managed by backend
resources:
- name: "ge_interface"
base_type: "string"
resource_type: "interface"
external: false # RM generates all combinations
constraints:
pattern: "ge{seq:start=0:end=7:reusable=true}/{seq:start=0:end=3:reusable=true}/{seq:start=0:end=47:reusable=true}"
exclude: ["ge0/0/0", "ge0/0/1"] # Reserve management ports
workflows:
allocation: "configure-ge-interface-workflow"
deallocation: "cleanup-ge-interface-workflow"
# Results: ge0/0/2, ge0/0/3, ..., ge0/1/0, ge0/1/1, ..., ge7/3/47
# Total combinations: 8×4×48 = 1,536 interfaces (minus excludes)
```
### Example 4: Timestamped Log Files Template
```yaml
name: "system-logs"
scope: "production"
description: "System log file naming with timestamps"
# version and status are managed by backend
resources:
- name: "log_filename"
base_type: "string"
resource_type: "logfile"
external: false # RM generates timestamped filenames
constraints:
pattern: "log_{time:yyyymmdd}_{time:unix}_{choice:options=error,warn,info,debug}"
workflows:
allocation: "create-log-file-workflow"
deallocation: "archive-log-file-workflow"
# Result: "log_20240115_1705327800_error", "log_20240115_1705327850_warn"
# Each allocation gets current date and timestamp
```
### Example 5: Cloud Instance Template (Externally-Managed)
```yaml
name: "cloud-instances"
scope: "aws-us-east"
description: "AWS EC2 instance allocation"
# version and status are managed by backend
resources:
- name: "instance_id"
base_type: "string"
resource_type: "ec2_instance"
external: true # AWS provides instance IDs, RM tracks only
constraints: {} # No constraints - external system controls values
workflows:
allocation: "provision-ec2-instance-workflow"
deallocation: "terminate-ec2-instance-workflow"
```
---
## Workflow Integration
### Allocation Flow
1. **Client** creates allocation request via gRPC API
2. **Resource Manager** starts allocation workflow
3. **Workflow** calls `GetCandidates()` to get available values (for self-managed resources)
4. **Workflow** applies custom selection logic to choose preferred candidate
5. **Workflow** attempts allocation (may involve external systems)
6. **Workflow** calls `SelectCandidate()` with success/failure result
7. **Resource Manager** completes allocation and returns result
### Custom Selection Examples
Workflows can implement business logic to select specific candidates from the `GetCandidates()` response:
**Regional VLAN Selection:**
```python
# Workflow: vlan-east-allocation (choose odd VLANs)
candidates_response = resource_manager.GetCandidates(correlation_id)
candidates = json.loads(candidates_response.candidates) # [100, 101, 102, 103, ...]
selected_vlan = None
for candidate in candidates:
if candidate % 2 == 1: # Select odd VLAN for east datacenter
selected_vlan = candidate
break
# Workflow: vlan-west-allocation (choose even VLANs)
for candidate in candidates:
if candidate % 2 == 0: # Select even VLAN for west datacenter
selected_vlan = candidate
break
```
**Priority-Based Selection:**
```python
# Select lowest available value
selected = min(candidates)
# Select highest available value
selected = max(candidates)
# Select value within preferred range
preferred_candidates = [c for c in candidates if 200 <= c <= 300]
selected = preferred_candidates[0] if preferred_candidates else candidates[0]
```
**Filter-Based Candidate Retrieval:**
```python
# Integer pool filters - restrict range at query time
candidates_response = resource_manager.GetCandidates(
correlation_id="alloc-12345",
limit=10,
filters=json.dumps({"start": 200, "end": 300, "exclude": [250, 275]})
)
candidates = json.loads(candidates_response.candidates) # [201, 202, ..., 249, 251, ..., 299] (excluding 250, 275)
# String pool exclude filters - exclude specific values
candidates_response = resource_manager.GetCandidates(
correlation_id="alloc-67890",
limit=10,
filters=json.dumps({"exclude": ["lo1", "lo5", "lo10"]}) # Only exclude filter supported
)
candidates = json.loads(candidates_response.candidates) # ["lo0", "lo2", "lo3", "lo4", "lo6", "lo7", "lo8", "lo9", "lo11", "lo12"]
```
**Pattern-Based Selection:**
```python
# Select hostname with specific pattern
candidates = ["web_srv_001", "api_srv_002", "db_srv_003"]
selected = next(c for c in candidates if c.startswith("web_"))
# Select interface with specific number
candidates = ["eth0", "eth1", "eth2", "eth3"]
selected = next(c for c in candidates if int(c[3:]) % 2 == 0) # Even interfaces
```
**Time-Based Resource Selection:**
```python
# Template: "backup_{time:yyyymmdd}_{seq:start=1:end=999:reusable=false}_{choice:options=full,incremental}"
# Workflow: backup-allocation-workflow
candidates_response = resource_manager.GetCandidates(correlation_id)
candidates = json.loads(candidates_response.candidates)
# candidates = ["backup_20240115_1_full", "backup_20240115_2_incremental", ...]
# Custom logic: Prefer full backups on weekends
import datetime
is_weekend = datetime.datetime.now().weekday() >= 5 # Saturday/Sunday
selected = None
for candidate in candidates:
if is_weekend and "full" in candidate:
selected = candidate # Prefer full backups on weekends
break
elif not is_weekend and "incremental" in candidate:
selected = candidate # Prefer incremental on weekdays
break
if not selected:
selected = candidates[0] # Fallback to first available
resource_manager.SelectCandidate(
correlation_id=correlation_id,
success=True,
selected_value=json.dumps(selected)
)
# Result: "backup_20240115_1_full" (if weekend)
```
### Error Handling
Workflows should handle failures gracefully:
```python
# Workflow pseudocode
candidates_response = resource_manager.GetCandidates(correlation_id)
candidates = json.loads(candidates_response.candidates)
for candidate in candidates:
try:
# Attempt allocation in external system
external_system.allocate(candidate)
# Success - select this candidate
resource_manager.SelectCandidate(
correlation_id=correlation_id,
success=True,
selected_value=json.dumps(candidate)
)
break
except ExternalSystemError as e:
# Try next candidate
continue
else:
# All candidates failed
resource_manager.SelectCandidate(
correlation_id=correlation_id,
success=False,
error_message="All candidates rejected by external system"
)
```
This tutorial provides the foundation for using the Resource Manager effectively for various resource allocation scenarios.
