Workflow Reference¶

Quick links:

DSL Reference — YAML syntax for scenario definition
CLI Reference — command-line tools for running scenarios
API Reference — Python API for programmatic scenario creation
Auto-Generated API Reference — complete class and method documentation

This document describes NetGraph workflows – analysis execution pipelines that perform capacity analysis, failure simulation, and network statistics computation.

Overview¶

Workflows are lists of analysis steps executed sequentially on network scenarios. Each step performs a specific operation like computing statistics, running Monte Carlo failure simulations, or exporting graph data.

workflow:
  - step_type: MaxFlow
    name: "cap"
    source_path: "^datacenter/.*"
    sink_path: "^edge/.*"
    iterations: 1000

Execution Model¶

Network State Management:

Scenario state: Permanent network configuration (e.g., disabled: true nodes/links)
Analysis state: Temporary failure exclusions during Monte Carlo simulation

NetworkView: Provides isolated analysis contexts without modifying the base network, enabling concurrent failure scenario analysis.

Core Workflow Steps¶

BuildGraph¶

Exports the network graph to JSON (node-link format) for external analysis. Not required for other steps, which construct internal views.

- step_type: BuildGraph
  name: build_graph

NetworkStats¶

Computes network statistics (capacity, degree metrics).

- step_type: NetworkStats
  name: baseline_stats

MaxFlow¶

Monte Carlo flow capacity analysis with optional failure simulation.

- step_type: MaxFlow
  name: capacity_analysis
  source_path: "^servers/.*"
  sink_path: "^storage/.*"
  mode: "combine"              # combine | pairwise
  failure_policy: random_failures
  iterations: 1000
  parallelism: auto             # or an integer
  baseline: true
  include_flow_details: false   # cost_distribution
  include_min_cut: false        # min-cut edges list

TrafficMatrixPlacement¶

Monte Carlo placement of a named traffic matrix with optional alpha scaling.

- step_type: TrafficMatrixPlacement
  name: tm_placement
  matrix_name: default
  iterations: 100
  parallelism: auto
  baseline: false
  include_flow_details: true      # cost_distribution per flow
  include_used_edges: false       # include per-demand used edge lists
  # Alpha scaling – explicit or from another step
  alpha: 1.0
  # alpha_from_step: msd_default
  # alpha_from_field: data.alpha_star

MaximumSupportedDemand¶

Search for the maximum uniform traffic multiplier alpha_star that is fully placeable.

- step_type: MaximumSupportedDemand
  name: msd_default
  matrix_name: default
  acceptance_rule: hard
  alpha_start: 1.0
  growth_factor: 2.0
  resolution: 0.01
  max_bracket_iters: 32
  max_bisect_iters: 32
  seeds_per_alpha: 1
  placement_rounds: auto

Node Selection Mechanism¶

Source and sink nodes are selected using regex patterns matched against full node names with Python's re.match() (anchored at start).

Basic Pattern Matching¶

# Exact match
source_path: "spine-1"

# Prefix match
source_path: "datacenter/servers/"

# Pattern match
source_path: "^pod[1-3]/leaf/.*$"

Capturing Groups for Node Grouping¶

No Capturing Groups: All matching nodes form one group labeled by the pattern.

source_path: "edge/.*"
# Creates one group: "edge/.*" containing all matching nodes

Single Capturing Group: Each unique captured value creates a separate group.

source_path: "(dc[1-3])/servers/.*"
# Creates groups: "dc1", "dc2", "dc3"
# Each group contains servers from that datacenter

Multiple Capturing Groups: Group labels join captured values with |.

source_path: "(dc[1-3])/(spine|leaf)/switch-(\d+)"
# Creates groups: "dc1|spine|1", "dc1|leaf|2", "dc2|spine|1", etc.

Flow Analysis Modes¶

combine Mode: Aggregates all source matches into one virtual source, all sink matches into one virtual sink. Produces single flow value.

pairwise Mode: Computes flow between each source group and sink group pair. Produces flow matrix keyed by (source_group, sink_group).

MaxFlow Parameters¶

Required Parameters¶

source_path: Regex pattern for source node selection
sink_path: Regex pattern for sink node selection

Analysis Configuration¶

mode: combine                    # combine | pairwise (default: combine)
iterations: 1000                 # Monte Carlo trials (default: 1)
failure_policy: policy_name      # From failure_policy_set (default: null)
baseline: true                   # Include baseline (default: false)
parallelism: auto                # Worker processes (default: auto)
shortest_path: false             # Limit to shortest paths (default: false)
flow_placement: PROPORTIONAL     # PROPORTIONAL | EQUAL_BALANCED
include_flow_details: false      # Emit cost_distribution per flow
include_min_cut: false           # Emit min-cut edge list per flow

Performance Tuning¶

parallelism: 8                     # Worker processes (default: 1)
seed: 42                          # Reproducible results
shortest_path: false              # Shortest paths only (default: false)
flow_placement: "PROPORTIONAL"    # "PROPORTIONAL" or "EQUAL_BALANCED"

Results Export Shape¶

Exported results have a fixed top-level structure:

{
  "workflow": { "<step>": { "step_type": "...", "execution_order": 0, ... } },
  "steps": {
    "<step_name>": {
      "metadata": { ... },
      "data": { ... }
    }
  },
  "scenario": { "seed": 1, "failure_policy_set": { ... }, "traffic_matrices": { ... } }
}

MaxFlow and TrafficMatrixPlacement store data.flow_results as a list of per-iteration results:

{
  "flow_results": [
    {
      "failure_id": "baseline",
      "failure_state": null,
      "flows": [
        {
          "source": "A", "destination": "B", "priority": 0,
          "demand": 10.0, "placed": 10.0, "dropped": 0.0,
          "cost_distribution": { "2": 6.0, "4": 4.0 },
          "data": { "edges": ["(u,v,k)"] }
        }
      ],
      "summary": {
        "total_demand": 10.0, "total_placed": 10.0,
        "overall_ratio": 1.0, "dropped_flows": 0, "num_flows": 1
      },
      "data": { }
    },
    {
      "failure_id": "d0eea3f4d06413a2",
      "failure_state": null,
      "flows": [],
      "summary": {
        "total_demand": 0.0, "total_placed": 0.0,
        "overall_ratio": 1.0, "dropped_flows": 0, "num_flows": 0
      },
      "data": { }
    }
  ],
  "context": { ... }
}

Common Workflow Patterns¶

Single Deterministic Analysis¶

workflow:
  - step_type: MaxFlow
    source_path: "^servers/.*"
    sink_path: "^storage/.*"

Monte Carlo Failure Analysis¶

workflow:
  - step_type: MaxFlow
    source_path: "^pod1/.*"
    sink_path: "^pod2/.*"
    failure_policy: random_link_failures
    iterations: 10000
    parallelism: 8
    baseline: true
    seed: 42

Comparative Analysis¶

workflow:
  # Baseline capacity
  - step_type: MaxFlow
    name: baseline
    source_path: "^dc1/.*"
    sink_path: "^dc2/.*"
    iterations: 1

  # Single failure impact
  - step_type: MaxFlow
    name: single_failure
    source_path: "^dc1/.*"
    sink_path: "^dc2/.*"
    failure_policy: single_link_failure
    iterations: 1000
    baseline: true

Report Generation¶

Generate analysis reports using the CLI:

# Jupyter notebook (defaults to <results_name>.ipynb)
python -m ngraph report baseline_scenario.json

# HTML report (defaults to <results_name>.html when --html is used)
python -m ngraph report baseline_scenario.json --html

See CLI Reference for complete options.

Best Practices¶

Performance¶

Use parallelism for Monte Carlo analysis (typically CPU core count)
Set store_failure_patterns: false for large-scale analysis
Start with fewer iterations for initial validation

Analysis Strategy¶

Begin with single deterministic analysis before Monte Carlo
Use combine mode for aggregate analysis, pairwise for detailed flows
Include baseline: true for failure scenario comparison
Use descriptive name parameters for each step

Node Selection¶

Test regex patterns against actual node names before large runs
Use capturing groups to analyze multiple node groups simultaneously
Anchor patterns with ^ and $ for precise matching
Leverage hierarchical naming conventions for effective grouping

Integration¶

Reference failure policies from failure_policy_set
Ensure failure policies exist before workflow execution
Include BuildGraph only when graph export to JSON is needed for external analysis