Validation Tool

Syntax-check generated Prefect flows and return both sources for LLM comparison.

Overview

The validate tool helps verify conversions by:

Syntax checking — Parsing the generated Python code for syntax errors
Source comparison — Returning both original DAG and generated flow side-by-side
Comparison guidance — A checklist for the LLM to verify structural fidelity

The LLM performs the structural comparison — the tool provides the inputs.

Usage

Via MCP

{
  "tool": "validate",
  "args": {
    "original_dag": "dags/etl_pipeline.py",
    "converted_flow": "from prefect import flow, task\n..."
  }
}

Parameters

Parameter	Type	Required	Description
`original_dag`	string	Yes	Path to DAG file, or inline DAG code
`converted_flow`	string	Yes	Path to flow file, or inline flow code

Both parameters accept either a file path or inline source code.

Output Format

{
  "original_source": "from airflow import DAG\n...",
  "converted_source": "from prefect import flow, task\n...",
  "syntax_valid": true,
  "syntax_errors": null,
  "comparison_guidance": "Compare the original Airflow DAG with the generated Prefect flow. Verify:\n1. All tasks from the DAG are represented in the flow\n2. Task dependencies are preserved\n..."
}

Fields

Field	Type	Description
`original_source`	string	Full source code of the original DAG
`converted_source`	string	Full source code of the generated flow
`syntax_valid`	boolean	Whether the generated code parses without errors
`syntax_errors`	array or null	List of `{line, column, message}` if invalid
`comparison_guidance`	string	Checklist for the LLM to verify the conversion

Comparison Guidance

The comparison_guidance field includes a checklist:

All tasks from the DAG are represented in the flow
Task dependencies are preserved (>> chains become function call order)
XCom push/pull is replaced with task return values and parameters
Connections are mapped to Prefect blocks
Variables are mapped to Prefect variables or environment config
Schedule, retries, and other DAG config is carried over
Sensors are converted to polling tasks with retries or event triggers
TaskGroups are converted to subflows
Trigger rules are handled with state inspection

Examples

Valid Conversion

{
  "original_source": "with DAG('etl') as dag:\n    extract = PythonOperator(...)\n...",
  "converted_source": "@flow(name='etl')\ndef etl():\n    data = extract()\n...",
  "syntax_valid": true,
  "syntax_errors": null,
  "comparison_guidance": "Compare the original..."
}

Syntax Error in Generated Code

{
  "original_source": "...",
  "converted_source": "@flow\ndef etl(\n    data = extract()",
  "syntax_valid": false,
  "syntax_errors": [
    {
      "line": 3,
      "column": 24,
      "message": "unexpected EOF while parsing"
    }
  ],
  "comparison_guidance": "..."
}

Best Practices

Always validate after generation — Catch syntax errors before deployment
Pass inline content — Works with both file paths and inline code
Review the comparison guidance — Use the checklist to verify structural fidelity
Re-validate after fixes — If the LLM corrects issues, validate again
Test with real data — Validation checks syntax, not runtime behavior