Why I'm Building Crawl: The Migration Intelligence Gap Nobody Fills
12 years at Informatica taught me that every data migration fails at the same step: understanding what you have. AI is finally good enough to change that.
The problem I kept seeing
I spent 12 years at Informatica in various roles, working Fortune 500 data integration across APAC, MENA, and Europe. What's funny is that by the end as a Development Architect, I got to fix the bugs that I encountered when I was deploying the software as a Consultant. I was always part of the MDM vertical — data management, data enrichment, data stewardship, golden customer records, survivorship rules, fuzzy match rules — but I worked closely with the PowerCenter ETL teams and saw first hand the problems they had to deal with in terms of data migration. And when designing the landing tables for data ingestion into MDM, I saw first hand how difficult and painful it was to make even minor changes to those bridge tables.
Every migration project — Oracle to Snowflake, on-prem to cloud, legacy ETL to dbt — hit the same wall.
Not the technical translation. Tools exist for that. SnowConvert handles SQL dialects. Lakebridge maps Spark equivalents. dbt gives you a modern modeling layer.
The wall was earlier: understanding what you have.
Thousands of stored procedures encoding business rules nobody documented. ETL jobs with logic baked into transformation steps that predate anyone on the current team. Views referencing tables that were dropped two years ago. The tribal knowledge lived in the SQL, and nobody had time to read it all.
Every migration I worked on started the same way: consultants spending weeks manually reading stored procedures, writing documentation that was outdated before the project kicked off. It was the most expensive, least scalable part of every engagement.
Why now
When I left Informatica in 2023, AI wasn't strong enough to do this reliably. You could get an LLM to summarize a single procedure, but it couldn't synthesize across hundreds of objects — finding contradictions between procedures, detecting dead code, scoring migration risk based on vendor-specific syntax patterns.
That's changed. Models can now hold enough context to cross-reference a procedure against its dependencies. They can identify that sp_calculate_customer_churn uses a different LTV formula than sp_calculate_ltv. They can flag that a DATEADD call uses vendor-specific syntax that won't survive a platform change. They can do this at scale, across an entire codebase of stored procedures, and produce cogent summaries that a project manager can actually use to make triage decisions.
The gap hasn't closed — the tools to fill it have finally arrived.
What Crawl does
Crawl is the Step 0 layer: pre-migration intelligence that runs before you pick up any conversion tool. It connects to your database (read-only, catalog-only — it reads procedure source code, never your data), extracts business rules using hybrid AST + LLM analysis, and produces triage reports that tell you:
- What do we have? — Auto-generated business rule summaries
- Is it still alive? — Dead code detection, contradiction flagging
- What should we migrate first? — Triage by criticality, complexity, risk
- What breaks if we move? — Vendor-specific logic that won't survive a platform change
It's open-source (Apache 2.0), vendor-neutral, and designed to run with local LLMs so enterprise code never leaves your environment.
The SQL Explainer: a free taste
To validate that the core analysis works, I built a free SQL explainer tool. Paste a stored procedure, get a structured breakdown of what it does — business rules, risk flags, complexity score. It's Crawl's analysis engine running on a single procedure instead of an entire database.
Try it. Then imagine that across your whole data stack.
What's next
Crawl is in early development. Oracle Data Integrator (ODI) is the first supported source, PostgreSQL is in progress, with Snowflake, SQL Server, Oracle PL/SQL, and Informatica PowerCenter planned. If you've ever stared down a migration backlog and wished someone had documented the stored procedures before the original team left, this is for you.