How I used Orion to repair our GitHub data pipeline

This morning, I opened Orion and typed a sentence that probably sounds familiar to anyone who has worked around data infrastructure:

“Hey we just setup a fivetran pipeline for github data to replace our old janky manual script.”

We had connected GitHub to BigQuery through Fivetran. Fresh data was arriving. But the metrics we actually looked at still depended on the old ingestion path. Our legacy commits table stopped on October 6. Our pull request review data stopped on October 1. It was October 14.

Over the conversations that followed, I used Orion to build a shared commit view, migrate our saved metrics, reconcile conflicting results, correct their definitions, schedule the upstream refresh, and update the context explaining how the pipeline worked. We also found a missing source table and set up a workflow to bring me back when it arrived.

That is a substantial amount of data engineering. What made the morning feel magical was how much of it I could direct through a conversation while staying close to the evidence.

I started by asking Orion to break the work apart.

My instruction was fairly blunt: “Let’s not do everything in this single chat.” I wanted separate prompts, each with clear instructions and a comparison and verification step.

Orion proposed six workstreams: build the foundation view first, migrate the individual activity meters, migrate the leaderboards, repair the weekly comparison, migrate the largest-PR metric, and finish with an audit. The dependent work would use the same foundation. Each migration prompt required comparisons before changes, and the final audit was explicitly told to check what the other chats had actually done.

This gave me a way to manage the work. Each conversation had a bounded responsibility and a definition of success. I could inspect a discrepancy, supply a missing fact, or approve a change without carrying the entire migration in my head.

The foundation work immediately showed why this mattered.

A GitHub commit has more than one plausible date. People appear under different names and logins. Automation creates commits alongside humans. Repositories move into a monorepo, while their history still needs to exist. A new connector can collect a much wider population of activity than the script it replaces.

Orion inspected the schemas and built a view with one row per commit. It consolidated author identities, applied the bot exclusions I chose, and flagged the repositories that had migrated so their history could be distinguished from current work. The resulting view contained 93,773 unique commits.

It also compared the date fields against the old data. Across 9,783 shared commit identifiers, the legacy timestamp matched the author date exactly. The committer date matched only about 94% of the time. That gave us an evidence-based reason to choose the author date for historical continuity.

Before AI, I would have had to construct that investigation myself: inspect the schemas, write the joins, check uniqueness, reconcile identities, compare timestamps, and explain the result. With Orion, I could ask for the comparison and spend my attention on interpreting and guiding the AI.

Then we hit the most revealing discrepancy of the morning.

Some of the new metric tests showed two to four times as many commits as the old source. A fresh pipeline had produced very different numbers. Several explanations were possible.

I remembered something about the old implementation:

“I think the discrepancy might be that we only counted commits on main branches historically. If you apply that logic, what happens?”

Orion tested it.

For one engineer, the October 1 comparison went from 27 commits in the old table and 102 in the new data to 27 in both after applying the main/master filter. Another went from 12 versus 53 to 12 versus 12. For my own activity, 9 versus 27 became 9 versus 9.

That single piece of historical context explained the mismatch.

This is the collaboration that stood out to me. Orion made the differences visible and did the work to test an explanation. I supplied knowledge about what the original system was intended to measure. Together, we turned a suspicious result into a precise understanding of the migration.

A source change can quietly become a definition change. The SQL can run perfectly while the number starts answering a different question.

We worked through the downstream metrics with that in mind. Orion migrated the six existing individual meters and added a seventh that we noticed was missing. It consolidated people who had appeared under multiple identities in the leaderboards

The audit found a few more common problems: the individual meters could compare today's partial activity with full days in their baselines. It also found that the leaderboards were still counting all branches while the meters and weekly comparison counted main branch commits. The project as a whole remained inconsistent.

I asked Orion to fix those inconsistencies. It updated the metrics, reran them, and recorded the decisions in the project’s shared context.

Before AI, I would have carved out dedicated engineering time for this work. There were schemas to inspect, queries to rewrite, historical results to reconcile, schedules to coordinate, and documentation to maintain. Every discrepancy meant another investigation. With Orion, I could work through that process in a series of focused conversations, reviewing evidence and making decisions as we went.

The pace was what felt magical. When a result looked wrong, I could pursue it while the reasoning was still fresh. Orion handled the queries and comparisons, and I stayed focused on whether we were measuring what we intended. My understanding of the system kept turning into concrete changes.

Even a missing table became something I could manage conversationally. While its data synced, I asked Orion to watch for its arrival and bring me back to finish the remaining work. It built that follow-up into a workflow.

The most valuable thing I contributed throughout the morning was context: what our old numbers meant, which identities belonged together, and which decisions should carry forward. Orion helped turn that knowledge into queries, checks, and documentation. A fact that had lived in my memory could now inform the next analysis.

That is why I see data engineering and context engineering becoming inseparable. As AI takes on more implementation, knowing what the data should mean becomes even more valuable. Capturing that knowledge gives future work a better starting point. We can spend more of our attention asking better questions, investigating surprises, and deciding what to change.

I came to Orion with a set of stale metrics. By lunch, I had changed how those metrics worked, made the knowledge behind them explicit, and raised my expectations of what I could get done in a morning.