GitLost: How Researchers Tricked GitHub's AI Agent Into Leaking Private Repos

Security researchers at Noma Labs disclosed a critical vulnerability in GitHub's Agentic Workflows feature that allows unauthenticated attackers to extract data from private repositories. The attack requires nothing more than posting a crafted GitHub Issue in any public repository within an organization.

What Is GitLost?

GitLost is the name researchers gave to a prompt injection vulnerability affecting GitHub's new Agentic Workflows system. The core issue: insufficient trust boundary enforcement between untrusted user input and AI agent instructions.

When an organization enables Agentic Workflows with cross-repository access, the AI agent can read files from both public and private repositories. The problem is that the agent also processes the content of GitHub Issues - which anyone can create on public repos.

How the Attack Works

The attack chain is straightforward:

1. Identify target: Find an organization with GitHub Agentic Workflows enabled and cross-repository access configured
2. Create malicious issue: Post a GitHub Issue in any public repository within that organization
3. Trigger the agent: When the workflow assigns the issue, the AI agent activates
4. Extract data: Hidden instructions in the issue body direct the agent to fetch private repository contents
5. Exfiltrate: The agent posts the extracted data as a public comment

The researchers demonstrated successful extraction of README files and code from private repositories, with the agent dutifully posting the contents as public issue comments.

The "Additionally" Bypass

One detail from the disclosure stands out. GitHub appears to have implemented guardrails to prevent obvious prompt injection attempts. The researchers found that adding the word "Additionally" to their payload bypassed these protections, forcing the model to reframe output rather than refuse.

This highlights a fundamental problem with LLM guardrails - they are essentially more prompts, and prompts can be overridden with... more prompts.

What HN Is Saying

The Hacker News discussion (280+ comments at time of writing) is filled with developers debating whether this is a GitHub vulnerability or a user misconfiguration issue.

One commenter framed the core problem clearly:

"Who thought having a LLM with access to private information, with public access to ask it questions, would ever be a secure process?"

Several commenters pointed out this is analogous to setting up a CI job with access to secrets and running it on public PRs. If you configure GitHub to allow public code or LLM instructions to run in contexts with access to sensitive data, that data will leak.

The discussion around guardrails was particularly pointed:

"LLM guardrails are either just written prompts as in 'Please do not bad stuff :(' or other LLMs verifying that the first LLM didn't do some bs. Both methods do not work sufficiently as time shows again and again."

Another commenter offered a succinct take on the architectural issue:

"The answer is you should not allow LLMs access to untrusted input and sensitive data at the same time."

A few developers noted that the proper fix is for GitHub to prevent agentic workflows from executing in a public repo context if they also have private repo access. Several mentioned they're moving to self-hosted alternatives like Forgejo.

The SQL injection comparison came up repeatedly, with commenters pointing out a key difference: SQL injection is fully mitigated by prepared statements. There is no equivalent "prepared statement" solution for prompt injection.

Read the full thread at https://news.ycombinator.com/item?id=48827858.

Why This Matters

This vulnerability illustrates what security researcher Simon Willison calls the "Lethal Trifecta" - the dangerous combination of:

1. An AI agent with access to sensitive data
2. The ability to receive instructions from untrusted sources
3. The ability to take actions (like posting comments)

Any two of these might be acceptable. All three together creates an exploitable system.

GitHub's Agentic Workflows shipped with all three by default. Organizations that enabled cross-repository access effectively gave every GitHub user on the internet a channel to query their private repositories.

Recommendations

The researchers and HN commenters suggest several mitigations:

For organizations using GitHub Agentic Workflows:

• Review cross-repository permissions immediately
• Restrict agentic workflows to private repos only, or remove private repo access entirely
• Consider whether the workflow needs to respond to issue content at all

For anyone building AI agent systems:

• Never treat user-controlled content as trusted instructions
• Minimize agent permissions to the absolute minimum required
• Separate agents that handle untrusted input from agents with access to sensitive data
• Implement hard permission boundaries, not just prompt-based guardrails

For the industry:

• Stop shipping AI features with maximum permissions by default
• Recognize that guardrails are not security boundaries
• Accept that prompt injection is currently unsolvable at the model layer

The Bigger Picture

This is not the first AI agent security incident, and it will not be the last. As one HN commenter noted, we are in "the wild west phase of agent usage."

The pattern is now well-established: a company ships an AI feature with broad permissions, researchers find a prompt injection path, the company patches that specific attack, and researchers find another. The underlying architecture - mixing untrusted input with trusted instructions in the same context window - remains unchanged.

Until the industry develops architectural solutions (not just guardrails) for separating instructions from data in LLM contexts, every agent system that processes untrusted input while holding sensitive permissions is a vulnerability waiting to be discovered.

Sources

• GitLost: We Tricked GitHub's AI Agent into Leaking Private Repos - Noma Security
• Hacker News Discussion
• Simon Willison's Lethal Trifecta Talk - Referenced in HN comments