CodePulse AI started as a side project itch I had to scratch: paste a GitHub repository URL, get back a structured security audit. No setup, no CLI, no local tooling. Just a URL and a report. I built it using the Claude API as the analysis engine, and it taught me more about working with LLMs in production than any tutorial I've read.
This post covers the technical decisions behind integrating Claude into a real product not the Hello World examples, but the parts that only surface when actual code is running in production.
The Pipeline
The flow has three stages: fetch code from GitHub, chunk and prepare it for Claude's context window, then stream the analysis back to the client. Each stage has its own failure modes.
src/pipeline/analyze.ts
// High-level flow
export async function analyzeRepository(repoUrl: string): Promise<AnalysisStream> {
const { owner, repo, ref } = parseGitHubUrl(repoUrl);
const files = await fetchRepoFiles(owner, repo, ref);
const chunks = prepareChunks(files);
return streamAnalysis(chunks);
}
Stage 1: Fetching Code from GitHub
GitHub's REST API lets you fetch a repo's file tree and then the contents of individual files. For analysis purposes, I only fetch files that are likely to contain security-relevant code: TypeScript, JavaScript, Python, PHP, SQL, and config files (.env.example, docker-compose.yml, etc.). Binary files, images, lock files, and node_modules are excluded.
src/github/fetch.ts
const INCLUDE_EXTENSIONS = new Set([
".ts", ".tsx", ".js", ".jsx", ".mjs", ".cjs",
".py", ".php", ".rb", ".go", ".rs",
".sql", ".graphql",
".env.example", ".yml", ".yaml", ".json", ".toml",
".sh", ".Dockerfile",
]);
const EXCLUDE_PATHS = [
"node_modules/", "vendor/", ".git/", "dist/", "build/",
"*.min.js", "*.lock", "package-lock.json", "yarn.lock",
];
export async function fetchRepoFiles(
owner: string,
repo: string,
ref = "HEAD"
): Promise<{ path: string; content: string }[]> {
// Get file tree recursively via GitHub Trees API
const tree = await octokit.git.getTree({
owner,
repo,
tree_sha: ref,
recursive: "1",
});
const relevantFiles = tree.data.tree.filter((item) => {
if (item.type !== "blob") return false;
const ext = path.extname(item.path ?? "");
if (!INCLUDE_EXTENSIONS.has(ext)) return false;
return !EXCLUDE_PATHS.some((p) => item.path?.includes(p));
});
// Fetch content in parallel (respect GitHub's 5000 req/hr limit)
const contents = await pMap(
relevantFiles,
async (file) => {
const { data } = await octokit.git.getBlob({ owner, repo, file_sha: file.sha! });
return {
path: file.path!,
content: Buffer.from(data.content, "base64").toString("utf-8"),
};
},
{ concurrency: 10 }
);
return contents;
}
WarningAlways check data.encoding before decoding. GitHub returns base64 for text files but none if the blob exceeds 100MB (it won't send the content at all). Handle this case or your pipeline silently drops large files.
Stage 2: Chunking for the Context Window
A typical repository has far more code than fits in Claude's context window in one shot. My approach: analyze in logical chunks, then synthesize. Security issues are usually local to a file or a small set of related files, so per-file analysis works well. For very large files (> 500 lines), I split at function/class boundaries rather than arbitrary line counts.
src/pipeline/chunks.ts
const MAX_TOKENS_PER_CHUNK = 30_000; // ~120KB of code, conservative
export function prepareChunks(
files: { path: string; content: string }[]
): AnalysisChunk[] {
const chunks: AnalysisChunk[] = [];
let current: typeof files = [];
let currentTokens = 0;
for (const file of files) {
// Rough token estimate: 1 token ≈ 4 chars for code
const fileTokens = Math.ceil(file.content.length / 4);
if (currentTokens + fileTokens > MAX_TOKENS_PER_CHUNK && current.length > 0) {
chunks.push({ files: current });
current = [];
currentTokens = 0;
}
current.push(file);
currentTokens += fileTokens;
}
if (current.length > 0) chunks.push({ files: current });
return chunks;
}
Stage 3: The System Prompt and Structured Output
Getting Claude to return structured, machine-parseable output was the most important prompt engineering challenge. I ask for JSON with a specific schema and provide a concrete example in the prompt. Claude honors this reliably but you must validate the response before trusting it.
src/claude/prompt.ts
export const SYSTEM_PROMPT = `You are a senior security engineer reviewing code for a penetration testing firm.
Analyze the provided code files and return a JSON object matching this exact schema:
{
"summary": "one paragraph overview",
"severity": "critical" | "high" | "medium" | "low" | "info",
"findings": [
{
"id": "unique string",
"title": "short title",
"severity": "critical" | "high" | "medium" | "low",
"file": "path/to/file.ts",
"line": number | null,
"description": "detailed explanation",
"recommendation": "how to fix",
"cwe": "CWE-XXX" | null
}
],
"positives": ["what the code does well"],
"score": number between 0-100
}
Focus on: SQL injection, XSS, authentication bypass, insecure deserialization, hardcoded secrets, SSRF, path traversal, IDOR, and broken access control.
Return ONLY valid JSON. No markdown, no explanation outside the JSON object.`;
Streaming the Response
Security analysis takes 10-30 seconds per chunk. Streaming is not optional it's the difference between a product that feels responsive and one that looks broken while the spinner turns.
src/claude/stream.ts
import Anthropic from "@anthropic-ai/sdk";
const anthropic = new Anthropic({ apiKey: process.env.ANTHROPIC_API_KEY });
export async function* streamAnalysis(chunk: AnalysisChunk): AsyncGenerator<string> {
const fileContents = chunk.files
.map((f) => `\`\`\`// ${f.path}\n${f.content}\`\`\``)
.join("\n\n");
const stream = await anthropic.messages.stream({
model: "claude-sonnet-4-6",
max_tokens: 4096,
system: SYSTEM_PROMPT,
messages: [
{
role: "user",
content: `Analyze these files:\n\n${fileContents}`,
},
],
});
for await (const event of stream) {
if (
event.type === "content_block_delta" &&
event.delta.type === "text_delta"
) {
yield event.delta.text;
}
}
}
Parsing Structured Output Defensively
Claude almost always returns valid JSON when you ask for it clearly. But "almost always" is not "always". I wrap the output in a Zod schema and handle parse failures gracefully rather than crashing.
src/claude/parse.ts
import { z } from "zod";
const FindingSchema = z.object({
id: z.string(),
title: z.string(),
severity: z.enum(["critical", "high", "medium", "low"]),
file: z.string(),
line: z.number().nullable(),
description: z.string(),
recommendation: z.string(),
cwe: z.string().nullable(),
});
const AnalysisSchema = z.object({
summary: z.string(),
severity: z.enum(["critical", "high", "medium", "low", "info"]),
findings: z.array(FindingSchema),
positives: z.array(z.string()),
score: z.number().min(0).max(100),
});
export function parseAnalysis(raw: string): z.infer<typeof AnalysisSchema> | null {
try {
// Claude sometimes wraps JSON in backtick code fences despite instructions
const cleaned = raw.replace(/^```json\n?/, "").replace(/\n?```$/, "").trim();
return AnalysisSchema.parse(JSON.parse(cleaned));
} catch {
return null; // Caller handles null gracefully
}
}
Cost Management
This is the part nobody covers in tutorials but everyone discovers in production. LLM costs scale with usage. For CodePulse AI I implemented three guards:
- 1.Token estimation before sending: count approximate tokens in the request. If it exceeds a threshold, warn the user and let them confirm before running the analysis.
- 2.Repository size cap: refuse repos over 50MB or 500 files. This is as much a quality gate as a cost gate huge repos produce noisy analysis.
- 3.Result caching by commit SHA: if the same repo at the same commit SHA was analyzed in the last 24 hours, return the cached result. Most re-analysis requests happen within minutes of a first run.
“The prototype already caught real bugs on the first repo I tested it on — a hardcoded AWS key in a config file that had been sitting in a public repo for 14 months. That was the moment I knew the product was worth building.”