← Back
Home › Systems › Agentic AI

codebase-indexing

codebase context indexing for agents

Cursor index

SystemTechniques · 9Primitives · 36 codebase-indexing Code symbol indexingcompilers · medium: Extract definitions and references per file, give each symbol a stable string id, and store symbol to occurrences, re-indexing only files whose content hash changed.›Incremental parsingcompilers · medium: Reparse only the region an edit damaged and splice the new subtrees into the old tree, sharing everything else; O(edit) instead of O(file).›Language server protocolcompilers · medium: One JSON-RPC conversation over a pipe lets any editor drive any language's analysis server; the cost is keeping a live, edited document in sync and answering before it changes.›HNSWsearch-sort · medium: Hierarchical Navigable Small World: a layered proximity graph searched greedily from a sparse top layer; the default in-memory approximate nearest-neighbour index.›Linear-time regular expression matc…search-sort · medium: Match a regular expression in O(pattern x text) worst case by tracking all live parse positions at once, then cache the state sets as a lazily built DFA for speed.›Trigram indexsearch-sort · medium: Trigram posting lists accelerating regex and substring searchInverted indexstorage · medium: Map each term to a sorted, compressed list of the documents containing it; the core of full-text search and other fast set queries.›BM25 rankingsearch-sort · low: The standard lexical relevance score: IDF-weighted term frequency with saturation (k1) and length normalization (b); default in Lucene, Elasticsearch and SQLite FTS5.›Code context retrievalagentic-ai · unknown: Roaring bitmapscompression · low: Compressed set of 32-bit integers in 64K-value chunks, each a sorted array, bitmap or run list; fast intersections, the standard for bitmap indexes.›Merkle treedata-structures · low: Hash tree for verifying and diffing large datasets with logarithmic proofsFile system watchinghardware-os · low: Treat OS change notices as hints, debounce them, and rescan and diff a snapshot whenever notices overflow or may be lost.›Embeddingsmachine-learning · low: A learned table mapping discrete ids to dense vectors: gather forward, sparse scatter-add backward, row-wise optimizers to keep huge tables cheap.›Text chunkingmachine-learning · low: Cut documents into token-bounded pieces along structure boundaries, keeping offsets and section paths, so each piece embeds well and can be cited.›Glob matchingsearch-sort · low: Glob/gitignore pattern matching without backtracking blowupHybrid search and rank fusionsearch-sort · low: Merge a keyword (BM25) result list and a vector (ANN) list into one ranking, by ranks (RRF) or by normalized scores (convex combination).›Cryptographic hashes (SHA-2, BLAKE3)security · low: Collision-resistant fingerprints of data; SHA-256 for interoperability, BLAKE3 for speed, HMAC/HKDF for keyed uses.›Git object modelstorage · low: Content-addressed blobs, trees, commits and refs; packfiles with delta compression; worktreesName resolutioncompilers · low: Bind every use of a name to its declaration with a scope stack, handling shadowing, hoisting and closure capture in one AST walk.›Protobuf wire formatnetworking · low: Protocol Buffers wire format: tags, varints, length-delimited fields and schema evolutionRecursive-descent parsingcompilers · low: Hand-written top-down parser with one function per grammar rule; simple, fast, with full control over error messages and recovery.›Ropegraphics · low: A balanced tree of small string chunks with cached byte and newline counts, so seek, insert, delete and snapshot cost O(log n) instead of O(n).›d-ary heapdata-structures · low: Array heap with d children per node (d = 4 typical): half the binary heap's depth, siblings in one cache line, faster priority queues.›+ 2 more techniques+ 20 more primitives

Similar systems

Share the most building blocks with this one.

code-search 26 %agent-memory 21 %product-search 19 %coding-agent 18 %rag 18 %