Our client is building a next-generation search infrastructure product for AI agents: an external, paid search API that enables agents and AI applications to retrieve fresh, relevant, and hard-to-find information from the web.
The product is being built as a large-scale search system consisting of several core layers: web crawling, offline indexing and document processing, ML-based ranking, and high-load runtime serving.
The Runtime team owns the online serving layer: the performance-critical service that receives search queries, retrieves relevant documents from a massive corpus, ranks them, selects the best results, and returns them through an external API.
This is a greenfield infrastructure challenge at search-engine scale, with a small but very senior engineering team and a strong focus on deep systems engineering, performance, latency, and cost efficiency.
Key Responsibilities:
Design and build the high-load runtime serving layer of a search engine for AI agents.
Develop performance-critical C++ components responsible for query processing, retrieval, ranking, result selection, and response serving.
Work on systems that must handle very high throughput and strict latency requirements.
Profile and optimize CPU usage, memory usage, allocations, concurrency, network, IO, serialization, cache behavior, and other low-level bottlenecks.
Make architecture and implementation decisions that balance latency, throughput, reliability, and infrastructure cost.
Collaborate with engineers working on crawling, indexing, offline processing, and ML ranking to build an end-to-end search pipeline.
Build core infrastructure rather than simply integrating existing tools or assembling ready-made services.
Contribute to the technical direction of the Runtime layer in a highly senior, hands-on engineering environment.
Requirements:
Strong hands-on C++ engineering experience and seniority level: Staff / Principal / Tech Lead / Google L6+ equivalent.
Experience building high-load, low-latency, performance-critical systems.
Deep understanding of systems internals, performance optimization, profiling, and debugging.
Proven personal ownership of components that handled significant production load, ideally tens of thousands of RPS, thousands of RPS per node, or hundreds of thousands of RPS at service level.
Ability to explain concrete performance bottlenecks you identified and fixed, including trade-offs around latency, throughput, memory, CPU, and cost.
Experience working on internals of complex systems, not only integrating ready-made infrastructure such as Kafka, Redis, Elasticsearch, or databases.