Perplexity Publishes Research on Fast GPU Embeddings

Perplexity AI posted research on its serving infrastructure behind embedding and ranking models. The company described components including Ivy, Tulip, and ROSE for handling batching, latency, and throughput on GPUs. Founder Aravind Srinivas called it a deep dive into how Perplexity serves search results at scale: embeddings for ranking, GPU-based model inference, request batching, running inference servers, and handling latency/throughput trade-offs. The post links to a blog post titled Fast Embeddings on GPUs. Replies on X mixed praise for the technical details with criticism of the product.