Monthly Archives: August 2014

Ten times faster

One week ago I complained about Solr’s two-phase distributed faceting being slow in the second phase – ten times slower than the first phase. The culprit was the fine-counting of top-X terms, with each term-count being done as an intersection … Continue reading

Posted in eskildsen, Faceting, Hacking, Low-level, open source, Performance, Solr, Uncategorized | 5 Comments

Ten times slower

I jumped the gun on our current web index status with our whale hunting safari, but it turns out that there are other fish to kill (sorry, whales are not fish and I’ll stop with the nautical metaphors immediately). This … Continue reading

Posted in eskildsen, Faceting, Low-level, Performance, Solr | Leave a comment

Whale hunting with Solr

Our web archive index passed the 10TB mark a few days ago, so it was time for new performance measurements. To recap: 12 shards @ 900 GB, a total of 10.7TB or 3.6 billion documents. Served from a single 256GB … Continue reading

Posted in eskildsen, Performance, Solr | 2 Comments