Chapter 28 References

Sources for Web Crawler at Scale, Distributed Search Engine & Google PageRank, Distributed Message Queue, and Distributed S3-Like Object Storage. Return to the Chapter 28 overview for chapter scope.

Books

• Designing Data-Intensive Applications by Martin Kleppmann.
• The Design of Future Things by D. Allan Simmering.
• Computer Networks by Andrew S. Tanenbaum and David J. Wetherall.
• Mining of Massive Datasets by Jeffrey Dean and Anand Rajaraman.

Websites

• Robots Exclusion Protocol
• Google Search Central: crawling and indexing
• The PageRank Citation Ranking: Bringing Order to the Web
• Apache Kafka documentation
• Amazon S3 multipart upload
• S3 API: Object Lambda and storage classes
• Apache Ozone: Object Store on Apache Hadoop
• MinIO documentation
• Elasticsearch: Search and index APIs
• OpenSearch documentation