Information retrieval techniques: Lecture 34. The main topics covered in this chapter include: basic crawl architecture; communication between nodes; URL frontier: mercator scheme; duplicate documents; shingles + set intersection;... Please refer to the content of document.