Search engines rely on programs known as crawlers (or spiders) that gather information by following the trails of hyperlinks that tie the Web together.

搜索引擎使用一种称为搜寻器(或称蜘蛛机器人)的自动程序随着将网页连接一起的超级链接的足迹来收集资料。

youdao

The movement of the web may tell potential intruders that they are in the vicinity of a large animal, dissuading it from getting closer to the vulnerable baby spiders.

网的震动会使入侵者认为，它们即将面对的是庞然大物，脆弱的幼蛛们以此来让入侵者乖乖远离。

youdao

Web spiders are like a text-based browser, they can't read the text embedded in the graphic image or Flash.

网络蜘蛛就像一个以文本为基础(text - based)的浏览器，它们无法阅读嵌入图像或Flash中的内容。

youdao

All the search engines follow these robot directives, which, while not a perfect security solution, limit the entry points used by the bots and spiders that index the Web.

FORBES: Google Me Not