v0.4.0

robot-directives

Parse robot directives within HTML meta and/or HTTP headers.

crawlers header html http meta metadata nofollow noindex robots robots.txt seo spiders

Website Repository

Click the AI button to generate exact installation commands and usage examples for your environment.

Dependencies (3)

deep-freeze-node ^1.1.2
isbot ^2.0.3
useragent ^2.1.13

Dev Dependencies (4)

chai ^3.5.0
coveralls ^2.13.1
mocha ^3.3.0
nyc ^10.3.2

Package Info

License

MIT

Last Publish

June 26, 2022

Unpacked Size

0 Bytes

Total Files

?

Issue Tracker

Maintainers (1)

stevenvachon

stevenvachon contact@svachon.com