Change #254054
| Category | None |
| Changed by | Tim Rühsen <tim.ruehsen@gmx.de> |
| Changed at | Sat 03 Jan 2026 20:09:51 |
| Repository | https://api.github.com/repos/rockdaboot/wget2 |
| Project | rockdaboot/wget2 |
| Branch | master |
| Revision | be7415fc71e16e43aacc1896041f07df8882a2b0 |
Comments
Don't fetch robots.txt with --no-robots and --no-follow-sitemaps * src/wget.c (queue_url_from_local): Skip fetching robots.txt. * tests/test-robots-off.c: New test case for --no-robots --no-follow-sitemaps.
Changed files
- src/wget.c
- tests/test-robots-off.c