Tag Archives: wget

  • Spider a website with wget

    This command might be useful if you want to auto-generate the Boost module cache files on a Drupal site wget -r -l4 –spider -D thesite.com http://www.thesite.com Let’s analyse the options… -r indicates it’s recursive (so “follow the links” and look for more than one page) -l indicates the number of levels we want to recurse. […]