XenForo's built-in sitemap tells crawlers about every thread and when it changed. It should be building automatically.