• GoDaddy Community
  • Websites + Marketing | Website Builders
  • Websites + Marketing | Website Builders

    Showing results for 
    Show  only  | Search instead for 
    Did you mean: 

    crawling gocentral created site creates a > 2MB logfile on semrush, one line > 2 million chars


    I use semrush to audit my site and its been failing for over a week now. They sent me the logfile from their crawl, and sure enough on line 385 of the log, it extends to over 2 million chars. 

    this is the command they are using:

    curl -i -sS -L --proto-redir -all,http,https --max-time 5 -A 'Mozilla/5.0 (compatible; GoogleBot/2.1; +http://www.google.com/bot.html)' -H 'Accept-Encoding: gzip, deflate' -H 'Accept: */*' --compressed theocgrillcleaner.com

    and this is the beginning of line 385 in the generated logfile. (too long to post)

    from what I can see on the + 2mil line, it appears related to the booking button function but not 100% sure but many references to date/time booking.

    <script type="text/javascript">!function(M){var e={};function b(p){if(e[p])return e[p].exports;var o=e[p]={i:p,l:!1,exports:{}};return M[p].call(o.exports,o,o.exports,b),o.l=!0,o.exports}b

    and goes on +2 million chars away.

    Is it possible to have this looked at as it does not seem correct and is recent within the last 2 weeks I believe from what I gather from my logs.

    Any ideas?




    Update 12/17
    I moved the Appointment widget off the homepage and low and behold, the SEMrush crawl worked just fine. 

    0 REPLIES 0