Observation on Bots spiders crawling this friendly little B6G Forum

[privateer35]

I do SEO for a local agency. This forum produces a ton of content (mostly complaining ???)

in order for google to keep the most up to date information, frequent crawling of the forums is necessary. How else do they have news articles in their search results minutes after they are posted!

Came here to say this ^

Sponsored

 

[jayhawkco]

Not to make everyone sad, but a "robot" or a "crawler" is just a script or program that someone has run, possibly a large website like Google, that "scrapes" or reads the data on the publicly viewed pages to make it readable by that site. So Google would scrape to make it a part of their search engine. If I wrote my own script for a personal web page, it might pull info such as new posts (or B&P notices). Nothing to get too excited about alas.

Chris

 

[lenitech]

I find it crazy that a small dealership in Iowa has a tech savvy guy that figured out how to take advantage of the modern internet to drive mega sales to his dealership from around the country and none of the big dealers in Silicon Valley have figured this out. Instead of them using modern marketing tools they are whining to Ford that it's not fair. I have to give mega props to Albert and Granger for figuring this out.

 

[MaverickMan]

Are you saying robots are watching me? Where's my pants?

 

[Kogi]

Robots in disguise?

 

[Dads_bronze_bronco]

I always feel like somebody's watching me...

 

[Blksn955.o]

Ha, as I sit waiting for a search analytics report finish. Will be running an internal and Google analytics reporting today.

 

[vrtical]

good ole google crawling the site.

User-agent: *
Disallow: /

 

[Monster79]

Just when I thought your avatar couldn't get any better...

Well done sir, you have won

 

[Johnny Mo]

This

 

[MrJoe]

Google is putting spiders where?

 

[Granger Ford]

Those were some funny replies, yes we do analytics on our sites as well...

But I did not realize how often and quickly they are gathering the new data was...

Makes the Google buy words more affective...

 

[BroncOBX]

Did someone say B&P?

DRINK!

Laugh out loud for sure. I'll drink to that!

Sponsored