fresiCrawler 1.3 crawler information

What is fresiCrawler 1.3 and what is it doing?

fresiCrawler 1.3 is a web crawler that is surfing the web automatically. It has been developed by the fresi project team. It finds websites that are publicly reachable and indexes real estate ads/classifieds from them on the "fast real estate search index" available on fresi.it and fresi.eu in order to unify as many real estate ads/classifieds as possible in one public search index.

Identification

fresiCrawler 1.3 usually identifies himself using the following user agent: "fresiCrawler 1.3 - fresi.it"

Exclusion standards respected

fresiCrawler 1.3 respects the NOINDEX statement in the robots meta tag content. (ex.: <META NAME="robots" CONTENT="NOINDEX">). This statement must be made in each page's head area that is meant to be excluded from being indexed.

How to contact us

In case of any requests regarding fresiCrawler 1.3, feel free to contact us at info@fresi.it please. Thank you.