BetfrikaBot, Betfrika's web crawler

Who we are

BetfrikaBot is the crawler of Betfrika ASBL/e.V., a non-profit organization present in Europe and Africa.

It reads public pages of organizations (churches, associations, reuse centers, businesses…) to find their official contact details, a short description of their activity and the leaders they present themselves on their website (name, role, work email address), so that we can offer them a partnership: books, collections, donations.

It always identifies itself with this User-Agent:

BetfrikaBot/1.0 (+https://www.betfrika.org/bot)

What it does

• It reads only a few pages per site: the home page, the legal notice or Impressum, the contact page, the “about” page and, if there is one, the page that presents the team or the leaders. It may use your sitemap to find them, and re-reads them periodically to keep this information up to date.
• It reads one page at a time, with a pause of at least a few seconds between two pages of the same site. When a website asks it to slow down, it spaces out its visits, then stops for several days if the request persists.
• When a page needs JavaScript to display, it may open it in a headless browser, always under the name BetfrikaBot.
• It follows your robots.txt: the rules written for User-agent: BetfrikaBot or, failing that, for User-agent: *, as well as Crawl-delay.
• It follows Content-Signal: BetfrikaBot does not pass the text of your pages to any AI model when your site forbids it (ai-input=no).

What it never does

• Work around a refusal: a 401, 403 or 429 response, a captcha or anti-bot page, a Disallow rule.
• Pretend to be a browser or another crawler.
• Create an account, fill in a form or get around a paywall.
• Use a person's photo, date of birth, home address or personal phone number to contact them.

What we keep, and for how long

It collects, in particular, the organization's business contact details (generic email address, phone number, postal address, website), its legal form and registration number, its social media accounts, the languages of its website, a short description of its activity and, depending on the type of organization, what the website publishes about the organization itself, for example a church's denomination or the origin of its community. This information is about the organization.

It also collects the leaders the organization presents itself on its website, for example the pastor, the chair or the manager: their name, their role and the email address published next to their name. When no address is published, we infer one in the format of the organization's addresses (for example firstname.lastname@its-domain), which we verify before any email is sent; we do the same for people of the organization already in our records. For each of them, we keep the page where they are presented and the date it was last read; if that page no longer names them, has disappeared or has not been read again for four months, we do not contact them before checking again.

The pages it reads and the raw responses of our tools and data providers, which may contain a photo or a personal number, are kept for at most 90 days (the pages' HTML code, 14 days), so that we can check what was taken from them.

The contact details of the organization and of its leaders are kept as long as they serve our relationship with it. They are checked again regularly and erased on request.

A personal address or number found elsewhere on a page is marked as such, is never used to contact you, and is erased at the latest 90 days after the last reading that saw it. The only exception: the address a leader has published next to their own name in order to be reached.

When you reply to one of our emails, your message is kept in our email service and in our outreach tool for as long as our exchange continues; on request, we erase it from our outreach tool. A request not to be contacted again is always respected.

Legal basis: the association's legitimate interest in offering a partnership to an organization and to its leaders (Article 6(1)(f) GDPR). When a church's page reveals a religious belief, we only process what the person has made public themselves (Article 9(2)(e) GDPR).

Where this data comes from

• The public pages of the organization's website.
• Public directories, for example those of church federations.
• Public registers: the French SIRENE register of businesses and associations, and ADEME's directory of reuse organizations.
• Search services, to find the website's address.
• What we already knew about the organization (our records, a meeting, an exchange).
• The format of the organization's email addresses, to infer a leader's address.
• Imported files and professional data providers (Bright Data, Apify, Apollo), from public LinkedIn profiles and business directories (name, role, organization, city, country, LinkedIn profile and, sometimes, work email address). Bright Data also regularly reads again the public LinkedIn profile of people in our records, to know whether they have changed organization: if so, we link them to their new organization and stop using their former work address. These providers collect these profiles with their own tools, not with BetfrikaBot.
• Replies to our emails and non-delivery notices.

Who receives it

• Our servers, in the European Union (Microsoft Azure, in Germany and Ireland).
• The search services Serper and Google receive the organization's name and city, and a word that states its type (for example “church”), to find its website, never the content of its pages.
• Anthropic (United States) receives the text of a few pages to describe the organization: its activity and, depending on its type, its denomination or the origin of its community. We first remove email addresses, phone numbers and, as far as possible, names of people from that text. Never when the website forbids it (ai-input=no). The page that presents the team is never sent to it: leaders are collected by our own rules, without AI. When you reply to one of our emails, your reply may also be passed to it: automatically, to link it to the right record, only its new part and its subject, with addresses and numbers masked, together with your name as displayed, the domain of your address, and the name, role and organization of a few people in our records who might have written it; and, at the request of a person from the association, the exchange without addresses or numbers, to summarize it or prepare a reply, in the latter case with your name, role, organization and country.
• Bouncer (European Union) receives email addresses to check that they exist: before any email is sent, and a few test addresses to learn an organization's address format; never the content of the pages.
• Bright Data receives the address of the public LinkedIn profile of people in our records, to read it again; Bright Data and Apify receive the criteria of our searches (country, role, sector). Neither receives the content of your website's pages.
• Salesforce, our contact management tool, receives the first name, last name, email address, organization and country of the contacts we follow up with.
• Our email service (Google Workspace) carries the emails we send you and stores your replies.
• Anthropic, Google and Salesforce are companies based in the United States.

How to block it

Add these two lines to your site's robots.txt file:

User-agent: BetfrikaBot
Disallow: /

BetfrikaBot reads your robots.txt before reading your site, then re-reads it within 24 hours at most: the rule applies from that new check. To stop it as soon as possible, write to us (see below).

Contact, objection and erasure

We record your objection: BetfrikaBot no longer reads your site, and your organization is removed from our outreach. You can also ask for access to the contact details we collected, their rectification, their erasure or the restriction of their processing. If you are a leader presented on your organization's website, you can object for yourself alone: we no longer contact you, erase what our outreach tool knows about you, including in Salesforce, and keep only your email address on our objection list and a fingerprint (a one-way hash) of your name, so that we never add you again to your organization's records.

You can also lodge a complaint with a data protection authority, for example the one in your country.

Your rights and how we process data: Privacy Policy