AI agent · Professional work
Tender Scraper Agent
Give it a tender website and it explores the site, writes a scraper, tests it and fixes its own errors until it works. Used on 280+ real government tender websites.
The problem
A tender-aggregation platform tracks hundreds of government and bank tender sites. Each needed a hand-written scraper, and sites change often.
What I built
An agent that, given a URL, surveys the site, finds the tender list, chooses fields (title, deadline, value, documents), writes a scraping recipe, runs it on the real scraping engine and repairs it from structured errors. Admins approve with one click.
How it works
- Explore — survey the website and locate the tender list, detail pages and documents.
- Write — choose the fields and produce a scraping recipe for the existing engine.
- Test and repair — run the recipe on the real scraping engine, read structured errors and fix the recipe within the job budget.
- Approve — hand the working recipe to an admin for one-click approval.
The hard parts
Pagination, detail pages, document downloads, pop-ups and many date formats all needed handling without site-specific code. Image CAPTCHAs use an AI solver with a human fallback. Every job has a 2-hour budget.
The result
Shipped to the team's dev environment across 3 codebases; QA uses it on 280+ tender websites. The agent produced working recipes for the majority of sites without human help.
Built with
Explore the project
Professional work. This write-up describes the workflow; source code and internal data are private.
Slack HR Onboarding Agent
Let’s make something useful
Have a project in mind?
Tell me about it. I reply within 24 hours.