How we rank tenders for a firm

This page exists so you can decide whether to believe our rankings. It has the method, the held-out numbers, and the biases we know about and have not solved.

Stage 1: deterministic filters

Before anything is scored, notices are removed for reasons that need no model: the tender has closed, it is outside the regions the firm works in, or it is outside their trades. These are rules, they are auditable, and a firm can see exactly why something was excluded.

Trade matching runs on a hand-maintained vocabulary of 20 trades with English and French keyword sets — 557 rules at last count. It is deliberately deterministic: when it is wrong we can fix it with a rule and pin the fix with a test, which is not true of a model.

Stage 2: the learned ranker

Surviving notices are scored by a gradient-boosted ranker (LightGBM LambdaRank) trained on real bidding behaviour: 756,872 observed bids from Québec’s public procurement record, across 11,182 firms.

Thirty features in three groups — what the firm has done before, what the opportunity is, and how the two relate. The single most important is the cosine similarity between a multilingual sentence embedding of the notice title and a centroid of the tenders the firm has actually bid on. SEAO is French and CanadaBuys is English, so the embedding model has to share one vector space across both languages.

The numbers

2.3× the deterministic baseline on recall@10 and 2.3× on MRR. We report the full-pool number because an earlier version of our evaluation sampled 200–400 negatives per firm and inflated every metric roughly fourfold. Sampled-candidate metrics are not comparable to these, and you should treat any ranking claim that does not say which it used as unmeasured.

Splits are temporal, never random

Every evaluation trains on one time window and tests on a later one. A random split would let the model learn from bids placed after the ones it is asked to predict, which inflates scores and predicts nothing about live use. Every firm-history feature also takes an as-of date and excludes anything dated on or after it, with a strict inequality — a tender closing on a given day must not see bids placed that day.

Three biases we have not solved

• Incumbency is self-reinforcing. Prior bids with a buyer is a strong feature, so a firm that has worked for a municipality is ranked toward working for it again. That reflects reality and it also entrenches it.
• Single-bidder procurements are excluded. 62% of SEAO procurements name only the winner, who won 98% of the time; training on them would teach who wins, not who bids. The blind spot: a firm may legitimately want sole-source-shaped work, and this data cannot teach us that.
• The training market is Québec. Nearly all labelled bidding behaviour is Québec, so behaviour is learned there and applied elsewhere.

What the public demo does and does not do

The demo ranks from a description, which means it has no bidding history to work with. Every firm-history feature is zero and the firm vector is assembled from matched trades rather than from real bids. It is the same model, given much less. The demo says so on the page rather than presenting itself as the product.

One consequence we publish because it is easy to miss: since both sides of the similarity feature derive from the same trade assignment in that mode, the demo leans harder on the keyword rules than the real product does. Demo behaviour is not model behaviour.

Scores are bid fit, never a chance of winning

The model predicts bid propensity — how likely this notice is to be something a firm like yours pursues. It says nothing about whether you would win, and we do not let the word “probability” near the output, because a number that looks like a win chance will be read as one.

Method aside: the ranking is only half the product. The other half is the qualification check that reads a document and quotes the clause that would disqualify you — covered in the clauses that disqualify compliant bids.