Imported from yousefamar/console (
.claude/skills/property/SKILL.md). Install upstream withnpx skills add yousefamar/console --skill property. Copyright stays with the author.
name: property
description: Console's multi-portal house-hunt poller (server/src/property): inventory + kind/tier layers, the eight portal clients (Rightmove, OnTheMarket, IS24, Sparkasse, immobiliare, wikicasa, subito, smallholdings, Kleinanzeigen), cross-portal dedupe, detail enrichment + farmland classifier, place filter, tenure, review state, backfill, nearest-airport lookup, notify layer/criteria, two-tier bar, and every portal's protocol traps. Use when changing property search/poll/alert code, adding a portal, debugging pins or pushes, or running con map property.
paths: server/src/property/**, server/src/routes/property.ts, cli/src/commands/map-property.ts, src/map/property-review.ts
user-invocable: false
metadata:
card_keywords: "rightmove, onthemarket, immoscout, is24, immobiliare, wikicasa, subito, sparkasse, kleinanzeigen, house-hunt, house hunt, property search, property listing, property listings, house listings, farmland, smallholding, isochrone, where-to-move, con map property"
Property search
Moved out of CLAUDE.md on 2026-09-08 (the CLAUDE.md diet, ^rosy-owl) — the text below is the trimmed reference that used to load into every request. The CLAUDE.md stanza for this subsystem keeps only the cross-cutting invariants; treat THIS file as the authoritative detail and keep it current the way you would CLAUDE.md.
Per-portal protocol notes (primary source for wire formats): ~/sync/brain/root/projects/home/<portal>-api.md.
- Inventory + kind layers. The hourly poll skims the newest
FETCH_LIMIT=50 per ring and used to REPLACE the map with them. Now every search also has an inventory (server/src/property/inventory.ts,PropertyInventoryStore→ one file per search under~/.config/console/property-inventory/):PropertySync.fullSync(id)runsfetchAll()— every page of every ring, and when a portal caps a single query (Rightmoveindex>1000, IS24 page≥50, immobiliareisResultsLimitReached) it halves the price range and recurses (MIN_BAND_WIDTH5k). Runs everyFULL_SYNC_INTERVAL_MS=6h fromtick()(after the skims) orPOST /property/searches/:id/sync/con map property sync <id>; the skim upserts into the inventory too (full:false, never marks removed); a full pull marks what the portal no longer returnedremovedAt(kept, hidden). Inventory rows are COARSE — only what the portal applied server-side — so the fine filters (postFilterovers.unsupported,clipToLayer,applyOutsideBar, dismissals) run at draw time and a criteria tweak redraws without a re-pull; a coarse edit (criteria/layer/country) drops the inventory viasync.update()(routes call that, notstore.update()), and the next tick re-pulls.recordInventory()marks every pulled id seen (full pulls never notify);SEEN_LIMITraised 4k→20k so the hourly trim can't evict them. Layers are sliced by KIND only (Yousef: "do not slice by country … nor by source. Only slice by type"):PropertySearch.kind('house'default |'farmland'|'plot'), oneproperty/<kind>layer fed by every search of that kind,country+portalare popup fields;MAX_PINS600→25k.GET /property/searches/:id/inventory[?removed=1], CLIcon map property inventory <id> [--removed]. Clients pauseDEEP_PAGE_DELAY_MS=300 between pages on deep pulls (limit > 4 pages) only. - Several portals per country + cross-portal dedupe.
PropertySearch.portalpicks the client explicitly (portalOf(s)=s.portal ?? PORTAL_BY_COUNTRY[s.country]; a portal change re-seeds like a criteria change), so aggregators/specialist sources are just more searches feeding the same kind layer —con map property add --country UK --portal onthemarket …,count --portal. Because agents syndicate,updateKindLayer()runs every fine-filtered candidate throughgroupDuplicates()(server/src/property/dedupe.ts: ≤60 m + price within 1.5%/€1.5k, bedrooms as fallback; grid-bucketed O(n); never groups two rows from the same search — a portal's ids are the identity there, so co-located flats / IS24 marker-snapped rows stay separate) BEFORE verdicts: the group is drawn once from the primary-portal copy (alsoOnpopup field names the others), any dismissed copy hides the whole group, any interested copy marks it interested. - Detail-page enrichment + farmland classifier.
PortalClient.detail(listing)(optional) fetches the listing's own page;PropertySync.enrich(id, limit)runs it over inventory rows lackingdetailAt(or older thanDETAIL_TTL_MS30d), newest first,DETAIL_DELAY_MS1.1 s apart,ENRICH_PER_TICK400 per search per hourly tick (UK's ~8k rows ≈ 20 ticks from cold),null⇒ row marked removed, a throw (403/429) ends the batch.POST /property/searches/:id/enrich?limit=N,con map property enrich <id> [--limit N]. Rightmove's detail page is a SvelteKitdevalue-flattened model (window.__PAGE_MODEL = {"data":"[…]"}— indexes into a flat array), unflattened byunflatten()in rightmove.ts;detailFields()keepskeyFeatures,description(entities decoded), exactpropertySubType, full postcode, "Added on" date, and land size parsed from prose only (land.tsplotAreaFromText: acres/ha/"plot of N sq m", largest plausible figure) —sizingsis the FLOOR area restated in sqft/sqm/acres (100 m² → 0.0247 ac), never land.postFilterkeywords now search keyFeatures + description too. Layers split per LISTING, not per search:listingKind()(land.ts) sends a row toproperty/farmlandwhen its search is farmland, its type matches the smallholding/equestrian/farmhouse/Bauernhaus/Resthof/rustico/casale… regex, or its plot ≥FARMLAND_MIN_PLOT_M2(1,000 m² — Yousef's call: UK house-with-land ≤ £300k barely exists above a quarter acre), or — when no size is stated at all — its text matchesFARMLAND_TEXT_RE(paddock/orchard/stables/equestrian/smallholding/woodland/pasture/acre(s)/hectare; DE Streuobst/Weide/Pferdestall stems; IT frutteto/uliveto/pascolo; deliberately not "garden"/"land"/"plot" alone). A stated size below the floor beats a keyword.updateLayer()therefore redraws every kind. Classifier rules that came from the first real UK pass (236 of 8,039 houses promoted → 7):FARMLAND_TEXT_REhas no woodland / bare acre(s) / stable(s) / equestrian; every keyword and every prose acreage passeshasLandKeyword-style context checks (proper names via street/house/business suffixes, neighbourhood via before/after word lists); prose figures over 25 acres are the surroundings, not the plot; word fractions ("one third of an acre", "0.13 of an acre") parse; postcodes ("WV14 8HA") never read as hectares. Rightmove/OnTheMarket have no plot field, solistingKindignores a storedplotAreafrom those portals and re-derives it from prose under the current rules; portals with a real plot field are trusted at any size. - Eight portal clients. Besides Rightmove / IS24 / immobiliare:
onthemarket.ts(UK; JSON, polygon-native, server-side keywords/garden/parking/auction; ~98% overlap with Rightmove, kept for lag + keyword search),sparkasse.ts(DE; unprotected JSON API republishing ~90% of Immowelt, radius ≤100 km viacoverRingWithCircles, plot + coords per row; numeric fields are German-formatted strings with separators stripped —parseGermanNumber),wikicasa.ts(IT; WKTpolygonFromMaphonoured,cityDtocoords, 48% hidden-address rows →coordsPrecision:'area'),subito.ts(IT;hadesAPI needsx-subito-channel: web, radius-tiled, c=29 houses + c=30 rustici whenfarmhouserequested,geo.mapis the geocoded address at everyzoom→'exact'; comune point →'area'only without a map point or within 250 m of it,agent:'privato'),smallholdings.ts(UK farmland feed; WP REST + Nominatim geocode cached at~/.cache/console/geocode-uk.json). Each has its own__tests__/property-<portal>.test.ts+ recorded fixture and a protocol note in the vault (~/sync/brain/root/projects/home/<portal>-api.md). Surveyed and NOT built: Zoopla/PrimeLocation (Cloudflare + terms; 0 unique houses), Idealista (DataDome hard block), Immowelt direct (DataDome escalates to captcha; Sparkasse covers it), Kleinanzeigen (hair-trigger IP block — needs a ≤2 req/min crawler, deferred), Casa.it (needs real Chrome — deferred). Shared fixes that came out of the builds:postFilterenforcesminBedrooms/maxBedrooms/houseSubtypeslocally when a portal reports them unsupported (fail-open on missing data);normaliseHouseTypeknows semindipendente (semi) and terratetto (terraced); dedupe skips the bedroom check for locali-based portals (LOCALI_PORTALS);nearGeometry's bbox pre-filter pads longitude in real degrees (the oldbufferKm/100rejected points 4–6 km east/west at UK latitudes). - Tiered searches → their own layers.
PropertySearch.tier(e.g.gold) routes a search's pins toproperty/<kind>-<tier>(gold colour) instead of the sharedproperty/<kind>; untiered searches never draw there and tiered ones never draw to the defaults. Why: Yousef's gold-town searches (Brighton & Hove, Edinburgh coast) run at a lifted £400k ceiling and "must go in their own map layer so it doesn't get confusing" — the kind-only layer rule stands, the tier is a second axis he asked for.PropertySync.layerTargets()enumerates kinds × tiers in use,pruneLayers()dropsproperty/*layers nothing feeds (also whatmigrateLayers()now uses), andremove()prunes so a retired tier's layer disappears with its last search. CLI:--tier <name|none>onadd/set.layerNameFor(kind, tier)in store.ts is the one place the slug is built. - Bot walls:
browser-fetch.tsruns a portal's requests inside a Chromium page. immobiliare'sapi-nexthas been behind DataDome since 2026-09-07 (HTTP 403 with ageo.captcha-delivery.comJSON body) and DataDome fingerprints the TLS handshake, so a browser-minteddatadomecookie is useless to undici (verified: same request 200 in-page, 403 from Node with the page's cookies).BrowserFetch(one browser, one page parked on the origin, requests serialised at ≥1 s, lazy launch, idle close after 2 min, one relaunch on a 403) exposes afetch-shaped function thatindex.tshands toImmobiliareClient— the client is untouched and still fixture-tested with a plain stub. It needs Playwright's FULL Chromium in new-headless mode (channel: 'chromium'): the defaultheadless_shellis served the captcha on the homepage, as is curl from the VPS. Playwright itself is resolved from other repos on the machine (playwright.ts, shared with the IS24 WAF-token minter); none found → the client throws "no Playwright install found" and the search records it aslastError. Load: ~50 requests/h (skims + liveness probes) + the 6-hourly full pull (~100 pages) — 10 pages at 1.5 s passed in the probe; there is nodetail()for immobiliare so enrichment never hits it. Casa.it ("needs real Chrome") would be the same pattern. Tests:property-browser-fetch.test.ts(fake Chromium). - Place filter —
criteria.maxHighStreetM+ localexcludeNewBuild. Yousef's definition of the suburbia he hates is "rows of houses where the only way to a high street is by car"; his bar is a real high street within ~5 min on foot. No portal can express that, soserver/src/property/place.tsdoes it at draw time next to the zone clip:HighStreetIndexloads the vault'sdata/high-streets.geojson(one point per ~100 m cell whose 300 m neighbourhood holds ≥ 20 OSM shops + cafés + restaurants + bars/pubs —amenity=barbecause the Italian bar IS the café; built by the vault'sfetch-shops.sh+fetch-bars.sh→build-high-streets.mjs). Rows withcoordsPrecision: 'area'split by what the centroid IS (PLACE_CENTRE_PORTALSinsync.ts,walkTestable()): a comune/zone CENTRE (Subito's town point, Wikicasa's cityDto) is tested at the centre with the same bar and labelledtown centre 120 m to shops— a centre with no high street rules out every house in the comune (64% of Italian centroid rows, 2026-09-13; the old skip let 3,162 untestable pins through and made Italy 90% of the house layer); a PLZ average (Kleinanzeigen) or a free geocode (smallholdings) proves nothing either way and still passes unlabelled. The residual leak is a car-only outskirt of a comune that does have a centre. Grid-hashes it, memoises per coordinate and re-reads the file when its mtime changes (no restart for new shop data);nearestM()returns straight-line metres (capped at 5 km, null without data).PropertySync.applyPlaceFilter()drops rows beyondmaxHighStreetMand, when the portal reportedexcludeNewBuildunsupported, rows whose text has developer/new-build tells (newBuildLike: developer names, "show home", "help to buy", "plot 12", "The Orchard at …"; never "modern" or the street type). Pins carryhighStreet: "320 m to shops". Path fromPROPERTY_HIGH_STREETSor~/sync/brain/root/projects/home/data/high-streets.geojson.LOCAL_ONLY_CRITERIA(store.ts) exempts draw-time-only keys from the re-pull rule. CLI:--max-high-street-m <m|none>. - Leasehold is an automatic disqualification (Yousef's rule).
Listing.tenure(normalised byland.ts normaliseTenure: freehold / share-of-freehold / leasehold / commonhold) is set from OTM'sTenure:features bullet and from Rightmove's detail-pagetenure.tenureType;postFilterenforcesfreeholdOnlyon EVERY portal regardless ofunsupported(drops leasehold; drops share-of-freehold/commonhold too whenexcludeCommonhold), falling back to prose only when it says leasehold and never freehold; unknown tenure passes. Rightmove also filters server-side (tenureTypes). - Plot kind + fixer-upper glyph + sliver rings (2026-09-11, ^brisk-owl).
PropertyKindgained'plot'— bare land WITH planning consent for a dwelling (Yousef: "bare land with planning permission as a separate layer, why not"; self-build is the alternative-living route he kept, houseboats he didn't). A plot search (--kind plot --type land;Criteria.propertyType: 'land'→ RightmovepropertyTypes=land, no other client has a land type) feedsproperty/plotONLY and nothing else ever promotes into it (listingKindreturnsplotfor a plot search first;updateKindLayerseals the direction both ways). At draw time a plot row must passplanningLike()(land.ts) — sentence-level, FAIL-CLOSED: a sentence with "subject to", "planning potential", "potential/scope for", "lapsed/refused/no planning", "holiday" is discarded, then the rest must state consent ("full/outline planning", "planning permission for a 4 bed", "with planning", "permission in principle", "planning ref", "consented site"). Interested rows bypass it like every filter. Pins are 🏗️. Fixer-uppers:fixerLike()(FIXER_RE, calibrated on 15,035 live UK rows — 1,439 hits after dropping "blank canvas" (garden cliché), "structural defects" (new-build warranty boilerplate) and bare "subsidence" (material-information "History of subsidence: No")) marks house pins with 🏚️ + popupcondition: needs work; interested still wins the glyph. Why: "a fixer-upper in a gold town is also an option" — 13 of the 69 gold-town inventory rows are fixer-uppers. Sliver rings:outerRings()drops rings underMIN_RING_AREA1e-5 deg² (~0.1 km²) — the zone union produced 259 slivers under 1 km² (of 352 polygons), each costing a portal request per search, and a 3-point one made Rightmove 400 an entirecount; the vault'sintersect-livable.mjsnow drops them at source too (93 polygons remain, area unchanged). - Pins are just an agent map-layer —
updateKindLayer()upsertsproperty/<kind>(orproperty/<kind>-<tier>) viaMapLayerStore, so the SPA renders + toggles them with near-zero client-side change (MapTab'sBUILTIN_SUBLAYERSis only for location/geocaches/meetup; everything else flows throughlayers-subscribe.ts; the one exception isLayerFeaturePanel's verdict buttons, which POST/reviewdirectly). Notifications go out viaPushServer(one per fresh listing up toMAX_ALERTS=5, else a summary), index deltas viaSyncBus. KeepRESULTS_LIMIT(store) ≥MAX_PINS(sync) — the store cap caps the map, not just the API payload. dismissis a separate persisted set fromseenIds, on purpose.PropertySearch.dismissedIds— hides a listing's pin permanently, survives future polls and criteria edits (unlikeseenIds, whichstore.update()deliberately clears on a criteria/layer/country change). A dismissal is a judgment about the listing, not about whether today's query happens to match it.sync.dismiss(id, listingId, dismissed?)(now a thin wrapper oversync.review(), see the review stanza below) writes through the store then callsupdateLayer()immediately — the pin vanishes on the spot, not on the next hourly tick.POST /property/searches/:id/dismiss {listingId, dismissed?}, CLIcon map property dismiss <id> <listingId> [--undo], SPA button inLayerFeaturePanel(only rendered when the layer's feature carrieslistingId/searchId— added to pinpropertiesinupdateLayer(), excluded from the panel's generic key→value rows viaPANEL_SPECIAL).- Two-tier bar —
outsideCriteria(Yousef's rule: outside the Heathrow catchment needs a much higher bar; terraced = automatically out).PropertySearch.outsideCriteriais aNotifyCriteria-shaped gate that is a search filter: inpollSearch/backfill,applyOutsideBar()keeps a listing if it's insidenotifyLayer's geofence OR passeswithoutAirportGate(outsideCriteria)(never a Routes lookup — this runs over the whole snapshot); a failing outside listing never reacheslastResults/the map. Inside the geofence the plaincriteriaare the only bar. RequiresnotifyLayer; no gate or no geofence = pass-through. Interested listings bypass it (carried bywithInterestedCarried, a verdict beats a filter). Notifications are untouched (still inside-geofence + notify gate). Deliberately its own object rather than reusingnotifyCriteria, so the push bar and the map bar can drift apart later; seeded identical (UK: ≤£250k + detached/bungalow/villa/farmhouse/land). CLIcon map property set <id> --outside-criteria notify|none|<json>(notifycopies the current notify gate).PATCH /property/searches/:idalready passes it through (untyped body).criteria.excludeHouseSubtypesis the terraced rule: an exclusion list enforced LOCALLY on every portal inpostFiltervianormaliseHouseType(listing.propertyType), regardless ofunsupported— Rightmove'spropertyTypesomits terraced yet still returns town houses; fail-open on blank type text. CLI--exclude-house-subtypes terraced(a criteria flag — criteria flags MERGE into the search's current criteria;--notify-*gate flags replace the whole gate).excludeHouseSubtypesis inLOCAL_ONLY_CRITERIA(store.ts, withmaxHighStreetM), so editing it no longer re-seeds or drops the inventory; every OTHER criteria key still counts as coarse and costs a full re-pull (con map property sync <id>to force it). - Interested listings bypass every draw-time filter (2026-09-09).
updateKindLayer()re-adds a search'sinterestedIdsrows from the inventory afterpostFilter/ zone clip / outside bar / place filter — a verdict beats a filter on the inventory→layer path too, not just the poll path (withInterestedCarried). Why: the 400 m walk bar silently removed 3 of Yousef's 4 interested houses. Only the portal removing the listing (pruneGone→removeListing+markRemoved) takes an interested pin off the map. - Review state — interested / not interested / unreviewed.
PropertySearch.interestedIdsbesidedismissedIds, mutually exclusive, both survive criteria edits;store.review(id, listingId, 'interested'|'dismissed'|'none')is the one write path (dismiss()delegates to it). Opening a listing never changes state — only a verdict does. Pins are house emoji, not dots (Yousef's call): every feature carries_icon— 🏠 unreviewed, 🏡 interested (an emoji can't be recoloured, so the verdict is a different house) — via the renderer's existing per-feature_icon→em:<emoji>symbol path; an interested feature also carriesreview: 'interested'+_color: INTERESTED_COLOR(green — inert on the SPA symbol layer, kept for circle-drawing renderers such as Android), a dismissed one is absent. Map tab count = unreviewed pins —countUnreviewedListings()insrc/map/property-review.tsoveruseMapStore.layerData(property/*layers,listingIdpresent,review !== 'interested'), wired inLayout.tsx'sPaneTab; works from any pane becausewireMapLayersSubscriptionfetches every layer's geojson at boot. Interested listings outlive the snapshot:recordPollreplaceslastResultswith the newest-per-ring snapshot (so an unreviewed listing leaves the map once 50 newer ones appear in its ring, live or not), butwithInterestedCarried()carries interested listings forward;PropertySync.pruneGone()then probes each carried one the poll didn't re-surface viaPortalClient.isLive()(≤LIVENESS_MAX_PROBES=30/search/poll) andremoveListing()s a definitefalse;null(WAF/network/no coords) keeps it. Probes are per-portal because the pages differ: RightmoveGET /properties/<id>→ 410 = gone (200 + "has been removed" text also gone); IS24 expose page via the WAF token → 410 = gone, 200 with"isDeactivated…":true= withdrawn; immobiliare pages 403 anything but a browser and api-next has no per-id endpoint, so it's a targeted re-query —newest([boxAround(lat, lon, 120 m)], {channel})and look for the id (IT coords are exact).POST /property/searches/:id/review {listingId, state}, CLIcon map property interested <id> <listingId> [--undo];/property/listingsnow omits dismissed rows and flagsreview: 'interested'. SPA panel (LayerFeaturePanel) has both verdict buttons and is keyed per feature — it holds optimistic state that must not leak between pins. Android (2026-09-12): the pin panel has both verdicts (MapRepository.reviewListing), the Map toolbar chip + grid tile carry the unreviewed count, and the swipe deck (next stanza) is the fast path. - Interested ⇒ a vetting card (2026-09-14, ^pink-frog). Yousef: "When I mark a home as interested, kick off a card that investigates the listing more deeply and tries to see if there's a catch" — he had been filing
What's the catch? <url>cards by hand (^trim-bass, ^cosy-pony, ^gold-kiwi).PropertySync.onInterestedis a settable hook (index.ts assigns it onceBoardOpsexists; tests leave it null) thatreview()fires ONLY on the not-interested → interested edge — a repeat verdict, a dismissal or a clear never fire; re-interesting after a clear fires again and the board dedupes. The listing row comes from the inventory first (enriched: keyFeatures/description/tenure), the poll snapshot as fallback; a listing in neither is logged and skipped.property/interest-card.ts(pure,property-interest-card.test.ts):buildInterestCard()→ textWhat's the catch? <url>(his phrasing verbatim so hand- and auto-filed cards look alike) + two detail lines (price · beds/type · tenure · plot · address · kind/tier/search/portal id; thejqone-liner for the stored row, the vetting doc, the report pathresearch/listings/<portal>-<id>.md);findInterestCard(columns, url)= one card per listing EVER, any column incl. Done (the Done card is the answer), matched onlistingUrlKey()(fragment/query/trailing slash stripped, lower-cased — Rightmove's#/paste vs the client's bare URL) over text + detail. index.ts adds it to boardhome, columnIn Progress,@home, top — so the BoardWatcher forks a fresh vetter per listing (queued FIFO past the fork cap);PROPERTY_INTEREST_BOARD/PROPERTY_INTEREST_OWNERenv override. The vetter's procedure is the vault'sprojects/home/listing-vetting.md(why-is-it-cheap first, template fit second, report + two-line hand-back). - Review deck —
GET /property/deck?kind=house|farmland|plot&limit=N(2026-09-12, ^bold-kiwi). The phone's Tinder-style triage (AndroidPropertyDeckScreen, routemap/deck, 🏠 chip on the Map toolbar; SPA has no deck — its verdicts stay inLayerFeaturePanel).PropertySync.reviewable(kind, tier)is the ONE computation of "what is on the map" — the fine-filtered, zone-clipped, outside-barred, place-filtered, cross-portal-grouped candidates minus dismissed groups — and bothupdateKindLayer()(pins) anddeck()(cards) consume it, so the deck can never disagree with the pins.deck()walks every layer target of the kind (tiers ride along with atierbadge), drops interested rows (decided ≠ triage), sorts newest first with undated rows (immobiliare) last in stable order (sortDeck), and reportstotal+ per-kindcountsfor the kind picker. Cards (property/deck.tsDeckCard,toDeckCard) carry what the popup omits — rawprice+currency,keyFeatures,descriptioncapped atDECK_DESCRIPTION_MAX2000 chars,tenure,propertyType,fixer,alsoOn, the pin'shighStreet/airportlabels (airportLabel()is shared with the popup + push),lat/lon.limitdefaults 30, capped 200 (clampDeckLimit). Verdicts are the existingPOST /property/searches/:id/review; the phone queues them through its outbox and filters ids it has judged out of every refetch because the hub keeps serving a card until the verdict lands. CLIcon map property deck [--kind] [--limit]. Unit-tested inproperty.test.ts("the deck is exactly the drawn pins minus verdicts"). backfillisnewest()with a much biggerlimit, nothing more — each client already self-limits at its own real ceiling (Rightmoveindex>1000, IS24page>=50, immobiliareisResultsLimitReached/maxPages), so passingBACKFILL_LIMIT=5000instead of the hourlyFETCH_LIMIT=50just lets a client walk further before hitting a wall it already respects — no new pagination logic needed. The part that had to be careful:PropertySearchStore.recordBackfill()is a distinct code path fromrecordPoll()— it merges intolastResults(dedup by id, existing pins the backfill doesn't re-surface survive) and marks every touched id as seen, but never computes or returns afreshlist, so a backfill can never trigger a notification regardless of how many "new" ids it turns up.POST /property/searches/:id/backfill, CLIcon map property backfill <id>.- Nearest-airport drive+transit time, computed once per genuinely-new listing (
property/airport-distance.ts). The isochrone that gates the search polygon is a boolean pass/fail with no number attached and no record of which airport; this answers the follow-up question for one specific listing that's about to be shown.AIRPORTS_BY_COUNTRYis a static per-country list curated from the vault'smanifest.json(UK) /fra-reachable.json(DE/IT) — the exact airports whose isochrones were unioned into the search polygon, so "nearest of these" is the right question by construction, not a separate global airport lookup. Narrows to the 3 straight-line-closest airports first (free), then asks the Routes API for real DRIVE time on just those (a straight-line-nearest airport isn't always the fastest to actually reach — motorway layout, river crossings) and takes the minimum; TRANSIT is then queried for that one winning airport only. Requires a configured Google Maps key (con map gmaps credentials, plus the Routes API enabled on the GCP project — Places and Routes are separate toggles, a key working for search can still 403 on directions) —attachAirportDistances()no-ops silently ifgmaps.isConfigured()is false, so distances just don't appear rather than erroring the poll.- TRANSIT is schedule-dependent, so it's never queried for "now." Google defaults
computeRoutesto the current instant when nodepartureTimeis given — for an hourly poller that's indistinguishable from a live 2am query with zero service.nextWeekdayMorningUtc()always resolves to the next weekday 09:00 UTC that's still in the future (rolls a Friday-evening or weekend "now" straight to Monday), giving a reproducible "typical commute-ish morning" figure instead of whatever the wall clock happened to be.departureTimeis deliberately only ever sent onTRANSITcalls (DRIVE already models "now" viaTRAFFIC_AWARE). - Only computed for listings that get their own individual push —
fresh.length > MAX_ALERTS(the >5 summary-push path) skips the lookup entirely, since that path shows no single listing's detail to attach a distance to. Keeps the API-call volume tied to "how many listings did we actually show," not "how many listings exist." - Persisted onto
Listing.nearestAirportviaPropertySearchStore.setNearestAirport()— a separate write path fromrecordPoll(), called after the poll already landed, so the pin (drawn fromlastResults) and a fresh push both carry the figure without re-running the whole poll. Surfaces in three places: the pin'sairportpopup field, the per-listing push body (airportSuffix()), andListing.nearestAirportitself for anything reading the JSON directly.
- TRANSIT is schedule-dependent, so it's never queried for "now." Google defaults
notifyLayer— a push-noise filter, distinct from the search polygon. UK/DE/IT-wide searches generate far more genuinely-new listings than Yousef wants buzzing his phone for;PropertySearch.notifyLayernarrows notifications only —PropertySync.filterForNotify()runspointInGeometry()(geo.ts, dependency-free ray-casting; no turf in this repo) against every geometry in the resolved geofence before deciding whether/how to notify. Listings outside it are still merged intolastResultsand drawn on the map by the normal poll path — nothing about search/display changes, only whether the phone buzzes. The>MAX_ALERTSsummary-vs-per-listing split happens on the filtered count, not the raw fresh count, so a UK-wide batch of 18 fresh listings with only 2 inside the notify layer correctly sends 2 individual pushes rather than one noisy "18 new" summary. Unset (the default) notifies on everything, unchanged from before.- Two resolution modes,
geometriesForNotify(): a baregroup/namemap-layer slug, or a raw geojson file path (/…,~/…, or ending.json/.geojson) read straight off disk. The path mode exists specifically so a one-off geofence (e.g. one airport's isochrone) doesn't have to be re-published as a visible Map-tab layer just to be usable here; the live UK search pointsnotifyLayerat the vault's pre-existingdata/iso/51.46775,-0.45477-60min-auto.geojson(Heathrow's own isochrone from the airport-fetch pipeline, never re-fetched — same file the mergedlivable-zoneunion was built from) instead. POST/PATCH /property/searches[/…]passnotifyLayerstraight through (untyped JSON body →store.update()'s spread), CLI--notify-layer <slug-or-path>onadd/set(set ... --notify-layer noneclears it — sent as JSONnull, sinceundefinedis dropped byJSON.stringifyand would never reach the server to clear anything).
- Two resolution modes,
notify+notifyCriteria— WHETHER and WHAT to push, besidenotifyLayer's WHERE. Both notification-only: polling,lastResultsand map pins are untouched, andstore.update()never re-seeds on them (only criteria/layer/country do).notify: false= poll + draw, never push — set on DE + IT so only the UK search buzzes.notifyCriteria(server/src/property/notify-filter.ts, pure + unit-tested) is a stricter gate evaluated on each fresh in-geofence listing:maxPrice,minBedrooms,minPlotArea,minFloorArea,houseSubtypes(matched against the listing's ownpropertyTypetext vianormaliseHouseType— a semi never passes a gate that omitssemi-detached, whatever else its text says; unclassifiable text fails),keywords(any-of over title/summary/address),maxAirportDriveMinutes(the one gate needing a lookup —filterByAirportDriveruns the Routes lookup on the survivors of everything else, capped atDRIVE_GATE_MAX_LOOKUPS=10per poll). Strictness rule, opposite topostFilter: a field the listing omits FAILS the gate — a push is a positive claim, unknown isn't evidence. Consequence: Rightmove never exposes plot area, so the search's ownminPlotArea: 800is unenforced on UK (the feed is effectively "any ≤£300k freehold house", 71 % semis) andnotifyCriteria.minPlotAreawould silence the UK search outright. UK gate set tohouseSubtypes: [detached, bungalow, villa, farmhouse, land]+maxPrice: 250000. The summary-vs-per-listing split happens on the GATED count. CLI:con map property set <id> --notify on|off,--notify-max-price/--notify-min-beds/--notify-min-plot/--notify-min-area/--notify-house-subtypes/--notify-keywords/--notify-max-drive(any gate flag replaces the whole gate as one object),--notify-criteria noneclears it.- All three portals fail OPEN on unknown params (a typo'd filter is silently ignored, count unchanged). Always verify a new filter actually moved the count. IS24 alone hard-400s a bad value on a known enum.
- Rightmove needs both endpoints:
/api/property-search/map/searchis clean JSON, 121 rows/page, but ignoressortType;/property-for-sale/find.html(__NEXT_DATA__, 24 rows/page) honourssortType=6= newest first. Socount()uses the JSON one,newest()the HTML one.index > 1000→ 400.keywordsis a sort, not a filter → reported unsupported. AccumulateresultCountonce per ring (if (index === 0)), not per page. - immobiliare
/api-next/*needs no auth or cookies.vrt=lat,lng;…takes an arbitrary polygon. A cosmeticpath=param is mandatory (omit → 500). Sort iscriterio=data&ordine=desc—criterioOrdinamentofails open silently.isResultsLimitReachedself-reports truncation. It exposes no listing/creation date at all — only anisNewboolean, so IT rows havelistedAt: undefinedand can't be locally sorted (the portal-side sort still delivers newest-first, so the seen-id diff is unaffected). - ImmoScout24 has a split personality:
POST /Suche/controller/search/{resultCountWithModel,change.go}are cookie-free from plain fetch, but theGET /Suche/shape/…HTML holding the listings is AWS-WAF gated (401).waf-token.tsdrives a headless Chromium at the homepage (its first response IS the 401 challenge; it self-solves in ~3 s), harvests theaws-waf-tokencookie (4-day TTL), and caches it 0600 for use from plain fetch. Playwright is not a dependency — resolved from other repos on this machine, absent → counts still work, listing reads degrade to an error. - IS24
&pagenumber=1is a WAF 401.sorting=2alone → 200,sorting=2&pagenumber=2→ 200,sorting=2&pagenumber=1→ 401. Page 1 must omit the param entirely. - A WAF token can 401 days before its stated
expires, so TTL is not a validity test — only a real 401 is.newest()catchesWafRejected, callsWafTokenStore.invalidate()(drops cache + file) and re-mints once before giving up. Without this a single early rejection silently killed DE results for the rest of the 4-day TTL. change.gocompiles our query model into IS24's own short-param URL (price=-600000.0,ground=800.0-, base64-swapped-alphabetshape=…) — never reverse-engineer that grammar, just POST the model and use the returnedurl.- IS24 coords come from
mapMarkers, not the listing:address.wgs84Coordinateexists on only ~35% of rows (the rest hide the street address), butresultlist.resultlist.mapMarkers.results— shipped in the same response — covers 100%, giving hidden-address listings an approximate point. Markers group co-located listings, so expandgroupedListings[].realEstateIdor you'll match a third of them. Prefer the exact address coord, fall back to the marker. - IS24 misc:
accept: application/jsonis mandatory on the POSTs (else it replies ISO-8859-1 and umlauts corrupt a UTF-8 parse);price.value: 0means "Preis auf Anfrage", not free — confirmed live 2026-08-15 by fetching a flagged listing's own expose page and finding<span class="is24-preis-value">Auf Anfrage</span>, not a data error;onlyWithGardenis a no-op forHOUSE_BUY(688→688) —lotSizeRangeis the real plot-size filter;numberOfRoomsis Zimmer (all rooms), so it reads ~1 above bedrooms. It's the only portal that filters plot size (lotSizeRange) and broadband (minimumInternetSpeed). excludePriceOnRequestdrops any listing with no price at all — real signal, not noise: one DE poll had 37.5% price-on-request. Genuinely unfilterable server-side on any of the three portals (no query param anywhere expresses it), so it's always post-filtered onListing.price == null.- Tenure is asymmetric. Rightmove
tenureTypes=FREEHOLD,SHARE_OF_FREEHOLD(share-of-freehold is functionally freehold for us;COMMONHOLDreturns 0 rows anywhere yet). immobiliaretipoProprieta=1= full ownership, which is what excludes nuda proprietà (seller keeps lifetime use) and multiproprietà — both look like bargains and aren't. Germany has no filter at all: Erbbaurecht is the analogue but IS24 exposes nothing, sofreeholdOnlyis simply unenforceable in DE (declared unsupported, and not post-filterable either since the listing doesn't carry it). - Auctions and schemes are separate
Criteriafields,excludeAuctionsandexcludeSchemes— an auction isn't inherently undesirable (just a faster ~28-day cash-ready completion, no survey contingency), so it's a distinct opt-out from retirement/shared-ownership stock, not bundled with it. Rightmove'sdontShow=auctionis behind a feature switch and silently does nothing, soexcludeAuctionsis always declared unsupported on every portal and matched on listing text (auctionLike()insync.ts,AUCTION_RE— word-bounded since 2026-09-13: the old substringastamatched "fantastica"/"castagno"/"Bastia"; notribunale, that is a location) — enforced inpostFilterregardless ofunsupported, and inLOCAL_ONLY_CRITERIA, so toggling it never drops an inventory (turning it OFF on a portal that filtered server-side shows auction rows only from the next full pull);excludeSchemesusesSCHEME_TERMSand is a real Rightmove filter (dontShow=retirement,sharedOwnership+includeSSTC=false) but unsupported on DE/IT. immobiliare'snoAsteis the auction flag specifically — a real bug had it wired toexcludeSchemesinstead ofexcludeAuctionsuntil 2026-08-15. An auction row'spriceis the GUIDE, not an asking price — the pin popup and notification text print it asguide £220,000(priceLabel()insync.ts) and the deck card carriesauction: true. A UK guide is the seller's minimum expectation with the reserve up to 10% above it; 14 Tyle Road, Tilehurst (RM 92428200, Heathrow tier, 2026-09-14) guided £220k against a £295k 2018 sale and £325k+ terraces on the same street, and only passed the ≤£300k coarse filter because of it. houseSubtypesnarrowspropertyType: 'house'to a portable subset (detached | semi-detached | terraced | bungalow | villa | farmhouse | land), mapped per portal — RightmovepropertyTypes(villa/farmhouse hard-400, no value exists), IS24houseTypesenum (land has norealEstateTypeat all —SITE_BUY/LAND/PLOT_BUYall 400), immobiliareidTipologia[](no bungalow or land id in the taxonomy). Requesting an unmappable subtype alone (e.g.bungalowon immobiliare) must fall back to the default house set, never to "no type filter" — an earlier version of the immobiliare client sent literally nothing in that case, silently widening the query to every residential type including flats.numberOfRoomsRangemust be shifted byROOM_OFFSET = 1— IS24 counts Zimmer (all habitable rooms incl. the living room), so a 3-bed house is a 4-Zimmer-Haus. Feeding bedroom counts straight in makes every DE search a whole room stricter than the UK/IT one, invisibly.- Because IS24 fails open on unknown fields, an invented field name is worse than an error — it looks like a working filter and silently isn't. Never emit a query-model field that isn't in the api notes: bathrooms has no verified field, so
minBathroomsis post-filtered on the listing'snumberOfBathRooms. - Parse embedded JS object literals by brace-matching, never regex —
pageModel(html, marker)is exported fromrightmove.tsand reused for IS24'sresultListModel:. (Related: regexing minified JS bundles catastrophically backtracks; use Pythonre.finditer+ manual slicing.)