The evidence gate: why a third of my hotels don't get a quiet score
I launched a register of quiet hotels this month. Every hotel on it gets a score for how quiet guests actually found it, built from what guests wrote in reviews, not from a star rating or a marketing description.
The rule that decides who gets a score at all is simple: a hotel needs at least 3 genuine mentions of quiet or noise across its reviews. Below that, it gets nothing. No score, no placeholder, no "not enough data yet" number that looks like a real one. It just doesn't appear.
What that rule actually excluded
I scanned 3,276 candidate hotels, meaning every hotel with at least one sentence that touched on quiet or noise. Of those, 2,436 didn't clear the 3-mention bar. That's about three in four candidates gone before scoring even started.
857 passed the gate. After I cleaned a batch of rows where the location data was broken (a scrape had leaked postal codes and address fragments into fields that should have held a city name), 813 hotels made the final register.
That's the honest shape of the dataset: 3,276 hotels had someone mention quiet at all, and less than a quarter had enough of a paper trail to trust.
The hand-check
Automated extraction can look right in aggregate and still be wrong sentence by sentence, so I read a sample myself. 100 sentences, picked by a fixed method so I couldn't cherry-pick the good ones. I read each one and judged whether it was actually about room or hotel quiet, and whether the polarity (quiet vs noisy) was right.
97 out of 100 were correct. One was a clean miss: a sentence about staying quiet during a class, attached to the wrong hotel's review data. Two were borderline calls where the words were ambiguous even to me on a slow read.
That check also caught a real bug. One sentence read "total silence from management," meaning no response to a complaint, not a quiet room. The extraction had scored it as a positive quiet signal because it was looking a fixed number of characters backward for a negation word and crossed into the wrong clause. I fixed the boundary logic. The one hotel that bug affected dropped out of the register entirely once the fix ran, because without that one sentence it no longer cleared the 3-mention gate. That's the correct outcome, not a loss.
Showing the noise, not hiding it
Every hotel on the register shows both counts: how many quiet mentions, how many noise mentions. A hotel with 9 quiet mentions and 2 noise mentions still shows the 2. I could have only surfaced the positive count and let the score speak for itself. I didn't, because a score that hides its own contradicting evidence isn't one I'd trust from someone else.
The point
Anyone can build a page for every hotel. The harder decision is refusing to. 2,436 hotels had someone mention quiet in a review and still didn't get a score, because "someone mentioned it once" isn't evidence, it's a data point. A scoring system earns trust less by what it publishes and more by what it's willing to leave blank.
What I'd tell you to steal from this: set a minimum evidence bar before you build anything, make it a hard gate with no override, and check your own extraction by hand on a sample big enough to catch the bugs that only show up in real sentences.