Hotel Technology Pilot Program: 90 Days, Five Numbers

A 90 day technology pilot at one property works when three things are written down before the tool is switched on: five baseline numbers the property already produces, the threshold the owner will accept as success for each, and the name of the person who owns the pilot. Everything after that is a weekly 20 minute review of the same five numbers and a one page readout at day 90. Pilots fail, as RAND found in 2024, mostly because leaders misstate the problem or optimize the wrong metric, not because the technology was wrong ^[1]. Whether the tool is one of the platforms in our guide to the best hotel AI concierge platforms or an upsell engine or a new messaging inbox, the structure below is the same.

I will say this once and then stop: nothing in this post promises an outcome. Every worked number is labeled illustrative, the readout mock shows a layout with placeholders, and the only thing I am confident of is that a pilot with a baseline beats a pilot without one.

Why do most technology pilots end in a shrug?

Because nobody wrote down where the property started, so nobody can say where it ended up. The vendor sends a dashboard. The owner asks "did it work?" The general manager says "the team likes it," which is not an answer an owner can take to a budget meeting.

RAND's 2024 report, based on interviews with 65 data scientists and engineers, lists five root causes of AI project failure: leadership misstating the problem to be solved or optimizing the wrong metric, insufficient data, prioritizing the latest technology over the business problem, underinvestment in infrastructure, and applying AI to problems beyond the current state of the art ^[1]. The report also repeats an outside estimate that more than 80% of AI projects fail, which it attributes to a 2022 Fortune article; that is an estimate RAND cites, not RAND's finding, and I would quote it that way or not at all.

The broader picture is sobering too. McKinsey's 2026 state of AI survey of 1,719 participants across 97 nations found 37% reporting enterprise-level EBIT impact from AI and about 6% qualifying as high performers who attribute at least 5% of EBIT to it, even as nearly nine in ten use AI in at least one function ^[2]. Lots of use, little measured impact. At a hotel, the causes translate neatly: no baseline, success defined by the vendor's metrics, scope added mid-pilot (which is how one tool becomes hotel tech fragmentation in a quarter), nobody owning it, and staff adoption never measured. Each has a written fix below.

What should you measure before the pilot starts?

Five numbers, for two weeks, by hand if the property has no report. Each gets a source, the two week figure, and the threshold the owner has agreed in advance will count as success. The threshold is the owner's number, not the vendor's and not yours, which is what makes the readout impossible to argue with later.

Baseline worksheet for a 90 day hotel technology pilot, 2026. The external references are context for the owner, not targets, and the Shiji figure is a vendor benchmark.

Two weeks is the minimum that smooths out one bad Saturday. The hand count is not beneath anyone: a tally sheet at the desk with hour bands across the top is how a property finds out where its contacts actually cluster, which is rarely the hour anyone guessed. These five numbers are the same ones that survive an owner meeting after the pilot, and the five ROI numbers that survive an owner meeting explains how to turn them into an ROI figure once the readout exists.

How many properties and how long?

One property, 90 days, with a start date chosen for the season. A second property does not double the signal; it doubles the noise, because two properties have two guest mixes, two teams and two sets of excuses. Prove it once, then expand with a second baseline.

Ninety days is long enough for a weekly series to show a trend and short enough that the owner is still paying attention. Two weeks of baseline come before day zero. A midpoint check at day 45 reviews the kill rule and the adoption check. The readout date at day 90 goes into the owner's calendar before launch, so the pilot cannot quietly become a second quarter. Choose the start so the 90 days do not straddle your biggest seasonal shift; a pilot that begins in the last two weeks of high season and ends in the quiet months will show a drop in contacts that has nothing to do with the tool.

What does the weekly rhythm look like?

Twenty minutes, the same five numbers, every week, with the pilot owner in the chair. The agenda does not change, which is the point: a pilot review that becomes a general operations meeting has lost its purpose.

Weekly 20 minute pilot review agenda. Practice checklist, 2026.

The adoption check is the item most pilots skip and most need. A tool nobody uses produces beautiful dashboards and no change. Ask who is still handling things the old way and why. The staff feedback line matters for the same reason, and because staff time is the scarcest input you have: in the AHLA and Hireology survey of 282 US hoteliers over December 2024 and January 2025, 65% reported staffing shortages and 71% had openings they could not fill ^[6]. A pilot that costs a stretched team an extra hour a day without saying so is not being measured honestly.

How do you count costs and staff time honestly?

Everything, including the hours. License, setup, training, and the pilot owner's weekly time, with the hourly assumption written next to it. Vendors count the license. Owners count all of it, and they are right to.

Illustrative cost and staff time table for a hypothetical pilot, 2026. Every figure is an invented assumption to show the structure; use your own loaded rates and the vendor's actual price.

Budget context helps the owner place the number. In Canary Technologies' early 2026 survey of more than 400 hotel technology decision makers, 85% expected to allocate at least 5% of IT budget to AI tools and 82% expected AI usage to increase within the year ^[3]. That is a vendor survey, so treat it as a sign of where the industry says it is heading rather than a benchmark. What a specific category of tool tends to cost, and the hidden lines to ask about, is in what an AI concierge costs in 2026.

If you want a sense of how the costs above relate to the measures in the worksheet, the ROI calculator will take your assumptions; just keep the pilot's own readout free of anything the property did not measure itself.

How do you present the readout to the owner?

One page, and the owner has seen its skeleton before day zero. The same five numbers, baseline and day 90 side by side, the agreed threshold, and a met or not met column. Under that: what changed and what did not, in plain sentences. Costs incurred, from the table above. What it took from staff, honestly. Three risks. And a recommendation with a next step: expand, extend or stop.

The mock above is a layout for a hypothetical 150 room resort with placeholder bars where the values go. I have deliberately not filled it in, because a filled example would imply a result, and no result exists until your property produces one. What the page should feel like is a financial statement, not a sales deck: no vendor screenshots, no adjectives, no number the baseline cannot support. For the question of what the owner and the general manager each need from a tool like this, what general managers need from an AI concierge frames the conversation from the GM's side.

Years of helping people make hard decisions taught me that the person deciding wants two things: to trust the numbers and to not be surprised. The readout handles the first. Sending the owner the weekly five numbers from week one handles the second.

When should you stop a pilot early?

On three triggers, and stopping is a result, not a failure. First, guest harm: the tool caused or missed a safety, medical or trust problem. Stop the same day, write it up, and decide whether a fix is credible. Second, staff refusal after real support: training was given, the owner of the pilot sat with the team, and they are still routing around it in week six. That is information about fit, not about effort. Third, the vendor fails a fix it committed to in writing by the date it committed to.

The hardest part is admitting it before day 90. A pilot that limps to the readout to avoid an awkward conversation has spent the owner's quarter protecting someone's feelings. Write the early stop up with the same five numbers and the same honesty as a success, and the owner will trust the next pilot more, not less. The vendor questions that make the third trigger enforceable, including what a committed fix should look like in a contract, are in questions a GM should ask any hotel software vendor.

What happens after day 90?

Expand, extend or stop, and each has a next step. Expand means a second property, which means a second baseline, because the first property's numbers tell you nothing about the second's starting point. It also means contract terms negotiated with the readout in hand, which is the strongest position a buyer ever has.

Extend means the signal was promising but not conclusive: a seasonal shift muddied two of the five numbers, or adoption only settled in month three. Extend for a defined period with the same five numbers and a new readout date, not indefinitely. Stop means the thresholds were not met or a trigger fired, and the write-up goes in the file so the next pilot starts smarter.

Whatever the decision, re-baseline before any expansion. A tool that worked at the beach property in the quiet season has not yet worked at the city property in conference season, and the owner knows it.

Where ResortConcierge helps

The general practice is five numbers counted by hand before launch and reported every week after it. Once the pilot starts, ResortConcierge AI's analytics report contacts, response times and escalations by week, which supplies three of the five baseline numbers without manual tallying; the pre-pilot baseline still needs the hand count, because the tool was not there yet.

The escalation queue shows how many conversations needed a person and how fast they were picked up, which is the adoption check and the safety check in one view.

Review reply suggestions and sentiment flags let the pilot include review response rate and time as a measured line without adding a second tool. Two honest limits: occupied room nights for the per-room measures still come from your property management system (PMS) report, because PMS connectors are planned, not shipped, and a supervised mode where a person approves every AI reply before it sends is also planned, so a week one review period would use the escalation queue rather than a pre-send approval screen.

Print the worksheet, pick the five numbers and start the tally sheet this week, before anyone signs anything. The two weeks of baseline are the part of the pilot nobody can do for you, and they are the only reason the day 90 conversation will have an answer.