M - Travel Resort in Cabo

Prompt: Find and book a 4-star all-inclusive resort in Cabo for 5 days in May (I'm flexible)

Personalization Setting: None

Parameter

Description

UX - Information Exchange

No specific questions asked on if there are specific dates in May that I would like to travel with, even though I mentioned that I am flexible with dates.

No specific ask on whether I should be traveling from a specific day to another one.

No specific questions on whether I am by myself or traveling with a family, for those to

UX - Ease of Use

The browser instance opens as an in-line chat interface which often has to be enlarged to be able to see it clearly. Using a canvas style UX pattern here would yield better visibility for the user than an in-line browser instance

Clarification & Higher-Order Thinking

Didn’t ask me if I already had flights booked for a specific date or had holidays on a specific dates which would have been helpful in drilling down to a specific date.

The higher order thinking on looking at the map and evaluating options was interested by it kept switching back and forth between maps and results for a long time without scrolling / refreshing which is why I thought it was sort of stuck.

When asked to put the options together after searching everything, the model hallucinated on the initial flight price for the onward flight as $35 while it was $155 and thus gave me the wrong trip estimate.

Task Decomposition & Modularity

Could have potentially asked if there was a broader task around flight planning or rental car booking along with hotel booking as well, which would serve as a broader user intent with the agent solving for related intents as well.

Application Identification

It primarily focussed on Booking.com since that was selected as the application of choice.

Personalization

One of the key tasks in deciding such hotels is to give users a set of options, especially for the more savvy users. However, for the less savvy ones, they may want to get their travel booked directly without options and so understanding whethe the user likes choice of options and comparison or not is usually very helpful.

Browser Navigation & Integration / Explainability

In a medium complexity use case, involving checking various hotels for a trip to Cabo, Operator constantly

switched back and forth

⁠

between the maps and the search results tab while evidently not showcasing any net new scrolling or zooming within the maps, potentially highlighting that it could have been in an infinite loop, however it was able to successfully get out of it. We are also not sure how it inferred the visual map for the recommendations that it was scraping to understanding proximity to amenities but this seems like an issue that stems from inadequate post-training treatment around trade offs between multiple options, where the model may not know how to juxtapose between these options.

Credential Management, Risk Management & Handover

For booking, it still gave back control to the user for providing transaction information, which is largely done for risk management purposes but being able to store user’s credit card information, and only executing on it after the price has been confirmed will be a better way to manage risk than letting the user enter the details.

There are no rows in this table

⁠

Want to print your doc?
This is not the way.

Try clicking the ⋯ next to your doc name or using a keyboard shortcut (

CtrlP

) instead.