Something that is perhaps “underreported” is that a lot of AI solutions aren’t easier to do than just doing the thing.
My budget app, for example: It’s not easier for me to ask the AI bot “show me all my transactions from McDonald’s” than it is to click on the search bar/payee filter and type “McDonald’s.”
The UX experience for basically all LLMs that I've interacted with is having the AI sneak in where I don't want it and end up taking 10 extra seconds to do what used to be a 0.1ms operation.
The worst place for AI is in search bars, yet that seems to be the first place every product manager wants to put it. I hate waiting forever to get results because AI is poorly searching in the foreground.
I sat through an egregious demo of an AI add on for documentation. The flow was load doc (e.g. pdf) -> data filled into a specialized form view. To really wow me, they showed how I could type in the AI chat interface "change the title to 'abc'" and then it would make the change in the form view.
Why can I not just type the new name into the title field itself? Well, this is supposedly easier and in time the AI will learn and get the title right 100% of the time, so there won't be a need for me to edit it.
This is a major player in the industry, not some random kid.
This is where I see a lot of AI stuff right now honestly.
It's just "I want the computer to figure this out for me even if it takes longer than the old tools, because I cannot be bothered to learn how to use the old tools"
its a classical effect of having a hammer and recognising everything as a nail to whack. its pretty tricky to find a problem well suited to a tool like LLMs.
i programming stuff using LLM of which the resultant programs are deterministic and dont use LLMs or tokens is a better or easier approach to having gains. because the resulting program can be easier to use and doesnt cost a lot to run. Automations are a good target, or integratig libraries into other programs. tedious things u know well how to do but take time to do manually. then once done fix a problem or workflow that u need fixed by the solution yielded.
ive not seen 1 app that does calls to an LLM that i find useful, apart from harnesses that produce code (which is only recently becoming more appealing and might depend on how good and fast u can write code, at which im not the best nor the fastest.)
My wife has to submit a bunch of translated documents for procurement processes. Noone really reads those but they are a hard requirement. These are usually in DOCX or sometimes PDF with tables, product images etc. So I built her a little applet which (after clicking two buttons) sends the XML or PDF to an LLM in chunks and it spits back the same structure but with translated text. Of course there are edge cases where it doesn’t work perfectly but I’d say it works 95% of the time in a good enough quality to submit them. Saves her roughly 0.5-1 hour per job and much cheaper than other alternatives.
Recently laid off and not a big AI guy… but to illustrate your points:
Tailoring + proofing resumes to _manually_ apply to jobs is a great application of LLMs in my experience. But I wouldn’t want an AI to apply for jobs for me.
Or they just flat out don't work. I was on an insurance website recently, and getting big annoying popups to use their AI agent for help with the site or my plan, even encouraging me to ask questions about my plan. I did have one, very simple, "what's my plan's copay?"
"Sorry, I do not have that information."
So then, "Ok, where do I find it?"
"Sorry, I do not have that information."
What the hell? What is this for? Of course, someone will point out "it's because of regulations" or some technical limitation, sure, I can understand that, but then I must ask the question then why did you think it was worth building?
I was dealing with a LLM help bot on a website where I had to upload some documents and asked the bot how to get through a step where I simply don't have the right things and it kept telling me I could sidestep that (but I could not) or upload the thing (which I told it I could not to begin with!).
It's not like the previous tree based bots were good but at least they had a path to speaking with a person.
I've had the bot of a "modern" holiday home tell me some rubbish about luggage storage. Fortunately, I knew that there would be a human fallback during working hours so it worked out OK. But who thought that it was a good idea to have a shitty bot to ask about important things?
I called my dentist because I was sick and wanted to move my appointment. You can't talk to a human anymore, it's all ai, after a bit of back and forth it assured me it had canceled my appointment and booked a new one 2 days later. I was actually surprised it went so well
The next day I get an email at 7pm telling my appointment was successfully rescheduled for 1pm (the same day), none of these match either my og appointment nor the rescheduled one
When I went to the dentist for the appointment the ai assistant supposedly rebooked, they told me there was a mistake and I was assigned a kids dentist.
In the end I had to rebook a third appointment with the receptionist
I think to some extent it's possible to reduce cognitive load even on these simpler actions just because you have to see less unrelated data on what you asked about. I say this as someone who doesn't really use LLM integrations.
And these type of requests consume very little tokens in comparison to almost any programming work.
But asking yourself "can I trust this information?" is the baseline amount of cognitive load added to the interaction.
Depending on the level of trust or importance of the decision this may already be a fairly high bar to clear before being able to claim any overall reductions in the interactions itself.
Okay, now do "all my transactions this year for retail stores"? Or "All luxury / deductible purchases"?
Of course when you arbitrarily limit the problem domain to some specific example, you can make any tool seem pointless and stupid. It's when you expand your goals to things that aren't trivial that you get benefits.
Do we need not Mathematica or NumPy because calculators can add just fine?
I would not trust the AI to order food off of these websites for me because the labeled data itself is pretty much shit. Lots of dishes are unlabeled, have really bad pictures or no pictures at all, and the worse is the ones that have AI generated descriptions which are often wrong or not helpful. I'm totally fine with letting it just use my saved credit cards and delivery address (I assume the AI doesn't see them) but I can't trust it to order the food I actually want.
E: As an example (n = 1 etc): I am vegetarian. I have never ordered meat dishes, ever. Yet all of these food delivery apps suggest meat dishes for me.
doordash is regularly audited for PCI, they almost certainly store raw CC#s and follow modern tokenization standards. The best the agent would see is the name you stored for the card if you have multiple cards it can pick from in the account.
You can pull the PCI compliance documentation from their trust center if you are a partner
https://trust.doordash.com/
For addresses, it's a almost certain the agent would see those since those are visible in the UI when ordering and required for selecting a destination for the order
It's plausible, but also, AI systems have a long history of being plugged in on the wrong side of security boundaries.
Remember when you were able to take over any Instagram account by asking the helpful support AI to send a password reset to an email address of your choice?
> AI, I asked you to check the docs for the latest version of the API, yet you didn't. Why?
> You are completely right! That was my miss, I have no excuse. I've now looked at the docs and implemented it.
Now imagine how triggered you'd be when the AI decides to get you a vegan burger because the chicken wrap is on the second page and it know you're a health conscious person.
It's better they don't because considering how Claude reacts when you give it a meaningless SSH login password for a local box, the moment it learns your CC# and CVV2 it will probably call your bank and cancel the card instead of ordering any food.
Worse than that. Some product manager thought that if they shipped a bunch of buttons that said "Tide" users would keep pressing them and Tide sales would skyrocket. Bet he got promoted for the idea too.
Agreed. I can't imagine trusting it enough to make a purchase automatically, and that's before getting to the reality of how awkward it is. Which brand? Which quantity? Which style (extra strong, extra soft, etc)? Did the price go up since last time? All stuff that you can visually spot in a half second, but would be a huge PITA verbally.
That AI is clearly a product imagined by wealthy tech executives who have personal assistants who shop for them, who trust those personal assistants to buy what they expect for them. The use case doesn't resonate for us normal people, because we do our own shopping, are price sensitive, and typically don't just buy things without doing basic research about what they cost.
Is that true? Amazon reports massive growth in AI and voice commerce. While Echo users appear uncomfortable making big purchases, the segment who routinely use it for stuff like paper towels has been taking off.
I don't know many Echo owners, but the few that I know (n=2) used it to reorder stuff like paper towels.
> There are other signs that voice shopping has never taken off. Fewer than 25% of Amazon Echo owners have ever directly used their devices to make a purchase and fewer than 5% are repeat Alexa shoppers, according to a report last year from Consumer Intelligence Research Partners, which conducts consumer surveys (an Amazon spokesperson said the firm's data was inaccurate and the majority of customers who shop with Alexa are repeat shoppers). And in August 2018, The Information reported that Amazon’s own internal data showed that only 2% of Alexa users had made a purchase so far that year with their voices.
from Amazon Faces Moment of Truth on Alexa as ChatGPT Steals Its Thunder (2023) https://archive.is/2jzve
They are currently showing buy links on Amazon prime in ads they show during their shows after being ad free for years. I can't see anyone being interrupted during their favourite show and then suddenly be interested in buying washing up liquid. Amazon also did those buttons that you clicked to reorder stuff as well.
> DoorDash only rolled out its Model Context Protocol (MCP) connector less than a month ago. ... Without that kind of linkup—which is still only available to a small number of “early beta partners” and mainly used to streamline corporate lunch orders
So it's available to almost no one anyway. Misleading title.
The article does say that AI agents could go through the web interface, but it's not clear if Doordash is counting those orders as AI orders or not. I agree, there's no real information here.
The problem with AI assistants in general is that humans barely ever know what they want and the promise is that AI can intuit/predict what we want. It can't. People like to shop. It's easier to shop with our eyes than our ears. Personal assistants, as far as I can tell, have one use case and that is doing mundane chores like e-mail summaries. But to do that, you have to give them access to everything, so it becomes a transaction of privacy. And I think a lot of people are paranoid. Plus each one is a little token bonfire that is probably costing them money.
And I feel like the labs must all know this, so it makes me wonder why they're actually trying to push these assistants onto us. The answer I'm left with is training data — data theft. They'd all violate their privacy terms if it earned them an extra nickel.
When I have demoed the use of agents, I usually do it with an agent combining multiple applications' results. Something like: "We're having a family gathering at grandma's. Which of us can stop in at the drugstore to pick up her prescriptions on the way without adding a lot to our travel time?"
Agents are great at moderately complex combinations of data and apps. If I had to do this every day or every week even, it might be worth creating a script suited for a task like that using some automation system. Agents are great at providing a disposable saw when it doesn't make sense to stop and sharpen the saw.
I'm not a doorDash user, but they've been doing it long enough to have an app that's probably highly polished. There's not going to be a lot of upside to speaking or typing an English language command to get what you want. Trying to do that is a waste of a well designed app and the agent's token budget.
I would be extremely cross if I heard that someone gave my location information to some company that runs this stuff. That level of privacy violation should lead to disinheritance.
For me, the best use of AI is making bespoke software connecting systems and devices that have no business being connected but I find value in so I can have a unique interface for all my tech and hobbies. Not sure how that gets translated to the everyday man but it sure isn't "bring me food".
Agents are great for that middle ground between just doing it manually and creating an automation to do it. Which strikes me as yet another reason AI is so well suited to software development tasks, but hard to find useful applications outside of software development.
This has always been an ai use case that made no sense to me. The UI for ordering food is good, it's visual, it shows you the food you're getting and how much it costs. There's no reason for a chat interface to be an upgrade here. Chat interfaces only have an advantage when you need the nuance of language. It's good at the things where trying to build a UI for it would collapse into an incoherent jumble of visual language.
Some of these use case feel so imaginary that lot of people just don't get why would some other people do them. Like with food delivery I want to select specifically what I want and evaluate if the price is right. Same goes for ordering most stuff or deciding on vacation.
Maybe I could get train ticket from AI... Or order taxi. At least until it went wrong for the first time...
I would use it if they let me, but you have to be approved for access to their cli and mcp is only for companies. I've had my agent place orders but clicking around with computer use takes longer than ordering by hand.
Bots and tech are testing humanity's levels of desire for laziness and how much they are ready to pay for buying that laziness. Turns out that the desire is limitless. As someone said, the ultimate luxury is in having maximum laziness - not having to move, talk, think or use senses - the state of being a rock.
I mean even if I were an AI maximalist and very lazy, I think clicking “reorder” in the DoorDash app is infinitely easier than trying to tell Muse or whatever to order me me last DoorDash meal.
It's not a super repetitive problem (e.g.respond to 100s of messages) and human judgment is highly valued here (what am I hungry for?) and visuals are sometimes a factor.
Agents are good for complex purchases with lots of hidden fees like hotels. Less useful for burgers, though I'd definitely try it for a large catering order.
This is kind of a moot article because up until a month a go when DoorDash released their MCP, their bot detection was getting in the way of agentic ordering.
While I'm sure there is a place for AI on the food prep side of things, I think it misses the point on the ordering side. While a lot of people want something different, they don't want it to be that different and it's just easier and faster to choose it themselves.
DoorDash seems like a resource anti-optimization that would be made irrelevant by an AI agent managing one's meals. An unofficial HEB MCP [0]—that's cooking with gas!
Because the DoorDash app is so simple, delightful, intuitive?
Look, there's a lot of AI excitement and a lot of it is dumb. But if there's a number I can text "hey get me a pizza" and it re-orders the same pizza order I make every Friday night, I'm going to use it. Would I pay for it? As a standalone service? Obviously not.
But if I could delete half the apps on my phone with their horrid dark-pattern-filled interfaces and constant upsells, I'd love that. I'll happily delegate the interfacing to an agent.
And I'd 100% pay that agent to drop the ads and upsells when DoorDash tells it to pass them along.
Not to mention, searching for food joints is awful. If I want to find authentic Chinese or somewhere with real Bao, I'm going to have to trudge through a hundred pages of Google, Reddit, Yelp, online menus. 90% of the results are garbage and sponsored fast food slop or shadow kitchens.
If I could just ask a single sentence and get back pre-filtered, high-quality answers I would be stoked.
My budget app, for example: It’s not easier for me to ask the AI bot “show me all my transactions from McDonald’s” than it is to click on the search bar/payee filter and type “McDonald’s.”
But if I use the AI bot, I’m burning tokens.
The UX experience for basically all LLMs that I've interacted with is having the AI sneak in where I don't want it and end up taking 10 extra seconds to do what used to be a 0.1ms operation.
The worst place for AI is in search bars, yet that seems to be the first place every product manager wants to put it. I hate waiting forever to get results because AI is poorly searching in the foreground.
... One wonders if that is due to the great implementation of the bot, or due to the less than stellar results of confluence search.
Why can I not just type the new name into the title field itself? Well, this is supposedly easier and in time the AI will learn and get the title right 100% of the time, so there won't be a need for me to edit it.
This is a major player in the industry, not some random kid.
It's just "I want the computer to figure this out for me even if it takes longer than the old tools, because I cannot be bothered to learn how to use the old tools"
I've encountered this a lot
i programming stuff using LLM of which the resultant programs are deterministic and dont use LLMs or tokens is a better or easier approach to having gains. because the resulting program can be easier to use and doesnt cost a lot to run. Automations are a good target, or integratig libraries into other programs. tedious things u know well how to do but take time to do manually. then once done fix a problem or workflow that u need fixed by the solution yielded.
ive not seen 1 app that does calls to an LLM that i find useful, apart from harnesses that produce code (which is only recently becoming more appealing and might depend on how good and fast u can write code, at which im not the best nor the fastest.)
Tailoring + proofing resumes to _manually_ apply to jobs is a great application of LLMs in my experience. But I wouldn’t want an AI to apply for jobs for me.
"Sorry, I do not have that information."
So then, "Ok, where do I find it?"
"Sorry, I do not have that information."
What the hell? What is this for? Of course, someone will point out "it's because of regulations" or some technical limitation, sure, I can understand that, but then I must ask the question then why did you think it was worth building?
It's not like the previous tree based bots were good but at least they had a path to speaking with a person.
I suspect nobody ever tried to use the bot.
The next day I get an email at 7pm telling my appointment was successfully rescheduled for 1pm (the same day), none of these match either my og appointment nor the rescheduled one
When I went to the dentist for the appointment the ai assistant supposedly rebooked, they told me there was a mistake and I was assigned a kids dentist.
In the end I had to rebook a third appointment with the receptionist
except it's worse, if you manage to get some info out of it, it could all be just hallucinated, you were told a lie and no one is responsible.
it's like the only purpose is to erect a barrier between the company and its clients.
And these type of requests consume very little tokens in comparison to almost any programming work.
Depending on the level of trust or importance of the decision this may already be a fairly high bar to clear before being able to claim any overall reductions in the interactions itself.
Of course when you arbitrarily limit the problem domain to some specific example, you can make any tool seem pointless and stupid. It's when you expand your goals to things that aren't trivial that you get benefits.
Do we need not Mathematica or NumPy because calculators can add just fine?
Eh, I could see a case where an AI makes ordering for a group easier.
E: As an example (n = 1 etc): I am vegetarian. I have never ordered meat dishes, ever. Yet all of these food delivery apps suggest meat dishes for me.
Why?
You can pull the PCI compliance documentation from their trust center if you are a partner https://trust.doordash.com/
For addresses, it's a almost certain the agent would see those since those are visible in the UI when ordering and required for selecting a destination for the order
Remember when you were able to take over any Instagram account by asking the helpful support AI to send a password reset to an email address of your choice?
> You are completely right! That was my miss, I have no excuse. I've now looked at the docs and implemented it.
Now imagine how triggered you'd be when the AI decides to get you a vegan burger because the chicken wrap is on the second page and it know you're a health conscious person.
I don't know many Echo owners, but the few that I know (n=2) used it to reorder stuff like paper towels.
from Amazon Faces Moment of Truth on Alexa as ChatGPT Steals Its Thunder (2023) https://archive.is/2jzve
The success of ChatGPT in this space would support the point that AI ordering is a valuable market segment, no?
So it's available to almost no one anyway. Misleading title.
And I feel like the labs must all know this, so it makes me wonder why they're actually trying to push these assistants onto us. The answer I'm left with is training data — data theft. They'd all violate their privacy terms if it earned them an extra nickel.
Agents are great at moderately complex combinations of data and apps. If I had to do this every day or every week even, it might be worth creating a script suited for a task like that using some automation system. Agents are great at providing a disposable saw when it doesn't make sense to stop and sharpen the saw.
I'm not a doorDash user, but they've been doing it long enough to have an app that's probably highly polished. There's not going to be a lot of upside to speaking or typing an English language command to get what you want. Trying to do that is a waste of a well designed app and the agent's token budget.
Maybe I could get train ticket from AI... Or order taxi. At least until it went wrong for the first time...
It's not a super repetitive problem (e.g.respond to 100s of messages) and human judgment is highly valued here (what am I hungry for?) and visuals are sometimes a factor.
0. https://github.com/mgwalkerjr95/texas-grocery-mcp
* No affiliation with the above repo, but I'm considering trying it out.
Look, there's a lot of AI excitement and a lot of it is dumb. But if there's a number I can text "hey get me a pizza" and it re-orders the same pizza order I make every Friday night, I'm going to use it. Would I pay for it? As a standalone service? Obviously not.
But if I could delete half the apps on my phone with their horrid dark-pattern-filled interfaces and constant upsells, I'd love that. I'll happily delegate the interfacing to an agent.
And I'd 100% pay that agent to drop the ads and upsells when DoorDash tells it to pass them along.
If I could just ask a single sentence and get back pre-filtered, high-quality answers I would be stoked.