During Toronto Tech Week, Wealthsimple's CTO talked about how they can't use AI to give targeted advice to users like "you should buy this stock" because there are regulations against brokers telling investors to do so. Specifically outside of when the client asks, like when they let Wealthsimple manage their account.
After going to Shopify's event, I wondered how it works with other domains with less regulations.
What are the risks in using AI and potential regulatory stuff that prevents them from doing anything? I would assume that trust with their users is important, so how much trust do they put in AI?
This question I had for Shopify because unlike in financial institutions, they don't have the risk of AI making financial decisions for customers or using language that is illegal (banks cannot legally tell someone "you should buy this stock" unless that person has given the bank permission to manage their account or provide such advice).
Looking it up, Shopify is allowed to make any and all recommendations on their platform, so my question still remains. If we contrast to a bank, they would lose all of the trust that they have built over more than a century if a user makes a bad decision because a bank AI told them to. If a user makes a bad purchase from AI from Shopify, what kind of effect will that have on the user? What will they think? Is that a risk Shopify is willing to take?
I think the interesting part is that I don't know what it would look like in Shopify's ecosystem. They help companies build ecommerce websites, so would they say that the AI falls under the company's purview or theirs? Let's assume that it falls under Shopify's, and the AI can look at a catalogue of every single item listed on Shopify from every single vendor on their platform (which I believe is the AI tool they showcased at Toronto Tech Week). AI makes some recommendations based on what the user wants to buy. User sees options and chooses one. If the user asks AI for what they think, and the item is bad, where does the blame fall? Not legally or morally, but in the user's mind. Will it prevent them from using Shopify's AI assistant again? Honestly, after writing it out, I see minimal downside for Shop with good upside from the experience of having an assistant helping you make purchases.
Even if Shopify doesn't place full trust in their AI and recognizes that it makes mistakes sometimes, I think their domain is very low risk and they wouldn't have a problem with deploying it unless they couldn't eliminate a failure mode, such as it promising to sell an expensive item for $1 and that expense coming from Shop's budget (or the small business who they gets mad and stops using Shop, potentially spreading the news to other businesses). Honestly, that scenario carries far more risk, but given that they'd never allow their checkout functionality to read the price from the chat, I don't see it happening.