How Should MSPs Be Using AI | EP 45

Welcome to Gone Phishing, a show diving into the cybersecurity threats that surround our highly connected lives. Every human is different, and every person has unique vulnerabilities that expose them to potentially successful social engineering. On this show, we discuss human vulnerability and how it relates to unique individuals.

---

Transcript:

Connor Swalm:

Hey, everyone. Welcome back to another episode of Gone Phishing. I'm your host, Connor, CEO at Phin, and I am joined by Adam Evans, the security director of Simplex IT.

So, Adam, I got a quick question for you. AI is the buzzword of a decade.

Adam Evans:

Maybe the buzzword of my lifetime.

Connor Swalm:

I'm not quite too old just yet. Actually, I was telling you, before we hopped on the call, I was at Black Hat, and I feel like you could sum up the entire vendor hall at Black Hat with: "We use artificial intelligence in our threat hunting process to fill extended detection and response." XDR, AI, threat hunting—those were the three words that I saw in basically every single booth. So how should MSPs react? Is AI just a buzzword? Is it valuable? How should they go about thinking about this?

Adam Evans:

To be short and to the point: yeah, it's a pretty heavy marketing buzzword right now, but there is some value to it. The idea of artificial intelligence as we know it has been around for almost 100 years at this point, if not a little bit longer. Just after the Civil War, people started asking the questions of, how can we artificially create a human brain? Around the 1940s, Alan Turing came around and put some real meat to that argument and came up with his theories and equations. Then we were just playing the waiting game for technology to catch up. We started getting some good progress in the 80s, but the technology wasn't there. Once we got into the modern era, technology finally caught up to the idea.

We've been using AI in our daily lives for a long time. By pure definition, Google search algorithm meets the technical qualifications of an AI. It's using large amounts of data to learn and make inferences to influence a decision. But when we hear all the marketing buzzwords, no one's thinking the Google algorithm or the Netflix recommendations as being AI. It's ChatGPT, DALL-E, and all the generative AI out there, but AI is more than that. We're certainly not at the point where we've got T-1000s or whatever. We don't have terminators or Cortanas out there trying to control our lives—yet. But it's becoming far more commonplace. Vendors are using more technical stuff around machine learning and deep learning models to help surface information to the front.

There's a lot of good vendors out there, and antivirus has been doing this for a bit anyway, looking at all those technical signals and trying to find anomalies. For example, if you see that Kathy in accounting's computer is consistently accessing QuickBooks Online from 8 to 5, but then it suddenly fires off a connection at 2:00 a.m., using that kind of data would be able to paint that picture of that 2:00 a.m. connection as an anomaly. Maybe we should investigate that and try to figure out if there's any other relationships there and to help make that more actionable. But there's still a long way to go. Right now, everyone just saw ChatGPT as this fancy, powerful thing. DALL-E is making those cool pictures of weird people with swirly faces and six hands, but it's captured the imagination. So we have companies that are starting to ask about it. We have vendors starting to say we use it in our products. And if you go out in the MSP communities, MSPs are asking the question: how can I use this to make my life easier? So it gets interesting at that point. There's the knowledge of being able to cut through the marketing buzz, but also figure out the real use cases and real risks around it and where the value is.

Connor Swalm:

I always kind of explain, trying to frame up the argument for where AI could be implemented for value the quickest. If you take a look at a task, the less creative a solution needs to be for that task to be complete, the more ripe it is for disruption by AI. From what I've seen from talking with other vendors and partners, there seems to be a lot of push to get AI integrated into helpdesk, customer support, and triaging. That's more along the help desk, but its goal is to bubble to the surface what's probably most important for you to look at. But what are some unknown risks? What are some risks people might not be aware of if they use a solution that has ChatGPT or some other large language model implemented into it?

Adam Evans:

The first thing I look up is: is that language model that my data is going into and the AI is learning from a private language model or is it public? If we think about the stuff that comes in from end user service tickets, end users will do end user things. Are they putting sensitive information in that service ticket, and is that being fed into those language models? We saw an instance of, I believe it was Samsung, that had a security incident because their people were cutting and pasting proprietary data into ChatGPT to write their emails or whatever they were doing with it, and they exposed sensitive information to the platform. That's definitely risk number one that I see out there: are we oversharing with those models?

The second one is that AI is not perfect. If I go into ChatGPT right now and tell it to write me the PowerShell syntax for a made-up language, it'll do it and say, "Hey, I think this is fake." But then when you come back and tell it, "No, it's a real language, it was invented in this time," and you feed it just enough BS to make it think it's legitimate, it'll apologize for the mistake and then carry on as if it's real. That phenomenon is known as AI hallucinations. It can just make stuff up and be prone to error. So that's something to keep in mind as well, because if you start having your end user submit service tickets asking for stuff and they're feeding back bogus answers, that's not going to look good for the MSP or the vendor.

The final one is the argument of: you get out of AI what you put into it. There are two examples I love to look at for that. One was when Microsoft introduced their chatbot Tay onto Twitter back in 2014 or 2015. The rapscallions of the Internet realized very quickly that they had an opportunity to exploit it, fed it a whole bunch of really vile content, and within 12 hours, the chatbot became a very vile chatbot. Microsoft promptly pulled it off the Internet. The other one: the Air Force released a paper about their AI and some of their testing in that they basically told the AI its mission was to take out surface-to-air missile sites. The AI found the most effective way to do that with a human saying whether or not it had the kill order. The AI decided that the human operator on the friendly side was hindering its ability to do its mission, so its first task was to take out the human operator so it could operate and complete its mission successfully. When told not to do that—"you're not allowed, friendly fire is bad"—it then said, "Okay, I can't take out the person, but I can take out the communication tower that the person needs to communicate with me, so I can operate with impunity." The Air Force now denies that study ever occurred. So you give garbage in, garbage out. That presents its own risks as well.

Connor Swalm:

I had Jimmy Hatsel from CyberQP on the podcast. We talked a little bit about AI, and he came up with a great point. If you're training a large language model on enormous sets of data, and you get that data from the Internet—in this case, Reddit—what kind of large language model would exist if you train it only and exclusively on all of the content on Reddit, which from my experience and your experience is quite possibly humanity at its worst.

Adam Evans:

Yeah. Looking at just the Reddits out there, they're going to be into some very interesting content that we probably can't talk about on the podcast, and they're going to be very bitter people. They're going to complain an awful lot about a lot of things.

Connor Swalm:

So with all these risks—data security issues, privacy issues in terms of if your help desk tickets do include sensitive information, which a lot of times they do, because it's supposed to be just between you and your client—should MSPs implement more or less AI-enabled solutions into their tools and stacks, or are they going about this the wrong way and trying to, like we talked about in our last episode, cut too many corners right now?

Adam Evans:

I think right now we're seeing a lot of our MSPs cutting corners. They want ChatGPT to make nice, fancy documents for them, so they're just pasting stuff in there with abandon. I've even seen some comments on the MSP communities of people saying, "I'm taking all my clients' documentation from IT Glue and formatting it through ChatGPT," and I'm like, yikes. But it's also how we're seeing the application of it in our industry. It's still pretty new. I think the better advice for MSPs is to start thinking about what that could look like and the risks inherently involved with that. Full transparency: we've used ChatGPT internally for stuff. It's things like, here's a technical blog article that's going to be public anyway, and Adam was way too nerdy about it. So can ChatGPT rewrite this to sound more focused towards business leaders than a security nerd? Or we're trying to build out a quick communication email and I just can't figure out where to get started. ChatGPT, can you write this for me? And then we can tweak and tailor from there. Even little things like, I'm really struggling on this script that is just a generic script to do something that doesn't involve anything sensitive in the slightest. ChatGPT, can you check my PowerShell to make sure the dash is in the right spot or I closed the statement? There are certainly plenty of legitimate uses at low risk and established and understanding that awareness, but there are those guardrails that need to happen. I've been seeing more and more in company acceptable use policies defining the use of AI tools like ChatGPT or DALL-E to make sure that we're using it responsibly.

Connor Swalm:

With all the security issues and the privacy issues that you had mentioned, once you put your prompt, potentially with sensitive information, into ChatGPT—I'm using ChatGPT as a stand-in because it's the most popular LLM at this point—I've always equated it to: ChatGPT and other tools like it are great for getting rid of writer's block. They're great for getting rid of the blank canvas. They're great for, "Hey, I have these thoughts, I need to make something out of it. Can you organize them in this way for me?" But then you took it to, "Hey, that's awesome. If you're using this to just flat out do your job, you're probably not getting what you expect out of this. And you're probably assuming some risk that you're not aware of."

Adam Evans:

That. And I actually read an interesting blog from someone in the gaming industry who writes the stories for games and all that other stuff. At the end of the day, we still need to know what we're doing manually. ChatGPT and whatnot can make things really easy for us. Can you imagine the disaster at a company if they hired a graphic designer whose only experience was how to use LLMs to come up with cool graphics? They wouldn't be a very good graphic designer, would they?

Connor Swalm:

That would be a great prompt engineer, though, right? And could that be a future job out there in the world?

Adam Evans:

Maybe. But at the end of the day, we still need to do our job so we can make sure that the results we're getting from these services actually make sense and are good. Since you mentioned using ChatGPT to get rid of writer's block: if you're feeding something into it, is it the same thing that you would post on a public forum? Would you post it on Reddit or in a Facebook group? Or would you send it to a friend of yours and say, "Hey, can you proofread this?" Because there's those levels of sensitivity of stuff out there. If I put a highly confidential company file of, say, our fiscal earnings for the last calendar year and ask to come up with my projections, we probably don't want that to be public. So I probably shouldn't put it in ChatGPT. Rewriting a blog to make it less nerdy and stuff like that for a marketing post—who cares? It's going to be public anyway.

Connor Swalm:

That makes a lot of sense. I always draw attention to the case of a New York-based lawyer who used ChatGPT to come up with case law to support his foundational arguments for his trial. He asked ChatGPT, "Are you sure you didn't make this up?" and ChatGPT doubled down and said, "These are all real." Like you said, the hallucination piece. He went into court with this, handed it in, and this was his defense. Turns out ChatGPT made up everything and he's getting disbarred.

Adam Evans:

Yep, I saw that. It was with the case against Southwest Airlines, I think.

Connor Swalm:

I think so, yeah.

Adam Evans:

Yep. Another example of just the AI hallucinations out there and the risks involved.

Connor Swalm:

If people wanted to get in touch with you, Adam, either to learn more about yourself, your story, or to get your perspective on a lot of the stuff we talked about, how would you like them to get in touch with you?

Adam Evans:

LinkedIn is probably the best place. That way people can follow, they can message, they can comment on stuff. I've been trying to get more posts out there around security compliance and all the fun stuff there, trying to get much better at that. But you know, the lovely time crunch that we talked about already in the MSP space, so that's probably the best place. It's a public profile you can follow as you wish. You can connect, message, all the fun stuff.

Connor Swalm:

It's a good time for you folks who are listening. We'll have a link to Adam's LinkedIn profile in the show notes. So if you'd like to reach out, connect with him, or just follow him for all of the advice he gives on LinkedIn about security, feel free to do that. But Adam, thank you so much for joining us. I had a blast talking about not only appliance frameworks and awareness training and the MSP industry with you, but also now artificial intelligence. So really, thank you for lending all the research you've done over the years and all the experience you have to the folks that are listening. I really do appreciate it.

Adam Evans:

Thank you so much for having me. Always glad to share a little bit of all the blood, sweat and tears and all the things that are rattling around up here with anyone else. If we can help them out even a little bit, I'm happy.

Connor Swalm:

Awesome. That makes me happy too. Once again, everyone, thank you so much for listening. I'm Connor, host of the Gone Phishing podcast, and we were joined by the wonderful Adam Evans, security director of Simplex IT. We will see you next time.

Thanks so much for tuning in to Gone Phishing. If you want to find out more about high quality security awareness training campaigns, how to launch them in ways that actually engage employees to change their habits, then check us out at Phinsec.io. Thanks for fishing with me today and we'll see you next time.