In this episode of Cyber Security Inside, Camille and Tom discuss scaling artificial intelligence with Kavitha Prasad, VP & GM Datacenter, AI and Cloud Execution and Strategy. The conversation covers:
- What your AI strategy should include, especially since artificial intelligence is now so pervasive.
- Why AI should be viewed as a system, and how it is connected to deep learning.
- The ecosystem around your AI and how to meet needs without compromising security or performance.
- Where AI is headed in the next decade and what that means for us.
And more. Don’t miss it!
The views and opinions expressed are those of the guests and author and do not necessarily reflect the official policy or position of Intel Corporation.
Here are some key takeaways:
- Your AI strategy should include several things, especially if you are interested in how big companies like Intel use a system view for it. It should consider how you manage memory, how you move data and the challenge behind that, cost, edge to cloud connections, and what customers actually want as well.
- Artificial intelligence is very pervasive. From the phone in your hand to the cloud, it is prevalent in everyone’s lives now. That is why aligning the direction AI is going with the direction that companies have set for themselves is important.
- Kavitha considers AI as a system, because it is a heterogeneous workload. AI is connected to deep learning, but it is more than that. There is so much application to AI, and there are multiple aspects to making it work successfully.
- There are many constraints to making it work well. It could be latency, thermal, power, or more. How you deploy the solution is just as important as what the solution is. The two bottlenecks to doing that are memory management and data movement, and we have to solve those problems at a system level.
- The customer KPIs are different, though. They are more focused on how many faces or pictures you detected, how many units are functional versus charging, and power constraint costs. These are different from other needs, but they need to be accounted for as well when defining your AI system.
- Another way to look at how to deploy these AI models is the edge-to-cloud continuum and creating meaningful compute across the pipeline. It involves considering bandwidth and connectivity when transferring lots of data.
- AI is just huge. From software to hardware, from IoT to cloud devices, and so much more. There are distributed models and it touches everything. And ecosystem plays a huge role.
- The development to deployment is a key part of that ecosystem. We need to get the performance right out of the box, and it isn’t about what you pack into the system or raw computer or the peak power. It’s all about performance and what you get out of your software tool chain.
- Where is AI going in the near future? Multimodal intelligence and connections. Being able to connect a picture of a cat to the word cat in multiple languages is something that is possible and that will emerge. It is about building a composite out of different sets of information to make meaningful results.
- Artificial general intelligence is also likely going to become popular. This is the version of AI that has the intellectual capability of a human being, who could solve math problems like a human would.
- Explainable AI or Responsible AI is all about the “why” of decisions. When a human makes a mistake, it is understandable. But it is less accepted when a machine makes a mistake. So knowing why a machine made a decision, and if it is a trustworthy decision without biases is something that is going to be more and more important in coming years.
- Cognitive AI is also something that is being talked about. This is also about the “why” behind decision making and how it relates to human behavior. The landscape of AI is going to change a lot, but it is hard to predict exactly how.
Some interesting quotes from today’s episode:
“If you think about AI, it’s a very pervasive workload, right? Wherever there is compute, you can have AI around it.” – Kavitha Prasad
“If you really look at real world application, you could have multiple sensors or multiple data sources. You take all that data, you do pre-processing, or you do whatever data mining, data wrangling, whatever you need to do… then you do a compute on it – which could be deep learning training, or deep learning deployment. And then you have statistical ways of deploying that artificial intelligence.” – Kavitha Prasad
“You could take actions, you could do actuation, you could put a brake in your car, for example, to dropping a box off to somebody using a robot or, you know, completing a sentence in certain cases. So there’s a lot of heterogeneity to this workload.” – Kavitha Prasad
“Any workload, you start off from the software, Python coding all the way to the bottom of the stack, there are only two things that matter: memory management and data movement. And this has been a problem since ages. It’s not a new problem for AI. Since compute started, these were the two bottlenecks, and they continue to be bottlenecks even now.” – Kavitha Prasad
“How do you make sure that you are creating meaningful compute across the entire pipeline? Because that defines your cost, too, in the end. If you think about the petabytes worth of data that is getting generated, I have to transfer all of it to the cloud. It’s a huge cost. And what happens in areas where you don’t have internet connectivity or there is a shortage of bandwidth?” – Kavitha Prasad
“AI is a system whether you slice it down vertically from an application all the way to silicon, or whether you look at deployment all the way from the client to the cloud, to the edge to network into cloud. It has a system level problem. And it needs to work cohesively for us to make sure that AI is deployed effectively in the market.” – Kavitha Prasad
“If you think about multiple input data types – images, video, speech – how do I put that together and create something meaningful out of it? Because, for example, a cat is a cat is a cat. If you have a picture of a cat, or if you say ‘a cat,’ or if you say ‘a cat’ in Chinese or whatever language, it’s a cat. How do I bring that multimodal knowledge to make something meaningful out of it? This is something that is going to emerge.” – Kavitha Prasad
“In general, humans are forgiving when humans make mistakes, but they’re not forgiving when machines make mistakes. So the question becomes, how do I explain what decisions I took? What is the reasoning behind what I took? And put it into the human-centric: am I doing it for social goodness?” – Kavitha Prasad