EvoChip claims its new AltiCore software reduces power requirements and costs of AI inferencing to boost performance and reduce power needs by orders of magnitude, with testing by Dell Titanium channel partner Sidepath highlighting potential channel opportunities around lower-cost, GPU-free AI deployments on laptops, servers and microcontrollers.
EvoChip, a startup developer of technology aimed at significantly increasing AI performance while slashing AI power requirements, is looking to bring its AltiCore ecosystem to market via PC and server vendors and their channel partners in the near future.
EvoChip’s AltiCore technology includes a core mathematical framework that uses logic gates on standard processors instead of neural networks to handle AI inferencing, said Patrick O’Neill, chief technology officer and co-founder for the Dana Point, Calif.-based company and the inventor of the technology.
EvoChip was founded to overcome the pervasive issue of power consumption used for compute in AI by replacing heavy arithmetic with logic-dominant operator chains via software, O’Neill said.
[Related: Nvidia Vera Rubin: 9 Hardware, Cloud Companies Building Out Ecosystem]
“We wanted to reinvent that stack from the ground up,” he said. “We wanted something purpose-built for AI, not applying general-purpose compute towards AI, and in the process come up with an efficient way of reducing requirements for power, compute, memory, processing time, all that stuff.”
EvoChip now has the technology to do just that, O’Neill said.
“It takes the place of neural networks for a very large percentage of AI use cases, scaling all the way from half-million-dollar server racks and data centers down to teeny, tiny MCUs (microcontroller units),” he said.
AI today is built on neural networks, which are machine learning models based on how the human brain works, O’Neill said.
“When talking AI, it’s usually some form of a neural network, whether it’s a dense network or a convolutional neural network, but if you talk to ChatGPT, use image generation, use image classification, or use self-driving cars, those pretty much all are neural network-based models,” he said.
Neural networks rely on large matrices of weights, coefficients and interconnections to turn data inputs into outputs such as predictions or classifications, O’Neill said, O’Neill said.
“Computers are not natively suited for doing that sort of thing,” he said. “We’ve adapted them to do continuous real numbers, to approximate those numbers by using some very brilliant ways of applying digital logic. But in the end, it’s just billions of little switches turning on or off. These are the native elements of digital computing.”
When a human goes golfing, they don’t think of things like wind vectors or how many newtons of force are needed to move their deltoid muscle in one angle and their elbow at another angle, O’Neill said.
“They just walk up and hit the ball, and it goes in the hole,” he said. “Our hypothesis was that neural network mathematics, which are very complicated, very human-esque, are poorly suited to the digital environment. There has to be a more efficient way to accomplish that without going through all the things we are used to doing that may be are not the most efficient.”
EvoChip is using a completely different way of doing the math, he said.
“We went back to the fundamentals,” he said. “We used the basic building blocks of digital logic—logic gates, ANDs and Ors, and NANDs and NORs, and bit shifts and primitives that are very fast and lightweight in the digital silicon transistor substrate. To that end, we eliminated a huge amount of mathematics required to do a lot of decision making and classification.”
As a result, O’Neill said, an AI model that normally requires a top-of-the-line server to run can now fit into a small chip such as an STMicroelectronics MPU with no modifications.
“Our technology is simple enough that it runs on the smallest thing, and it runs on the biggest thing,” he said. “It’s the same model, same speed, same lightweight creature, if you will.”
Conventional mathematics is built on tens of thousands of logic gates, but EvoChip uses only six, he said.
The result is a massive increase in AI performance from a low-cost system which can be designed to run inference without GPUs, O’Neill said.
EvoChip and Sidepath, a Laguna Hills, Calif.-based solution provider and elite Dell Technologies Titanium partner, tested the technology against a neural network model on Dell server and laptop hardware, O’Neill said
“As far as performance goes, inferences per second is the benchmark you use to compare different algorithms across different things,” he said. “We got higher inferences per second on the Dell notebook than the neural networks were getting on the $60,000 or whatever it cost rack.”
Using a University of California, Irvine, data set used to predict the chance someone might default on their credit, EvoChip compared its performance with a neural network of equivalent accuracy. The neural network required 8,323 arithmetic operations per inference versus 116 for EvoChip, O’Neill said. In another test, the comparison was 21,000 operations versus 110, he said
O’Neill said the test results were not audited by a third party, but they are reproduceable by anyone. The company has a test report that shows the exact settings and processes used to compare the two technologies should anyone wish to independently verify the results, he said.
EvoChip’s technology has yet to be applied to large scale models, O’Neill said.
“Our market focus right now is microcontrollers because we can put models in there where a neural network can’t fit at all. … Ultimately, we would like to replace Nvidia GPUs,” he said. “If we can make this thing outperform a $3,000 GPU on a desktop, that’s a real value in the marketplace.”
EvoChip is close to moving AltiCore to general availability as it builds documentation, refines the user experience, and tests the technology on more platforms in addition to the STMicroelectronics MCU, O’Neill said.
Solution providers working with system vendors and others who will build systems with the EvoChip AltiCore technology will find multiple ways to add value, O’Neill said.
“They have customers and use cases that we can address in software, and to them, anything that says, ‘OK, you need 10 fewer servers’ is a benefit,” he said. “As I mentioned, we’re 27 times faster than existing technology. That may not sound like a lot, but if you’re doing ChatGPT and this one server can do the same as 27 servers with a legacy algorithm, that’s an enormous savings in costs, thermal constraints, mitigation, less cooling, less electricity, less hardware expense, all those things.”
Patrick Mulvee, a partner at Sidepath, called the EvoChip technology a game-changer.
The solution provider, working with EvoChip, tested the technology on both the latest Dell Pro Max laptop and on an older model, as well as using the newest Dell R670 servers to show it worked with and without using virtual machines, he said.
“When we ran them using the EvoChip technology, they perform exponentially better, faster, quicker, and pull less power,” he said. “It makes a server perform like it’s on steroids.”
The tests were run with no hardware changes, just the addition of the EvoChip software, Mulvee said.
“I think manufacturers will pick this technology up and run it through their channels or direct,” he said. “It’s incredible technology the world needs. It doesn’t make sense for people to just keep throwing hardware at the AI problem when this software can make the hardware perform better.”
The EvoChip technology has applications throughout the business world because it significantly lowers the entry barrier to AI, said Matt Musial, solutions architect at Sidepath.
“We could apply this technology to all sorts of things now because it doesn’t have a $100,000 or a million-dollar starting point,” Musial said. “The alternative is a $100,000 card or a million-dollar rack of 10 servers to create a starting point for evaluating AI, and 95 percent of those projects fail. Now we can apply AI rapidly. We’ll be able to move at a much faster pace than we do today because of the accessibility.”
That will be a boon to customers, including large customers, who need to analyze their data holdings to improve decision making to get started in their AI inference journey, Musial said.
“This brings a lot more affordability to that process,” he said. “You could start on your laptop. You can start on a much smaller initial investment and then figure out what works, what you need in a model. And then you can start to say, ‘How do we scale this? Does it need to be scaled? Do I need to pay for it?’ I could see a lot of our customers using this technology to cut their teeth in the AI world. And with the results we’re seeing, they may not have to buy more going forward.”
Gerald Conrad, founding investor at EvoChip, told CRN that EvoChip has the potential to lead to a massive shift in AI similar to what happened when the first PCs showed they could compete with IBM mainframes by focusing on providing the functions to do their jobs instead of having the capabilities needed to send a man to the moon.
“EvoChip can do 90 percent of what needs to be done for under $100 on a laptop,” Conrad said.
