Post from Google that provides insight into the difference between CPUs, GPUs and TPUs for AI processing AI. Useful insight if you are considering on-prem AI.
When Google designed the TPU, we built a domain-specific architecture. That means, instead of designing a general purpose processor, we designed it as a matrix processor specialized for neural network work loads. TPUs can’t run word processors, control rocket engines, or execute bank transactions, but they can handle the massive multiplications and additions for neural networks, at blazingly fast speeds while consuming much less power and inside a smaller physical footprint.
What makes TPUs fine-tuned for deep learning? | Google Cloud Blog : https://cloud.google.com/blog/products/ai-machine-learning/what-makes-tpus-fine-tuned-for-deep-learning