What are the mathematical prerequisites for an AI researcher?

Question

What are the mathematical prerequisites for understanding the core part of various algorithms involved in artificial intelligence and developing one's own algorithms?

Please, refer to some specific books.

Answer 1

Good Mathematics Foundation

Begin by ensuring full competency with intermediate algebra and some other foundations of calculus and discrete math, including the terminology and basic concepts within these topics.

• Infinite series
• Logical proofs
• Linear algebra and matrices
• Analytic geometry, especially the distinction between local and global extremes (minima and maxima), saddle points, and points of inflection
• Set theory
• Probability
• Statistics

Foundations of Cybernetics

Norbert Wiener, Cybernetics, 1948, MIT Press, contains time series and feedback concepts with clarity and command not seen in subsequent works; it also contains an introduction to information theory beginning with Shannon's log₂ formula for a defining the amount of information in a bit. This is important to understand the expansion of the information entropy concept.

Calculus

Find a good calculus book and make sure you have clarity around key theory and application in these categories.

• Time series
• Infinite series
• Convergence — Artificial networks ideally converge to an optimum during learning.
• Partial differentials
• Jacobian and Hessian matrices
• Multivariate math
• Boundary regions
• Discrete math

Much of that is in Calculus, Strang, MIT, Wellesley-Cambridge Press. Although the PDF is available on the web, it is basic and not particularly deep. The one in our laboratory's library is Intermediate Calculus, Hurley, Holt Rinehart & Winston, 1980. It is comprehensive and in some ways better laid out than the one I have in my home library, which Princeton uses for sophomores.

Ensure you are comfortable working in spaces beyond ℝ² (beyond 2D). For instance, RNNs are often in spaces such as ℝ⁴ thorugh ℝ⁷ because of the horizontal, vertical, pixel depth, and movie frame dimensions.

Finite Math

It is unfortunate that no combination of any three books I can think of has all of these.

• Directed graphs — Learn this BEFORE trees or circuits (artificial nets) because it is the superset topography of all those configurations
• Abstract symbol trees (ASTs)
• Advanced set theory
• Decision trees
• Markov chains
• Chaos theory (especially the difference between random and pseudo-random)
• Game Theory starting with Von Neumann and Morgenstern's Game Theory, the seminal work in that field
• Convergence in discrete systems especially the application of theory to signal saturation in integer, fixed point, or floating-point arithmetic
• Statistical means, deviations, correlation, and the more progressive concepts of entropy, relative entropy, and cross-entropy
• Curve fitting
• Convolution
• Probability especially Bayes' Theorem
• Algorithmic theory (Gödel's uncertainty theorems and Turing completeness)

Chemistry and Neurology

It is good to recall chemical equilibria from high school chemistry. Balance plays a key role in more sophisticated AI designs. Understanding the symbiotic relationship between generative and discriminative models in GANs will help a student further this understanding.

The control functions within biological systems remain a primary source of proofs of concept in artificial intelligence research. As researchers become more creative in imagining forms of adaptation that do not directly mimic some aspect of biology (still a distance off as of this writing) creativity may play a larger role in AI research objective formulation.

Even so, AI will probably remain a largely interdisciplinary field.

Answer 2

I work as a professor, and recently designed the mathematics requirements for a new AI major, in consultation with many of my colleagues at other institutions.

The other answers, particularly this one do a good job of cataloging all the specific topics that might be useful somewhere in AI, but not all of them are equally useful for understanding core topics. In other cases, understanding the topic is essentially the same as understanding the related AI algorithms, so we usually just teach them together instead of assuming prerequisite knowledge. For instance, Markov Decision Processes aren't hard to teach to someone who already knows the basics of graph theory and probabilities, so we usually just cover them when we teach reinforcement learning in an AI course, rather than as a separate topic in a mathematics course.

The mathematics requirements we settled on look like:

• A one or two semester course in discrete mathematics. This is as much to establish comfort with proof and mathematical rigor as with any specific topic in the area. It's mostly just "foundational" knowledge, but bits of it turn out to be very useful. Comfort with infinite summations, the basics of graphs, combinatorics, and asymptotic analysis are perhaps the most directly applicable parts. I like Susanna Epp's book.

• A one or two semester course in linear algebra, which is useful across a wide variety of topics in AI, especially machine learning and data mining. Lay & Lay is an okay book, but probably not the absolute best. Shilov is a recommendation from Ian Goodfellow and others, but I've not tried it myself.

• A course in probability, and possibly a modern course in statistics (i.e. with a Bayesian focus). An older course in statistics, or one targeting social scientists, is not very useful though. My statistician colleagues are using Lock5 right now, and having good experiences with it.

• At least differential and integral calculus, and preferably at least partial derivatives in vector calculus, but perhaps the whole course. This is useful in optimization, machine learning, and economics-based approaches to AI. Stewart is the most common textbook. It's comprehensive, and can be used for all three courses, but it's explanations aren't always the very best. I'd still recommend it though.

Those are the core topics. If you don't also have a traditional background in programming, then a course in graph theory and the basics of asymptotic complexity or algorithm design and analysis might be good supplements. Usually AI'ers come from a standard computer science background though, which covers all those things very well.

Answer 3

As far as simple algorithms like Gradient Descent are concerned, you need to have a good grasp of partial derivatives. Especially if you want to implement neural networks. Also most algorithms are vectorised to improve computing speed and so you need to be comfortable with matrix math. This involves being really quick and comfy with dimensions of matrices, dimensions of products, multiplication of matrices, transpose and so on. Very rarely, you might use matrix calculus to directly arrive at optimal solutions, so a few results from this area should do. Moving on, you need to understand some function analysis. this is needed to get an intuition on what activation functions like sigmoid and tanh, log are doing. A grasp of probability and expectations is also really useful. You should also be clear with orthogonal vectors and inner products.

That being said, I would suggest you grasp basic calculus and matrix operations and try learning AI concepts. If you can't figure something out, explore the math.

Note: again this is only for starting.