Functionals & Functional Derivatives | Calculus of Variations | Visualizations

We can minimize a Functional (Function of a Function) by setting the first Functional Derivative (=Gâteaux Derivative) to zero. Here are the notes: https://raw.githubusercontent.com/Cey...

A Function maps a scalar/vector/matrix to a scalar/vector/matrix. We have seen it multiple times, we know how to take derivatives etc. But now imagine something takes in a function and outputs a scalar/vector/matrix? At first this seems more complicated. Situations like these arise for instance in Lagrangian and Hamiltonian Mechanics or when deriving probability density functions from a maximum entropy principle.

But a more intuitive example: Say you want to take your car from Berlin to Munich. There are quite a lot of possible routes to take, each with a potentially different velocity and height profile. Now imagine you have a function that associates each point in time over the route with a position on the map. You could use this to deduce the height-and velocity profile. A Functional would now be a function that takes in the route and outputs the fuel consumption, i.e. mapping from a function to a scalar.

Then, you might be interested in minimizing your fuel consumption, so you seek the minimum of a Functional. First Derivative equals zero, right? But how do you take the functional derivative.

All of this and more will be answered in the video. ;)

-------

📝 : Check out the GitHub Repository of the channel, where I upload all the handwritten notes and source-code files (contributions are very welcome): https://github.com/Ceyron/machine-lea...

📢 : Follow me on LinkedIn or Twitter for updates on the channel and other cool Machine Learning & Simulation stuff:   / felix-koehler   and   / felix_m_koehler  

💸 : If you want to support my work on the channel, you can become a Patreon here:   / mlsim  

-------

Timestamps:
00:00 Introduction
00:49 Can't we just use Newtonian Mechanics?
01:27 Defining Energies and Parameters
04:21 "Average Difference in Energy"
06:20 A Functional
07:11 Example 1
08:46 Example 2
09:56 Example 3
11:18 Comparing the Examples
12:20 Visualizing the Examples
13:23 Mathematical Definition of a Functional
15:22 Concept of Minimizing a Functional
16:22 Intro to the Functional Derivative
19:43 Example: Minimizing the Functional
22:53 Rearrange for y
25:38 Fundamental Lemma of Calculus of Variations
26:55 Rediscovering Newtonian Mechanics
28:07 Solving the ODE
29:31 Summary: Functional Derivatives
30:35 Outro