Posts

Lagrangian duality in convex analysis: A (not so) summarized account

I've been dabbling in some convex optimization and decided to write this post to summarize the ideas in a (hopefully) clear and linear fashion. The material is mostly straight out of Boyd and Vandenberghe (freely available), inspired by this very helpful MSE post. Let's get started. We'll work with $\mathbb{R}^n$ for concreteness, though the theory extends to much more general contexts (such as Banach spaces). I presume you know a convex set $C \subseteq \mathbb{R}^n$ is one for which $x, y \in C$ and $0 \leq t \leq 1$ implies $tx + (1-t)y \in C$, and that a real-valued function $f$ is convex if its domain is convex and \[ f(tx + (1-t)y) \leq tf(x) + (1-t)f(y) \] for $x, y \in \text{dom}(f)$ and $0 \leq t \leq 1$ as before. I said $f$ is real-valued, but we also allow it to take on $\pm \infty$, adopting the standard conventions for arithmetic and inequalities with infinities, which will be useful later. Because of that, we define a proper convex function to be one wh

Differential forms, integration and the generalized Stokes theorem (part III)

Image
The last ingredient we need for the generalized Stokes theorem is the exterior derivative of a differential form. Consider a $k$-form $\omega$ in $\mathbb{R}^n$. Its exterior derivative $d\omega$ will be a $(k+1)$-form, i.e. a function of a point $p$ and $k+1$ vectors $v_1, \ldots, v_{k+1}$, which we can think of as forming the parallelogram \[ P_p(v_1, \ldots, v_{k+1}) = \{p + t_1 v_1 + \cdots + t_{k+1} v_{k+1}\ |\ 0 \leq t_i \leq 1, \ i = 1, \ldots, k+1\} \] By this point it should not be particularly surprising to see the definition: \[ d\omega_p(v_1, \ldots, v_{k+1}) = \lim_{h \to 0} \frac{1}{h^{k+1}} \int_{\partial P_p(hv_1, \ldots, hv_{k+1})} \omega \] Observe that the boundary of the $(k+1)$-parallelogram $P_p(v_1, \ldots, v_{k+1})$ is a union of $k$-dimensional parallelograms, and so it makes sense to integrate the $k$-form $\omega$ over it. Furthermore note that, if $v_1, \ldots, v_{k+1}$ is an orthonormal set, then \[ h^{k+1} = \text{vol}(P_p(hv_1, \ldots, hv_{k+1})) \]

Differential forms, integration and the generalized Stokes theorem (part II)

Today let's see how to integrate differential forms. We begin with $\mathbb{R}^n$ and then move on to manifolds. The $n$-form $dx^1 \wedge \cdots \wedge dx^n$ is a basis for the space of linear $n$-forms on $\mathbb{R}^n$; therefore any differential $n$-form on $\mathbb{R}^n$ is given by \[ \omega = f(x^1, \ldots, x^n) dx^1 \wedge \cdots \wedge dx^n \] To recover the function $f$ from $\omega$ we can just evaluate it on the standard basis: \[ f(p) = \omega_p(e_1, \ldots, e_n)\] Therefore $n$-forms in $\mathbb{R}^n$ are in direct correspondence to scalar functions. Integrating $\omega$ is easy: we set \[ \int_{\mathbb{R}^n} \omega = \int_{\mathbb{R}^n} f(x^1, \ldots, x^n) dx^1 \cdots dx^n \] where the integral on the right hand side is the ordinary Riemann integral of the function $f$ [1] . The same definition holds for integrating $\omega$ over any domain $D \subseteq \mathbb{R}^n$ (some reasonably well-behaved subset, possibly with boundary). Now let $M$ be an oriented $n$-d

Differential forms, integration and the generalized Stokes theorem (part I)

If you've read the previous couple of posts, you were certainly struck by how similar the Gauss, Green and Stokes theorems are. They are all particular instances of the generalized Stokes theorem which we discuss today, in the language of differential forms. First, let me suggest a couple references: Hubbard & Hubbard - Vector Calculus, Linear Algebra and Differential Forms: A Unified Approach. This is an outstanding book on multivariable calculus, with the best presentation of differential forms I've seen anywhere. It has detailed proofs of the ideas sketched in this series of posts. Terence Tao - Differential Forms and Integration . Lovely article which really helped me gain some intuition about differential forms. Baez & Munian - Gauge Fields, Knots and Gravity . Don't be misled by the physics title -- this book has one of the clearest expositions of basic differential geometry I've seen (rivaled perhaps by Spivak's books). So let's get started. On

Green, Gauss, Stokes: the classical theorems of integral calculus (part II)

Image
This time we'll go over the Green and Stokes theorems in a similar way. In the interpretation of a vector field $V$ as the velocity field of some fluid, while the divergence measures the rate at which the fluid diverges (i.e. flows away) from a point $p$, the curl measures the rate at which the fluid curls (i.e. circulates) around $p$. This leads to a subtler notion because a fluid could circulate around any axis through $p$ (or, in dimensions greater than $3$, circulate within any plane through $p$), so we'll begin by looking at the plane only. Green's theorem So let $V$ be a vector field on the plane $\mathbb{R}^2$, and $p$ a point in $\mathbb{R}^2$. How do we quantify this curling around $p$? We take a disk $D(p; \epsilon)$, with boundary circle oriented counter-clockwise, and look at \[ \int_{\partial D(p; \epsilon)} V \cdot ds \] The integrand $V \cdot ds$ gives us the tangential component of velocity, which is what makes the fluid go around the circle -- so integrat

Green, Gauss, Stokes: the classical theorems of integral calculus (part I)

Image
In this series of posts I will present the classical theorems of integral calculus in a unified, geometric perspective; later I will show how this generalizes to differential forms, the exterior derivative and the general Stokes theorem. The objective here is not to be rigorous, but to emphasize the geometric intuition in these topics. Let's start off easy, with the ordinary derivative of a function $f : \mathbb{R} \to \mathbb{R}$. Here's an unusual way to think about it: given an interval $[a, b]$, oriented positively (from $a$ to $b$), we can think of its boundary $\partial [a, b]$ as the discrete collection of points $\{a, b\}$, with a twist: $b$ is positively oriented, and $a$ is negatively oriented (it'll become clear what this means). If we integrate the function $f$ along the boundary of $[a, b]$, since an integral over a discrete set is really just a sum we get (keeping in mind the orientations) \[ \int_{\partial [a, b]} f = f(b) - f(a) \] The 'volume' of

The joy of quaternions

Image
Quaternions are a lovely topic that usually don't get much attention in the standard undergraduate and graduate textbooks on mathematics. In this post I'll try to go over the basic ideas and motivations of the theory in a hopefully illuminating way; let's get started. The complex numbers $\mathbb{C}$ form a wonderful tool to describe geometry in a plane, since (among other things) complex multiplication describes at once both rotations and scalings in a neatly concise way. Hamilton's plan, which eventually led him to the discovery of quaternions, was to develop an analogous number system that would help describe geometry in three-dimensional space. So how might you do that? Naively, you might try to just extend the complex numbers and take numbers of the form \[ a + bi + cj \] where $a,b,c \in \mathbb{R}$, $i$ is the usual imaginary unit and $j$ is a new quantity which also satisfies $j^2 = -1$. To obtain a multiplication of these new numbers, essentially we just nee