These are quick notes on the Kolmogorov-Arnold Network (KAN) introduced in [LWV+24], with a few questions here and there.
<aside> 💡 Ignore this section if you know spline functions well enough.
</aside>
A spline function is simply a way to parameterize a generic curve (we restrict the discussion to 1D functions), which is similar to a polynomial expansion of the variable but avoids weird phenomena happening with polynomials of very high degrees (e.g., oscillations at the borders).
A spline function of order $n$ is defined by first dividing the domain into $k$ intervals (when the intervals are of equal length, this is called a uniform spline). These intervals are described by a grid of $k+1$ points on the $x$-axis called knots:
$$ t_1, \ldots, t_{k}, t_{k+1} $$
For each interval $[t_i, t_{i+1}]$ the spline is defined by a polynomial of order at max $n$ acting on the interval, such that the overall function is continuous at the knots. Splines are interesting because any spline can be described by an expansion into so-called B-splines basis functions $B_{i,n}$:
$$ S_n(x)=\sum_{i=1}^kc_iB_{i,n}(x) $$
where the index $i$ runs on the different intervals. The parameters $c_i$ together with the order $n$ and the location of the knots fully determines the shape of the function. For any given order $n$ the basis functions $B_{i,n}$ can be computed recursively using the Cox–de Boor recursion formula. For order $0$, the B-spline is simply $1$ over the corresponding interval:
$$ B_{i,0}(x)=\begin{cases} 1 & x \in [t_i, t_{i+1}] \\ 0 & \text{otherwise} \end{cases} $$
For a generic order $n$, the spline is a linear combination of the splines of order $n-1$ at the $i$-th and $i+1$-th intervals. In particular, let us define a first linear term as:
$$ t^+{i,n}(x) = \frac{x - t_i}{t{i+k} - t_i} $$
This is a ramp that goes from $0$ to $1$ in the interval $[t_i, t_{i+n}]$. We also define:
$$ t^-{i,k}(x)=\frac{t{i+k+1} - x}{t_{i+k+1}-t_{i-1}} $$
This is a second ramp that goes from $1$ to $0$ in the interval $[t_{i+1}, t_{i+k+1}]$. Then:
$$ B_{i,n}(x)=t_{i,k}^+(x)B_{n-1,i}(x)+t^-{i,n}(x)B{n-1, i+1}(x) $$
It is helpful to get a feeling about B-splines by visualizing them (see Colab). We do this by exploiting the functions in the spline module of the official KAN repository. First, we plot the basis splines of order $n=0,1,2$ for $5$ equi-spaced knots in $[0,1]$.
Basis splines of order $0$.
Basis splines of order $1$.