Consider the ball $B_r$ of radius $r$. This is a manifold with boundary whose boundary is the sphere of radius $r$. Now consider specifically the difference between this ball and a slightly larger ball $B_{r + dr}$. The difference is a thin spherical shell which is $dr$ thick, and as $dr \to 0$ it approaches, in some sense, the boundary sphere of $B_r$.
So the boundary is something like a "derivative." This is made somewhat more precise by Stokes' theorem, and is also related to ideas like cobordism. A simple example of the way in which the boundary behaves like a derivative is that if $M, N$ are manifolds with boundary, then loosely speaking we have the "product rule"
$$\partial(M \times N) = \left( \partial(M) \times N \right) \cup \left( M \times \partial(N) \right).$$
There's some subtlety to making this precise because $M \times N$ is generally a manifold with corners (consider for example $M = N = I$, which is perhaps the easiest case to visualize).