Symmetries are a bit deeper than just functions or maps. They actually act upon functions or maps. This answer mentions a category theory definition for symmetries:
A symmetry of a morphism $\phi:A\to B$ means a pair $(\alpha,\beta)$ of automorphisms of $A$ and $B$ respectively, such that $\beta\circ\phi=\phi\circ\alpha$.
Of course, category theory is impenetrable if one doesn't already know it. The general idea is that a symmetry is captured not just in the mapping of sets, but of the structure of those sets. If I have a triangle with vertices A, B, and C, it not only maps A B and C to A' B' and C', but also maps the relationships between them. If A and B are connected, A' and B' are connected. And it doesn't matter if you apply the relationship first and then map the result with $\phi$, or if you start with mapping with $\phi$ and then apply the relationship. A map has such a relationship with the terrain it represents. It doesn't matter if you walk 10 miles north, and then find the point where you are, or if you start by seeing how many map-inches 10 miles is, and then walk the corresponding amount. It doesn't matter if you draw a 5" line to the left, starting at your nose, or draw a 5" line to the right of your mirror image. The result is the same.
As for why they have an inverse, that's more of a question of why did we name "symmetry" "symmetry." They have it because we found a useful class of things to talk about had these behaviors. We could talk about something which does not need to be an isomorphism (which would mean it does not need to have an inverse). It just turned out that the things that had inverses were far more useful to talk about. Indeed, there is an article out there, Semigroup Theory of Symmetry, which goes into detail about this more generalized concept of symmetry, and even goes beyond this to look at an even more concept: groupoids of symmetry. (Unfortunately, this is behind a paywall, so I have not actually been able to access it. You might!)
In a very vague sense, we are interested in mappings that preserve all structure. To say that, we typically need to be able to map it back, to show that the structure did not change. If we were to remove this requirement, we could have "symmetries" involving relationships that can never be demonstrated to preserve structure. That just turned out not to be that useful.
But let's widen the net. Let's consider what I will call semi-symmetries, which obey the above definition except using endomorphisms rather than automorphisms. What sort of structures do we see.
In the finite case, where these endomorphisms are just functions, we see that the endomorphisms must change the cardinality of the set we are looking at. If the cardinality of the domain and range are the same then the mapping must be bijective, and this is an invertable function (an automorphism). So in the finite case, any semi-symmetry operation is going to involve either expanding or contracting the number of objects which can be explored. In a sense, these maps may be "given groups of people such that everyone is married to one other, map people to the taller person in the married couple." They can be done, but they lack the feel of "symmetry."
So what about the infinite cases? I came across one very interesting example of this sort of semi-symmetry: Hilbert's Grand Hotel. This is a very common paradox that one has to work through when learning set theory to understand the counter-intuitive math to resolve the paradox. The problem looks like this:
Hilbert has a Grand Hotel with an infinite number of rooms, each numbered with a natural number. A new client shows up, looking for the room, but is dejected because the sign out front reads "No Vacancy." Hilbert assures the client that this is not a problem, and gives instructions to the bellhops to ask each party to please pack up, and move into the room with a room number one higher than they were originally in.
All the guests oblige, and now room 0 is empty. Hilbert happily books the new client, and hands them the keys.
In this example, there is a semi-symmetry. $\alpha$ is "every client moves to a room one higher in number" and $\beta$ is $n^\prime = n + 1$. Its easy to see that the $\beta$ equation is non-invertible over natural numbers, because there is no $n$ such that $n^\prime = 0$.
This problem is interesting because it highlights that there are some injective mappings in set theory which do not change the cardinality of the set. I would argue that this Hilbert Hotel situation would earn the intuitive sense of a "symmetry," although the actual math shows it to be a semi-symmetry. This would suggest that there is some validity to thinking of at least some of the non-invertible mappings as being part of a symmetry.
So having shown that it makes some sense, we are left with the question of "why not?" There must be a "soft" reason for why we preferred to focus on the invertible mappings and not the non-invertible ones. The best answer I found for this is from another stack exchange question focusing on the underlying algebraic structures: Why are groups more important than semigroups?. In the wining answer:
One thing to bear in mind is what you hope to achieve by considering the group/semigroup of automorphisms/endomorphisms.
A typical advantage of groups is that they admit a surprisingly rigid theory (e.g. semisimple Lie groups can be completely classified; finite simple groups can be completely classified), and so if you discover a group lurking in your particular mathematical context, it might be an already well-known object, or at least there might be a lot of known theory that you can apply to it to obtain greater insight into your particular situation.
Semigroups are much less rigid, and there is often correspondingly less that can be leveraged out of discovering a semigroup lurking in your particular context. But this is not always true; rings are certainly well-studied, and the appearance of a given ring in some context can often be leveraged to much advantage.
A dynamical system involving just one process can be thought of as an action of the semigroup N
. Here there is not that much to be obtained from the general theory of semigroups, but this is a frequently studied context. (Just to give a perhaps non-standard example, the Frobenius endomorphism of a char. p ring is such a dynamical system.) But, in such contexts, precisely because general semigroup theory doesn't help much, the tools used will be different.
To sum this quote up: groups provide a very powerful platform for deriving insight from these relationships. Semigroups are simply not as powerful, making it harder to make useful inferences. Of course, given that semigroups naturally occur in dynamic systems, there are clearly useful semi-symmetries to be had. However, they are not useful because they are semi-symmetries, but rather useful because of their association with dynamic systems. Accordingly, their value is not phrased using the language of symmetries, but rather in the language of dynamic systems. The fact that they are semi-symmetries is more of an afterthought. Similarly, from what I can glean from the research, semi-symmetries are interesting for those who study rings, but those analyses are done with respect to the additional structure a ring provides.
