For EU citizens it might not be strictly necessary to carry an ID when traveling around the Schengen area but you have to make several distinctions here. The fact that the police can and does check ID does not mean it's mandatory for everybody to have one on them. Conversely, the fact that there should be no systematic check at the border does not mean that it cannot be mandatory to carry ID (it's the case in the Netherlands for example and yet actual checks seem rare in my experience). Rules can also be different for locals, EU citizens and third-country nationals.
For example, regarding the first incident, French citizens are not even required to hold a national ID card at all but it's still perfectly legal for the police to do ID checks under certain conditions, even far away from the border (more details here or on service-public.fr). I think that even for third-country nationals (and certainly for French citizens), not having any means to prove your identity on you is not per se a punishable offense. The police might keep you for a few hours to ascertain who you are by other means or be generally unpleasant but you cannot be charged with anything just because you left your passport at home.
Furthermore it should be pointed out that racial profiling has long been a problem in France. I have no way to independently assess how widespread it really is but I heard many anecdotes and some organizations regularly complain about it. There are even a few official reports that acknowledged the problem and, back when he was still running for office, the current president promised a number of specific measures to deal with it. The details are complex but this gives you some idea of the context.
Now, under the Schengen agreement, France committed herself not to perform systematic checks at her borders. But anything else still depends on national law. For example, being part of the Schengen area does not mean that nobody can ever be asked for ID on French territory. Nor does it provide you with any practical recourse if you are victim of racial profiling or if you suspect French police is bending the rules. If France really is in breach of the Schengen agreement, this is something other EU member states and especially the EU Commission might deal with in several ways (potentially escalating to an infringement procedure in front of the EU court of justice) but that does not immediately help you in practice.
In fact, French law even specifically allows the police to ask people for ID at all times in border areas, on international trains and in a number of train stations including Metz. In 2011, the Commission looked at what France was doing and issued a critical-sounding press-release but did not go further or challenge the law itself. This guy has been writing extensively about checks at the France-Italy border and elsewhere so it seems to be a relatively common occurrence.
All this also means that there are no EU or Schengen-wide rule that would definitively answer your question. It's certainly not mandatory to always carry ID everywhere but since police checks do happen, carrying your passport seems like a good idea in practice, especially as a non-EU citizen.
