For example, in the 115th Congress, 33 Representatives and three
Senators missed 10% or more of votes, across both parties.
This isn't too bad. So, 97 Senators and 402 Representatives miss less than 10% of floor votes. The median percentage of missed votes was 1.0% in the U.S. Senate and 2.6% in the House.
Also, keep in mind that if you are in the minority party in the House, the odds that your vote will be outcome determinative are nearly zero. Sixteen of the 33 Representatives who missed more than 10% of the votes were Republicans who were in the minority and lost when then voted differently from the Democratic majority pretty much 100% of the time. Two of the three Senators who missed more than 10% of the votes were Democrats, who were in the minority in that chamber.
Most measures are adopted with supermajority support (and the parties sometimes agree to have offsetting absences so that absence don't effect which party has a majority).
Furthermore, in any group of this size, there are going to be some people who suffer a serious illness for some period of time who must unavoidably miss a significant number of votes.
Elijah Cummings, for example, who missed the most votes in the House, did so mostly because he died two and a half months before his term expired, because he was hospitalized for two months during his final term for heart surgery and recovery from it, and because he missed a few more days for knee surgery. You really can't really fault a guy for missing some votes for reasons like those.
Sometimes members are running for other offices. For example, Jared Polis, a Democrat who missed 14.5% of his House votes, was running for Governor in his home state of Colorado near the end of his term (and won).
There may be isolated individuals who are serious offenders in terms of missing votes that actually made a difference, but it isn't a huge issue based on these statistics.
Of course, the bottom line is that the only way a member of Congress can be removed is by the voters once every two years in the House and once every six years in the Senate (in a usually quite safe district where an incumbent almost automatically wins a primary election), or by a two-thirds majority of his or her own chamber (which almost never happens).
Quite a few of the members of Congress with lots of missed votes were not re-elected. Once a member announced his or her retirement, loses of primary election, or loses the general election and is a lame duck, even the voters cannot hold that individual accountable.
So, there is really no way to punish someone, if they don't stray too far from reasonable participation without a good excuse (as determined by same party colleagues) for not doing so.