In S02, we learn that Elliot was arrested for "unauthorized use of a computer, computer trespass, and grand larceny". The arresting officer says, "Grab the computers ... Get the hard drives. All the USB drives. Any digital media." (S02E09)
In S02E10, we see Elliot return to his apartment for the first time where indeed all his computers and electronic equipment are gone (and he has had to go to Micro Center to buy a bunch of gear including some gear for making an antenna with a Pringles can to steal someone's WiFi).
I believe the next time we see Elliot's CD collection is in S03E08 when he burns information about Trenton/Shama Biswas and Mobley/Sunil Markesh onto CDs, puts the CDs into the CD collection, then slides the CD collection to its usual spot on the floor under the chest of drawers.
Why didn't the police confiscate this? (Even if they were unable to decrypt the CDs, surely they would have kept the CDs confiscated as they did with all his other equipment and wouldn't have just returned the CDs to him?)