I think the first thing I would do is trace the original source of the data and ask for the original CAD data used to create the drawings. If it was done by a third-party under contract to your organisation, then they may have an obligation to supply the data. Even if it was done by a third-party, a polite phone call often works wonders. Even if all they can supply you with is a list of coordinates for the points you can do a table join and move your points that way.
This may not be possible. So the second thing I would do is determine which (if either) is correctly georeferenced, the aerial image or the CAD drawing. If the aerial image is commercially purchased and the CAD drawings were scanned by some guy on work-experience in a poorly maintained office scanner from crumpled paper, my bet is that the CAD scans are at fault, but you can't be certain until you check them both against a trusted reference (e.g. known survey points). However, if your aerial image is from a mass-market source such as Google or Bing, then I would take the internal rubber-sheeting of the image with a pinch of salt. If you can't trust either image, then there is nothing for it but to start again and georeference/rubber-sheet the CAD images yourself.
If you can trust the aerial image then you can use the Affine Transformation tool in the plugins. Working out the transformation matrix could be tricky depending on how the georeferencing of the CAD images went wrong. Hopefully it is just a bit of a scale and rotate issue, but your statement 'off by varying amounts' is worrying. You can try determining this by re-georeferencing the CAD image.
Another tool to consider in your case could be the Vector Bender plugin. This post shows somebody had a very similar problem to yours and created a tool to solve the problem. I have not used this tool but it does sound promising.
