Assuming you're using Windows, part of the problem is that there is no "Date Taken" tag. Windows fills this property with different metadata, depending upon what it can find. The mostly likely tag (and the one Windows gives highest priority to) is the EXIF:DateTimeOriginal tag. So the best command for you to try would be:
ExifTool "-FileModifyDate<EXIF:DateTimeOriginal" DIR
Edit: I made a more complete command. This will set the FileModifyDate by trying all the various metadata Windows uses for the "Date Taken" property, in order of priority. It uses the fact that when ExifTool has two assignments that affect the same tag, the latter takes precedence. From what I've discovered, the XMP:DateTimeOriginal has the least priority, so it's used first, while the EXIF:DateTimeOriginal has the highest priority, so it's used last.
ExifTool "-FileModifyDate<XMP:DateTimeOriginal" "-FileModifyDate<EXIF:CreateDate" "-FileModifyDate<XMP:CreateDate" "-FileModifyDate<$IPTC:DateCreated $IPTC:TimeCreated" "-FileModifyDate<EXIF:DateTimeOriginal" DIR
exiftool -time:all -a -G0:1 -s yourfile.jpgand give the exact "date modified" tag (and group) you want to change? – mattdm Sep 11 '15 at 14:06Edit: Found link, Last Access Timestamp - Enable or Disable in Windows
– StarGeek Sep 11 '15 at 16:41