I recently took a picture where I instinctively tilted the frame slightly while composing:

Just to see, I straightened it out in an editing program, like so:

In looking at them both for a while, I prefer the first one. There's a greater dynamic sense, and it has a more casual aesthetic — but the parallel line of her arm and the right edge of the image provide a stop and bit of static balance. The leveled image is too much "okay, I'm standing here".
This isn't a 45° angle that would make the image a composition about diagonals, but I think it illustrates the value of non-squared lines as a element in an overall composition.
Other than the issue of presentation, there's another thing to consider in the digital age. If you want to print a slanted film photograph turned to be straight (or vice versa), the only loss is the cropped corners/edges. If you rotate a digital file by an arbitrary angle, pixels have to be re-interpolated, which is inherently a very lossy operation. It's like running a blur filter over the whole thing.
In my example above, notice how the sparkle has gone out of Anya's eye. I didn't treat these two images differently in any way except for the rotation. After that, they're both scaled down with an identical filter. I did the work really quickly and from a JPEG; I'm sure more careful work could avoid that particular damage, but there is a very concrete example of the loss I'm talking about.
That doesn't break shooting in this way, but makes it a bigger decision. Of course, one can always keep the digital file as-is and turn and crop after printing, but that different from the workflow many people who shoot digitally are accustomed to.
In response to Matt Grum's comment below: the test of rotating and then rotating back by the exact same amount doesn't show the whole problem, in that the operation causes blur, but that blur is largely reversible if you do the exact same thing in reverse. If you just leave the image rotated and continue to do other things (like, say, print), the loss is made permanent.