The problem is that your default aurora installation has a user 'user'@'%' with all grants and the password you set.
If you manually do a GRANT ... TO 'user'@'ip', it's adding a new user, user@ip, that will be used when you connect from this ip (while the default user@% is used from all other ips).
And this new user will have only SELECT INTO S3 permission. (and most probably, if you you didn't add IDENTIFIED BY xxx in your GRANT request, the new user won't even have a password set so you won't be able to connect from this ip).
To solve this, just run:
GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, DROP, RELOAD, PROCESS, REFERENCES, INDEX, ALTER, SHOW DATABASES, CREATE TEMPORARY TABLES, LOCK TABLES, EXECUTE, REPLICATION SLAVE, REPLICATION CLIENT, CREATE VIEW, SHOW VIEW, CREATE ROUTINE, ALTER ROUTINE, CREATE USER, EVENT, TRIGGER, LOAD FROM S3, SELECT INTO S3 ON *.* TO 'user'@'ip' IDENTIFIED BY PASSWORD 'xxxx' WITH GRANT OPTION;
(you can connect from another ip (so it defaults to the default profile), and run SHOW GRANTS to see what are the default grants and password for your default user)