0

I am trying to deploy a grails app (to production environment) to an Ubuntu 14.04 server running on an AWS t2.micro instance. This application and deploying the Wars to the server have been working fine for over a year until this last few weeks. Now whenever I try to deploy the War I get the following errors in Tomcat log files and I am unable to start the application.This started without me making any changed to the server configuration myself.

 07-Jun-2016 18:01:51.553 SEVERE [localhost-startStop-3] org.apache.catalina.core.StandardContext.startInternal Error listenerStart
07-Jun-2016 18:01:51.559 SEVERE [localhost-startStop-3] org.apache.catalina.core.StandardContext.startInternal Context [/appname] startup failed due to previous errors
07-Jun-2016 18:01:51.570 WARNING [localhost-startStop-3] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesJdbc The web application [appname] registered the JDBC driver [com.mysql.jdbc.Driver] but failed to unregister it when the web application was stopped. To prevent a memory leak, the JDBC Driver has been forcibly unregistered.
07-Jun-2016 18:01:51.571 WARNING [localhost-startStop-3] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [appname] appears to have started a thread named [Abandoned connection cleanup thread] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:

Further Details:

  • Grails 2.5.4 (have also tried downgrading to 2.5.0 and 2.4.4)
  • Mysql 5.6 running on AWS RDS (securitygroups set up correctly and working)
  • Tomcat 8.0.21 running on server
  • Java 1.8.0 Oracle version (have also tried Openjava version)

I have searched a lot (I do not considered this a duplicate just because the error message is the same though as the solutions that fixed others do not work for me) and tried a lot of different possible configuration changes and solutions but nothing has worked so far I have tried :

From this question stackoverflow.com/questions/3320400

I have tried moving the JDBC driver to Tomcat's /lib folder and removing it from the \WEB-INF\lib\ in the War file before deploying it.

I have tried downgrading to tomcat version 6 (also tried installing 7, 8 and 9 versions)

I have a version in a war file that is currently live on the production server and I also have that code in github that was committed when I went live with that version BUT if I checkout that version and create a War from it to deploy it I get the errors (but deploying the original war still works. I have also tried checking out earlier version well before I had these issues and I get teh same problem. This suggests to me it is something to do with the environment either on my machine or the server but I have the same version of Java, Tomcat Mysql etc on both. The only real differences are that I develop on a Windows Machine and production is Ubuntu, and I have local Mysql server on same machine and production is separate RDS instance.

I have also tried deleting tomcat temp directory and restarting as sugggested in Tomcat fails to start because of jdbc driver loading

UPDATE

Checking the tomcat localhost log files when deploying I get the following entries:

08-Jun-2016 09:52:24.860 INFO [http-nio-8080-exec-10] org.apache.catalina.core.ApplicationContext.log Destroying Spring FrameworkServlet 'gsp'
08-Jun-2016 09:52:24.984 INFO [http-nio-8080-exec-10] org.apache.catalina.core.ApplicationContext.log Closing Spring root WebApplicationContext
08-Jun-2016 09:52:57.928 INFO [localhost-startStop-2] org.apache.catalina.core.ApplicationContext.log No Spring WebApplicationInitializer types detected on classpath
08-Jun-2016 09:52:58.863 INFO [localhost-startStop-2] org.apache.catalina.core.ApplicationContext.log Initializing Spring root WebApplicationContext

I do not know if this is relevant or not though (will research)

UPDATE 2 I didnt find out exactly what the issue was but it was caused by the plugins / BuildConfig file. I went back to a barebones configuration by creating a new TestApplication which deployed without problems. I then used that Buildconfig and one by one added the plugins I required for my live app to work until I found one that caused the deployment issue. I could recreate the issue by enabling rest-client-builder:2.1.1 but also by using hibernate4:4.3.8.1 instead of Hibernate:3.6.10.19. I also found a much (over 8 months old) older version of the code base that I could create a war from and deploy with out problems so I have been readding all the newer features to that codebase , testing , deploying carefully to try to get back to where I was and so far without problems. It has meant I am now on grails 2.4.4 but if things are working I dont have a need to upgrade to a newer version.

Community
  • 1
  • 1
boliviab
  • 83
  • 2
  • 14
  • Have you purchased paid AWS Support? If so, you could try deploying your WAR to Elastic Beanstalk in a preconfigured Tomcat container. We made this switch and it's overall reduced deployment complexity and also allowed us to get help directly from AWS Support regarding any startup errors. – Marty Chang Jun 08 '16 at 01:49
  • Also when you say "deploying the original WAR still works", do you mean that if you deploy the original WAR to the EC2 instance, it runs fine without errors? If so wouldn't it be a code problem, not an environment problem? – Marty Chang Jun 08 '16 at 01:50
  • Check if localhost.log in tomcat log folder contains any exceptions – Sangram Jadhav Jun 08 '16 at 05:50
  • @MartyChang - i have tried elastic beanstalk before and it didnt support grails very well (cant remember exactly why now as it was a while ago) as it can only be used with Amazon Linux - that is why I reverted to using a EC2 instance with Ubuntu – boliviab Jun 08 '16 at 13:45
  • @MartyChang - i have a live version running that is deployed from a war named war34 - every war I have tried after war34 does not deploy anymore and (this is why I think it is enviroment) if I use the exact same codebase that was used to create war34 to create a new war that also will not deploy and gives the same errors. Same with earlier commits - wont deploy (basically none of my codebase at any point of time deploy correctly anymore -I can only use wars created from war34 and before) – boliviab Jun 08 '16 at 13:48
  • @sangram jadhav - i have checked and there are a few entries that appear when deploying but I am not sure if they are relevant or not. I will post in an update in the Question now – boliviab Jun 08 '16 at 13:54
  • I am thinking my next test is to upload the code base and try to create a war directly on the production server enviroment so I am 100% sure the war creation enviroment is the same as the one to deploy to.I will try that this evening – boliviab Jun 08 '16 at 14:36
  • It's still confusing why the same code to create **war34** builds a new WAR file that doesn't deploy. So by "environment difference" that seems to be a difference in your _build_ environment, not your _runtime_ environment. What if you build a new WAR using the same source for **war34**, explode both WAR files and perform a full `diff` between the two resulting folders? – Marty Chang Jun 08 '16 at 20:15
  • Yeah that is what I was referring to but maybe didnt explain clearly sorry. Yeah thats a good idea to run a diff test like that - I will try to get some time to do that tomorrow (along with building a war on the production server instead of my pc) to see what happens (ive not had a chance today to work on this at all) – boliviab Jun 09 '16 at 01:24

0 Answers0