Cannot update Docker single image past v1.8.15

novanglus · January 20, 2023, 9:05pm

Hello all. I recently tried to upgrade my docker instance of community-ce:v1.7.8 to the latest version but it failed. I back tracked and tried incremental updates, first to 1.8 which was succesful, and then to v1.8.15 which was successful. Howeverm trying to go to version 1.9 or above results in a failure. I tried v.1.9, v.19.1 and v1.9.2 as well as the latest and they all result in the same error: the end of my log looks as follows:

backend stdout | at reactor.core.publisher.MonoIgnoreThen$ThenIgnoreMain.onComplete(MonoIgnoreThen.java:209)
backend stdout | at reactor.core.publisher.MonoIgnoreThen$ThenIgnoreMain.subscribeNext(MonoIgnoreThen.java:238)
backend stdout | at reactor.core.publisher.MonoIgnoreThen$ThenIgnoreMain.onComplete(MonoIgnoreThen.java:203)
backend stdout | at com.appsmith.server.configurations.MDCConfig$MdcContextLifter.onComplete(MDCConfig.java:63)
backend stdout | at com.appsmith.server.configurations.MDCConfig$MdcContextLifter.onComplete(MDCConfig.java:63)
backend stdout | at reactor.core.publisher.Operators$MultiSubscriptionSubscriber.onComplete(Operators.java:2058)
backend stdout | at com.appsmith.server.configurations.MDCConfig$MdcContextLifter.onComplete(MDCConfig.java:63)
backend stdout | at reactor.core.publisher.MonoPeekTerminal$MonoTerminalPeekSubscriber.onComplete(MonoPeekTerminal.java:299)
backend stdout | at com.appsmith.server.configurations.MDCConfig$MdcContextLifter.onComplete(MDCConfig.java:63)
backend stdout | at reactor.core.publisher.MonoIgnoreElements$IgnoreElementsSubscriber.onComplete(MonoIgnoreElements.java:89)
backend stdout | at com.appsmith.server.configurations.MDCConfig$MdcContextLifter.onComplete(MDCConfig.java:63)
backend stdout | at io.lettuce.core.RedisPublisher$ImmediateSubscriber.onComplete(RedisPublisher.java:896)
backend stdout | at io.lettuce.core.RedisPublisher$State.onAllDataRead(RedisPublisher.java:698)
backend stdout | at io.lettuce.core.RedisPublisher$State$3.read(RedisPublisher.java:608)
backend stdout | at io.lettuce.core.RedisPublisher$State$3.onDataAvailable(RedisPublisher.java:565)
backend stdout | at io.lettuce.core.RedisPublisher$RedisSubscription.onDataAvailable(RedisPublisher.java:326)
backend stdout | at io.lettuce.core.RedisPublisher$RedisSubscription.onAllDataRead(RedisPublisher.java:341)
backend stdout | at io.lettuce.core.RedisPublisher$SubscriptionCommand.doOnComplete(RedisPublisher.java:778)
backend stdout | at io.lettuce.core.protocol.CommandWrapper.complete(CommandWrapper.java:65)
backend stdout | at io.lettuce.core.protocol.CommandWrapper.complete(CommandWrapper.java:63)
backend stdout | at io.lettuce.core.protocol.CommandHandler.complete(CommandHandler.java:746)
backend stdout | at io.lettuce.core.pubsub.PubSubCommandHandler.complete(PubSubCommandHandler.java:167)
backend stdout | at io.lettuce.core.protocol.CommandHandler.decode(CommandHandler.java:681)
backend stdout | at io.lettuce.core.pubsub.PubSubCommandHandler.decode(PubSubCommandHandler.java:112)
backend stdout | at io.lettuce.core.protocol.CommandHandler.channelRead(CommandHandler.java:598)
backend stdout | at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
backend stdout | at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
backend stdout | at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
backend stdout | at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
backend stdout | at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
backend stdout | at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
backend stdout | at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
backend stdout | at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:800)
backend stdout | at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:487)
backend stdout | at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:385)
backend stdout | at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
backend stdout | at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
backend stdout | at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
backend stdout | at java.base/java.lang.Thread.run(Thread.java:829)
backend stdout |
backend stdout | [2023-01-20 20:51:29,144] -
backend stdout |
backend stdout | Error starting ApplicationContext. To display the conditions report re-run your application with ‘debug’ enabled.
backend stdout |
backend stdout | [2023-01-20 20:51:29,191] - Application run failed
backend stdout | org.springframework.beans.factory.BeanCreationException: Error creating bean with name ‘mongockInitializingBeanRunner’ defined in class path resource [com/appsmith/server/configurations/MongoConfig.class]: Invocation of init method failed; nested exception is com.github.cloudyrock.mongock.exception.MongockException: com.mongodb.MongoTimeoutException: Timed out after 30000 ms while waiting to connect. Client view of cluster state is {type=UNKNOWN, servers=[{address=localhost:27017, type=UNKNOWN, state=CONNECTING, exception={com.mongodb.MongoSocketOpenException: Exception opening socket}, caused by {io.netty.channel.AbstractChannel$AnnotatedConnectException: Connection refused: localhost/127.0.0.1:27017}, caused by {java.net.ConnectException: Connection refused}}]
backend stdout | at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1804)
backend stdout | at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:620)
backend stdout | at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:542)
backend stdout | at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:335)
backend stdout | at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234)
backend stdout | at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:333)
backend stdout | at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:208)
backend stdout | at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:955)
backend stdout | at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:918)
backend stdout | at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:583)
backend stdout | at org.springframework.boot.web.reactive.context.ReactiveWebServerApplicationContext.refresh(ReactiveWebServerApplicationContext.java:66)
backend stdout | at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:734)
backend stdout | at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:408)
backend stdout | at org.springframework.boot.SpringApplication.run(SpringApplication.java:308)
backend stdout | at org.springframework.boot.builder.SpringApplicationBuilder.run(SpringApplicationBuilder.java:164)
backend stdout | at com.appsmith.server.ServerApplication.main(ServerApplication.java:15)
backend stdout | at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
backend stdout | at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
backend stdout | at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
backend stdout | at java.base/java.lang.reflect.Method.invoke(Method.java:566)
backend stdout | at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:49)
backend stdout | at org.springframework.boot.loader.Launcher.launch(Launcher.java:108)
backend stdout | at org.springframework.boot.loader.Launcher.launch(Launcher.java:58)
backend stdout | at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:65)
backend stdout | Caused by: com.github.cloudyrock.mongock.exception.MongockException: com.mongodb.MongoTimeoutException: Timed out after 30000 ms while waiting to connect. Client view of cluster state is {type=UNKNOWN, servers=[{address=localhost:27017, type=UNKNOWN, state=CONNECTING, exception={com.mongodb.MongoSocketOpenException: Exception opening socket}, caused by {io.netty.channel.AbstractChannel$AnnotatedConnectException: Connection refused: localhost/127.0.0.1:27017}, caused by {java.net.ConnectException: Connection refused}}]
backend stdout | at com.github.cloudyrock.mongock.runner.core.executor.MongockRunnerBase.execute(MongockRunnerBase.java:68)
backend stdout | at com.github.cloudyrock.spring.v5.MongockSpring5$MongockInitializingBeanRunner.afterPropertiesSet(MongockSpring5.java:84)
backend stdout | at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1863)
backend stdout | at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1800)
backend stdout | … 23 common frames omitted
backend stdout | Caused by: com.mongodb.MongoTimeoutException: Timed out after 30000 ms while waiting to connect. Client view of cluster state is {type=UNKNOWN, servers=[{address=localhost:27017, type=UNKNOWN, state=CONNECTING, exception={com.mongodb.MongoSocketOpenException: Exception opening socket}, caused by {io.netty.channel.AbstractChannel$AnnotatedConnectException: Connection refused: localhost/127.0.0.1:27017}, caused by {java.net.ConnectException: Connection refused}}]
backend stdout | at com.mongodb.internal.connection.BaseCluster.getDescription(BaseCluster.java:180)
backend stdout | at com.mongodb.internal.connection.SingleServerCluster.getDescription(SingleServerCluster.java:44)
backend stdout | at com.mongodb.client.internal.MongoClientDelegate.getConnectedClusterDescription(MongoClientDelegate.java:144)
backend stdout | at com.mongodb.client.internal.MongoClientDelegate.createClientSession(MongoClientDelegate.java:101)
backend stdout | at com.mongodb.client.internal.MongoClientDelegate$DelegateOperationExecutor.getClientSession(MongoClientDelegate.java:291)
backend stdout | at com.mongodb.client.internal.MongoClientDelegate$DelegateOperationExecutor.execute(MongoClientDelegate.java:183)
backend stdout | at com.mongodb.client.internal.MongoIterableImpl.execute(MongoIterableImpl.java:135)
backend stdout | at com.mongodb.client.internal.MongoIterableImpl.iterator(MongoIterableImpl.java:92)
backend stdout | at com.mongodb.client.internal.MongoIterableImpl.forEach(MongoIterableImpl.java:121)
backend stdout | at com.github.cloudyrock.mongock.driver.mongodb.sync.v4.repository.MongoSync4RepositoryBase.getResidualKeys(MongoSync4RepositoryBase.java:77)
backend stdout | at com.github.cloudyrock.mongock.driver.mongodb.sync.v4.repository.MongoSync4RepositoryBase.isIndexFine(MongoSync4RepositoryBase.java:65)
backend stdout | at com.github.cloudyrock.mongock.driver.mongodb.sync.v4.repository.MongoSync4RepositoryBase.ensureIndex(MongoSync4RepositoryBase.java:52)
backend stdout | at com.github.cloudyrock.mongock.driver.mongodb.sync.v4.repository.MongoSync4RepositoryBase.initialize(MongoSync4RepositoryBase.java:43)
backend stdout | at com.github.cloudyrock.mongock.driver.core.driver.ConnectionDriverBase.initialize(ConnectionDriverBase.java:40)
backend stdout | at com.github.cloudyrock.mongock.runner.core.executor.MigrationExecutor.initializationAndValidation(MigrationExecutor.java:225)
backend stdout | at com.github.cloudyrock.spring.v5.core.SpringMigrationExecutor.initializationAndValidation(SpringMigrationExecutor.java:31)
backend stdout | at com.github.cloudyrock.mongock.runner.core.executor.MigrationExecutor.executeMigration(MigrationExecutor.java:63)
backend stdout | at com.github.cloudyrock.spring.v5.core.SpringMigrationExecutor.executeMigration(SpringMigrationExecutor.java:37)
backend stdout | at com.github.cloudyrock.mongock.runner.core.executor.MongockRunnerBase.execute(MongockRunnerBase.java:53)
backend stdout | … 26 common frames omitted
backend stdout |
editor stdout | 127.0.0.1 - - [20/Jan/2023:20:51:38 +0000] “GET / HTTP/1.1” 200 4519 “-” “curl/7.68.0”
editor stdout |

Any help would be greatly appreciated.

Amelia · January 20, 2023, 9:15pm

Hello! Please follow the steps in this Discord post. It contains all necessary information on how to update your instance and fix the issue. Let us know if this helps you get unblocked or if you face further errors.

novanglus · January 20, 2023, 9:27pm

Hey thanks for your help. I can’t seem to see that discord post. Can you send me it another way?

Amelia · January 20, 2023, 9:55pm

Getting error 503 (on the self-hosted instance) after upgrading to v1.9.x

Are you upgrading your instance from version < v1.9.2 to > v1.9.3?

If yes, then first upgrade to v1.9.2 first and then upgrade to the latest version.

Do you have a custom MongoDB setup for your instance?

If yes, you must upgrade your MongoDB to v5.0.14 and restart the Appsmith server.

Are you using a multi-container setup? (separate containers for server and client etc.)

Migrate to a single container setup and then try again. (docker : Migrate from Multi-Container setup | Appsmith, K8s: Migrate From Multi-Container Setup (k8s) | Appsmith)

If you’re still reading, that means you have a single container setup, and even though you’re on v1.9.2 you’re still getting 503.

This could mean your machine does not support mongoV5. A couple of symptoms are listed below:
- You can check if the appsmith container is unhealthy.
- The list of supported processors is listed here - Advanced Vector Extensions - Wikipedia.
- Server logs have an error like - Exception in monitor thread while connecting to server localhost:27017
Solution: Upgrade your machines to CPUs that support Mongo V5. If you can’t, unfortunately, you’ll have to revert to the older version. For now, you can only upgrade till v1.8.15.
- You can do this by changing the appsmith-ce to appsmith-ce:v1.8.15 in your installation/upgrade steps

novanglus · January 21, 2023, 12:31am

Thank you for this. Apparently my CPU does not support mongoDB 5 so I’m stuck on 1.8.15 for now. Thanks for your help!

arminus · February 18, 2023, 10:18am

I have the same problem since I run one of my Appsmith instances on a Pi4.

Apparently however, there is a solution to build MongoDB 5.x for a Pi4 here and a binary “release” here and here - couldn’t this be pulled into the arm64 images for the appsmith images?

arminus · February 21, 2023, 7:55am

Created [Bug]: Cannot upgrade beyond 1.8.5 on ARM due to MongoDB 5 · Issue #20818 · appsmithorg/appsmith · GitHub for this.