2025-01-03后端服务崩溃

起因:

甲方反馈前端软件下载页面无法正常使用

排查:
通过观察前端异常现象很容易确定排查方向,前端一直在Loading,控制筛选异步请求发现后端服务无法正常请求

登录服务器查看:

发现服务容器崩了已停止,观察最后的控制台日志信息如下:

2025-01-02 19:02:33 [32m[schedule-pool-2] WARN  c.zaxxer.hikari.pool.ProxyConnection
 - master - Connection com.mysql.cj.jdbc.ConnectionImpl@68769506 marked as broken because of SQLSTATE(08S01), ErrorCode(0)
com.mysql.cj.jdbc.exceptions.CommunicationsException: Communications link failure

The last packet successfully received from the server was 2 milliseconds ago. The last packet sent successfully to the server was 2 milliseconds ago.
...
Caused by: java.io.EOFException: Can not read response from server. Expected to read 4 bytes, read 0 bytes before connection was unexpectedly lost.
...

2025-01-02 19:02:33 [schedule-pool-2] ERROR o.s.s.s.TaskUtils$LoggingErrorHandler
 - Unexpected error occurred in scheduled task
org.springframework.transaction.CannotCreateTransactionException: Could not open JDBC Connection for transaction
...

2025-01-02 19:02:33 [master housekeeper] WARN  com.zaxxer.hikari.pool.PoolBase
 - master - Failed to validate connection com.mysql.cj.jdbc.ConnectionImpl@4743b8fb (No operations allowed after connection closed.). Possibly consider using a shorter maxLifetime value.

2025-01-02 19:02:34 [master housekeeper] WARN  com.zaxxer.hikari.pool.PoolBase
 - master - Failed to validate connection com.mysql.cj.jdbc.ConnectionImpl@6f9c485a (No operations allowed after connection closed.). Possibly consider using a shorter maxLifetime value.

2025-01-02 19:02:37 [master housekeeper] WARN  com.zaxxer.hikari.pool.PoolBase
 - master - Failed to validate connection com.mysql.cj.jdbc.ConnectionImpl@6ca27b00 (No operations allowed after connection closed.). Possibly consider using a shorter maxLifetime value.

2025-01-02 19:02:38 [schedule-pool-2] WARN  com.zaxxer.hikari.pool.PoolBase
 - master - Failed to validate connection com.mysql.cj.jdbc.ConnectionImpl@1732e7e9 (No operations allowed after connection closed.). Possibly consider using a shorter maxLifetime value.

2025-01-02 19:02:38 [schedule-pool-2] WARN  com.zaxxer.hikari.pool.PoolBase
 - master - Failed to validate connection com.mysql.cj.jdbc.ConnectionImpl@4489348e (No operations allowed after connection closed.). Possibly consider using a shorter maxLifetime value.

2025-01-02 19:02:38 [schedule-pool-2] [31mWARN  com.zaxxer.hikari.pool.PoolBase
 - master - Failed to validate connection com.mysql.cj.jdbc.ConnectionImpl@75e9aebf (No operations allowed after connection closed.). Possibly consider using a shorter maxLifetime value.

2025-01-02 19:02:38 [schedule-pool-2] WARN  com.zaxxer.hikari.pool.PoolBase
 - master - Failed to validate connection com.mysql.cj.jdbc.ConnectionImpl@6ff9c556 (No operations allowed after connection closed.). Possibly consider using a shorter maxLifetime value.

2025-01-02 19:02:38 [schedule-pool-2] WARN  com.zaxxer.hikari.pool.PoolBase
 - master - Failed to validate connection com.mysql.cj.jdbc.ConnectionImpl@62a61719 (No operations allowed after connection closed.). Possibly consider using a shorter maxLifetime value.

2025-01-02 19:02:38 [schedule-pool-2] WARN  com.zaxxer.hikari.pool.PoolBase
 - master - Failed to validate connection com.mysql.cj.jdbc.ConnectionImpl@239b1d22 (No operations allowed after connection closed.). Possibly consider using a shorter maxLifetime value

可能的原因:
…

快速解决:

重启后端Docker容器 恢复正常

问题排查:

通过详细的日志发现并未出现异常流量

……

发现本地的MySql端口映射也分布不上了,时好时坏

各种尝试….

最后在1Panel 同步了一下数据库就能连上了…奇怪的

….