dss linkis启动的时候。cg-engineplugin 不能上传bml。Hadoop 3.3.0

[yongster]

Hadoop版本为 3.3.0 Linkis是重新编译好了的。

当启动的时候就会报错。bml提交不到hdfs

报错日志和HDFS配置已上传

2022-08-05 11:41:55.620 [ERROR] [Linkis-Default-Scheduler-Thread-1       ] o.a.l.e.s.s.DefaultEngineConnResourceService (91) [$anonfun$run$4] - Failed to upload engine conn to bml, now exit! java.net.SocketException: Connection reset
	at java.net.SocketInputStream.read(SocketInputStream.java:209) ~[?:1.8.0_65]
	at java.net.SocketInputStream.read(SocketInputStream.java:141) ~[?:1.8.0_65]
	at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:137) ~[httpcore-4.4.14.jar:4.4.14]
	at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:153) ~[httpcore-4.4.14.jar:4.4.14]
	at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:280) ~[httpcore-4.4.14.jar:4.4.14]
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) ~[httpclient-4.5.13.jar:4.5.13]
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) ~[httpclient-4.5.13.jar:4.5.13]
	at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259) ~[httpcore-4.4.14.jar:4.4.14]
	at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:163) ~[httpcore-4.4.14.jar:4.4.14]
	at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:157) ~[httpclient-4.5.13.jar:4.5.13]
	at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273) ~[httpcore-4.4.14.jar:4.4.14]
	at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125) ~[httpcore-4.4.14.jar:4.4.14]
	at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) ~[httpclient-4.5.13.jar:4.5.13]
	at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) ~[httpclient-4.5.13.jar:4.5.13]
	at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) ~[httpclient-4.5.13.jar:4.5.13]
	at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) ~[httpclient-4.5.13.jar:4.5.13]
	at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) ~[httpclient-4.5.13.jar:4.5.13]
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) ~[httpclient-4.5.13.jar:4.5.13]
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:108) ~[httpclient-4.5.13.jar:4.5.13]
	at org.apache.linkis.httpclient.AbstractHttpClient.executeRequest(AbstractHttpClient.scala:349) ~[linkis-httpclient-1.1.1.jar:1.1.1]
	at org.apache.linkis.httpclient.AbstractHttpClient.addAttempt$1(AbstractHttpClient.scala:107) ~[linkis-httpclient-1.1.1.jar:1.1.1]
	at org.apache.linkis.httpclient.AbstractHttpClient.execute(AbstractHttpClient.scala:123) ~[linkis-httpclient-1.1.1.jar:1.1.1]
	at org.apache.linkis.httpclient.AbstractHttpClient.execute(AbstractHttpClient.scala:92) ~[linkis-httpclient-1.1.1.jar:1.1.1]
	at org.apache.linkis.bml.client.impl.HttpBmlClient.uploadResource(HttpBmlClient.scala:313) ~[linkis-bml-client-1.1.1.jar:1.1.1]
	at org.apache.linkis.engineplugin.server.service.DefaultEngineConnResourceService.uploadToBml(DefaultEngineConnResourceService.scala:59) ~[linkis-engineconn-plugin-server-1.1.1.jar:1.1.1]
	at org.apache.linkis.engineplugin.server.service.DefaultEngineConnResourceService.$anonfun$refresh$6(DefaultEngineConnResourceService.scala:145) ~[linkis-engineconn-plugin-server-1.1.1.jar:1.1.1]
	at org.apache.linkis.engineplugin.server.service.DefaultEngineConnResourceService.$anonfun$refresh$6$adapted(DefaultEngineConnResourceService.scala:141) ~[linkis-engineconn-plugin-server-1.1.1.jar:1.1.1]
	at scala.collection.IndexedSeqOptimized.foreach(IndexedSeqOptimized.scala:36) ~[scala-library-2.12.15.jar:?]
	at scala.collection.IndexedSeqOptimized.foreach$(IndexedSeqOptimized.scala:33) ~[scala-library-2.12.15.jar:?]
	at scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:198) ~[scala-library-2.12.15.jar:?]
	at org.apache.linkis.engineplugin.server.service.DefaultEngineConnResourceService.org$apache$linkis$engineplugin$server$service$DefaultEngineConnResourceService$$refresh(DefaultEngineConnResourceService.scala:141) ~[linkis-engineconn-plugin-server-1.1.1.jar:1.1.1]
	at org.apache.linkis.engineplugin.server.service.DefaultEngineConnResourceService$$anon$1.$anonfun$run$3(DefaultEngineConnResourceService.scala:87) ~[linkis-engineconn-plugin-server-1.1.1.jar:1.1.1]
	at org.apache.linkis.engineplugin.server.service.DefaultEngineConnResourceService$$anon$1.$anonfun$run$3$adapted(DefaultEngineConnResourceService.scala:85) ~[linkis-engineconn-plugin-server-1.1.1.jar:1.1.1]
	at scala.collection.immutable.Map$Map1.foreach(Map.scala:193) ~[scala-library-2.12.15.jar:?]
	at org.apache.linkis.engineplugin.server.service.DefaultEngineConnResourceService$$anon$1.$anonfun$run$2(DefaultEngineConnResourceService.scala:85) ~[linkis-engineconn-plugin-server-1.1.1.jar:1.1.1]
	at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23) ~[scala-library-2.12.15.jar:?]
	at org.apache.linkis.common.utils.Utils$.tryCatch(Utils.scala:40) ~[linkis-common-1.1.1.jar:1.1.1]
	at org.apache.linkis.engineplugin.server.service.DefaultEngineConnResourceService$$anon$1.$anonfun$run$1(DefaultEngineConnResourceService.scala:89) ~[linkis-engineconn-plugin-server-1.1.1.jar:1.1.1]
	at org.apache.linkis.engineplugin.server.service.DefaultEngineConnResourceService$$anon$1.$anonfun$run$1$adapted(DefaultEngineConnResourceService.scala:82) ~[linkis-engineconn-plugin-server-1.1.1.jar:1.1.1]
	at scala.collection.IndexedSeqOptimized.foreach(IndexedSeqOptimized.scala:36) ~[scala-library-2.12.15.jar:?]
	at scala.collection.IndexedSeqOptimized.foreach$(IndexedSeqOptimized.scala:33) ~[scala-library-2.12.15.jar:?]
	at scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:198) ~[scala-library-2.12.15.jar:?]
	at org.apache.linkis.engineplugin.server.service.DefaultEngineConnResourceService$$anon$1.run(DefaultEngineConnResourceService.scala:82) ~[linkis-engineconn-plugin-server-1.1.1.jar:1.1.1]
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_65]
	at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_65]
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) ~[?:1.8.0_65]
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) ~[?:1.8.0_65]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) ~[?:1.8.0_65]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) ~[?:1.8.0_65]
	at java.lang.Thread.run(Thread.java:745) ~[?:1.8.0_65]

linkis-cg-engineplugin.log
hdfs-site.txt
core-site.txt
core-site.txt
hdfs-site.txt

[ruY9527]

sh linkis-daemon.sh restart cg-engineplugin , restart the service separately, and then look at the log of the service and the log of linkis PS publicservice

---

单独重新启动下服务,再看该服务的log和linkis-ps-publicservice的log

[yongster]

重启了 还是报这个错误。 linkis-ps-publicservice 什么错误都没报。

linkis-ps-cs 报下面的错误。但是Eureka 是正常的。
2022-08-08 09:17:11.507 [INFO ] [DiscoveryClient-HeartbeatExecutor-0 ] c.n.d.s.t.d.RedirectingEurekaHttpClient (93) [execute] - Request execution error. endpoint=DefaultEndpoint{ serviceUrl='http://10.18.104.131:9600/eureka/} exception=org.apache.http.NoHttpResponseException: 10.18.104.131:9600 failed to respond stacktrace=com.sun.jersey.api.client.ClientHandlerException: org.apache.http.NoHttpResponseException: 10.18.104.131:9600 failed to respond
at com.sun.jersey.client.apache4.ApacheHttpClient4Handler.handle(ApacheHttpClient4Handler.java:187)
at com.sun.jersey.api.client.filter.GZIPContentEncodingFilter.handle(GZIPContentEncodingFilter.java:123)
at com.netflix.discovery.EurekaIdentityHeaderFilter.handle(EurekaIdentityHeaderFilter.java:27)
at com.sun.jersey.api.client.Client.handle(Client.java:652)
at com.sun.jersey.api.client.WebResource.handle(WebResource.java:682)
at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74)
at com.sun.jersey.api.client.WebResource$Builder.put(WebResource.java:529)
at com.netflix.discovery.shared.transport.jersey.AbstractJerseyEurekaHttpClient.sendHeartBeat(AbstractJerseyEurekaHttpClient.java:103)
at com.netflix.discovery.shared.transport.decorator.EurekaHttpClientDecorator$3.execute(EurekaHttpClientDecorator.java:92)
at com.netflix.discovery.shared.transport.decorator.MetricsCollectingEurekaHttpClient.execute(MetricsCollectingEurekaHttpClient.java:73)
at com.netflix.discovery.shared.transport.decorator.EurekaHttpClientDecorator.sendHeartBeat(EurekaHttpClientDecorator.java:89)
at com.netflix.discovery.shared.transport.decorator.EurekaHttpClientDecorator$3.execute(EurekaHttpClientDecorator.java:92)
at com.netflix.discovery.shared.transport.decorator.RedirectingEurekaHttpClient.execute(RedirectingEurekaHttpClient.java:91)
at com.netflix.discovery.shared.transport.decorator.EurekaHttpClientDecorator.sendHeartBeat(EurekaHttpClientDecorator.java:89)
at com.netflix.discovery.shared.transport.decorator.EurekaHttpClientDecorator$3.execute(EurekaHttpClientDecorator.java:92)
at com.netflix.discovery.shared.transport.decorator.RetryableEurekaHttpClient.execute(RetryableEurekaHttpClient.java:120)
at com.netflix.discovery.shared.transport.decorator.EurekaHttpClientDecorator.sendHeartBeat(EurekaHttpClientDecorator.java:89)
at com.netflix.discovery.shared.transport.decorator.EurekaHttpClientDecorator$3.execute(EurekaHttpClientDecorator.java:92)
at com.netflix.discovery.shared.transport.decorator.SessionedEurekaHttpClient.execute(SessionedEurekaHttpClient.java:77)
at com.netflix.discovery.shared.transport.decorator.EurekaHttpClientDecorator.sendHeartBeat(EurekaHttpClientDecorator.java:89)
at com.netflix.discovery.DiscoveryClient.renew(DiscoveryClient.java:893)
at com.netflix.discovery.DiscoveryClient$HeartbeatThread.run(DiscoveryClient.java:1457)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
2022-08-08 09:17:11.507 [INFO ] [DiscoveryClient-HeartbeatExecutor-0 ] c.n.d.s.t.d.RedirectingEurekaHttpClient (93) [execute] - Request execution error. endpoint=DefaultEndpoint{ serviceUrl='http://10.18.104.131:9600/eureka/} exception=org.apache.http.NoHttpResponseException: 10.18.104.131:9600 failed to respond stacktrace=com.sun.jersey.api.client.ClientHandlerException: org.apache.http.NoHttpResponseException: 10.18.104.131:9600 failed to respond
at com.sun.jersey.client.apache4.ApacheHttpClient4Handler.handle(ApacheHttpClient4Handler.java:187)
at com.sun.jersey.api.client.filter.GZIPContentEncodingFilter.handle(GZIPContentEncodingFilter.java:123)
at com.netflix.discovery.EurekaIdentityHeaderFilter.handle(EurekaIdentityHeaderFilter.java:27)
at com.sun.jersey.api.client.Client.handle(Client.java:652)
at com.sun.jersey.api.client.WebResource.handle(WebResource.java:682)
at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74)
at com.sun.jersey.api.client.WebResource$Builder.put(WebResource.java:529)
at com.netflix.discovery.shared.transport.jersey.AbstractJerseyEurekaHttpClient.sendHeartBeat(AbstractJerseyEurekaHttpClient.java:103)
at com.netflix.discovery.shared.transport.decorator.EurekaHttpClientDecorator$3.execute(EurekaHttpClientDecorator.java:92)
at com.netflix.discovery.shared.transport.decorator.MetricsCollectingEurekaHttpClient.execute(MetricsCollectingEurekaHttpClient.java:73)
at com.netflix.discovery.shared.transport.decorator.EurekaHttpClientDecorator.sendHeartBeat(EurekaHttpClientDecorator.java:89)
at com.netflix.discovery.shared.transport.decorator.EurekaHttpClientDecorator$3.execute(EurekaHttpClientDecorator.java:92)
at com.netflix.discovery.shared.transport.decorator.RedirectingEurekaHttpClient.execute(RedirectingEurekaHttpClient.java:91)
at com.netflix.discovery.shared.transport.decorator.EurekaHttpClientDecorator.sendHeartBeat(EurekaHttpClientDecorator.java:89)
at com.netflix.discovery.shared.transport.decorator.EurekaHttpClientDecorator$3.execute(EurekaHttpClientDecorator.java:92)
at com.netflix.discovery.shared.transport.decorator.RetryableEurekaHttpClient.execute(RetryableEurekaHttpClient.java:120)
at com.netflix.discovery.shared.transport.decorator.EurekaHttpClientDecorator.sendHeartBeat(EurekaHttpClientDecorator.java:89)
at com.netflix.discovery.shared.transport.decorator.EurekaHttpClientDecorator$3.execute(EurekaHttpClientDecorator.java:92)
at com.netflix.discovery.shared.transport.decorator.SessionedEurekaHttpClient.execute(SessionedEurekaHttpClient.java:77)
at com.netflix.discovery.shared.transport.decorator.EurekaHttpClientDecorator.sendHeartBeat(EurekaHttpClientDecorator.java:89)
at com.netflix.discovery.DiscoveryClient.renew(DiscoveryClient.java:893)
at com.netflix.discovery.DiscoveryClient$HeartbeatThread.run(DiscoveryClient.java:1457)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)

[yongster]

linkis 下的所有日志
log.zip

[ruY9527]

linkis 下的所有日志 log.zip

hi,Please check dss-gateway-support scala version and linkis scala agreement.