There is no problem running the DDP based pytorch training program on the host computer,
After entering docker and running, the error “unhandled system error, NCCL version 2.7.8” appears.
Solution:
Add NCCL_DEBUG=INFO before the python -m torch.distributed.launch --nproc_per_node=4
You can see:
s215:623:649 [3] include/shm.h:48 NCCL WARN Error while creating shared memory segment nccl-shm-send-404da1ec128dc62d-0-3-2 (size 4104)When entering docker, just add --ipc=host.
Error message:
Error: VM option 'UseG1GC' is experimental and must be enabled via -XX:+UnlockExperimentalVMOptions.
Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.
According to the error message, go to the kafka decompression directory bin/kafka-run-class.sh file, open it with vim, then use /UseG1GC to search for this piece of configuration information, and delete this piece of configuration
and then restart Kafka. If there are similar problems, you can see if there are abnormal similar configurations, you can try to delete them, and remember to backup and save them.
scalac: Error: Error compiling the sbt component ‘compiler-interface-2.10.0-52.0’
sbt.internal.inc.CompileFailed: Error compiling the sbt component ‘compiler-interface-2.10.0-52.0’
at sbt.internal.inc.AnalyzingCompiler$.handleCompilationError$1(AnalyzingCompiler.scala:424)
at sbt.internal.inc.AnalyzingCompiler$.$anonfun$compileSources$5(AnalyzingCompiler.scala:441)
at sbt.internal.inc.AnalyzingCompiler$.$anonfun$compileSources$5$adapted(AnalyzingCompiler.scala:436)
at sbt.io.IO$.withTemporaryDirectory(IO.scala:490)
at sbt.io.IO$.withTemporaryDirectory(IO.scala:500)
at sbt.internal.inc.AnalyzingCompiler$.$anonfun$compileSources$2(AnalyzingCompiler.scala:436)
at sbt.internal.inc.AnalyzingCompiler$.$anonfun$compileSources$2$adapted(AnalyzingCompiler.scala:428)
at sbt.io.IO$.withTemporaryDirectory(IO.scala:490)
at sbt.io.IO$.withTemporaryDirectory(IO.scala:500)
at sbt.internal.inc.AnalyzingCompiler$.compileSources(AnalyzingCompiler.scala:428)
at org.jetbrains.jps.incremental.scala.local.CompilerFactoryImpl$.org$jetbrains$jps$incremental$scala$local$CompilerFactoryImpl$$getOrCompileInterfaceJar(CompilerFactoryImpl.scala:154)
at org.jetbrains.jps.incremental.scala.local.CompilerFactoryImpl.$anonfun$getScalac$1(CompilerFactoryImpl.scala:54)
at scala.Option.map(Option.scala:242)
at org.jetbrains.jps.incremental.scala.local.CompilerFactoryImpl.getScalac(CompilerFactoryImpl.scala:47)
at org.jetbrains.jps.incremental.scala.local.CompilerFactoryImpl.createCompiler(CompilerFactoryImpl.scala:25)
at org.jetbrains.jps.incremental.scala.local.CachingFactory.$anonfun$createCompiler$3(CachingFactory.scala:24)
at org.jetbrains.jps.incremental.scala.local.Cache.$anonfun$getOrUpdate$2(Cache.scala:20)
at scala.Option.getOrElse(Option.scala:201)
at org.jetbrains.jps.incremental.scala.local.Cache.getOrUpdate(Cache.scala:19)
at org.jetbrains.jps.incremental.scala.local.CachingFactory.createCompiler(CachingFactory.scala:24)
at org.jetbrains.jps.incremental.scala.local.LocalServer.doCompile(LocalServer.scala:43)
at org.jetbrains.jps.incremental.scala.local.LocalServer.compile(LocalServer.scala:30)
at org.jetbrains.jps.incremental.scala.remote.Main$.compileLogic(Main.scala:207)
at org.jetbrains.jps.incremental.scala.remote.Main$.$anonfun$handleCommand$1(Main.scala:190)
at org.jetbrains.jps.incremental.scala.remote.Main$.decorated$1(Main.scala:180)
at org.jetbrains.jps.incremental.scala.remote.Main$.handleCommand(Main.scala:187)
at org.jetbrains.jps.incremental.scala.remote.Main$.serverLogic(Main.scala:163)
at org.jetbrains.jps.incremental.scala.remote.Main$.nailMain(Main.scala:103)
at org.jetbrains.jps.incremental.scala.remote.Main.nailMain(Main.scala)
at sun.reflect.GeneratedMethodAccessor1.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.martiansoftware.nailgun.NGSession.run(NGSession.java:319)
Reason: The configuration scala.version in pom.xml is inconsistent with the locally installed Scala version, which is: 2.12.15.
Solution: Replace the version number.
Error reporting information
org.apache.ibatis.exceptions.PersistenceException:
### Error building SqlSession.
### The error may exist in com/itrs/mapper/OrderMapper.xml
### The error occurred while processing mapper_resultMap[orderMap]
### Cause: org.apache.ibatis.builder.BuilderException: Error parsing SQL Mapper Configuration. Cause: org.apache.ibatis.builder.BuilderException: Error parsing Mapper XML. The XML location is 'com/itrs/mapper/OrderMapper.xml'. Cause: org.apache.ibatis.builder.BuilderException: Error resolving class. Cause: org.apache.ibatis.type.TypeException: Could not resolve type alias 'order'. Cause: java.lang.ClassNotFoundException: Cannot find class: order
at org.apache.ibatis.exceptions.ExceptionFactory.wrapException(ExceptionFactory.java:30)
at org.apache.ibatis.session.SqlSessionFactoryBuilder.build(SqlSessionFactoryBuilder.java:80)
at org.apache.ibatis.session.SqlSessionFactoryBuilder.build(SqlSessionFactoryBuilder.java:64)
at com.itrs.test.MybatisTest.test1(MybatisTest.java:33)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:69)
at com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:33)
at com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:220)
at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:53)
Caused by: org.apache.ibatis.builder.BuilderException: Error parsing SQL Mapper Configuration. Cause: org.apache.ibatis.builder.BuilderException: Error parsing Mapper XML. The XML location is 'com/itrs/mapper/OrderMapper.xml'. Cause: org.apache.ibatis.builder.BuilderException: Error resolving class. Cause: org.apache.ibatis.type.TypeException: Could not resolve type alias 'order'. Cause: java.lang.ClassNotFoundException: Cannot find class: order
at org.apache.ibatis.builder.xml.XMLConfigBuilder.parseConfiguration(XMLConfigBuilder.java:121)
at org.apache.ibatis.builder.xml.XMLConfigBuilder.parse(XMLConfigBuilder.java:98)
at org.apache.ibatis.session.SqlSessionFactoryBuilder.build(SqlSessionFactoryBuilder.java:78)
... 24 more
Caused by: org.apache.ibatis.builder.BuilderException: Error parsing Mapper XML. The XML location is 'com/itrs/mapper/OrderMapper.xml'. Cause: org.apache.ibatis.builder.BuilderException: Error resolving class. Cause: org.apache.ibatis.type.TypeException: Could not resolve type alias 'order'. Cause: java.lang.ClassNotFoundException: Cannot find class: order
at org.apache.ibatis.builder.xml.XMLMapperBuilder.configurationElement(XMLMapperBuilder.java:122)
at org.apache.ibatis.builder.xml.XMLMapperBuilder.parse(XMLMapperBuilder.java:94)
at org.apache.ibatis.builder.xml.XMLConfigBuilder.mapperElement(XMLConfigBuilder.java:374)
at org.apache.ibatis.builder.xml.XMLConfigBuilder.parseConfiguration(XMLConfigBuilder.java:119)
... 26 more
Caused by: org.apache.ibatis.builder.BuilderException: Error resolving class. Cause: org.apache.ibatis.type.TypeException: Could not resolve type alias 'order'. Cause: java.lang.ClassNotFoundException: Cannot find class: order
at org.apache.ibatis.builder.BaseBuilder.resolveClass(BaseBuilder.java:118)
at org.apache.ibatis.builder.xml.XMLMapperBuilder.resultMapElement(XMLMapperBuilder.java:262)
at org.apache.ibatis.builder.xml.XMLMapperBuilder.resultMapElement(XMLMapperBuilder.java:253)
at org.apache.ibatis.builder.xml.XMLMapperBuilder.resultMapElements(XMLMapperBuilder.java:245)
at org.apache.ibatis.builder.xml.XMLMapperBuilder.configurationElement(XMLMapperBuilder.java:118)
... 29 more
Caused by: org.apache.ibatis.type.TypeException: Could not resolve type alias 'order'. Cause: java.lang.ClassNotFoundException: Cannot find class: order
at org.apache.ibatis.type.TypeAliasRegistry.resolveAlias(TypeAliasRegistry.java:120)
at org.apache.ibatis.builder.BaseBuilder.resolveAlias(BaseBuilder.java:149)
at org.apache.ibatis.builder.BaseBuilder.resolveClass(BaseBuilder.java:116)
... 33 more
Caused by: java.lang.ClassNotFoundException: Cannot find class: order
at org.apache.ibatis.io.ClassLoaderWrapper.classForName(ClassLoaderWrapper.java:200)
at org.apache.ibatis.io.ClassLoaderWrapper.classForName(ClassLoaderWrapper.java:89)
at org.apache.ibatis.io.Resources.classForName(Resources.java:261)
at org.apache.ibatis.type.TypeAliasRegistry.resolveAlias(TypeAliasRegistry.java:116)
... 35 more
Problem Description:
Here are: error parsing SQL mapper configuration, error parsing mapper XML and type typeexception: unable to resolve type alias “order”.
Cause analysis:
Unable to resolve type name ‘ order ‘, please see
sqlMapConfig.xml. It is found that the custom name order is omitted
Solution:
<!--Add a custom name-->
<typeAliases>
<typeAlias type="com.itrs.domain.User" alias="user"></typeAlias>
<typeAlias type="com.itrs.domain.Order" alias="order"></typeAlias>
</typeAliases>
Add a custom name to solve the problem
Project scenario:
Mybatis
Problem Description:
org.apache.ibatis.exceptions.PersistenceException:
### Error building SqlSession.
### The error may exist in com/itrs/mapper/UserMapper.xml
### Cause: org.apache.ibatis.builder.BuilderException: Error parsing SQL Mapper Configuration. Cause: java.io.IOException: Could not find resource com/itrs/mapper/UserMapper.xml
at org.apache.ibatis.exceptions.ExceptionFactory.wrapException(ExceptionFactory.java:30)
at org.apache.ibatis.session.SqlSessionFactoryBuilder.build(SqlSessionFactoryBuilder.java:80)
at org.apache.ibatis.session.SqlSessionFactoryBuilder.build(SqlSessionFactoryBuilder.java:64)
at com.itrs.test.MybatisTest.test1(MybatisTest.java:21)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:69)
at com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:33)
at com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:220)
at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:53)
Caused by: org.apache.ibatis.builder.BuilderException: Error parsing SQL Mapper Configuration. Cause: java.io.IOException: Could not find resource com/itrs/mapper/UserMapper.xml
at org.apache.ibatis.builder.xml.XMLConfigBuilder.parseConfiguration(XMLConfigBuilder.java:121)
at org.apache.ibatis.builder.xml.XMLConfigBuilder.parse(XMLConfigBuilder.java:98)
at org.apache.ibatis.session.SqlSessionFactoryBuilder.build(SqlSessionFactoryBuilder.java:78)
... 24 more
Caused by: java.io.IOException: Could not find resource com/itrs/mapper/UserMapper.xml
at org.apache.ibatis.io.Resources.getResourceAsStream(Resources.java:114)
at org.apache.ibatis.io.Resources.getResourceAsStream(Resources.java:100)
at org.apache.ibatis.builder.xml.XMLConfigBuilder.mapperElement(XMLConfigBuilder.java:372)
at org.apache.ibatis.builder.xml.XMLConfigBuilder.parseConfiguration(XMLConfigBuilder.java:119)
... 26 more
Cause analysis:
it is the path problem.
Solution:
When creating files under resources, use the following methods to create files
Don’t write it as com.xx.mapper.xxxxx.xml
1. Problem
HDL locationing Error:
Error: TF_DENORMALIZED_QUATERNION: Ignoring transform for
child_frame_id “odom” from authority “unknown_publisher” because of an
invalid quaternion in the transform (0.0 0.0 0.983641 0.122190)
2. Solution
From the error report can be seen, the tip is a quadratic number of problems, the key is to say that there is no normalization, the release of TF has strict requirements, the quadratic number must be normalized, that is, xyzw squared and equal to 1, if not equal to the error reported above, a simple calculation, the above error tip quadratic number is: (0.0 0.0 0.983641 0.122190), that is, 0.98364 1 2 + 0.12219 0 2 = 0.982480012981 0.983641^2 + 0.122190 ^2 = 0.9824800129810.983641^2 + 0.122190^2 = 0.982480012981, obviously not equal to 1, so there will be the above problem, the solution is also simple, before the output Do a normalization process can be.
Eigen::Quaterniond quatern = {0.0 0.0 0.983641 0.122190}; // w x y z
quatern.normalize();
odom_trans.transform.rotation.x = quatern.x();
odom_trans.transform.rotation.y = quatern.y();
odom_trans.transform.rotation.z = quatern.z();
odom_trans.transform.rotation.w = quatern.w();
odom_trans.header.stamp = stamp;
odom_trans.header.frame_id = "map";
odom_trans.child_frame_id = odom_child_frame_id;
tf_broadcaster.sendTransform(odom_trans);Kafka startup error
Solution:
server.properties file
listeners=PLAINTEXT://:9092(Note that there is no IP address in this line, I just had a problem with the IP) advertised.listeners=PLAINTEXT://114.115.20.100:9092 (need to let go of the comment)
when compiling the library on Linux, an error “xdo.c:29:34: fatal error: X11/extensions/XTest.h” occurred
Solution:
Install the dependent library as follows:
sudo apt-get install xorg-devSolution:
1. Make sure that the table is not less nor more, the judgment scheme gives 2 points
(1) the simplest, compare the number and size of files
(2) compare the contents of the file md5 and directory name, and the number of – (in my computer g:/md5 – to their own words)
Copy the original data away
2. use hbase shell to create 1 new table, the table structure should be the same as the one you had problems with before, and the same name
3. Go to the data directory under your new directory and find the .tabledesc
(local is hidden files, hdfs is not)
4. go into this directory will be . tableinfo.xxxx copy out
5. delete the data in your table, copy the data you copied, and put the tableinfo data into .tabledesc.
6. Execute hbase hbck -repair to repair the table, if one repair is not successful, you can execute it several times
Error message:
Using the network weights trained by FCN, UNET and deeplab, an error is reported when loading the model:
RuntimeError: Error(s) in loading state_dict for Model: Missing key(s) in state_dict....
Training environment:
CPU:Intel E5
GPU: 3090*2
Pytorch1. ten
Solution:
Solve the mismatch problem
method 1 (invalid): the dictionary cannot match strictly. When the model is loaded, use the following code to add false to solve the mismatch problem
net.load_state_dict(t.load(ckpt_path),False)
Note: this method unlocks the strict matching and can ignore the error report. However, the model loaded by this method has problems in the actual segmentation effect due to the mismatch of parameters. Use it with caution!
Method 2: the network is the same, the only difference is that NN is used in the training process The dataparallel() method calls two graphics cards for training, so before loading the model weight, the model is also suitable for dataparallel packaging, which can solve the above error reporting problem.
net = nn.DataParallel(net)
net = net.to(device)
Error: L6915E: Library reports error: __ use_no_semihosting was requested, but _
How to solve such problems?
The steps are as follows:
Find the target and select use MicroLIB
Mac command line execution:
brew update
The following error is reported:
Error:
homebrew-core is a shallow clone.
To `brew update`, first run:
git -C /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core fetch --unshallow
This command may take a few minutes to run due to the large size of the repository.
This restriction has been made on GitHub's request because updating shallow
clones is an extremely expensive operation due to the tree layout and traffic of
Homebrew/homebrew-core and Homebrew/homebrew-cask. We don't do this for you
automatically to avoid repeatedly performing an expensive unshallow operation in
CI systems (which should instead be fixed to not use shallow clones). Sorry for
the inconvenience!
The cause of this problem is usually the problem caused by the system update.
Solution: delete the homebrew core and then update it
cd /usr/local/Homebrew/Library/Taps/homebrew
rm -rf homebrew-core
brew upgrade