Author Archives: Robins

How to Solve Pytorch eval Stuck Error

Question

The single card training is very fast. When it comes to eval, it doesn’t move after running a batch, and there is no error.

Tried, still not moving

1, change the pin_memory of valid_loader to False. if it is True, it will automatically load the data into pin_memory, which speeds up the data
transfer speed to GPU.
2, change num_workers to 1, some people say too many workers may lead to multi-process interlock, can reduce or not

 

Final Solution:

valid_loader:
pin_memory = true # this is very important. Before, people on the Internet said that changing false might solve the problem. My experiment proved that if you do not work, you can run normally by changing back to true.
num_workers=4
batch_size=8

train_loader:
pin_memory=True
num_workers=4
batch_size = 8
these parameters are the same as valid_loader

In general, first of all, the pin_memory of valid_loader is kept True, which is well understood, the data is automatically loaded into pin_memory, which speeds up the data transfer to the GPU and naturally speeds up the inference process. Then, the number of workers and batch_size is reduced, and both valid_loader and train_loader are reduced. pin_memory of train_loader is also kept True.

[Solved] AndroidStudio package Invalid keystore format Error: Execution failed for task ‘:examplecases:packageRelease’…

Post error information first

Execution failed for task ':examplecases:packageRelease'.
> A failure occurred while executing com.android.build.gradle.tasks.PackageAndroidArtifact$IncrementalSplitterRunnable
   > com.android.ide.common.signing.KeytoolException: Failed to read key aaa from store "C:\Users\a\Desktop\NewProject\aaa.jks": Invalid keystore format

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

 

 

Solution:

Open file -> settings.

Set the JDK version to 11, repack and solve the problem.

[Solved] LeNet Script Train Error: AttributeError: ‘DictIterator’ object has no attribute ‘get_next’

My training environment:

Windows10 64bit;

MindSpore1.5.0-beta;

CPU;

python3.9;

When training Mnist data set with LeNet , the following error occurs

How to solve this problem??

It’s true that the version is a bit old – the above use case needs to be modified if implemented on a newer version – from the current implementation of Iterator’s code, it no longer has the get_next method: https://gitee.com/mindspore/mindspore/blob/master/mindspore/python/mindspore/dataset/engine/iterators.py#L59

But it has __next__ method, so the above line can be modified, you can try it: > original: data = ds.get_next() > modified: data = next(ds)

[Solved] el-date-picker Error: Avoid mutating a prop directly since the value will be overwritten whenever the parent component re-renders.

Error Messages:
[Vue warn]: Avoid mutating a prop directly since the value will be overwritten whenever the parent component re-renders. Instead, use a data or computed property based on the prop’s value. Prop being mutated: “placement”
Reason: Version issue
Solution:
1. Downgrade the version.
2. Add align=”center” in el-date-picker.

<el-date-picker
          :v-model="time"
          type="daterange"
          range-separator="to"
          start-placeholder="Start Date"
          end-placeholder="End date"
        >
</el-date-picker>

[Solved] Jupyter Notebook Error: SparkException: Python worker failed to connect back

report errors

---------------------------------------------------------------------------
Py4JJavaError                             Traceback (most recent call last)
<ipython-input-24-bafca16b0526> in <module>
      8     return jobitem, ratingsRDD
      9 jobitem, jobRDD = preparJobdata(sc)
---> 10 jobRDD.collect() 

G:\Projects\python-3.6.4-amd64\lib\site-packages\pyspark\rdd.py in collect(self)
    947         """
    948         with SCCallSiteSync(self.context) as css:
--> 949             sock_info = self.ctx._jvm.PythonRDD.collectAndServe(self._jrdd.rdd())
    950         return list(_load_from_socket(sock_info, self._jrdd_deserializer))
    951 

G:\Projects\python-3.6.4-amd64\lib\site-packages\py4j\java_gateway.py in __call__(self, *args)
   1303         answer = self.gateway_client.send_command(command)
   1304         return_value = get_return_value(
-> 1305             answer, self.gateway_client, self.target_id, self.name)
   1306 
   1307         for temp_arg in temp_args:

G:\Projects\python-3.6.4-amd64\lib\site-packages\py4j\protocol.py in get_return_value(answer, gateway_client, target_id, name)
    326                 raise Py4JJavaError(
    327                     "An error occurred while calling {0}{1}{2}.\n".
--> 328                     format(target_id, ".", name), value)
    329             else:
    330                 raise Py4JError(

Py4JJavaError: An error occurred while calling z:org.apache.spark.api.python.PythonRDD.collectAndServe.
: org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 0.0 failed 1 times, most recent failure: Lost task 0.0 in stage 0.0 (TID 0) (192.168.101.68 executor driver): org.apache.spark.SparkException: Python worker failed to connect back.
	at org.apache.spark.api.python.PythonWorkerFactory.createSimpleWorker(PythonWorkerFactory.scala:182)
	at org.apache.spark.api.python.PythonWorkerFactory.create(PythonWorkerFactory.scala:107)
	at org.apache.spark.SparkEnv.createPythonWorker(SparkEnv.scala:119)
	at org.apache.spark.api.python.BasePythonRunner.compute(PythonRunner.scala:145)
	at org.apache.spark.api.python.PythonRDD.compute(PythonRDD.scala:65)
	at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:373)
	at org.apache.spark.rdd.RDD.iterator(RDD.scala:337)
	at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)
	at org.apache.spark.scheduler.Task.run(Task.scala:131)
	at org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$3(Executor.scala:497)
	at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1439)
	at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:500)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.net.SocketTimeoutException: Accept timed out
	at java.net.DualStackPlainSocketImpl.waitForNewConnection(Native Method)
	at java.net.DualStackPlainSocketImpl.socketAccept(DualStackPlainSocketImpl.java:135)
	at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:409)
	at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:199)
	at java.net.ServerSocket.implAccept(ServerSocket.java:545)
	at java.net.ServerSocket.accept(ServerSocket.java:513)
	at org.apache.spark.api.python.PythonWorkerFactory.createSimpleWorker(PythonWorkerFactory.scala:174)
	... 14 more

Driver stacktrace:
	at org.apache.spark.scheduler.DAGScheduler.failJobAndIndependentStages(DAGScheduler.scala:2253)
	at org.apache.spark.scheduler.DAGScheduler.$anonfun$abortStage$2(DAGScheduler.scala:2202)
	at org.apache.spark.scheduler.DAGScheduler.$anonfun$abortStage$2$adapted(DAGScheduler.scala:2201)
	at scala.collection.mutable.ResizableArray.foreach(ResizableArray.scala:62)
	at scala.collection.mutable.ResizableArray.foreach$(ResizableArray.scala:55)
	at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:49)
	at org.apache.spark.scheduler.DAGScheduler.abortStage(DAGScheduler.scala:2201)
	at org.apache.spark.scheduler.DAGScheduler.$anonfun$handleTaskSetFailed$1(DAGScheduler.scala:1078)
	at org.apache.spark.scheduler.DAGScheduler.$anonfun$handleTaskSetFailed$1$adapted(DAGScheduler.scala:1078)
	at scala.Option.foreach(Option.scala:407)
	at org.apache.spark.scheduler.DAGScheduler.handleTaskSetFailed(DAGScheduler.scala:1078)
	at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.doOnReceive(DAGScheduler.scala:2440)
	at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:2382)
	at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:2371)
	at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:49)
	at org.apache.spark.scheduler.DAGScheduler.runJob(DAGScheduler.scala:868)
	at org.apache.spark.SparkContext.runJob(SparkContext.scala:2202)
	at org.apache.spark.SparkContext.runJob(SparkContext.scala:2223)
	at org.apache.spark.SparkContext.runJob(SparkContext.scala:2242)
	at org.apache.spark.SparkContext.runJob(SparkContext.scala:2267)
	at org.apache.spark.rdd.RDD.$anonfun$collect$1(RDD.scala:1030)
	at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
	at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112)
	at org.apache.spark.rdd.RDD.withScope(RDD.scala:414)
	at org.apache.spark.rdd.RDD.collect(RDD.scala:1029)
	at org.apache.spark.api.python.PythonRDD$.collectAndServe(PythonRDD.scala:180)
	at org.apache.spark.api.python.PythonRDD.collectAndServe(PythonRDD.scala)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
	at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
	at py4j.Gateway.invoke(Gateway.java:282)
	at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
	at py4j.commands.CallCommand.execute(CallCommand.java:79)
	at py4j.GatewayConnection.run(GatewayConnection.java:238)
	at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.spark.SparkException: Python worker failed to connect back.
	at org.apache.spark.api.python.PythonWorkerFactory.createSimpleWorker(PythonWorkerFactory.scala:182)
	at org.apache.spark.api.python.PythonWorkerFactory.create(PythonWorkerFactory.scala:107)
	at org.apache.spark.SparkEnv.createPythonWorker(SparkEnv.scala:119)
	at org.apache.spark.api.python.BasePythonRunner.compute(PythonRunner.scala:145)
	at org.apache.spark.api.python.PythonRDD.compute(PythonRDD.scala:65)
	at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:373)
	at org.apache.spark.rdd.RDD.iterator(RDD.scala:337)
	at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)
	at org.apache.spark.scheduler.Task.run(Task.scala:131)
	at org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$3(Executor.scala:497)
	at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1439)
	at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:500)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	... 1 more
Caused by: java.net.SocketTimeoutException: Accept timed out
	at java.net.DualStackPlainSocketImpl.waitForNewConnection(Native Method)
	at java.net.DualStackPlainSocketImpl.socketAccept(DualStackPlainSocketImpl.java:135)
	at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:409)
	at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:199)
	at java.net.ServerSocket.implAccept(ServerSocket.java:545)
	at java.net.ServerSocket.accept(ServerSocket.java:513)
	at org.apache.spark.api.python.PythonWorkerFactory.createSimpleWorker(PythonWorkerFactory.scala:174)
	... 14 more

Solution:

The following variable environments are configured:

# Windows Hadoop variable environments are configured
HADOOP_HOME = F:\hadoop-common-2.2.0-bin-master\hadoop-common-2.2.0-bin-master

# Windows JDKvariable environments are configured
JAVA_HOME = F:\jdk-8u121-windows-x64_8.0.1210.13

# Windows Pysparkvariable environments are configured
PYSPARK_DRIVER_PYTHON = jupyter
PYSPARK_DRIVER_PYTHON_OPTS = notebook
PYSPARK_PYTHON = python

Remember to restart the computer after the configuration is completed!

How to Solve pinia Error in ts File

Use Pinia in ts file report an error:

getActivePinia was called with no active Pinia. Did you forget to install pinia

 

1. Create the store.ts file

import { createPinia } from "pinia";

const store = createPinia();

export { store };

2、 Introduced in main.ts

import { createApp } from "vue";
import "./style.css";
import "./main.css";
import App from "./App.vue";
import ElementPlus from "element-plus";
import Avue from "@smallwei/avue";
import "@smallwei/avue/lib/index.css";
import "@/assets/css/mapbox-gl.css";
import { store } from "@/pinia/index";    // Introduce the created pinia
import router from "@/router/index";
import "element-plus/dist/index.css";

const app = createApp(App);

app.use(router);
app.use(store);  // use pinia
app.use(ElementPlus);
app.use(Avue);
app.mount("#app");

3、 Create your own Pinia file

import { defineStore } from "pinia";

interface State {
  nodeArr: any[] | [];
  stencilLoading: boolean;
  selectNodeId: number;
  lcNodeData: LCDataType | {}
}

const defaultState: State = {
  nodeArr: [], // Node data in the drag and drop bar
  stencilLoading: true, // Loading of the drag bar
  selectNodeId: -1, // The node id of the selected drag bar
  lcNodeData: {}, // The LC data in the selected canvas
};

export const lcDataStore = defineStore("lcData", {
  state: () => {
    return defaultState;
  },
  getters: {},
  actions: {},
});

4、 Using Pinia in the TS file

import { lcDataStore } from "@/pinia/lcAssign";
import { store } from "@/pinia/index";

const lcStore = lcDataStore(store);   // Here you have to pass in the store

// After that, you can use the properties and methods inside the lcStore

[Solved] Docker Elasticsearch8.4.0 Error: Exception in thread “main” java.nio.file.FileSystemException

Exception in thread "main" java.nio.file.FileSystemException: /usr/share/elasticsearch/config/elasticsearch.yml.Dym72YkCRZ-GMAliqWE2IA.tmp -> /usr/share/elasticsearch/config/elasticsearch.yml: Device or resource busy
	at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:100)
	at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:106)
	at java.base/sun.nio.fs.UnixCopyFile.move(UnixCopyFile.java:416)
	at java.base/sun.nio.fs.UnixFileSystemProvider.move(UnixFileSystemProvider.java:266)
	at java.base/java.nio.file.Files.move(Files.java:1432)
	at org.elasticsearch.xpack.security.cli.AutoConfigureNode.fullyWriteFile(AutoConfigureNode.java:1127)
	at org.elasticsearch.xpack.security.cli.AutoConfigureNode.fullyWriteFile(AutoConfigureNode.java:1139)
	at org.elasticsearch.xpack.security.cli.AutoConfigureNode.execute(AutoConfigureNode.java:687)
	at org.elasticsearch.server.cli.ServerCli.autoConfigureSecurity(ServerCli.java:161)
	at org.elasticsearch.server.cli.ServerCli.execute(ServerCli.java:85)
	at org.elasticsearch.common.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:54)
	at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:85)
	at org.elasticsearch.cli.Command.main(Command.java:50)
	at org.elasticsearch.launcher.CliToolLauncher.main(CliToolLauncher.java:64)

Cause: it is estimated that there is a problem with the mounting of the configuration file

My solution: when docker starts, it can run successfully without mounting the configuration file

1. The address bar cannot access port 9200

It needs to be added in elasticsearch.yml in the container

http.host: 0.0.0.0

2. After the above method is configured, you need to input the account password to access the 9200 port

After finding some solutions, you need to configure the following contents in the configuration file in the container

xpack.secruity.enabled: false

After restarting the container, ES can be run successfully

Note: VIM needs to be installed before editing the files in the container

[Solved] Tomcat startup error: sub-container startup failed

Tomcat Startup Error:

When reviewing Java Web, I want to make my workspace cleaner, so I cleaned up the web projects left in the Tomcat installation directory webapps, and double-click start Bat file, the CMD window flashes by, and I feel strange at once. This situation is usually JAVA_Home environment variable is not configured correctly.


Then open start.bat in text mode, add “pause” after the end statement at the end, and double-click start.bat again after saving, the following results appear:

It is found that the paths are correct, indicating that there is no problem with my environment variable configuration.


Open start.bat again as text, call call "%EXECUTABLE%" start %CMD_LINE_ARGS% statement is changed to run. After saving, double-click start.bat, the following message appears:

A series of errors are shown later, and I only cut one illustratively.

The main information observed is that the sub-container failed to start, and a specified resource set is invalid. The service component necessary Catalina.start fails to start; When you look carefully, you find that it shows an invalid resource set path (at the end of the figure above). You can see that there is no relevant folder under this path.

So I looked at the server.xml file in the conf directory and saw the following configuration (only the code was intercepted):

<Service name="Catalina">
    <Engine defaultHost="localhost" name="Catalina">
        <Host appBase="webapps" autoDeploy="true" name="localhost" unpackWARs="true">
            <Context docBase="E:apache-tomcat-9.0.20webappsSSM" path="/SSM" reloadable="true" source="org.eclipse.jst.jee.server:SSM"/>
            <Context docBase="E:apache-tomcat-9.0.20webappsSpring" path="/Spring" reloadable="true" source="org.eclipse.jst.jee.server:Spring"/>
            <Context docBase="E:apache-tomcat-9.0.20webappsPressRelease" path="/PressRelease" reloadable="true" source="org.eclipse.jst.jee.server:PressRelease"/>
        </Host>
    </Engine>
</Service>

In the Context tag, docBase specifies the path of the project, which is accessed through the virtual path path. Because I deleted these three folders together when cleaning the project before, there was an error when Tomcat started.

Delete the contents of the three Context tags in the server.xml file, double-click start.bat after saving, and find that Tomcat has started normally and can access port 8080;

Or create the corresponding SSM, Spring, and PressRelease folders in the webapps directory, and Tomcat can also be started normally.

[Solved] Tomcat Error: Failed to start componen

I make a crud project of servlet + JSP, and start Tomcat with an error:

org.apache.catalina.LifecycleException: Failed to start component [StandardEngine[Catalina].StandardHost[localhost].StandardContext[/oa3]]

It was found that the path was not added ‘/’

//@WebServlet("dept/list")
//Add slash!
@WebServlet("/dept/list")
public class DeptServlet extends HttpServlet {

 

[Solved] Jar error on Linux: no main manifest attribute

1: Running jar package in Linux environment, unable to find main

Reason: pom is missing configuration

Add the following to the pom file

<build>
        <plugins>
            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
                <executions>
                    <execution>
                        <goals>
                            <goal>repackage</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    </build>

Mat Failed to Open a Memory Dump Error [How to Solve]

1. When mat opens a memory dump file, the following error is reported:

An internal error occurred durina: “Parsing heap dump from

2. At the same time, the size of the dump file is relatively large, which indicates that the memory of the mat software is small, and it is not enough to open the analysis

Big problem. The configuration needs to be adjusted.

3. Solution: adjust the configuration by installing

1) right click display package content

2) Click to open this folder

3) Continue to click to open this folder

4) Right-click to open this profile

5) Change this configuration to 5120m and save the configuration. Just reopen MAT.

4. Done!