Tag Archives: Yarn

[Solved] Internal Error: EPERM: operation not permitted, open

When installing yarn on nodejs (my version: 16.17), after executing the command: corepack enable, I got an error: Internal Error: EPERM: operation not permitted, open 'Y:\nodejs-v16.17.0-x64\pnpm, The operation is not allowed, I thought of the previous execution of rm -rf /* with linux normal user will also prompt not allowed to execute the operation, and change to root (administrator) account to have higher privileges to execute the command, so I guess it should be the same problem, so I tried to run the cmd command window as administrator, and execute the command: corepack enable again, it succeeded It worked:

[Solved] MSBUILD : error MSB3428: Could not load the Visual C++ component “VCBuild.exe“

MSBUILD : error MSB3428: Could not load the Visual C++ component “VCBuild.exe”.

This problem occurs in the install front-end project

MSBUILD : error MSB3428: Could not load the Visual C++ component “VCBuild.exe”…

Solution:

1. The administrator opens CMD and sets the agent (optional, if you have one)

set HTTP_PROXY=http://127.0.0.1:1080
set HTTPS_PROXY=http://127.0.0.1:1080

2. Install windows build tool

npm install -g --production windows-build-tools

3. If the card is completely stuck, wait for one minute, and then enter C:\Users\Username\.windows-build-tools

If there is a file in the figure below, the download is successful, and if there is no file, the download fails

If the download fails, download it manually and extract it to C:\Users\Username\

The final status is shown in the figure:

5 Configure system environment variables (Setting -> advanced system settings -> environment variables)

Append to path item

​ C:\Users\13261.windows-build-tools

​ C:\Users\13261.windows-build-tools\python27

6. change the file name

Because the problem I encountered was MSBUILD : error MSB3428: Could not load the Visual C++ component “VCBuild.exe”…

The project will use the executable file VCBuild.exe to build

So I made a copy of vs_BuildTools.exe and modified it to VCBuild.exe, so that cmd can use “VCBuild”!

Because I have already installed python3 or higher, the command is also “python”

so python2.7 I will also modify the executable to python.exe -> python2.exe, pythonw.exe -> python2w.exe

TIP: If your front-end project will be built with commands like “python2” or “python3” instead of “python” to execute python code, then you also need to rename the executable

Translated with www.DeepL.com/Translator (free version)

[Solved] The main method caused an error: Could not deploy Yarn job cluster.

org.apache.flink.client.program.ProgramInvocationException: The main method caused an error: Could not deploy Yarn job cluster.

Caused by: org.apache.flink.client.deployment.ClusterDeploymentException: Could not deploy Yarn job cluster.

Caused by: org.apache.flink.yarn.YarnClusterDescriptor$YarnDeploymentException: The YARN application unexpectedly switched to state FAILED during deployment.

Diagnostics from YARN: Application application_1640140324841_0003 failed 1 times (global limit =4; local limit is =1) due to AM Container for appattempt_1640140324841_0003_000001 exited with  exitCode: 1
Failing this attempt.Diagnostics: [2021-12-22 11:23:34.422]Exception from container-launch.
Container id: container_e44_1640140324841_0003_01_000001
Exit code: 1
Shell output: main : command provided 1
main : run as user is etl_admin
main : requested yarn user is etl_admin
Getting exit code file…
Creating script paths…
Writing pid file…
Writing to tmp file /data1/yarn/nm/nmPrivate/application_1640140324841_0003/container_e44_1640140324841_0003_01_000001/container_e44_1640140324841_0003_01_000001.pid.tmp
Writing to cgroup task files…
Creating local dirs…
Launching container…

 

Solution:
Look is the flink version of the idea is 1.11.0, the flink version on the cluster is 1.13.1
Directly upgrade the flink version in the idea to 1.13.1, done!

Error in configuring Hadoop 3.1.3: attempting to operate on yarn nodemanager as root error

This may occur when HDFS and yarn services are turned on, and when HDFS and yarn services are turned off using scripts, the

solution may also occur

Add the following parameters to the top of start-dfs.sh and stop-dfs.sh (in SBIN of Hadoop installation directory)

HDFS_DATANODE_USER=root
HADOOP_SECURE_DN_USER=hdfs
HDFS_NAMENODE_USER=root
HDFS_SECONDARYNAMENODE_USER=root

Add the following parameters to the top of start-yarn.sh and stop-yarn.sh (in SBIN of Hadoop installation directory)

YARN_RESOURCEMANAGER_USER=root
HADOOP_SECURE_DN_USER=yarn
YARN_NODEMANAGER_USER=root

Successfully solved the problem

[Solved] gyp verb `which` failed Error: not found: python2

Problems with yarn install

1. No yarn
NPM install – G yarn
2. An error is reported and phthon is not found

yarn install v1.22.17
warning package-lock.json found. Your project contains lock files generated by tools other than Yarn. It is advised not to mix package managers in order to avoid resolution inconsistencies caused by unsynch
ronized lock files. To clear this warning, remove package-lock.json.
[1/4] Resolving packages...
[2/4] Fetching packages...
[3/4] Linking dependencies...
warning " > [email protected]" has unmet peer dependency "css-loader@*".
warning " > [email protected]" has unmet peer dependency "webpack@^3.0.0 || ^4.1.0 || ^5.0.0-0".
warning " > [email protected]" has unmet peer dependency "webpack@^2.0.0 || ^3.0.0 || ^4.0.0".
warning " > [email protected]" has unmet peer dependency "webpack@^3.0.0 || ^4.0.0".
[4/4] Building fresh packages...
[7/7] ⠈ ant-design-vue
[-/7] ⠁ waiting...
[6/7] ⠁ node-sass
[4/7] ⠁ yorkie
error E:\HuaDi\Projects\mrpc-inline\ant-design-vue-supermap\node_modules\node-sass: Command failed.
Exit code: 1
Command: node scripts/build.js
Arguments:
Directory: E:\HuaDi\Projects\mrpc-inline\ant-design-vue-supermap\node_modules\node-sass
Output:
Building: E:\develop\node.js\node.exe E:\HuaDi\Projects\mrpc-inline\ant-design-vue-supermap\node_modules\node-gyp\bin\node-gyp.js rebuild --verbose --libsass_ext= --libsass_cflags= --libsass_ldflags= --libs
ass_library=
gyp info it worked if it ends with ok
gyp verb cli [
gyp verb cli   'E:\\develop\\node.js\\node.exe',
gyp verb cli   'E:\\HuaDi\\Projects\\mrpc-inline\\ant-design-vue-supermap\\node_modules\\node-gyp\\bin\\node-gyp.js',
gyp verb cli   'rebuild',
gyp verb cli   '--verbose',
gyp verb cli   '--libsass_ext=',
gyp verb cli   '--libsass_cflags=',
gyp verb cli   '--libsass_ldflags=',
gyp verb cli   '--libsass_library='
gyp verb cli ]
gyp info using [email protected]
gyp info using [email protected] | win32 | x64
gyp verb command rebuild []
gyp verb command clean []
gyp verb clean removing "build" directory
gyp verb command configure []
gyp verb check python checking for Python executable "python2" in the PATH
gyp verb `which` failed Error: not found: python2
gyp verb `which` failed     at getNotFoundError (E:\HuaDi\Projects\mrpc-inline\ant-design-vue-supermap\node_modules\which\which.js:13:12)
gyp verb `which` failed     at F (E:\HuaDi\Projects\mrpc-inline\ant-design-vue-supermap\node_modules\which\which.js:68:19)
gyp verb `which` failed     at E (E:\HuaDi\Projects\mrpc-inline\ant-design-vue-supermap\node_modules\which\which.js:80:29)
gyp verb `which` failed     at E:\HuaDi\Projects\mrpc-inline\ant-design-vue-supermap\node_modules\which\which.js:89:16
gyp verb `which` failed     at E:\HuaDi\Projects\mrpc-inline\ant-design-vue-supermap\node_modules\isexe\index.js:42:5
gyp verb `which` failed     at E:\HuaDi\Projects\mrpc-inline\ant-design-vue-supermap\node_modules\isexe\windows.js:36:5
gyp verb `which` failed     at FSReqCallback.oncomplete (fs.js:175:21)
gyp verb `which` failed  python2 Error: not found: python2
gyp verb `which` failed     at getNotFoundError (E:\HuaDi\Projects\mrpc-inline\ant-design-vue-supermap\node_modules\which\which.js:13:12)
gyp verb `which` failed     at F (E:\HuaDi\Projects\mrpc-inline\ant-design-vue-supermap\node_modules\which\which.js:68:19)
gyp verb `which` failed     at E (E:\HuaDi\Projects\mrpc-inline\ant-design-vue-supermap\node_modules\which\which.js:80:29)
gyp verb `which` failed     at E:\HuaDi\Projects\mrpc-inline\ant-design-vue-supermap\node_modules\which\which.js:89:16
gyp verb `which` failed     at E:\HuaDi\Projects\mrpc-inline\ant-design-vue-supermap\node_modules\isexe\index.js:42:5
gyp verb `which` failed     at E:\HuaDi\Projects\mrpc-inline\ant-design-vue-supermap\node_modules\isexe\windows.js:36:5
gyp verb `which` failed     at FSReqCallback.oncomplete (fs.js:175:21) {
gyp verb `which` failed   code: 'ENOENT'
gyp verb `which` failed }
gyp verb check python checking for Python executable "python" in the PATH
gyp verb `which` failed Error: not found: python
gyp verb `which` failed     at getNotFoundError (E:\HuaDi\Projects\mrpc-inline\ant-design-vue-supermap\node_modules\which\which.js:13:12)
gyp verb `which` failed     at F (E:\HuaDi\Projects\mrpc-inline\ant-design-vue-supermap\node_modules\which\which.js:68:19)
gyp verb `which` failed     at E (E:\HuaDi\Projects\mrpc-inline\ant-design-vue-supermap\node_modules\which\which.js:80:29)
gyp verb `which` failed     at E:\HuaDi\Projects\mrpc-inline\ant-design-vue-supermap\node_modules\which\which.js:89:16
gyp verb `which` failed     at E:\HuaDi\Projects\mrpc-inline\ant-design-vue-supermap\node_modules\isexe\index.js:42:5
gyp verb `which` failed     at E:\HuaDi\Projects\mrpc-inline\ant-design-vue-supermap\node_modules\isexe\windows.js:36:5
gyp verb `which` failed     at FSReqCallback.oncomplete (fs.js:175:21)
gyp verb `which` failed  python Error: not found: python
gyp verb `which` failed     at getNotFoundError (E:\HuaDi\Projects\mrpc-inline\ant-design-vue-supermap\node_modules\which\which.js:13:12)
gyp verb `which` failed     at F (E:\HuaDi\Projects\mrpc-inline\ant-design-vue-supermap\node_modules\which\which.js:68:19)
gyp verb `which` failed     at E (E:\HuaDi\Projects\mrpc-inline\ant-design-vue-supermap\node_modules\which\which.js:80:29)
gyp verb `which` failed     at E:\HuaDi\Projects\mrpc-inline\ant-design-vue-supermap\node_modules\which\which.js:89:16
gyp verb `which` failed     at E:\HuaDi\Projects\mrpc-inline\ant-design-vue-supermap\node_modules\isexe\index.js:42:5
gyp verb `which` failed     at E:\HuaDi\Projects\mrpc-inline\ant-design-vue-supermap\node_modules\isexe\windows.js:36:5
gyp verb `which` failed     at FSReqCallback.oncomplete (fs.js:175:21) {
gyp verb `which` failed   code: 'ENOENT'
gyp verb `which` failed }
gyp verb could not find "python". checking python launcher
gyp verb could not find "python". guessing location
gyp verb ensuring that file exists: C:\Python27\python.exe
gyp ERR! configure error
gyp ERR! stack Error: Can't find Python executable "python", you can set the PYTHON env variable.
gyp ERR! stack     at PythonFinder.failNoPython (E:\HuaDi\Projects\mrpc-inline\ant-design-vue-supermap\node_modules\node-gyp\lib\configure.js:484:19)
gyp ERR! stack     at PythonFinder.<anonymous> (E:\HuaDi\Projects\mrpc-inline\ant-design-vue-supermap\node_modules\node-gyp\lib\configure.js:509:16)
gyp ERR! stack     at callback (E:\HuaDi\Projects\mrpc-inline\ant-design-vue-supermap\node_modules\graceful-fs\polyfills.js:295:20)
gyp ERR! stack     at FSReqCallback.oncomplete (fs.js:175:21)
gyp ERR! System Windows_NT 10.0.19042
gyp ERR! command "E:\\develop\\node.js\\node.exe" "E:\\HuaDi\\Projects\\mrpc-inline\\ant-design-vue-supermap\\node_modules\\node-gyp\\bin\\node-gyp.js" "rebuild" "--verbose" "--libsass_ext=" "--libsass_cfla
gs=" "--libsass_ldflags=" "--libsass_library="
gyp ERR! cwd E:\HuaDi\Projects\mrpc-inline\ant-design-vue-supermap\node_modules\node-sass
gyp ERR! node -v v14.1.0

Solution:
1. Install low version node.js
2.npm install -g node-gyp
3.yarn install

[Solved] Yarn install Error: “error Found incompatible module.”

Today, an error was reported when installing the dependency using yarn install. After consulting the online resources, the problem was solved. Here is a record.

The error content is as follows: no matching version was found

D:\Web Storm-Workspace\xxxx>yarn install
yarn install v1.22.15
[1/6] Validating package.json...
[2/6] Resolving packages...
[3/6] Fetching packages...
info [email protected]: The platform "win32" is incompatible with this module.

info "[email protected]" is an optional dependency and failed compatibility ch
eck. Excluding it from installation.
error [email protected]: The engine "node" is incompatible with this module.
 Expected version "^10.13.0 || ^12.13.0 || ^14.15.0 || >=15.0.0". Got "14.8
.0"
error Found incompatible module.
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about
this command.

Enter the following line of code: yarn config set ignore – engines true

The problem is solved.

D:\Web Storm-Workspace\xxxx>yarn config set ignore
-engines true

yarn config v1.22.15
success Set "ignore-engines" to "true".
Done in 0.11s.

Re-execute yarn Install, you will find that yarn is successfully installed!!!

[Solved] Error: unable to verify the first certificate

Many options have been set on win10, but the error is still reported: unable to verify the first certificate

$ yarn config list
yarn config v1.22.4
info yarn config
{
  'version-tag-prefix': 'v',
  'version-git-tag': true,
  'version-commit-hooks': true,
  'version-git-sign': false,
  'version-git-message': 'v%s',
  'init-version': '1.0.0',
  'init-license': 'MIT',
  'save-prefix': '^',
  'bin-links': true,
  'ignore-scripts': false,
  'ignore-optional': false,
  registry: 'http://npm.xxxxx.com',
  'strict-ssl': false,
  'user-agent': 'yarn/1.22.4 npm/?node/v12.22.1 win32 x64',
  NODE_TLS_REJECT_UNAUTHORIZED: 0,
  lastUpdateCheck: 1629361331003
}
info npm config
{
  registry: 'http://npm.xxxxx.com',
  CYPRESS_INSTALL_BINARY: 'https://npm.taobao.org/mirrors/cypress/5.6.0/darwin-x64/cypress.zip',
  '//arch': 'ia32',
  '//registry': 'https://registry.npm.taobao.org',
  electron_mirror: 'https://npm.taobao.org/mirrors/electron/',
  CHROMEDRIVER_CDNURL: 'http://npm.taobao.org/mirrors/chromedriver/',
  sass_binary_site: 'https://npm.taobao.org/mirrors/node-sass/',
  phantomjs_cdnurl: 'https://npm.taobao.org/mirrors/phantomjs',
  'strict-ssl': false,
  proxy: null,
  'https-proxy': null,
  '//npm.xxxxx.com/:_authToken': '8624c0fe-ea43-4f30-b113-617f22e9c6c6'
}

Solution:

NODE_TLS_REJECT_UNAUTHORIZED=0 yarn

Gyp err! Build error stack error

Error content:

gyp ERR! build error
gyp ERR! stack Error: `C:\\Program Files (x86)\\Microsoft Visual Studio\\2017\\BuildTools\\MSBuild\\15.0\\Bin\\MSBuild.exe` failed with exit code: 1

Repeat several times and still report an error.
Check the problem. The yarn version is normal, and the image is also a Taobao image.

solve:

1. Execute NPMI – G node gyp command
2. Delete directory/node_ All files under modules
3. Re install.

In addition, it can be seen on the Internet that it may be due to the node version. The node version needs to be lowered. I didn’t try this method. I don’t know if it’s feasible.

@Solution of error failed to get response from / Vue cli version marker

thank: CSDN@beeegood Questions provided.

Today, they encountered a very interesting bug. When they created a project with @ Vue/cli, they reported an error, which they had never seen before:

At first, I thought it was a version problem. After all, the error message said update, but the version of CLI was the latest. After asking, the node and NPM versions were also the latest (12.16.1, which was the latest as of the time I wrote this article). Most importantly, there was no old version of Vue cli

That’s very interesting. According to convention, when encountering the front-end problem, the first reaction is to unload and reload

npm uninstall -g @vue/cli
npm cache clean --force
npm install -g @vue/cli

However, it’s useless. After checking for a long time, we can’t find the relevant error report on the Internet, which is very embarrassing.

Later, I noticed that there is an output of yarn below. Is the built-in yarn in cli?But this shouldn’t be:

although I think it’s incredible, I still decided to look at the version of yarn. Yarn is highly suspected. Sure enough:

the problem was found. But what is this?Hadoop？

Later, I remembered that yarn is also a part of Hadoop, which is used to schedule resources:

Yarn: runtime.ContainerExecutionException : launch container failed

introduction:

After the spark submit submits the task, the code of the dirver side is executed normally, but the program gets stuck in the exciter stage and frequently reports errors until the task fails

 

location:

The log failed location prints a lot of warning:

The initial job did not accept any resources. Please check the cluster UI to make sure that the worker process is registered and has enough resources. The initial analysis is about resources. Then yarn logs pull down the logs to see:

The initial heap size of the JVM exceeds the maximum heap size. Check the task environment to find out the truth

 

solve:

The initial memory of the JVM – XMS (the minimum heap value of heap memory) requires 13g, but Excutor.memory Only 12g is given, so the above problem appears. Modify the script to keep it stable excutor.mermory =The size of – XMS is OK, the problem is solved~

Tips: generally – XMS – Xmx (the maximum heap value of heap memory) can be set the same.

Oracle recommends setting the minimum heap size (-Xms)equal to the maximum heap size (-Xmx) to minimize garbage collections.

 

Ambari failed to monitor yarn ResourceManager, restart invalid

Question:

Yarn ResourceManager restart is invalid.

analysis:

It could be Hadoop yarn monitored by ambari- resourcemanager.pid The file does not properly record the process ID of ResourceManager.

solve:

    enter the directory/var/run/Hadoop yarn/yarn; view Hadoop yarn- resourcemanager.pid The process ID of ResourceManager recorded in; JPS view the ID number of ResourceManager running; if it is not equal, copy and fill in the process ID of ResourceManager viewed by JPS to Hadoop yarn- resourcemanager.pid Wait a few seconds, cluster ambari The display is normal;