Incorrect Parameter Format Is Displayed When a Spark Task Is Submitted
Symptom
A user fails to submit a cluster task using Spark and a message is displayed indicating that the parameter format is incorrect.
Cause Analysis
- The command executed contains invalid characters.
- The owner and owner group of the uploaded JAR package are incorrect.
Procedure
- Run ./bin/spark-submit --class cn.interf.Test --master yarn-client Client installation directory/Spark/spark1-1.0-SNAPSHOT.jar; to check whether invalid characters are imported.
- If they are imported, modify the invalid characters and run the command again.
- After the command is executed again, other errors occur. Both the owner and the owner group of the JAR package are root.
- Change the owner and the owner group of the JAR package to omm:wheel.
Parent topic: Using Spark
- Symptom
- Cause Analysis
- Procedure