nav-img
Advanced

Why Can't I Find the Specified Python Environment After Adding the Python Package?

I cannot find the specified Python environment after adding the Python 3 package.

Set spark.yarn.appMasterEnv.PYSPARK_PYTHON to python3 in the conf file to specify the Python 3 environment for the compute cluster.

New clusters use the Python 3 environment by default.