How does S3 permissioning work with least privilege

[knarukulla]

Does all namespaces in K8s cluster has "full permissions" on S3 bucket? How does service accounts being configured?

spark.kubernetes.authenticate.driver.serviceAccountName=spark

Do we have single k8s SA mapped to single IAM role? and its API will submit the spark jobs with this spec?

[yuchaoran2011]

Does all namespaces in K8s cluster has "full permissions" on S3 bucket?

Only namespaces that run Spark pods need the permissions

Do we have single k8s SA mapped to single IAM role?

Yes that's one way of doing it. Another way is to have multiple Spark application namespaces and assign a unique service account to each namespace. Then for each service account, bind to a different IAM role