I am pretty naive to development in Spark and Scala.
I am able to set properties at runtime on spark session using the config method like below -
val spark = SparkSession.builder()
The above code will allow me to set properties on spark session level, but I want to set properties on a DataFrame level. Regarding this I have a few questions: