Spark Code -- Use date_format() to convert timestamp to String

Goal:

This article shares some Scala example codes to explain how to use date_format() to convert timestamp to String.

Solution:

data_format() is one function of org.apache.spark.sql.functions to convert data/timestamp to String.

This is the doc for datatime pattern.

Here is a simple example to show this in spark-sql way.

spark.sql("""
    SELECT current_timestamp() as ts, 
    date_format(current_timestamp(),"yyyy-MM-dd") as `yyyy-MM-dd`,
    date_format(current_timestamp(),"MMM") as `MMM`,
    date_format(current_timestamp(),"MMMM") as `MMMM`,
    date_format(current_timestamp(),"d") as `d`,
    date_format(current_timestamp(),"E") as `E`,
    date_format(current_timestamp(),"EEEE") as `EEEE`,
    date_format(current_timestamp(),"HH:mm:ss.S") as `HH:mm:ss.S`,
    date_format(current_timestamp(),"Z") as `Z`,
    date_format(current_timestamp(),"z") as `z`
""").show()

The output is:

+--------------------+----------+---+-------+---+---+--------+------------+-----+---+
|                  ts|yyyy-MM-dd|MMM|   MMMM|  d|  E|    EEEE|  HH:mm:ss.S|    Z|  z|
+--------------------+----------+---+-------+---+---+--------+------------+-----+---+
|2021-01-28 16:51:...|2021-01-28|Jan|January| 28|Thu|Thursday|16:51:48.802|-0800|PST|
+--------------------+----------+---+-------+---+---+--------+------------+-----+---+