pyspark.sql.DataFrameWriter.jdbc

DataFrameWriter.jdbc(url, table, mode=None, properties=None)[source]

Saves the content of the DataFrame to an external database table via JDBC.

New in version 1.4.0.

Parameters
tablestr

Name of the table in the external database.

modestr, optional

specifies the behavior of the save operation when data already exists.

  • append: Append contents of this DataFrame to existing data.

  • overwrite: Overwrite existing data.

  • ignore: Silently ignore this operation if data already exists.

  • error or errorifexists (default case): Throw an exception if data already exists.

propertiesdict

a dictionary of JDBC database connection arguments. Normally at least properties “user” and “password” with their corresponding values. For example { ‘user’ : ‘SYSTEM’, ‘password’ : ‘mypassword’ }

Other Parameters
Extra options

For the extra options, refer to Data Source Option in the version you use.

Notes

Don’t create too many partitions in parallel on a large cluster; otherwise Spark might crash your external database systems.

pyspark.sql.DataFrameWriter.insertInto pyspark.sql.DataFrameWriter.json