Ignoring quotes in CSV while working in Athena , hive, spark SQL

The example below will ignore the the quates i.e if your data looks like

“a”, “b”, “c”

it will be processed as

a,b,c

Notice the serde name and the propreties

CREATE EXTERNAL TABLE IF NOT EXISTS  some_table (

a string, b string, c string

)ROW FORMAT SERDE ‘org.apache.hadoop.hive.serde2.OpenCSVSerde’

WITH SERDEPROPERTIES (

  “separatorChar” = “,”,

  “quoteChar”     = “\””

)

LOCATION ‘s3://some_bucket/’;

 

 

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s