Wenn Sie com.bizo.hive.serde.csv.CSVSerde - hadoop, hive verwenden, wird alles, was als String importiert wurde, mit Hive versehen

Ich habe den stackoverflow-User-Dump heruntergeladen, damit ich mich an Hive gewöhnen konnte, und habe die XML-Datei in eine CSV-Datei konvertiert. Ich verwende Folgendes:

add jar /home/cloudera/csv-serde.jar;
drop table stackoverflow_users;

CREATE external TABLE IF NOT EXISTS stackoverflow_users (CreationDate timestamp, Views BIGINT,
AccountId BIGINT, AboutMe string,
WebsiteUrl string, LastAccessDate timestamp, upvotes bigint,
ProfileImageUrl string, DisplayName string,
Id BigInt, Reputation BIGINT, DownVotes bigint,
Age int, Location String)
ROW FORMAT SERDE "com.bizo.hive.serde.csv.CSVSerde"
location "/user/cloudera/users";

Und die Dateizeilen haben folgendes Format:

"2008-08-01T12:09:11.010","1347","14","","http://some.url","2016-01-15T01:44:05.733","369","","User name","20","6943","38","","Some location"
"2008-08-01T12:11:11.897","830","15","","http://some.url","2016-06-11T01:38:09.770","191","","User name","22","8727","5","30","Some location"

Wenn ich aber ein desc stackoverflow_usersIch sehe folgendes:

+------------------+------------+--------------------+--+
|     col_name     | data_type  |      comment       |
+------------------+------------+--------------------+--+
| creationdate     | string     | from deserializer  |
| views            | string     | from deserializer  |
| accountid        | string     | from deserializer  |
| aboutme          | string     | from deserializer  |
| websiteurl       | string     | from deserializer  |
| lastaccessdate   | string     | from deserializer  |
| upvotes          | string     | from deserializer  |
| profileimageurl  | string     | from deserializer  |
| displayname      | string     | from deserializer  |
| id               | string     | from deserializer  |
| reputation       | string     | from deserializer  |
| downvotes        | string     | from deserializer  |
| age              | string     | from deserializer  |
| location         | string     | from deserializer  |
+------------------+------------+--------------------+--+

Warum ist alles eine Zeichenfolge?

Antworten:

Das Problem liegt bei der verwendeten SerDe. Es wird auch berichtet Hier