各阶段的数据格式

数据格式支持

本附录列出了源端、处理器和目标端阶段支持的数据格式。

源端

下表列出了每个源端支持的数据格式。

Origin Avro Binary Datagram Delimited Excel JSON Log Protobuf SDC Record Text Whole File XML
Amazon S3
Amazon SQS Consumer
Azure Data Lake Storage Gen1
Aure Data Lake Storage Gen2
Azure IoT/Event Hub Consumer
CoAP Server
Directory
Elasticsearch * * * Not Applicable * * *
File Tail
Google BigQuery * * * Not Applicable * * *
Google Cloud Storage
Google Pub/Sub Subscriber
Groovy Scripting * * * Not Applicable * * *
gRPC Client
Hadoop FS
Hadoop FS Standalone
HTTP Client
HTTP Server
HTTP to Kafka * * * Not Applicable * * *
JavaScript Scripting * * * Not Applicable * * *
JDBC Multitable Consumer * * * Not Applicable * * *
JDBC Query Consumer * * * Not Applicable * * *
JMS Consumer
Jython Scripting * * * Not Applicable * * *
Kafka Consumer
Kafka Multitopic Consumer
Kinesis Consumer
MapR DB CDC * * * Not Applicable * * *
MapR DB JSON * * * Not Applicable * * *
MapR FS
MapR FS Standalone
MapR Multitopic Streams Consumer
MapR Streams Consumer
MongoDB * * * Not Applicable * * *
MongoDB Oplog * * * Not Applicable * * *
MQTT Subscriber
MySQL Binary Log * * * Not Applicable * * *
Omniture * * * Not Applicable * * *
OPC UA Client * * * Not Applicable * * *
Oracle Bulkload * * * Not Applicable * * *
Oracle CDC Client * * * Not Applicable * * *
PostgreSQL CDC Client * * * Not Applicable * * *
Pulsar Consumer
RabbitMQ Consumer
Redis Consumer
REST Service
Salesforce * * * Not Applicable * * *
SDC RPC
SDC RPC to Kafka
SFTP/FTP/FTPS Client
SQL Server 2019 BDC Multitable Consumer * * * Not Applicable * * *
SQL Server CDC Client * * * Not Applicable * * *
SQL Server Change Tracking * * * Not Applicable * * *
System Metrics * * * Not Applicable * * *
TCP Server
Teradata Consumer * * * Not Applicable * * *
UDP Multithreaded Source * * * Not Applicable * * *
UDP Source * * * Not Applicable * * *
UDP to Kafka * * * Not Applicable * * *
WebSocket Client
WebSocket Server
Windows Event Log * * * Not Applicable * * *

处理器

下表列出了读取所列格式数据的处理器:

Processor Avro Binary Datagram Delimited JSON Log Netflow Protobuf SDC Record Syslog Text XML
Data Parser
HTTP Client
JSON Parser
Log Parser
XML Parser

下表列出了将指定数据格式的数据写入字段的处理器:

Processor Avro Binary Delimited JSON Log Protobuf SDC Record Text XML
Data Generator
JSON Generator

目标端

下表列出了每个目标端支持的数据格式。

Destination Avro Binary Delimited JSON Protobuf SDC Record Text Whole File XML
Aerospike * * * Not Applicable * * *
Amazon S3
Azure Data Lake Storage
Azure Data Lake Storage Gen1
Azure Data Lake Storage Gen2
Azure Event Hub Producer
Azure IoT Hub Producer
Cassandra * * * Not Applicable * * *
CoAP Client
Couchbase
Elasticsearch * * * Not Applicable * * *
Einstein Analytics * * * Not Applicable * * *
Flume
Google BigQuery * * * Not Applicable * * *
Google Bigtable * * * Not Applicable * * *
Google Cloud Storage
Google Pub/Sub Publisher
GPSS Producer * * * Not Applicable * * *
Hadoop FS
HBase * * * Not Applicable * * *
Hive Metastore
Hive Streaming * * * Not Applicable * * *
HTTP Client
InfluxDB * * * Not Applicable * * *
JDBC Producer * * * Not Applicable * * *
JMS Producer
Kafka Producer
Kinesis Firehose
Kinesis Producer
KineticaDB * * * Not Applicable * * *
Kudu * * * Not Applicable * * *
Local FS
MapR DB * * * Not Applicable * * *
MapR DB JSON * * * Not Applicable * * *
MapR FS
MapR Streams Producer
MemSQL Fast Loader * * * Not Applicable * * *
MongoDB * * * Not Applicable * * *
MQTT Publisher
Named Pipe
Pulsar Producer
RabbitMQ Producer
Redis
Salesforce * * * Not Applicable * * *
Send Response to Origin
SDC RPC
SFTP/FTP/FTPS Client
Snowflake * * * Not Applicable * * *
Solr * * * Not Applicable * * *
Splunk * * * Not Applicable * * *
SQL Server 2019 BDC Bulk Loader * * * Not Applicable * * *
Syslog
To Error * * * Not Applicable * * *
Trash * * * Not Applicable * * *
WebSocket Client