SingleData Connector

The singleData connector supplements the OLAP capability for the openGauss, enhances the capability of data analysis.

The singleData mode can be ShardingSphere or tidRange

ShardingSphere Mode

Overview

Apache ShardingSphere is a distributed database ecosystem. It can convert any database into a distributed database. SingleData Connector can use ShardingSphere to shard openGauss data to improve analysis efficiency.

For more information, please visit: ShardingSphere official website

Configuration

To configure the singleData Connector for ShardingSphere mode, create a catalog properties file etc/catalog named, for example, shardingsphere.properties. Create the file with the following contents, replacing the connection properties as appropriate for your setup:

  • Basic Configuration
connector.name=singledata
singledata.mode=SHARDING_SPHERE
shardingsphere.database-name=sharding_db
shardingsphere.type=zookeeper
shardingsphere.namespace=governance_ds
shardingsphere.server-list=localhost:2181
  • Configuration Properties
Property NameDescriptionrequired
shardingsphere.database-nameName of the connected shardingSphere databaseYes
shardingsphere.typePersistent repository type of the registration center.Zookeeper and etcd are supported. Zookeeper is recommendedYes
shardingsphere.namespaceNameSpace of the registry centerYes
shardingsphere.server-listsConnection address list of the registry centerYes
  • Optional configuration when the persistent repository type is zookeeper
Property NameDescriptionDefault
shardingsphere.zookeeper.retry-interval-millisecondsRetry interval after connection failure in milliseconds500
shardingsphere.zookeeper.max-retriesMaximum retry connections3
shardingsphere.zookeeper.time-to-live-secondsTemporary node lifetime seconds60
shardingsphere.zookeeper.operation-timeout-millisecondsOperation timeout milliseconds500
shardingsphere.zookeeper.digestToken"”
  • Optional configuration when the persistent repository type is etcd
Property NameDescriptionDefault
shardingsphere.etcd.time-to-live-secondsTemporary node lifetime seconds30
shardingsphere.etcd.connection-timeoutConnection timeout seconds3

Note:

  • SingleData connector only supports ShardingSphere 5.2.0 now

TidRange Mode

Overview

OpenGauss data is stored in HEAP PAGE by row. Each row of data has a corresponding ctid (line number). The tidrangescan plugin of openGauss can use the range of ctid for query. The tidRange mode uses this plugin to implement parrallel data analysis.

To obtain the tidrangescan plugin, please visit: Plugin

Connection Configuration

To configure the singleData Connector for tidRange mode, create a catalog properties file etc/catalog named, for example, tidrange.properties. Create the file with the following contents, replacing the connection properties as appropriate for your setup:

connection.name=singledata
connection.mode=TID_RANGE
connection-url=jdbc:opengauss://master-host:port/database;jdbc:opengauss://slave-host:port/database
connection-user=user
connection-password=password
  • connection-url can be configured multiple JDBC connection addresses. The addresses are separated by ;. During query, each segment randomly selects a connection address for connection.
  • For other connection configurations, see the openGauss connector

Split Configuration

Property NameDescriptionRequiredDefault
tidrange.max-split-countMax count of split, that is, the maximum number of JDBC connections. The value must be less than or equal to the value of max_connections in openGaussNo100
tidrange.page-sizePage size of the openGauss. Ensure that the value is same as the value of block_size of the openGauss. Otherwise, the query result may be incorrectNo8kB
tidrange.default-split-sizeDefault size of each split. When the data size is small, the singleData connector fragments the data bases on this configuration. The value ranges from 1MB to 1GBNo32MB

NOTE:

  • TidRange mode needs to be used with tidrangescan plugin of the openGauss, without tidrangescan plugin, the singledata connector can be used for query, but the performance deteriorates greatly.
  • TidRange will be not enabled when an index exists in the table

Limitations

  • SingleData Connector only support select statement now, statement such as INSERT/UPDATE/DELETE that modify data or data structures are not supported.
  • The maximum decimal precision of openLookeng is 38. If the precision of Decimal or Numeric is greater than 38 in openGauss, it will be not supported.
  • The openGauss version required 3.0.0 or later

有奖捉虫

“有虫”文档片段

0/500

存在的问题

文档存在风险与错误

● 拼写,格式,无效链接等错误;

● 技术原理、功能、规格等描述和软件不一致,存在错误;

● 原理图、架构图等存在错误;

● 版本号不匹配:文档版本或内容描述和实际软件不一致;

● 对重要数据或系统存在风险的操作,缺少安全提示;

● 排版不美观,影响阅读;

内容描述不清晰

● 描述存在歧义;

● 图形、表格、文字等晦涩难懂;

● 逻辑不清晰,该分类、分项、分步骤的没有给出;

内容获取有困难

● 很难通过搜索引擎,openLooKeng官网,相关博客找到所需内容;

示例代码有错误

● 命令、命令参数等错误;

● 命令无法执行或无法完成对应功能;

内容有缺失

● 关键步骤错误或缺失,无法指导用户完成任务,比如安装、配置、部署等;

● 逻辑不清晰,该分类、分项、分步骤的没有给出

● 图形、表格、文字等晦涩难懂

● 缺少必要的前提条件、注意事项等;

● 描述存在歧义

0/500

您对文档的总体满意度

非常不满意
非常满意

请问是什么原因让您参与到这个问题中

您的邮箱

创Issue赢奖品
根据您的反馈,会自动生成issue模板。您只需点击按钮,创建issue即可。
有奖捉虫