You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I had searched in the issues and found no similar issues.
What happened
dinky task error log: 2025-01-15 15:03:58.229 INFO org.dinky.service.task.FlinkSqlTask(69): Initializing Flink job config... 2025-01-15 15:03:58.234 INFO org.dinky.trans.AbstractOperation(86): Start build CDCSOURCE Task... 2025-01-15 15:03:58.238 INFO org.dinky.trans.AbstractOperation(184): A total of 1 tables were detected... 2025-01-15 15:03:58.238 INFO org.dinky.trans.AbstractOperation(186): 1: mysql-database.mysql-table 2025-01-15 15:03:58.238 INFO org.dinky.trans.AbstractOperation(195): Set parallelism: 1 2025-01-15 15:03:58.238 INFO org.dinky.trans.AbstractOperation(199): Set checkpoint: 10000 2025-01-15 15:03:58.238 INFO org.dinky.trans.AbstractOperation(202): Build mysql-cdc successful... 2025-01-15 15:03:58.239 INFO org.dinky.cdc.AbstractSinkBuilder(190): Build deserialize successful... 2025-01-15 15:03:58.239 INFO org.dinky.cdc.AbstractSinkBuilder(203): Build partitionBy successful... 2025-01-15 15:03:58.240 INFO org.dinky.cdc.AbstractSinkBuilder(205): Build shunt successful... 2025-01-15 15:03:58.240 ERROR org.dinky.cdc.AbstractSinkBuilder(152): Build mysql-database.mysql-table cdc sync failed... 2025-01-15 15:03:58.240 ERROR org.dinky.cdc.AbstractSinkBuilder(153): org.apache.flink.table.api.ValidationException: Variable character string length must be between 1 and 2147483647 (both inclusive). at org.apache.flink.table.types.logical.VarCharType.<init>(VarCharType.java:75) at org.apache.flink.table.types.logical.VarCharType.<init>(VarCharType.java:83) at org.dinky.cdc.convert.DataTypeConverter.getLogicalType(DataTypeConverter.java:113) at org.dinky.cdc.AbstractSinkBuilder.buildColumn(AbstractSinkBuilder.java:327) at org.dinky.cdc.sql.AbstractSqlSinkBuilder.lambda$addTableSinkForTags$0(AbstractSqlSinkBuilder.java:143) at java.base/java.util.LinkedHashMap.forEach(Unknown Source) at org.dinky.cdc.sql.AbstractSqlSinkBuilder.addTableSinkForTags(AbstractSqlSinkBuilder.java:135) at org.dinky.cdc.sql.AbstractSqlSinkBuilder.buildPipeline(AbstractSqlSinkBuilder.java:206) at org.dinky.cdc.AbstractSinkBuilder.build(AbstractSinkBuilder.java:293) at org.dinky.trans.ddl.CreateCDCSourceOperation.execute(CreateCDCSourceOperation.java:203) at org.dinky.interceptor.FlinkInterceptor.build(FlinkInterceptor.java:53) at org.dinky.executor.Executor.pretreatExecute(Executor.java:247) at org.dinky.executor.Executor.executeSql(Executor.java:260) at org.dinky.job.runner.JobPipelineRunner.run(JobPipelineRunner.java:76) at org.dinky.job.JobManager.executeSql(JobManager.java:289) at org.dinky.service.task.FlinkSqlTask.execute(FlinkSqlTask.java:70) at org.dinky.service.impl.TaskServiceImpl.executeJob(TaskServiceImpl.java:211) at org.dinky.service.impl.TaskServiceImpl.executeJob(TaskServiceImpl.java:202) at org.dinky.service.impl.TaskServiceImpl$$FastClassBySpringCGLIB$$22087f7c.invoke(<generated>)
What you expected to happen
Task is running normally
How to reproduce
run this dinky job
`
SET 'table.exec.sink.upsert-materialize' = 'NONE';
Search before asking
What happened
dinky task error log:
2025-01-15 15:03:58.229 INFO org.dinky.service.task.FlinkSqlTask(69): Initializing Flink job config... 2025-01-15 15:03:58.234 INFO org.dinky.trans.AbstractOperation(86): Start build CDCSOURCE Task... 2025-01-15 15:03:58.238 INFO org.dinky.trans.AbstractOperation(184): A total of 1 tables were detected... 2025-01-15 15:03:58.238 INFO org.dinky.trans.AbstractOperation(186): 1: mysql-database.mysql-table 2025-01-15 15:03:58.238 INFO org.dinky.trans.AbstractOperation(195): Set parallelism: 1 2025-01-15 15:03:58.238 INFO org.dinky.trans.AbstractOperation(199): Set checkpoint: 10000 2025-01-15 15:03:58.238 INFO org.dinky.trans.AbstractOperation(202): Build mysql-cdc successful... 2025-01-15 15:03:58.239 INFO org.dinky.cdc.AbstractSinkBuilder(190): Build deserialize successful... 2025-01-15 15:03:58.239 INFO org.dinky.cdc.AbstractSinkBuilder(203): Build partitionBy successful... 2025-01-15 15:03:58.240 INFO org.dinky.cdc.AbstractSinkBuilder(205): Build shunt successful... 2025-01-15 15:03:58.240 ERROR org.dinky.cdc.AbstractSinkBuilder(152): Build mysql-database.mysql-table cdc sync failed... 2025-01-15 15:03:58.240 ERROR org.dinky.cdc.AbstractSinkBuilder(153): org.apache.flink.table.api.ValidationException: Variable character string length must be between 1 and 2147483647 (both inclusive). at org.apache.flink.table.types.logical.VarCharType.<init>(VarCharType.java:75) at org.apache.flink.table.types.logical.VarCharType.<init>(VarCharType.java:83) at org.dinky.cdc.convert.DataTypeConverter.getLogicalType(DataTypeConverter.java:113) at org.dinky.cdc.AbstractSinkBuilder.buildColumn(AbstractSinkBuilder.java:327) at org.dinky.cdc.sql.AbstractSqlSinkBuilder.lambda$addTableSinkForTags$0(AbstractSqlSinkBuilder.java:143) at java.base/java.util.LinkedHashMap.forEach(Unknown Source) at org.dinky.cdc.sql.AbstractSqlSinkBuilder.addTableSinkForTags(AbstractSqlSinkBuilder.java:135) at org.dinky.cdc.sql.AbstractSqlSinkBuilder.buildPipeline(AbstractSqlSinkBuilder.java:206) at org.dinky.cdc.AbstractSinkBuilder.build(AbstractSinkBuilder.java:293) at org.dinky.trans.ddl.CreateCDCSourceOperation.execute(CreateCDCSourceOperation.java:203) at org.dinky.interceptor.FlinkInterceptor.build(FlinkInterceptor.java:53) at org.dinky.executor.Executor.pretreatExecute(Executor.java:247) at org.dinky.executor.Executor.executeSql(Executor.java:260) at org.dinky.job.runner.JobPipelineRunner.run(JobPipelineRunner.java:76) at org.dinky.job.JobManager.executeSql(JobManager.java:289) at org.dinky.service.task.FlinkSqlTask.execute(FlinkSqlTask.java:70) at org.dinky.service.impl.TaskServiceImpl.executeJob(TaskServiceImpl.java:211) at org.dinky.service.impl.TaskServiceImpl.executeJob(TaskServiceImpl.java:202) at org.dinky.service.impl.TaskServiceImpl$$FastClassBySpringCGLIB$$22087f7c.invoke(<generated>)
What you expected to happen
Task is running normally
How to reproduce
run this dinky job
`
SET 'table.exec.sink.upsert-materialize' = 'NONE';
EXECUTE CDCSOURCE demo
WITH
(
'connector' = 'mysql-cdc',
'hostname' = 'mysql-host.com',
'port' = '3306',
'username' = 'mysql-username',
'password' = 'mysql-password',
'checkpoint' = '10000',
'parallelism' = '1',
'database-name' = 'mysql-database',
'table-name' = 'mysql-database.mysql-table',
'server-time-zone' = 'Asia/Shanghai',
'scan.startup.mode' = 'initial',
'server-id' = '100-110',
'sink.connector' = 'paimon',
'sink.database' = 'test',
'sink.metastore' = 'filesystem',
'sink.path' = 'oss://paimon/pre/',
'sink.warehouse' = 'oss://paimon/pre/',
'sink.fs.oss.endpoint' = 'oss-endpoint',
'sink.fs.oss.accessKeyId' = 'xxxx',
'sink.fs.oss.accessKeySecret' = 'xxx'
);
`
Anything else
No response
Version
1.2.0
Are you willing to submit PR?
Code of Conduct
The text was updated successfully, but these errors were encountered: