Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

routine load消费kakfa一条数据解析失败导致任务停止 #55052

Open
lukoou3 opened this issue Jan 14, 2025 · 0 comments
Open

routine load消费kakfa一条数据解析失败导致任务停止 #55052

lukoou3 opened this issue Jan 14, 2025 · 0 comments
Labels
type/bug Something isn't working

Comments

@lukoou3
Copy link

lukoou3 commented Jan 14, 2025

Steps to reproduce the behavior (Required)

create routine load tsg_olap.load_object_statistics on object_statistics columns (
    timestamp_ms,
    uuid,
    __time = from_unixtime(timestamp_ms/1000),
    type,
    bytes,
    bytes = ifnull(bytes, 0)
)
properties (
    'format' = 'json',
    'max_batch_interval' = '5',
    'max_batch_rows' = '200000',
    'max_batch_size' = '209715200',
    'max_filter_ratio' = '0.9',
    'max_error_number' = '9223372036854775807'
)
from kafka (
    -- ...
    'property.kafka_default_offsets' = 'offset_end'
);

发送端发送数据有些记录编码异常(object_type字段值编码异常),导致解析失败,导致整个任务失败。

{"uuid":"01944ecb-3214-7ea7-b647-56a96d67c610","type":"�  B.�","bytes":126,"timestamp_ms":1736488787000}
Error: Data quality error: Failed to iterate document stream as object. error: UTF8_ERROR: The input is not valid UTF-8. Row: parser current location: {"uuid":"01944add-fbff-75b0-892a-71b78d2037d8","ty

是否能有忽略解析错误的配置?

Expected behavior (Required)

有忽略解析错误的配置

Real behavior (Required)

StarRocks version (Required)

3.3.7

@lukoou3 lukoou3 added the type/bug Something isn't working label Jan 14, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type/bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant