We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
I am trying to follow the doc. but I face this problem.
I created the topic in Kafka and inserted some data on it. also, the schema is created, but the table is not.
my schema: { "schemaName": "events", "enableColumnBasedNullHandling": false, "dimensionFieldSpecs": [ { "name": "uuid", "dataType": "STRING", "notNull": false } ], "metricFieldSpecs": [ { "name": "count", "dataType": "INT", "notNull": false } ], "dateTimeFieldSpecs": [ { "name": "ts", "dataType": "TIMESTAMP", "notNull": false, "format": "TIMESTAMP", "granularity": "1:MILLISECONDS" } ] }
Table conf.: { "tableName": "events", "tableType": "REALTIME", "segmentsConfig": { "timeColumnName": "ts", "schemaName": "events", "replicasPerPartition": "1" }, "tenants": { "broker": "DefaultTenant", "server": "DefaultTenant" }, "tableIndexConfig": { "loadMode": "MMAP", "streamConfigs": { "streamType": "kafka", "stream.kafka.broker.list": "kafka:9092", "stream.kafka.topic.name": "events", "stream.kafka.consumer.type": "simple", "stream.kafka.decoder.class.name": "org.apache.pinot.plugin.stream.kafka.KafkaJSONMessageDecoder", "realtime.segment.flush.threshold.rows": "10000" } }, "metadata": { "customConfigs": {} } } the error message: { "code": 500, "error": "org.apache.pinot.spi.stream.TransientConsumerException: org.apache.pinot.shaded.org.apache.kafka.common.errors.TimeoutException: Failed to get offsets by times in 5000ms" }
Kafka topic: madhhar@madhharus:$ docker exec -it kafka kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic events --from-beginning {"ts": 1735112369985, "uuid": "c4456f4e-611b-48b6-b6da-550b4052c9d7", "count": 318} {"ts": 1735112370987, "uuid": "90727af9-0914-40b3-8af6-65f1c678d676", "count": 911} {"ts": 1735112371988, "uuid": "a280bbee-3e86-482f-839e-1872c0028995", "count": 768} {"ts": 1735112372990, "uuid": "3ee49eb9-b1b3-40b0-8173-938f32887e3e", "count": 78} {"ts": 1735112373991, "uuid": "065c78d7-c05d-4e9f-9d25-c5960b9d38cb", "count": 947} {"ts": 1735112374992, "uuid": "3d811d72-050f-4b10-a5aa-e332f6acb411", "count": 450} {"ts": 1735112375993, "uuid": "733eece5-2a57-48ef-a35c-0c3cf365f665", "count": 715} {"ts": 1735112376995, "uuid": "6be53521-969c-401c-a27a-8c2ef19bc0a3", "count": 759} {"ts": 1735112377995, "uuid": "b6075c33-50d8-4b5e-854a-45b14e0b9119", "count": 212} {"ts": 1735112378996, "uuid": "86734094-8e15-4376-9c3e-b56618e2bb09", "count": 875} {"ts": 1735112379998, "uuid": "fdfb2cce-bcce-4022-a3ee-827cb5f2170b", "count": 747} {"ts": 1735112380999, "uuid": "0c45bcec-161a-412d-bf38-e1c92f716a80", "count": 547} {"ts": 1735112382001, "uuid": "6fd0a307-d5d9-49fd-96af-67a5ce3f7d98", "count": 608} {"ts": 1735112383002, "uuid": "04e4a8dc-1de0-4f1a-88a5-af7a343f59a4", "count": 46} {"ts": 1735112384003, "uuid": "640944e6-abdf-49c5-8ef3-dd97e00dd676", "count": 224} {"ts": 1735112385005, "uuid": "7d41bd43-145b-4606-891f-fe1d04c5ab78", "count": 238} {"ts": 1735112386006, "uuid": "5a522e1c-cf89-47e4-9a51-07f9a800d3e0", "count": 244} {"ts": 1735112387007, "uuid": "0048f77f-38f4-4765-9c83-7d447b421095", "count": 779} {"ts": 1735112388008, "uuid": "31842f55-dd3a-44f8-beac-94fa7c7048a5", "count": 241} {"ts": 1735112389010, "uuid": "bbc6b245-d964-45ac-a106-13e2702130ab", "count": 994} {"ts": 1735112390011, "uuid": "1ad177f9-88ae-4559-8b70-bf37905d504e", "count": 122} {"ts": 1735112391013, "uuid": "bf2b002e-130e-42f8-be62-459a520727e3", "count": 394} {"ts": 1735112392014, "uuid": "d11d30f0-eeab-4efd-b94a-8a3258635178", "count": 600} {"ts": 1735112393015, "uuid": "9dd78e7a-96c0-4050-add0-4cceb5bc6bb6", "count": 114} {"ts": 1735112394016, "uuid": "6d3e629f-193a-400f-bd66-6d3fabeb1971", "count": 371} {"ts": 1735112395018, "uuid": "14f78ecb-fa2a-4417-84dc-23e408cd6705", "count": 272} {"ts": 1735112396019, "uuid": "fec5ca47-aa27-4a9b-851f-bcb275e1f3e7", "count": 323} {"ts": 1735112397021, "uuid": "56979da0-a18e-4178-81b2-6fbc6b41b861", "count": 888} {"ts": 1735112398022, "uuid": "1409767f-7745-4a70-a0e0-b23ef55888c8", "count": 658} {"ts": 1735112399024, "uuid": "1905899b-406b-499b-8abd-58a24f553b1e", "count": 151} {"ts": 1735112400025, "uuid": "45d6f861-5e94-4324-b6b9-d00e77cb60d9", "count": 433} {"ts": 1735112401026, "uuid": "7aae3d5b-399f-4a90-9fad-9e4bab167d75", "count": 860} {"ts": 1735112402028, "uuid": "19add349-d31c-44b3-ab8b-0cc1013bb037", "count": 300} {"ts": 1735112403029, "uuid": "1a878f82-302a-4895-bced-ae4b7ba7f48d", "count": 992} {"ts": 1735112404031, "uuid": "3dc66dad-e9e8-4ee6-95d9-d9a2e89c482d", "count": 352} ^CProcessed a total of 35 messages madhhar@madhharus:$
The text was updated successfully, but these errors were encountered:
Seems like a connection issue. Let me know if the problem is solved now. You may get more help from the Slack community.
Sorry, something went wrong.
No branches or pull requests
I am trying to follow the doc. but I face this problem.
I created the topic in Kafka and inserted some data on it. also, the schema is created, but the table is not.
my schema:
{
"schemaName": "events",
"enableColumnBasedNullHandling": false,
"dimensionFieldSpecs": [
{
"name": "uuid",
"dataType": "STRING",
"notNull": false
}
],
"metricFieldSpecs": [
{
"name": "count",
"dataType": "INT",
"notNull": false
}
],
"dateTimeFieldSpecs": [
{
"name": "ts",
"dataType": "TIMESTAMP",
"notNull": false,
"format": "TIMESTAMP",
"granularity": "1:MILLISECONDS"
}
]
}
Table conf.:
{
"tableName": "events",
"tableType": "REALTIME",
"segmentsConfig": {
"timeColumnName": "ts",
"schemaName": "events",
"replicasPerPartition": "1"
},
"tenants": {
"broker": "DefaultTenant",
"server": "DefaultTenant"
},
"tableIndexConfig": {
"loadMode": "MMAP",
"streamConfigs": {
"streamType": "kafka",
"stream.kafka.broker.list": "kafka:9092",
"stream.kafka.topic.name": "events",
"stream.kafka.consumer.type": "simple",
"stream.kafka.decoder.class.name": "org.apache.pinot.plugin.stream.kafka.KafkaJSONMessageDecoder",
"realtime.segment.flush.threshold.rows": "10000"
}
},
"metadata": {
"customConfigs": {}
}
}
the error message:
{
"code": 500,
"error": "org.apache.pinot.spi.stream.TransientConsumerException: org.apache.pinot.shaded.org.apache.kafka.common.errors.TimeoutException: Failed to get offsets by times in 5000ms"
}
Kafka topic:
madhhar@madhharus:
$ docker exec -it kafka kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic events --from-beginning${"ts": 1735112369985, "uuid": "c4456f4e-611b-48b6-b6da-550b4052c9d7", "count": 318}
{"ts": 1735112370987, "uuid": "90727af9-0914-40b3-8af6-65f1c678d676", "count": 911}
{"ts": 1735112371988, "uuid": "a280bbee-3e86-482f-839e-1872c0028995", "count": 768}
{"ts": 1735112372990, "uuid": "3ee49eb9-b1b3-40b0-8173-938f32887e3e", "count": 78}
{"ts": 1735112373991, "uuid": "065c78d7-c05d-4e9f-9d25-c5960b9d38cb", "count": 947}
{"ts": 1735112374992, "uuid": "3d811d72-050f-4b10-a5aa-e332f6acb411", "count": 450}
{"ts": 1735112375993, "uuid": "733eece5-2a57-48ef-a35c-0c3cf365f665", "count": 715}
{"ts": 1735112376995, "uuid": "6be53521-969c-401c-a27a-8c2ef19bc0a3", "count": 759}
{"ts": 1735112377995, "uuid": "b6075c33-50d8-4b5e-854a-45b14e0b9119", "count": 212}
{"ts": 1735112378996, "uuid": "86734094-8e15-4376-9c3e-b56618e2bb09", "count": 875}
{"ts": 1735112379998, "uuid": "fdfb2cce-bcce-4022-a3ee-827cb5f2170b", "count": 747}
{"ts": 1735112380999, "uuid": "0c45bcec-161a-412d-bf38-e1c92f716a80", "count": 547}
{"ts": 1735112382001, "uuid": "6fd0a307-d5d9-49fd-96af-67a5ce3f7d98", "count": 608}
{"ts": 1735112383002, "uuid": "04e4a8dc-1de0-4f1a-88a5-af7a343f59a4", "count": 46}
{"ts": 1735112384003, "uuid": "640944e6-abdf-49c5-8ef3-dd97e00dd676", "count": 224}
{"ts": 1735112385005, "uuid": "7d41bd43-145b-4606-891f-fe1d04c5ab78", "count": 238}
{"ts": 1735112386006, "uuid": "5a522e1c-cf89-47e4-9a51-07f9a800d3e0", "count": 244}
{"ts": 1735112387007, "uuid": "0048f77f-38f4-4765-9c83-7d447b421095", "count": 779}
{"ts": 1735112388008, "uuid": "31842f55-dd3a-44f8-beac-94fa7c7048a5", "count": 241}
{"ts": 1735112389010, "uuid": "bbc6b245-d964-45ac-a106-13e2702130ab", "count": 994}
{"ts": 1735112390011, "uuid": "1ad177f9-88ae-4559-8b70-bf37905d504e", "count": 122}
{"ts": 1735112391013, "uuid": "bf2b002e-130e-42f8-be62-459a520727e3", "count": 394}
{"ts": 1735112392014, "uuid": "d11d30f0-eeab-4efd-b94a-8a3258635178", "count": 600}
{"ts": 1735112393015, "uuid": "9dd78e7a-96c0-4050-add0-4cceb5bc6bb6", "count": 114}
{"ts": 1735112394016, "uuid": "6d3e629f-193a-400f-bd66-6d3fabeb1971", "count": 371}
{"ts": 1735112395018, "uuid": "14f78ecb-fa2a-4417-84dc-23e408cd6705", "count": 272}
{"ts": 1735112396019, "uuid": "fec5ca47-aa27-4a9b-851f-bcb275e1f3e7", "count": 323}
{"ts": 1735112397021, "uuid": "56979da0-a18e-4178-81b2-6fbc6b41b861", "count": 888}
{"ts": 1735112398022, "uuid": "1409767f-7745-4a70-a0e0-b23ef55888c8", "count": 658}
{"ts": 1735112399024, "uuid": "1905899b-406b-499b-8abd-58a24f553b1e", "count": 151}
{"ts": 1735112400025, "uuid": "45d6f861-5e94-4324-b6b9-d00e77cb60d9", "count": 433}
{"ts": 1735112401026, "uuid": "7aae3d5b-399f-4a90-9fad-9e4bab167d75", "count": 860}
{"ts": 1735112402028, "uuid": "19add349-d31c-44b3-ab8b-0cc1013bb037", "count": 300}
{"ts": 1735112403029, "uuid": "1a878f82-302a-4895-bced-ae4b7ba7f48d", "count": 992}
{"ts": 1735112404031, "uuid": "3dc66dad-e9e8-4ee6-95d9-d9a2e89c482d", "count": 352}
^CProcessed a total of 35 messages
madhhar@madhharus:
The text was updated successfully, but these errors were encountered: