Databricks now supports on-demand state repartitioning for Apache Spark Structured Streaming, allowing users to resize partitions without rebuilding checkpoint state, enabling more flexible tuning and scaling of stateful streaming queries. This feature is available in Databricks Runtime 18 and above with the RocksDB state store provider. AI summary
Firehose
Filtered to Companies, tagged “state repartitioning” · clear filters
Browse: People · Companies · Papers · Podcasts · Hacker News · Deep dives